Skip to content

fix(schedules): close log write stream after dokploy-server run - #5478

Open
mszortul wants to merge 2 commits into
Dokploy:canaryfrom
mszortul:fix/schedule-log-stream-close
Open

mszortul wants to merge 2 commits into
Dokploy:canaryfrom
mszortul:fix/schedule-log-stream-close

Conversation

@mszortul

@mszortul mszortul commented Sep 18, 2026

Copy link
Copy Markdown

runCommand opens a write stream for the deployment log in the dokploy-server branch and never closes it, so every scheduled run leaks one file descriptor for the lifetime of the process.

packages/server/src/utils/schedules/utils.ts (canary, L149):

const writeStream = createWriteStream(deployment.logPath, { flags: "a" });
const { SCHEDULES_PATH } = paths();
const fullPath = path.join(SCHEDULES_PATH, appName || "");

await spawnAsync("bash", ["-c", "./script.sh"], async (data) => { ... }, { cwd: fullPath });

No end(), close() or destroy() on any path. fs.WriteStream has no GC finalizer, so the fd is held until the process exits. The outer catch cannot close it either — the stream is scoped to the branch.

Impact

Three dokploy-server schedules on a one-minute cron, v0.26.3: ~3 fds/min. After roughly two months the panel hit EMFILE — the process stayed up and kept listening on 3000, but closed every connection without a reply (curl: (52) Empty reply from server), while the logs filled with:

Error: EMFILE: too many open files, open '/etc/dokploy/schedules/<slug>/<slug>-2026-09-18:11:09:00.log'

docker service ls reported 1/1 throughout. docker service update --force cleared it.

ls -l /proc/<pid>/fd showed one open log file per run, several marked (deleted): removeLastTenDeployments unlinks older logs while the fds stay open, so disk usage stays flat and only the fd table grows.

Reproduction

Same pattern (createWriteStream + spawnAsync + unlink of older logs), 10 iterations under node:20.16.0-slim: 10 leaked fds, 5 of them (deleted). With writeStream.end() in a finally: none. An explicit global.gc() reclaims nothing.

Fix

Wrap the spawn in try/finally and close the stream there — covers success, a failing script, and a throw from spawnAsync. Three lines plus re-indentation; no change to the surrounding control flow.

RetriggerConfidence Score: 4/5

The PR is not yet safe to merge because a pending PID-update callback can write after the newly added stream closure.

Summary

This PR closes the deployment-log write stream after a dokploy-server scheduled command finishes, addressing the per-run file-descriptor leak.

  • Wraps command execution in try/finally.
  • Ends the log stream on successful and failed command execution.
  • Introduces a race with asynchronous output callbacks that can still be pending when the stream is ended.

Reviews (1) · Last reviewed commit: "fix(schedules): close log write stream a..."

The dokploy-server branch of runCommand opens a write stream for the
deployment log and never closes it, so every scheduled run leaks one
file descriptor for the lifetime of the process. A schedule running each
minute reaches EMFILE in weeks: the panel stays up and listening but can
no longer accept connections, and the logs fill with EMFILE on the
schedule log path.

The catch that surrounds the call cannot close the stream either, since
the const is scoped inside the try. Wrapping the spawn in try/finally
closes it on success, on a failing script, and on a throw from
spawnAsync.
Comment on lines +174 to +176
} finally {
writeStream.end();
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Stream closes before callbacks

When an output chunk contains a PID, the async callback waits for updateDeployment before writing that chunk. However, spawnAsync does not wait for callback promises, so it can finish while the update is still pending. This finally block then ends the stream, and the callback can later write after end(), dropping log output and potentially emitting an unhandled ERR_STREAM_WRITE_AFTER_END. Wait for pending callbacks before closing the stream, or write the chunk before awaiting the update.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in fd18384 — the chunk is now written before the pid update is awaited, so nothing can write after end()

spawnAsync ignores the callback's promise, so the run can finish while
an update is still pending and the stream is closed underneath it.
Writing first keeps the chunk and avoids a write after end.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant