fix(mcp): a failed handshake must not strand the child process
Nightly Build / build (push) Successful in 4m5s

An MCP server whose `initialize` answer is an error — a broken or version-
mismatched connector — starts fine and then never exits. `McpServer::start`
returned `Err` correctly, but the `Child` lives in the read-loop task rather
than in the returned value, so `kill_on_drop` followed a task nothing ever
drops. Every retry therefore left a live process holding three pipes and a
pidfd, and the supervisor's retry ceiling is deliberately not permanent.

The end state was not a dead connector but a dead instance: the process hit
its 1024-descriptor limit, `accept()` began failing with EMFILE, and incoming
connections queued on a socket nobody could accept from — while the process,
the port and every other connector still looked healthy. Observed in
production at ~5h from the first bad handshake to unreachable, with 229
orphaned interpreters.

`stop_server`/`stop_all` had the same hole from the other side: they document
the dropped handle as killing the process, but the task holds its end of
stdin, so the child stayed blocked on a read that would never return.

Both close with one seam. `McpServer` now owns a oneshot sender whose receiver
the read-loop selects on; nothing ever sends, so the drop is the message. That
covers a deliberate stop, the last `Arc` going away, and a `?` in `start()`
unwinding past the local binding before it was ever returned — including the
caller's `timeout`, which drops the same future. The loop then kills and, as
importantly, reaps: an unreaped child trades the orphan for a zombie holding
the same pipes.

Both leaks are covered by tests that fail without the fix.

Also raise LimitNOFILE to 65536: the installers write it, and update.sh heals
an existing unit additively, leaving an admin's own value alone. The leak is
the bug, but 1024 for a process sharing descriptors between the listener,
every user's SQLite handles and three pipes per connector is thin regardless.
This commit is contained in:
Daniele
2026-08-24 17:34:44 +01:00
parent 72fa40708a
commit 67fc1455c5
7 changed files with 340 additions and 11 deletions
+5
View File
@@ -427,6 +427,11 @@ WorkingDirectory=${INSTALL_DIR}
# is unaffected — systemd never restarts after a requested stop.
Restart=always
RestartSec=5
# The default soft limit is 1024, which one process shares between the HTTP
# listener, every user's SQLite handles and three pipes per connector process.
# Running out does not degrade gracefully: accept() starts failing with EMFILE
# and the whole app stops answering while still looking healthy from outside.
LimitNOFILE=65536
Environment=SKALD_BIN=${INSTALL_DIR}/bin/skald
Environment=SKALD_SETUP_BIN=${INSTALL_DIR}/bin/skald-setup