fix(mcp): a failed handshake must not strand the child process

An MCP server whose `initialize` answer is an error — a broken or version-
mismatched connector — starts fine and then never exits. `McpServer::start`
returned `Err` correctly, but the `Child` lives in the read-loop task rather
than in the returned value, so `kill_on_drop` followed a task nothing ever
drops. Every retry therefore left a live process holding three pipes and a
pidfd, and the supervisor's retry ceiling is deliberately not permanent.

The end state was not a dead connector but a dead instance: the process hit
its 1024-descriptor limit, `accept()` began failing with EMFILE, and incoming
connections queued on a socket nobody could accept from — while the process,
the port and every other connector still looked healthy. Observed in
production at ~5h from the first bad handshake to unreachable, with 229
orphaned interpreters.

`stop_server`/`stop_all` had the same hole from the other side: they document
the dropped handle as killing the process, but the task holds its end of
stdin, so the child stayed blocked on a read that would never return.

Both close with one seam. `McpServer` now owns a oneshot sender whose receiver
the read-loop selects on; nothing ever sends, so the drop is the message. That
covers a deliberate stop, the last `Arc` going away, and a `?` in `start()`
unwinding past the local binding before it was ever returned — including the
caller's `timeout`, which drops the same future. The loop then kills and, as
importantly, reaps: an unreaped child trades the orphan for a zombie holding
the same pipes.

Both leaks are covered by tests that fail without the fix.

Also raise LimitNOFILE to 65536: the installers write it, and update.sh heals
an existing unit additively, leaving an admin's own value alone. The leak is
the bug, but 1024 for a process sharing descriptors between the listener,
every user's SQLite handles and three pipes per connector is thin regardless.

crates/mcp-client/src/server.rs

@@ -225,6 +225,31 @@ pub struct McpServer {
     /// manager can tell "this handle is dead" from "this call failed". Shared with
     /// that task, which is the only writer.
     alive: Arc<std::sync::atomic::AtomicBool>,
+    /// Ties the child process's life to this handle's.
+    ///
+    /// The read-loop owns the `Child` — it needs `wait()` for the exit status — so
+    /// `Command::kill_on_drop` follows *that task*, which nothing ever drops, rather
+    /// than this value. On its own that leaves two ways to strand a live child:
+    /// `stop_server`/`stop_all` drop a handle whose process then keeps running (the
+    /// task still holds its end of stdin, so the child blocks on a read that never
+    /// returns), and — the one that took an instance down — a `start()` that fails
+    /// *after* the spawn never produces a handle at all, so there is nothing to drop.
+    ///
+    /// That second case is the expensive one, because the natural failure is a server
+    /// which starts fine and answers `initialize` wrong: it never exits by itself, so
+    /// the supervisor mints one orphan (three pipes and a pidfd) per retry, and the
+    /// retry ceiling is deliberately not permanent. The end state is not a dead
+    /// connector but a dead *app* — the process hits its file-descriptor limit,
+    /// `accept()` begins failing with `EMFILE`, and connections pile up on a socket
+    /// nobody can accept from.
+    ///
+    /// Holding the sender here closes both: the read-loop selects on the matching
+    /// receiver, which resolves as soon as this field is dropped — whether that is a
+    /// deliberate stop, the last `Arc` going away, or a `?` in `start()` unwinding
+    /// past the local `server` binding before it was ever returned. The caller's
+    /// `timeout` is covered by the same mechanism, since dropping the `start()`
+    /// future drops that binding too.
+    _kill_on_drop: oneshot::Sender<()>,
 }
 
 impl McpServer {
@@ -328,6 +353,11 @@ impl McpServer {
             Arc::new(Mutex::new(HashMap::new()));
         let pending_elicitations = Arc::new(AtomicUsize::new(0));
 
+        // Created before the read-loop task, which is what holds the child and so is
+        // the only thing that can kill it. The sender goes into `server` below — see
+        // `McpServer::_kill_on_drop` for what that buys.
+        let (kill_tx, mut kill_rx) = oneshot::channel::<()>();
+
         let alive                    = Arc::new(std::sync::atomic::AtomicBool::new(true));
         let alive_bg                 = Arc::clone(&alive);
         let pending_bg               = pending.clone();
@@ -340,8 +370,26 @@ impl McpServer {
         tokio::spawn(async move {
             let mut child = child;
             let mut lines = BufReader::new(stdout).lines();
+            // Set when the handle went away, so the epitaph below tells a deliberate
+            // teardown apart from a server that died on its own.
+            let mut killed_by_client = false;
             loop {
-                match lines.next_line().await {
+                let next = tokio::select! {
+                    // A chatty server must not be able to starve the kill signal.
+                    biased;
+                    // Resolves when the `McpServer` holding the sender is dropped.
+                    // Nothing ever sends, so the value is always `Err(RecvError)` —
+                    // the drop *is* the message.
+                    _ = &mut kill_rx => {
+                        // `start_kill` only signals; the `wait()` below is what
+                        // reaps the child and releases its pipes.
+                        let _ = child.start_kill();
+                        killed_by_client = true;
+                        break;
+                    }
+                    line = lines.next_line() => line,
+                };
+                match next {
                     Ok(Some(line)) if !line.trim().is_empty() => {
                         if let Ok(msg) = serde_json::from_str::<Value>(&line) {
                             let has_method = msg.get("method").is_some();
@@ -380,12 +428,20 @@ impl McpServer {
                     _ => break,
                 }
             }
-            let exit_info = match child.wait().await {
-                Ok(status) if !status.success() => format!(
-                    "process exited with {}",
-                    status.code().map(|c| c.to_string()).unwrap_or_else(|| "signal".into())
-                ),
-                _ => "process exited unexpectedly".into(),
+            // Always reap, including after `start_kill`, which only signals: skipping
+            // this would trade the orphan for a zombie, and a zombie still holds the
+            // pipes that made the original leak fatal.
+            let status = child.wait().await;
+            let exit_info = if killed_by_client {
+                "stopped by the client".to_string()
+            } else {
+                match status {
+                    Ok(status) if !status.success() => format!(
+                        "process exited with {}",
+                        status.code().map(|c| c.to_string()).unwrap_or_else(|| "signal".into())
+                    ),
+                    _ => "process exited unexpectedly".into(),
+                }
             };
             // Publish the death *before* failing the pending calls: a caller woken
             // by the error below must find `is_alive() == false`, or it would
@@ -412,6 +468,10 @@ impl McpServer {
             pending_elicitations,
             server_capabilities: json!({}),
             alive,
+            // From here the child's life follows this binding: every `?` below drops
+            // it on the way out, which is what kills a server that started but never
+            // finished its handshake.
+            _kill_on_drop: kill_tx,
         };
 
         let init = server.request("initialize", json!({
@@ -465,6 +525,8 @@ impl McpServer {
             }
         }
 
+        // `..server` moves the kill sender into the returned value, so the child now
+        // outlives the handshake and dies with the handle instead.
         Ok(McpServer { tools, server_capabilities, ..server })
     }