fix: bring back an MCP connector whose process died
Nightly Build / build (push) Successful in 7m46s

A stdio connector *is* its child process, and nothing noticed when that
process went away. The handle stayed in the manager's map, so every later
tool call answered `MCP '<name>' disconnected: process exited with 139`,
and the connector's own background work stopped for good — until the user
happened to log in again.

The second half is the quiet one. A per-user connector is typically the
one that *pushes*: Gmail's poll thread produces the `event/new_email`
notifications that feed event triage. After a crash those simply stop,
with no call to fail and nothing in the UI to say so.

So a death is now reconciled, on the same terms as every other
reconciliation here: best-effort, bounded, settling at the next login if
it fails. `McpServerClient::is_alive` makes the death observable (the
read-loop clears the flag before failing the pending calls, so a caller
woken by the disconnect error finds a handle that admits it is dead), the
manager remembers the spec each server was started from, and one seam —
`restart_if_dead` — is driven from two places:

  - `call()`, which repairs the connector in time for the call that
    noticed it, so a crash costs one restart rather than a dead session;
  - a 10s sweep, which is the only thing that can bring back a connector
    nobody is calling.

The restart policy is a pure function so it can be tested without a DB
pool and a runtime. Backoff is enforced as a time gate, never a sleep: a
tool call that finds the gate shut fails immediately instead of parking a
waiting user behind a crash-loop, and the sweep retries later. Five
consecutive failures stop the attempts, and the reset window doubles as
the escape hatch — a box left running recovers from a transient outage
instead of staying dark.

`stop_server`/`stop_all` forget the spec, which is what keeps a stop a
stop: without it the sweep would resurrect a connector an admin had just
revoked, and a container remount would respawn into the container that
was being replaced.
This commit is contained in:
2026-08-07 12:27:20 +01:00
parent 31b4c76f51
commit c1177a934d
5 changed files with 346 additions and 0 deletions
+4
View File
@@ -167,6 +167,10 @@ impl Integrations {
"data",
crate::mcp::EventLog::Discard,
));
// Supervise the global connectors: a crashed one is restarted rather than
// staying dead until the process does. Spawned post-construction because the
// sweep reacts through the `Arc` it is watching (see `spawn_respawn_sweep`).
mcp.spawn_respawn_sweep(rt.shutdown_token.clone());
let mut plugin_manager = PluginManager::new(Arc::clone(&rt.db));
for plugin in plugins {