Skald-Circle/dev-docs/mcp-connectors.md

Skald dev-docs — architectural reference for coding agents. Index: README.md · Entry point: ../CLAUDE.md

Read this when: you touch MCP, connectors, the marketplace install path, OAuth or device login.

---

MCP connectors (blueprint §7/§14/§15)

MCP servers are surfaced to users as "Connectors" (UI naming; mcp/schema stays neutral, §0.1). The old single owner table mcp_servers, the agent-facing register_mcp/delete_mcp tools, and the mcp kinds of list_items/toggle_item are gone. Connectors are now admin-curated and user-activated through the Connectors UI/API — never written by the agent, which closes the §14 RCE vector (prompt-injection → agent writes+registers a local script → arbitrary code on the box).

Two runtimes, one view (§7). A session's MCP tools are the union of:

• Global runtime — shared, stateless connectors (web-search, Tavily…) that run on the host, connected at boot from mcp_global_servers by McpManager::initialize. Filtered per user by mcp_global_access.
• Per-user runtime — the connectors a user has activated, run inside their container, started at first login from that user's owner mcp_user_servers and living until restart (§9; the docker exec -i children die via kill_on_drop when the UserContext drops).

McpProvider (mcp/provider.rs) is the trait the session code talks to, so all_tool_defs / render_mcp_list / ActivateTools never learn which runtime owns a server. McpManager implements it directly (used for the inert ownerless bundle, §19); UserMcpView implements it as global ∪ user, where accessible_global is a snapshot of mcp_global_access captured when the UserContext is built (like fs membership). Both runtimes share McpManager::connect_all(specs, boot); McpServerSpec + global_row_spec/user_row_spec turn a DB row into a connectable spec (a per-user local_script spec targets the user's container).

Authorization is a capability on the role, not if role==admin (§0.1/§14 — db/role_capabilities.rs): mcp.register_remote + mcp.register_local_from_catalog are self-service (seeded on every new role by roles::create via seed_defaults); mcp.register_local_script + mcp.manage_catalog are admin-only. admin holds every capability by construction (short-circuit in has()). API handlers gate through require_cap.

Tables (see database.md) — registry: mcp_catalog (admin-vetted templates; holds only the schema of what an activation must supply, never live creds — plus, for OAuth, oauth_provider + oauth_scopes_json + deliver_json), mcp_global_servers + mcp_global_access, oauth_providers (per-provider client creds), role_capabilities. Owner: mcp_user_servers (per-user activations; api_key encrypted at rest — the refresh token for an OAuth one — catalog_name/oauth_provider/deliver_json bare TEXT snapshots).

Endpoints (src/frontend/api/mcp.rs, mounted in api/mod.rs) — admin: /mcp/catalog (GET/POST/DELETE), /mcp/global (list/enable/delete + /{id}/access GET/PUT), /mcp/providers (GET/POST + DELETE /{name} — OAuth provider creds, secret never returned to the browser). User: /mcp/available, /mcp/activate, /mcp/activated (+ DELETE /{id} to deactivate), /mcp/oauth/start + /mcp/oauth/complete (the §15 OAuth login), /mcp/login/status + /mcp/login/reset (the §15 QR/device login — see below). connectors.js (<connectors-page>) is the single Connectors surface — a row list, one row per connector (there is no separate catalog page): the user view (activate/deactivate + granted globals) always, plus the admin affordances when role_id === 'admin' — the Add connector dropdown (from the Marketplace, or manually via the #connectors/new sub-page), per-row removal from the catalog, and the Sign-in providers modal. The Marketplace stays its own page (marketplace.js), reached from that dropdown and linking back to #connectors. connector-detail.js (<connector-detail-page>) is a connector's own page and hosts both the OAuth login panel and the QR login panel.

Dependency reconciler (mcp::install::ensure_installed). Copying a local-script connector's files into a container never installed its deps. ensure_installed closes that: a content-hash reconciler keyed on the connector's source files (not a version string) that, when the hash changed, re-copies the files and installs deps inside the container — npm ci --omit=dev (node, from package.json) and/or pip install --target .pydeps (python, from requirements.txt, put on the server's PYTHONPATH by user_row_spec). Runs at activation and on every per-user startup path (UserContext build, remount) via mcp::prepare_local_connector, so a fresh container installs from scratch, an updated connector re-installs, and an unchanged one is a hash-match no-op. Deps are therefore never vendored — connectors ship package.json/requirements.txt, not node_modules/. Authoring contract for connectors lives in the marketplace repository — see below.

The host half has no reconciler, so its call sites are the contract. A global connector runs in the Skald process, not a container, and ensure_installed_host is not hash-guarded — it leans on pip/npm being idempotent, which is only safe as long as every path that lands new files also calls it. There are two: global_enable (the admin saving a connector's config) and, since it was missing, the global branch of Skald::refresh_connector_after_reinstall. Without the second, a marketplace Update that adds a requirements.txt copied the file and restarted the server without installing anything — the connector came back exactly as broken, and the only cure was re-saving its config. Note what that asymmetry cost: the per-user branch of the same function had always reinstalled (prepare_local_connector), so the bug was invisible on anything scope: user.

The verify runs with .pydeps on PYTHONPATH, and must (mcp::verify::verify_env). Only the server launch used to get that path (global_row_spec / user_row_spec); the verify is a bare sh -c inheriting nothing, so a python connector was rejected by its own verify for a dependency sitting installed one directory away — and global_enable installs before it verifies, so the deps were provably there at the moment the check denied them. The failure selected for well-written connectors: declaring no verify meant never meeting it. The workdir is the connector dir in both targets, so the path is derived, not plumbed, and set with or_insert — an explicit PYTHONPATH from the form is the author's. One gap left deliberately: POST /api/mcp/test (the Test button) shares run_verify but not ensure_installed_host, so testing a python connector that was never enabled on this box still fails on the missing deps. Making a "try it" button write to disk for minutes is the worse trade; enable first.

The connector specs live in the marketplace repository, not here. The feed and everything that authors for it are a separate repo, checked out beside this one at ../marketplace, and the authoring contract is CONNECTOR_MANIFEST_GUIDE.md at its root — same filename this repo used to carry a copy of, which is exactly why the copy is gone: two files with one name drift, and the one next to the connectors is the one an author reads. If the connector specification ever has to change, that is the file to consult and to edit — nothing in this repo restates it.

Connector versioning. mcp_catalog carries version (INTEGER — the update-comparison key), version_string (semver, display) and version_release_date (ISO, display), snapshotted from the feed on install. The marketplace list computes update_available = feed version > installed version (strict) and surfaces it as an "Update" button (marketplace.js). The integer is the UI signal; the actual re-install trigger is the reconciler's content-hash.

OAuth per-user connectors (blueprint §15 — copy-paste flow)

OAuth2 authorization-code + PKCE is wired for per-user connectors (Gmail is the first). The consent is a human copy-paste, not a headless action: no callback route into the (NAT'd, hostname-less) box, and no client secret on the public feed.

• Providers, not per-connector URLs. The client is per-provider (one Google app covers Gmail/Calendar/Drive): oauth_providers holds auth_url/token_url/client_id/client_secret/redirect_uri/extra_params, admin-entered via the Sign-in-providers modal (Google preset fills all but the two secrets; redirect_uri = the static oauth/show.html page, extra_params = access_type=offline+prompt=consent so Google returns a refresh token). The manifest only names auth.provider + auth.scopes + auth.deliver — never URLs or secrets (feed is remote data, §14).
• Flow (mcp/oauth.rs): activate on an OAuth catalog entry persists a pending mcp_user_servers row (files installed, command wired, no token) and returns needs_oauth — it does not start the server. /mcp/oauth/start builds the consent URL (PKCE S256 + opaque state) and stashes the verifier in a RAM-only, TTL'd flow store keyed by state; the user approves in a browser, the provider lands the code on oauth/show.html, they paste it back. /mcp/oauth/complete exchanges code+verifier for a refresh token (client_secret sent server-side), stores it in the row's api_key, flips to ready, and starts the server. PKCE makes an intercepted code worthless; a restart drops in-flight flows (mirrors the RAM-only session model).
• Credential delivery = env, nothing on disk. The manifest's deliver ({as,format,env}, parsed as mcp::DeliverSpec) says how the token reaches the server. user_row_spec_resolved assembles the credential (google_authorized_user JSON = client creds from the provider + refresh token) and injects it as an env var (GMAIL_CREDS_JSON) on the docker exec — never a file, coherent with §2 (the tempted admin doesn't read /proc). The server reads it via Credentials.from_authorized_user_info. Ran both at OAuth-complete and at login-time per-user startup.
• Google needs a Web-application client: a Desktop client rejects an https:// redirect (loopback only), so the oauth/show.html redirect must be registered on a Web app OAuth client, and exact-match under Authorized redirect URIs — redirect_uri_mismatch otherwise.

QR / interactive device login (blueprint §15 — polling flow)

For a per-user connector whose credential is produced by pairing (auth.type: "qr"; WhatsApp is the first, on Baileys — the slim skald-runtime image has no Chromium, so a browser-based client is out), there is no code to paste and the server must run to produce the QR. The seam is a generic tool contract, reusable for future device kinds (SSH…):

• login_status tool contract. A connector needing an interactive login exposes one tool, login_status, returning JSON {state, qr?, message} (state: connecting|need_scan|ready|logged_out; qr is a data-URL PNG only while need_scan). Skald calls it directly, never the agent.
• Flow. activate on a qr entry inserts a pending mcp_user_servers row and starts the server (unlike OAuth, which defers), returning needs_login/login_kind:"qr". /mcp/login/status ensures the server is running (restarts a pending one), calls login_status, and returns its state; on ready it flips the row's auth_state so all_startable picks it up next login. /mcp/login/reset calls the connector's logout tool to re-arm (link a different device). The connector-detail.js QR panel polls login/status and renders the QR.
• Credential = on-disk session, not a token. The connector persists its session inside its own dir (e.g. ./auth/), under the bind-mounted home so it survives a container recreate — the honest §4 gap (admin-root-readable), not memory_docs.
• Node 18 gotcha: the container ships Node 18; Baileys uses the Web Crypto global, so the server must globalThis.crypto ??= require('crypto').webcrypto or it dies pre-QR with "crypto is not defined".

Deferred: SSH and other §15 device kinds (would reuse the login_status contract), deliver.as=file, and non-Google OAuth providers are unimplemented paths that error clearly rather than half-work. No boot seed of catalog presets; the admin populates the catalog from the Marketplace.