ssh: fix sudo failing with "password required" (v6 / 1.1.0)
The connector asked for a sudo password on every privileged call and failed whenever nobody answered it, which is every unattended run. - Always probe `sudo -n` first, even for aliases set to sudo="prompt". `_sudo_prefix` used to elicit unconditionally, so a host granting this user NOPASSWD still opened an Agent Inbox prompt; with no human there it hit the client's 300s ELICITATION_DEADLINE, got back `cancel`, and surfaced as "sudo password required (user declined or timed out)". sudo refuses before running anything when it wants a password, so the probe is side-effect free. - Strip a leading `sudo` from `command` and turn it into sudo=true. Agents write `exec(command="sudo systemctl restart x")`: with sudo=false that ran a tty-less sudo, with sudo=true it nested `sudo -S ... sudo ...` whose inner prompt had no tty either. Handles -u/-n/-S/-E/-H/-i/-k/-p/--; an unknown flag leaves the command alone. sudo_user now implies sudo=true. - Run privileged commands as `sh -c '<command>'`, so `&&`, pipes and redirections are elevated too instead of only the first word. - Add SSH_MCP_SUDO_PASSWORD (optional, secret) for unattended runs. It is consulted only after `sudo -n` proved a password is needed, so on a NOPASSWD host it never lands in the command's own stdin. - Actionable errors for every sudo failure mode, and a `hint` on a nested sudo we could not peel off. General review of the same server: - Drain stdout and stderr together and make timeout_sec a real wall-clock deadline. Both streams share one SSH channel window, so reading stdout to EOF first stalled once a chatty stderr filled it. Command stdin is now closed after the optional password. - Queue messages that arrive while awaiting an elicitation reply instead of discarding them, so a concurrent tools/call is not lost. - Record the client's `elicitation` capability at initialize and fail fast when it is absent rather than blocking on a prompt nobody can answer. - Tolerate null/string integer arguments (depth, max_results, context_lines, timeout_sec). - Realign the version across both manifests: fragment.json said 5/1.0.4 while connector.json said 2/1.0.1, so the feed was permanently ahead of the installed version and offered an update forever. Also lands the pending docs work: CONNECTOR_MANIFEST_GUIDE.md as the single source of truth, docs/connector.manifest_guide.md retired to a pointer, CLAUDE.md audited against the repo, compile.py docstring fixed, and an opencode.json config.
This commit is contained in:
Daniele
+398
-202
@@ -1,156 +1,253 @@
|
||||
# Skald Connector Authoring Guide
|
||||
|
||||
Instructions for generating a **correct connector** for the Skald marketplace
|
||||
(`https://connectors.skaldagent.net`). Give this file to the agent that produces
|
||||
new connectors.
|
||||
**This file is the single source of truth for the connector format.** It lives in the
|
||||
marketplace repo (`connectors.skaldagent.net`) and is referenced — not copied — by the
|
||||
Skald application repo (`~/projects/skald-circle`). Change the format here, and here only.
|
||||
|
||||
A connector is a folder served by the marketplace. Skald installs it, verifies
|
||||
every file against a SHA-256 pinned in the index, then either runs it on the host
|
||||
(global connector) or copies it into the user's container and runs it there
|
||||
(per-user connector, blueprint §6/§7).
|
||||
Give this file to any agent that produces new connectors.
|
||||
|
||||
A connector is a folder served by the marketplace. Skald installs it, verifies every file
|
||||
against a SHA-256 pinned in the index, then either runs it on the host (global connector)
|
||||
or copies it into the user's container and runs it there (per-user connector, blueprint
|
||||
§6/§7).
|
||||
|
||||
Everything below marked **"the client enforces"** was checked against the implementation in
|
||||
`skald-circle` (`src/frontend/api/marketplace.rs`, `src/frontend/api/mcp.rs`,
|
||||
`crates/skald-core/src/mcp/{mod,install,verify,oauth}.rs`).
|
||||
|
||||
---
|
||||
|
||||
## 1. The two documents
|
||||
## 0. Folder layout
|
||||
|
||||
### 1a. The root index — `connectors.json`
|
||||
One folder per connector, named exactly like its `id`, **flat**:
|
||||
|
||||
One array of entries, each pointing at a connector folder. **The index is the
|
||||
signable root: it is the only place that lists a connector's files and their
|
||||
SHA-256 digests.** Skald refuses any file whose bytes do not match.
|
||||
```
|
||||
connectors/myconn/
|
||||
├── fragment.json ← index entry (compiler input, never served as-is)
|
||||
├── connector.json ← the manifest Skald reads
|
||||
├── server.py ← the entry file (or index.js…)
|
||||
├── requirements.txt ← or package.json
|
||||
├── verify.py ← optional
|
||||
├── icon_sm.svg
|
||||
└── icon_lg.svg
|
||||
```
|
||||
|
||||
**Keep the folder flat.** The client accepts a relative sub-path (`pkg/server.py`) but
|
||||
`scripts/compile.py` only scans the folder's **top level**: files in a subdirectory are
|
||||
silently left out of `files[]`, never downloaded, and the connector breaks at runtime with
|
||||
no error anywhere. If a tree is genuinely needed, teach the compiler to recurse first.
|
||||
|
||||
---
|
||||
|
||||
## 1. The three source documents (+ one compiler)
|
||||
|
||||
You maintain three files by hand. A fourth, `connectors.json`, is **generated**.
|
||||
|
||||
### 1a. `connectors/index.json` — the order
|
||||
|
||||
A flat JSON array of folder ids, in display order:
|
||||
|
||||
```json
|
||||
["gmail", "gcal", "myconn"]
|
||||
```
|
||||
|
||||
This is the compiler's input list. A connector missing from it does not exist.
|
||||
|
||||
### 1b. `connectors/<id>/fragment.json` — the index entry
|
||||
|
||||
The connector's entry in the compiled index, **with every field except `files[]`** (the
|
||||
compiler adds that one).
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"version": 1,
|
||||
"connectors": [
|
||||
{
|
||||
"id": "whatsapp", // unique slug = folder name
|
||||
"name": "WhatsApp",
|
||||
"version": 1, // INTEGER build number — the update key (§7)
|
||||
"version_string": "2.0.1", // semver, display only
|
||||
"version_release_date": "2026-07-19", // ISO date, display only
|
||||
"type": "mcp_local", // mcp_local | mcp_remote (see §3)
|
||||
"scope": "user", // user | global (see §3)
|
||||
"icon_small": "whatsapp/icon_sm.svg",
|
||||
"icon_large": "whatsapp/icon_lg.svg",
|
||||
"user_description": "Send and read WhatsApp messages from your linked account.",
|
||||
"requires": ["NODE"], // human hint: NODE | PYTHON | OAUTH | API_KEY
|
||||
"tags": ["messaging", "mcp", "local", "whatsapp", "qr"],
|
||||
"auth": { "type": "qr" }, // may be repeated here and in the manifest
|
||||
"folder": "whatsapp", // defaults to id
|
||||
"files": [
|
||||
{ "path": "index.js", "sha256": "…", "size": 21258 },
|
||||
{ "path": "package.json", "sha256": "…", "size": 302 },
|
||||
{ "path": "connector.json", "sha256": "…", "size": 620 },
|
||||
{ "path": "icon_sm.svg", "sha256": "…", "size": 306 },
|
||||
{ "path": "icon_lg.svg", "sha256": "…", "size": 308 }
|
||||
]
|
||||
}
|
||||
]
|
||||
"id": "whatsapp", // unique slug = folder name
|
||||
"name": "WhatsApp",
|
||||
"version": 8, // INTEGER build number — the update key (§7)
|
||||
"version_string": "2.2.0", // semver, display only
|
||||
"version_release_date": "2026-08-23", // ISO date, display only
|
||||
"type": "mcp_local", // mcp_local | mcp_remote (§3)
|
||||
"scope": "user", // user | global (§3)
|
||||
"icon_small": "whatsapp/icon_sm.png", // relative to the FEED ROOT, not the folder
|
||||
"icon_large": "whatsapp/icon_lg.png",
|
||||
"user_description": "Send and read WhatsApp messages from your linked account.",
|
||||
"requires": ["NODE"], // API_KEY | ENV | NODE | PYTHON | OAUTH | DOCKER
|
||||
"tags": ["messaging", "mcp", "local", "whatsapp", "qr"],
|
||||
"folder": "whatsapp", // defaults to id
|
||||
"auth": { "type": "qr" }, // informational here — see the table in §1e
|
||||
"tools": [ … ] // informational here — see §2a
|
||||
}
|
||||
```
|
||||
|
||||
**Rules**
|
||||
Note the **two icon vocabularies**: `icon_small` / `icon_large` in the index are relative to
|
||||
the feed root (`whatsapp/icon_sm.png`), while `files[]` and `connector.json` name them from
|
||||
inside the folder (`icon_sm.png`). The client reconciles the two, but only records an icon
|
||||
whose file was actually installed — so **icons must be shipped in the folder**, never hot-
|
||||
linked.
|
||||
|
||||
- `files[].path` is relative to the connector folder. List **every** file the
|
||||
connector ships (server code, `package.json`/`requirements.txt`, icons, and the
|
||||
`connector.json` itself). A missing or mismatched digest fails the install.
|
||||
- Compute `sha256` over the exact bytes served: `sha256sum <file>`.
|
||||
- Do **not** list `node_modules/` or any generated deps — those are installed on
|
||||
the box, not shipped (see §5).
|
||||
- `size` is optional but recommended.
|
||||
### 1c. `connectors/<id>/connector.json` — the manifest
|
||||
|
||||
### 1b. The per-connector manifest — `<folder>/connector.json`
|
||||
|
||||
The richer document. Fetched per connector and mapped into Skald's catalog.
|
||||
The richer document, fetched per connector and mapped into Skald's catalog.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"id": "whatsapp",
|
||||
"name": "WhatsApp",
|
||||
"version": 1, // INTEGER build number — the update key (§7)
|
||||
"version_string": "2.0.1", // semver, display only
|
||||
"version_release_date": "2026-07-19", // ISO date, display only
|
||||
"version": 8, // must match fragment.json (§7)
|
||||
"version_string": "2.2.0",
|
||||
"version_release_date": "2026-08-23",
|
||||
"type": "mcp_local",
|
||||
"scope": "user",
|
||||
"auth": { "type": "qr" }, // none | api_key | oauth2 | qr (see §4)
|
||||
"requires": ["NODE"],
|
||||
"tags": ["messaging", "mcp", "local", "whatsapp", "qr"],
|
||||
"auth": { "type": "qr" }, // none | api_key | oauth2 | qr | ssh_key (§4)
|
||||
"launch_command": "node index.js", // human-readable; mcp_config is what runs
|
||||
"transport": "stdio", // stdio | streamable-http (may also sit in mcp_config)
|
||||
"mcp_config": {
|
||||
"command": "node", // interpreter (local) …
|
||||
"args": ["index.js"], // … args[0] MUST name the entry file
|
||||
"transport": "stdio" // stdio (local) | streamable-http (remote)
|
||||
"args": ["index.js"] // … args[0] MUST name the entry file
|
||||
},
|
||||
"dependencies": ["@whiskeysockets/baileys@7.0.0-rc.14"], // display only (§5)
|
||||
"setup_instructions": ["Scan the QR code with WhatsApp → Linked devices"],
|
||||
"docs": [{
|
||||
"lang": "en",
|
||||
"description": "Human blurb shown in the UI.",
|
||||
"llm_short_description": "One line the model reads to decide whether to use this connector."
|
||||
}],
|
||||
"env": [], // form fields the user fills (see §4b)
|
||||
"tools": [ // OPTIONAL — friendly UI names per tool (§2a)
|
||||
{ "name": "send_message", "display_name": "Send Message" }
|
||||
],
|
||||
"homepage": "https://…",
|
||||
"icon_small": "icon_sm.svg", // relative to the folder here
|
||||
"icon_large": "icon_lg.svg",
|
||||
"tags": ["messaging", "mcp", "local", "whatsapp", "qr"]
|
||||
"env": [], // form fields the user fills (§4b)
|
||||
"verify": { "command": "python3 verify.py", "timeout_secs": 20 }, // optional (§6)
|
||||
"tools": [{ "name": "send_message", "display_name": "Send Message" }], // optional (§2a)
|
||||
"homepage": "https://web.whatsapp.com",
|
||||
"icon_small": "icon_sm.png", // relative to the FOLDER here
|
||||
"icon_large": "icon_lg.png"
|
||||
}
|
||||
```
|
||||
|
||||
**`mcp_config.args[0]` is load-bearing for a local connector:** it is how Skald
|
||||
learns which file to run. At activation Skald rewrites it to the file's path
|
||||
inside the user's container (`/root/.skald/mcp/<name>/<entry>`), so keep it a
|
||||
plain relative filename (`index.js`, `server.py`, `pkg/server.py`).
|
||||
**`mcp_config.args[0]` is load-bearing for a local connector:** it is how Skald learns which
|
||||
file to run — an install fails outright without it. At activation Skald rewrites it to the
|
||||
file's path inside the user's container (`/root/.skald/mcp/<name>/<entry>`), so keep it a
|
||||
plain relative filename (`index.js`, `server.py`).
|
||||
|
||||
**`transport` may sit either at the top level or inside `mcp_config`** — the client reads
|
||||
`mcp_config.transport` first, then the top-level one, then infers (`remote` → `http`,
|
||||
otherwise `stdio`). Both spellings are in use in this repo; pick one per connector and keep
|
||||
it consistent between the two files. `streamable-http` and `http` both normalise to HTTP;
|
||||
anything unrecognised silently becomes stdio, which for a remote connector means trying to
|
||||
spawn a command that does not exist.
|
||||
|
||||
**`llm_short_description` is model-facing:** it is the one-liner injected into the LLM's
|
||||
system prompt so the model knows what this connector does. Keep it short and functional
|
||||
("Weather — current conditions, 16-day forecast, and AQI data for any location"), **not** a
|
||||
list of tools (the model discovers those after `activate_tools`).
|
||||
|
||||
### 1d. `connectors/connectors.json` — the compiled index (generated)
|
||||
|
||||
**Never edit this file by hand.** `scripts/compile.py` reads `index.json`, loads each
|
||||
`fragment.json`, scans the folder's real files, computes their SHA-256 + size, and writes the
|
||||
result:
|
||||
|
||||
```bash
|
||||
python3 scripts/compile.py # regenerate
|
||||
python3 scripts/compile.py --verify # fail if the committed index is stale
|
||||
```
|
||||
|
||||
The compiler excludes `fragment.json`, `connectors.json`, `index.json`, `compile.sh`,
|
||||
`compile.py`, `update_hashes.py`, `.DS_Store` and every subdirectory. **Everything else is
|
||||
hashed, including `connector.json`.**
|
||||
|
||||
```jsonc
|
||||
"files": [
|
||||
{ "path": "index.js", "sha256": "…", "size": 21258 },
|
||||
{ "path": "package.json", "sha256": "…", "size": 302 },
|
||||
{ "path": "connector.json", "sha256": "…", "size": 620 },
|
||||
{ "path": "icon_sm.png", "sha256": "…", "size": 306 }
|
||||
]
|
||||
```
|
||||
|
||||
- `files[].path` is relative to the connector folder.
|
||||
- Digests and sizes are computed by the compiler. **Never write them by hand.**
|
||||
- The index is the **signable root**: it is the one document that names a connector's files
|
||||
and their digests. The client refuses any file whose bytes do not match, all-or-nothing —
|
||||
a mismatch leaves nothing on disk.
|
||||
- Do **not** ship `node_modules/` or vendored wheels (§5).
|
||||
|
||||
### 1e. Who reads what — the authority table
|
||||
|
||||
The client hydrates a card from **two** documents, and they are not interchangeable. Getting
|
||||
this wrong is the most common way a correct-looking connector misbehaves.
|
||||
|
||||
| Field | Read from | Notes |
|
||||
| --- | --- | --- |
|
||||
| `id`, `folder`, `tags` | **index** (`fragment.json`) | `folder` defaults to `id` |
|
||||
| `name` | index, falling back to manifest | |
|
||||
| `user_description` | **index**, falling back to `docs[0].description` | the human blurb |
|
||||
| `icon_small`, `icon_large` | **index** | feed-root-relative; the file must be in `files[]` |
|
||||
| `files[]` | **index** (manifest's is a legacy fallback) | the trust root |
|
||||
| `requires` | manifest, falling back to index | |
|
||||
| `type`, `scope` | manifest, falling back to index | keep them identical |
|
||||
| `version` trio | **manifest wins**, index is the fallback | a mismatch is logged as a desync warning (§7) |
|
||||
| `auth` | **manifest only** | the index's `auth` is *never parsed*; only `requires` is used as a coarse fallback |
|
||||
| `tools[]` | **manifest only** | a `tools[]` that lives only in `fragment.json` does nothing (§2a) |
|
||||
| `env[]`, `verify`, `mcp_config`, `docs`, `dependencies`, `setup_instructions`, `homepage` | **manifest only** | |
|
||||
|
||||
So: `auth` and `tools[]` in `fragment.json` are documentation for the catalog page and for
|
||||
whoever reads the index — harmless, worth keeping in sync, but **the manifest is what runs**.
|
||||
|
||||
---
|
||||
|
||||
## 2. Server contract (MCP over stdio)
|
||||
|
||||
A **local** connector is a program speaking JSON-RPC 2.0 over stdin/stdout. It
|
||||
MUST handle:
|
||||
A **local** connector is a program speaking JSON-RPC 2.0 over stdin/stdout. It MUST handle:
|
||||
|
||||
- `initialize` → `{ protocolVersion, capabilities: { tools: {} }, serverInfo }`
|
||||
- `notifications/initialized` → no response
|
||||
- `tools/list` → `{ tools: [ { name, description, inputSchema } ] }`
|
||||
- `initialize` → `{ protocolVersion: "2024-11-05", capabilities: { tools: {} }, serverInfo }`
|
||||
- `notifications/initialized` → **no response**
|
||||
- `ping` → `{}`
|
||||
- `tools/list` → `{ "tools": [ { name, description, inputSchema, title? } ] }` — an
|
||||
**object**, not a bare array
|
||||
- `tools/call` → `{ content: [ { type: "text", text } ], isError? }`
|
||||
|
||||
**stdout is reserved for JSON-RPC only.** Send all logs/diagnostics to **stderr**.
|
||||
Anything a library prints to stdout (a logger, a banner) corrupts the protocol —
|
||||
silence it (e.g. Baileys/pino → a silent logger; Python → `print(…, file=sys.stderr)`).
|
||||
**Any message without an `id` is a notification and must produce no output at all.**
|
||||
Answering one desynchronises a strict client.
|
||||
|
||||
A **remote** connector is an HTTP MCP endpoint (`mcp_config.url` +
|
||||
`transport: "streamable-http"`); no code runs on the box.
|
||||
**stdout is reserved for JSON-RPC only.** Send all logs/diagnostics to **stderr**. Anything a
|
||||
library prints to stdout (a logger, a banner) corrupts the protocol — silence it (Baileys/pino
|
||||
→ a silent logger; Python → `print(…, file=sys.stderr)`). Guard stdout writes with a lock if
|
||||
the server has background threads.
|
||||
|
||||
Report tool failures as a normal result carrying `isError: true` with a readable message —
|
||||
not as a JSON-RPC error, and not as a "successful" result with an `{"status":"error"}` body
|
||||
the model cannot distinguish from data.
|
||||
|
||||
A **remote** connector is an HTTP MCP endpoint (`mcp_config.url` + `transport:
|
||||
"streamable-http"`); no code runs on the box.
|
||||
|
||||
### 2a. Friendly tool names (`tools[]`) — optional
|
||||
|
||||
Raw MCP tool names are ugly in the chat UI (`search_files`, `send_message`). The
|
||||
optional top-level `tools[]` block gives each one a human title shown as the tool
|
||||
card's heading:
|
||||
Raw MCP tool names are ugly in the chat UI (`search_files`, `send_message`). Two ways to fix
|
||||
that, in order of preference:
|
||||
|
||||
1. **`title` in your `tools/list` entries** (preferred for servers we control) — Skald uses it
|
||||
automatically, no manifest change needed.
|
||||
2. **The manifest's `tools[]` block** — for remote connectors and third-party packages
|
||||
(`npx -y firecrawl-mcp`) whose `tools/list` we cannot edit:
|
||||
|
||||
```jsonc
|
||||
"tools": [
|
||||
{ "name": "send_message", "display_name": "Send Message" },
|
||||
{ "name": "list_chats", "display_name": "List Chats" },
|
||||
{ "name": "download_media", "display_name": "Download Media" }
|
||||
{ "name": "send_message", "display_name": "Send Message" }
|
||||
]
|
||||
```
|
||||
|
||||
- `name` — the **raw** tool name exactly as your server returns it from `tools/list`.
|
||||
- `name` — the **raw** tool name exactly as the server returns it from `tools/list`.
|
||||
- `display_name` — the friendly card title (English only; not internationalized).
|
||||
|
||||
**Resolution order** for a tool's card title is **`tools[].display_name` → the MCP
|
||||
`title` field → a prettified raw name**. So you have two ways to set a friendly
|
||||
name, and can skip `tools[]` entirely:
|
||||
**Resolution order:** `tools[].display_name` → the MCP `title` field → a prettified raw name
|
||||
(`send_message` → "Send Message").
|
||||
|
||||
1. **This block** — the authoritative override, curated in the manifest.
|
||||
2. **The MCP `title` field** — if your `tools/list` entries already carry a
|
||||
`title` (MCP 2025-06-18+), Skald uses it automatically; no manifest change
|
||||
needed. `tools[]` wins if both are present.
|
||||
3. If neither is set, Skald title-cases the raw name (`send_message` → "Send
|
||||
Message").
|
||||
⚠️ **`tools[]` is read from `connector.json`.** The client parses no `tools` field on the
|
||||
index entry, so a block placed only in `fragment.json` is inert. Put it in the manifest;
|
||||
mirroring it into `fragment.json` is optional and purely documentary.
|
||||
|
||||
**Icons are per connector, not per tool.** Every tool of a connector shows that
|
||||
connector's own `icon_small`; there is no per-tool icon field. Only list a tool in
|
||||
`tools[]` when its prettified name isn't good enough — partial lists are fine
|
||||
(unlisted tools fall through to steps 2–3).
|
||||
**Icons are per connector, not per tool.** Every tool shows its connector's `icon_small`;
|
||||
there is no per-tool icon. Partial `tools[]` lists are fine — unlisted tools fall through.
|
||||
|
||||
---
|
||||
|
||||
@@ -158,44 +255,87 @@ connector's own `icon_small`; there is no per-tool icon field. Only list a tool
|
||||
|
||||
| Manifest | Meaning |
|
||||
| --- | --- |
|
||||
| `scope: "user"` | runs **once per user**, inside their container. Personal creds. |
|
||||
| `scope: "user"` | runs **once per user**, inside their container. Personal creds. Stored as `per_user`. |
|
||||
| `scope: "global"` | runs **once for the household**, on the host. Shared, stateless. Admin enables it with a key. |
|
||||
| `type: "mcp_local"` | ships code that will **execute on the box** — installing needs the admin `mcp.register_local_script` capability (RCE-bearing act, §14). |
|
||||
| `type: "mcp_remote"` | just an HTTP URL; no local code. |
|
||||
| `type: "mcp_local"` | ships code that will **execute on the box** — installing needs the admin `mcp.register_local_script` capability (RCE-bearing act, §14). Stored as `local_script`. |
|
||||
| `type: "mcp_remote"` | just an HTTP URL; no local code. Stored as `remote`. |
|
||||
|
||||
Pick the narrowest: a personal messaging/email/calendar connector is
|
||||
`scope: "user"`; a shared search API is `scope: "global"`.
|
||||
Pick the narrowest: a personal messaging/email/calendar connector is `scope: "user"`; a shared
|
||||
search API is `scope: "global"`. Both axes **fail closed** — an unreadable `scope` becomes
|
||||
`per_user`, an unreadable `type` becomes `local_script`, the answer that demands *more*
|
||||
authority.
|
||||
|
||||
`requires[]` is a human hint rendered as tags on the catalog page: `API_KEY`, `ENV`, `NODE`,
|
||||
`PYTHON`, `OAUTH`, `DOCKER`. The client only inspects it as a fallback when the manifest
|
||||
declares no `auth` (`OAUTH` → oauth, `API_KEY` → api_key). `SECRETS_DIR` is **removed** — the
|
||||
`secrets/` folder is no longer part of the model.
|
||||
|
||||
---
|
||||
|
||||
## 4. Authentication (`auth.type`)
|
||||
|
||||
### 4a. The recognized values
|
||||
|
||||
| `auth.type` | Flow | Ships |
|
||||
| --- | --- | --- |
|
||||
| `none` | nothing to sign in | — |
|
||||
| `api_key` | user pastes a key/secret into a form | an `env[]` schema (§4b) |
|
||||
| `oauth2` | browser consent → paste code back | `auth.provider` + `auth.scopes` + `auth.deliver` (§4c) |
|
||||
| `oauth2` | browser consent → token injected as an env var | `provider` + `scopes` + `deliver` (§4c) |
|
||||
| `qr` | server shows a QR, user scans with a phone | a `login_status` tool (§4d) |
|
||||
| `ssh_key` | reserved for key-based remote access | — |
|
||||
|
||||
### 4b. `api_key` — the `env[]` schema
|
||||
**Anything else normalises to `none`.** `email` declares `auth.type: "password"`, which the
|
||||
client does not recognise and therefore treats as `none`; that connector works only because
|
||||
its credentials arrive through `env[]` like any other form field. Prefer `api_key` or `none`
|
||||
for new connectors and let `env[]` carry the credential.
|
||||
|
||||
Each entry drives one form field **and** is injected as an env var / URL token to
|
||||
the server:
|
||||
`auth.delivery` (`header` | `query` | `env`) is surfaced to the admin UI. For a remote
|
||||
connector the **URL placeholder is what actually routes the key** (§4b) — the `param` name is
|
||||
not read by the client.
|
||||
|
||||
### 4b. `env[]` — the activation form, and what reaches the process
|
||||
|
||||
Each entry drives one form field, and the collected values are injected into the server
|
||||
process environment:
|
||||
|
||||
```jsonc
|
||||
"env": [{
|
||||
"name": "tavilyApiKey",
|
||||
"label": "Tavily API key",
|
||||
"description": "Create one at https://app.tavily.com.",
|
||||
"name": "EMAIL_IMAP_HOST", // ← the ACTUAL environment variable name
|
||||
"label": "IMAP host",
|
||||
"description": "IMAP server hostname (e.g. imap.gmail.com).",
|
||||
"required": true,
|
||||
"secret": true, // rendered masked, stored encrypted
|
||||
"example": "tvly-xxxxxxxx"
|
||||
"secret": false, // true → masked in the form, stored encrypted
|
||||
"example": "imap.gmail.com",
|
||||
"default": "" // non-required fields only
|
||||
}]
|
||||
```
|
||||
|
||||
The server reads each value from `process.env.<name>` (or `os.environ`). For a
|
||||
**remote** connector that wants the key in the URL, use a placeholder:
|
||||
`"url": "https://mcp.example.com/?key={SECRET:tavilyApiKey}"`.
|
||||
> **`name` must be exactly the environment variable your server reads.** The process receives
|
||||
> the form's `name` → value map verbatim; there is no renaming layer. `mcp_config.env` is
|
||||
> **not** substituted at runtime — it survives only as a legacy fallback for the form schema,
|
||||
> and if it is ever the sole env source its `{ENV:…}` strings are injected *literally*. Do not
|
||||
> use it to map one name onto another.
|
||||
|
||||
The server reads each value from `os.environ` / `process.env`. Declare `requires: ["ENV"]`
|
||||
when the connector needs user-supplied config.
|
||||
|
||||
**Placeholders.** Two tokens, and exactly two places where they are substituted:
|
||||
|
||||
| Token | Meaning |
|
||||
| --- | --- |
|
||||
| `{ENV:NAME}` | non-sensitive value (host, port, username…) |
|
||||
| `{SECRET:NAME}` | sensitive value (password, API key, token) |
|
||||
|
||||
| Substituted in | Engine behaviour on an unknown name |
|
||||
| --- | --- |
|
||||
| `mcp_config.url` (remote connectors) | `{SECRET:x}` falls back to the connector's api_key; if that is absent the token is **left in the URL literally**, so a misconfiguration is visible |
|
||||
| `verify.command` (§6) | replaced with the **empty string** |
|
||||
|
||||
Nowhere else — not in `mcp_config.env`, not in `args`. Legacy `{key}` still resolves to the
|
||||
api_key in URLs; `{env:NAME}` and `{secrets}/…` are dead and are never substituted.
|
||||
|
||||
Remote example: `"url": "https://mcp.tavily.com/mcp/?tavilyApiKey={SECRET:tavilyApiKey}"`
|
||||
with a matching `env[]` entry named `tavilyApiKey`.
|
||||
|
||||
### 4c. `oauth2` — provider consent
|
||||
|
||||
@@ -208,20 +348,27 @@ The server reads each value from `process.env.<name>` (or `os.environ`). For a
|
||||
}
|
||||
```
|
||||
|
||||
The manifest names **only** the provider slug, scopes, and how the obtained token
|
||||
is delivered — never client secrets or endpoint URLs (those are admin-entered,
|
||||
kept off the public feed). Skald handles PKCE + code exchange and injects the
|
||||
credential as the named env var. `format`: `google_authorized_user` (Google) or
|
||||
`refresh_token`. Today only `as: "env"` is wired.
|
||||
The manifest names **only** the provider slug, the scopes, and how the obtained token is
|
||||
delivered — never client ids, client secrets, endpoint URLs or redirect URIs, which are
|
||||
admin-entered and stay off the public feed. Skald runs PKCE + code exchange and injects the
|
||||
credential as the named env var.
|
||||
|
||||
### 4d. `qr` / interactive device login — the generic contract
|
||||
- `deliver.as`: **only `"env"` is implemented** — `"file"` is rejected at activation with an
|
||||
explicit error rather than half-working. Do not ship it.
|
||||
- `deliver.format`: `google_authorized_user` (the JSON `from_authorized_user_file` reads) or
|
||||
`refresh_token`.
|
||||
- `deliver.env` must **not** also appear in `mcp_config.env` — Skald injects it at runtime.
|
||||
- An OAuth connector activates into a **pending** row; nothing starts until the consent
|
||||
round-trip completes. The admin must have configured the provider first, or activation
|
||||
fails early with a clear message.
|
||||
|
||||
For a connector whose credential is produced by **scanning/pairing** (WhatsApp
|
||||
today), there is no code to paste. The rule:
|
||||
### 4d. `qr` / interactive device login
|
||||
|
||||
> **Expose one extra tool, `login_status`, returning a JSON object** (as the
|
||||
> `text` of a normal text result). Skald calls it directly (never the agent) and a
|
||||
> login panel polls it.
|
||||
For a connector whose credential is produced by **scanning/pairing** (WhatsApp today) there is
|
||||
no code to paste. The contract:
|
||||
|
||||
> **Expose one extra tool, `login_status`, returning a JSON object** (as the `text` of a normal
|
||||
> text result). Skald calls it directly — never the agent — and a login panel polls it.
|
||||
|
||||
```jsonc
|
||||
// login_status result text (a JSON string):
|
||||
@@ -232,117 +379,166 @@ today), there is no code to paste. The rule:
|
||||
}
|
||||
```
|
||||
|
||||
- `activate` on a `qr` connector inserts a **pending** row and **starts the
|
||||
server** (so it can produce the QR), then hands off to the login panel.
|
||||
- The panel polls `POST /api/mcp/login/status`; when `state == "ready"` the
|
||||
connector is marked ready and starts automatically on later logins.
|
||||
- Also expose a `logout` tool (clears the session, forces a fresh QR) — the panel
|
||||
calls it via `POST /api/mcp/login/reset` to re-link a different phone.
|
||||
- The **credential is the on-disk session**, not a token. Persist it **inside the
|
||||
connector's own directory** (e.g. `./auth/` next to the entry file). That folder
|
||||
lives under the bind-mounted home, so it survives container recreates and
|
||||
connector updates. Never store it under a shared/global path.
|
||||
|
||||
Skald resolves `auth.type: "qr"` the same way whether it appears in the index
|
||||
entry or the manifest.
|
||||
- Activating a `qr` connector inserts a **pending** row and **starts the server** (so it can
|
||||
produce the QR), then hands off to the login panel.
|
||||
- The panel polls `POST /api/mcp/login/status`; when `state == "ready"` the connector is marked
|
||||
ready and starts automatically on later logins.
|
||||
- Also expose a **`logout`** tool (clears the session, forces a fresh QR) — the panel calls it
|
||||
via `POST /api/mcp/login/reset` to re-link a different phone.
|
||||
- The **credential is the on-disk session**, not a token. Persist it **inside the connector's
|
||||
own directory** (e.g. `./auth/` next to the entry file). That folder lives under the bind-
|
||||
mounted home, so it survives container recreates and connector updates. Never store it
|
||||
under a shared or global path.
|
||||
|
||||
---
|
||||
|
||||
## 5. Dependencies (node & python) — how they get installed
|
||||
|
||||
**Do not ship `node_modules/` or vendored wheels.** Declare deps as a standard
|
||||
manifest **file** and Skald installs them inside the container:
|
||||
**Do not ship `node_modules/` or vendored wheels.** Declare deps as a standard manifest **file**
|
||||
and Skald installs them where the server will run:
|
||||
|
||||
- **node:** ship a `package.json` with a `dependencies` map. Skald runs
|
||||
`npm ci --omit=dev` (falling back to `npm install --omit=dev`) in the connector
|
||||
dir. `node_modules/` resolves automatically beside the entry file.
|
||||
- **python:** ship a `requirements.txt`. Skald installs it with
|
||||
`pip install --target .pydeps` and puts `.pydeps` on the server's `PYTHONPATH`.
|
||||
`npm ci --omit=dev --no-audit --no-fund`, falling back to `npm install --omit=dev …` when
|
||||
there is no lockfile. `node_modules/` resolves automatically beside the entry file.
|
||||
- **python:** ship a `requirements.txt`. Skald runs
|
||||
`python3 -m pip install --break-system-packages --target .pydeps -r requirements.txt` and
|
||||
puts `.pydeps` on the server's `PYTHONPATH`.
|
||||
|
||||
This runs at activation **and** on every startup, guarded by a **content hash** of
|
||||
the connector's source files:
|
||||
A single install has a **300 s ceiling**; past that it fails rather than hanging a login.
|
||||
|
||||
This runs at activation **and** on every startup, guarded by a **content hash** of the
|
||||
connector's source files (`.skald-install.lock`):
|
||||
|
||||
- first activation / a brand-new container → full install,
|
||||
- a connector **update** (any shipped file changed) → re-copy + re-install,
|
||||
- unchanged → skipped in microseconds.
|
||||
|
||||
So you never write install steps into the manifest — just ship the dep file, list
|
||||
it in the index with its SHA-256, and set `requires: ["NODE"]` / `["PYTHON"]` as a
|
||||
human hint. Pin versions in `package.json` / `requirements.txt` for reproducible
|
||||
installs. Keep the dep tree lean (containers are slim; avoid native-heavy
|
||||
packages where a pure alternative exists — e.g. Baileys instead of a browser).
|
||||
So you never write install steps into the manifest — just ship the dep file. The manifest's
|
||||
`dependencies[]` and `setup_instructions[]` are **display metadata** for the admin card; they
|
||||
install nothing. Set `requires: ["NODE"]` / `["PYTHON"]` as a human hint. Pin versions for
|
||||
reproducible installs, and keep the tree lean (containers are slim; prefer a pure-JS/Python
|
||||
library over a native-heavy one — e.g. Baileys instead of a browser).
|
||||
|
||||
**Host assets are not copied into the container.** Icons (`.svg/.png/.jpg/.webp/.gif/.ico`) and
|
||||
`connector.json` stay on the host, so a server must never expect to read them at runtime.
|
||||
Everything else in the folder — entry file, deps file, helper modules, `verify.py` — is copied.
|
||||
|
||||
Where the files land:
|
||||
|
||||
| | Host | Per-user container |
|
||||
| --- | --- | --- |
|
||||
| installed folder | `<workdir>/connectors/<id>/` | `~/.skald/mcp/<name>/` (`/root/.skald/mcp/<name>/`) |
|
||||
| python deps | `connectors/<id>/.pydeps` | `~/.skald/mcp/<name>/.pydeps` |
|
||||
|
||||
---
|
||||
|
||||
## 6. Verify-before-save (optional but recommended)
|
||||
## 6. Verify-before-save (optional, strongly recommended)
|
||||
|
||||
Ship a `verify.py` / verify snippet and reference it:
|
||||
Ship a `verify.py` / `verify.js` (or an inline snippet) and reference it:
|
||||
|
||||
```jsonc
|
||||
"verify": { "command": "python3 verify.py", "timeout_secs": 15 }
|
||||
"verify": { "command": "python3 verify.py", "timeout_secs": 20 }
|
||||
```
|
||||
|
||||
It runs with the collected env/secret injected and must print **one JSON object**
|
||||
on stdout: `{"ok": bool, "message": string, "details"?: object}`, exit 0 on
|
||||
success. Used for `api_key`/`none` connectors to test creds before activating.
|
||||
(A `qr` connector needs no verify — its `login_status` is the live check.)
|
||||
It runs **after** the user fills the form and **before** the activation is persisted, with the
|
||||
collected env/secrets injected and `{ENV:…}` / `{SECRET:…}` substituted into the command.
|
||||
|
||||
**Output contract** — one JSON object on stdout and nothing else:
|
||||
|
||||
```json
|
||||
{"ok": true, "message": "IMAP and SMTP authentication successful", "details": {"imap": "…"}}
|
||||
{"ok": false, "message": "IMAP login failed: INVALID_CREDENTIALS"}
|
||||
```
|
||||
|
||||
Exit 0 on success. If the JSON fails to parse the client falls back to the exit code (0 = ok)
|
||||
and shows stderr. A timeout counts as a failure. **Never print credentials** in `message` or
|
||||
`details`.
|
||||
|
||||
Rules the client enforces:
|
||||
|
||||
- **The script must be one of the connector's shipped files.** The client resolves it by
|
||||
matching a **basename from `files[]`** against the command string — so `python3 verify.py`
|
||||
works because `verify.py` is in `files[]`. A script that is not shipped is never downloaded
|
||||
and activation fails; an inline command with no matching basename (firecrawl's `node -e "…"`)
|
||||
is fine and runs as-is.
|
||||
- **Where it runs:** inside the user's container, in the connector's directory, for a
|
||||
`per_user` connector; on the host in `connectors/<id>/` for a `global` one.
|
||||
- **Timeout:** declare `timeout_secs` for the record, but the runtime currently applies a fixed
|
||||
**20 s** to every verify. Keep the probe well under that.
|
||||
- Any `auth.type` may use verify — it is not limited to `api_key`. A `qr` connector needs none:
|
||||
its `login_status` is the live check.
|
||||
|
||||
**Without `verify` there is no test at all.** Activation goes straight to `ready` — including
|
||||
for remote connectors, which get no handshake fallback. The manifest author decides.
|
||||
|
||||
---
|
||||
|
||||
## 7. Versioning & updates
|
||||
|
||||
Three fields, in **both** the index entry and the `connector.json`, kept identical:
|
||||
Three fields, in **both** `fragment.json` and `connector.json`, kept identical:
|
||||
|
||||
| field | type | role |
|
||||
| --- | --- | --- |
|
||||
| `version` | **integer** | monotonic build number, **per connector** — the machine comparison key |
|
||||
| `version` | **integer** | monotonic build number, per connector — the machine comparison key |
|
||||
| `version_string` | string (semver) | display only |
|
||||
| `version_release_date` | ISO date `YYYY-MM-DD` | display only |
|
||||
|
||||
- `version` is a **number, not a string** (`1`, not `"1"` or `"2.0.1"`). Start at
|
||||
`1` for the first release under this scheme; **`+1` on every change** to any
|
||||
shipped file **or to any manifest metadata** (description, icons, `version_string`).
|
||||
Never reuse or decrement.
|
||||
- Skald stores the installed `version` and compares it to the feed's: a strictly
|
||||
greater feed `version` shows **"update available"** in the marketplace, and the
|
||||
Install button becomes **Update**. Clicking it re-downloads the files and rewrites
|
||||
the catalog row.
|
||||
- **The integer is the *only* "is there an update?" signal** — it is compared
|
||||
strictly (`feed > installed`). `version_string` (semver), icons and
|
||||
`llm_short_description` are **never** compared, so a change to any of them that
|
||||
does not also bump the integer is **invisible**: no "update available" badge
|
||||
appears. This is the common trap — a "content-only" edit (e.g. a better
|
||||
`llm_short_description`) that forgets the integer.
|
||||
- `version` is a **number, not a string** (`8`, not `"8"` or `"2.2.0"`). **`+1` on every
|
||||
change** to any shipped file **or** to any manifest metadata (description, icons,
|
||||
`version_string`). Never reuse or decrement.
|
||||
- **The integer is the only "is there an update?" signal**, compared strictly
|
||||
(`feed > installed`). `version_string`, icons and `llm_short_description` are never
|
||||
compared, so changing them without bumping the integer is **invisible** — no "update
|
||||
available" badge. This is the classic trap.
|
||||
- **The manifest wins over the index** when the two disagree, and the disagreement is only
|
||||
visible as a log line (`marketplace feed version desync`). If the index carries the *lower*
|
||||
number and the manifest the higher one, the strict comparison can never fire again and the
|
||||
connector silently stops offering updates. Keep them equal — `compile.py` does not check
|
||||
this for you.
|
||||
- **Two propagation paths, do not conflate them:**
|
||||
- *Per-user code + deps* (the scripts, `package.json`/`requirements.txt`) reconcile
|
||||
on a **content-hash** of the source files (§5), so new code lands at each user's
|
||||
next login even without a reinstall.
|
||||
- *Catalog metadata* (`llm_short_description` → the model's prompt, icons, friendly
|
||||
name) is **not** in that hash — it lives in the catalog row and is rewritten only
|
||||
by an explicit **reinstall/Update**. On reinstall Skald re-pulls the current feed
|
||||
(never the browse cache) and pushes the new description live: enabled global
|
||||
servers restart with it, and every logged-in user who activated the connector has
|
||||
it restarted with the fresh `llm_short_description` — no re-login needed.
|
||||
- So: to ship a new `llm_short_description`, **bump the integer** (so the admin sees
|
||||
"update available") and the admin clicks **Update**. Nothing auto-propagates a
|
||||
description change.
|
||||
- `version_string` and `version_release_date` are display metadata only — never
|
||||
compared. (Migration note: replace any legacy string `"version": "2.0.1"` with
|
||||
the integer `version` + `version_string`.)
|
||||
- *Per-user code + deps* reconcile on a **content hash** of the source files (§5), so new
|
||||
code lands at each user's next login even without a reinstall.
|
||||
- *Catalog metadata* (`llm_short_description` → the model's prompt, icons, friendly name) is
|
||||
**not** in that hash. It lives in the catalog row and is rewritten only by an explicit
|
||||
reinstall/Update, which re-pulls the current feed (never the 300 s browse cache) and
|
||||
restarts enabled global servers and every live user's copy with the fresh description.
|
||||
- So: to ship a new `llm_short_description`, **bump the integer** so the admin sees "update
|
||||
available" and clicks Update. Nothing auto-propagates a description change.
|
||||
|
||||
---
|
||||
|
||||
## 8. Checklist for a new connector
|
||||
## 8. Hard limits the client enforces
|
||||
|
||||
1. Folder `myconn/` with: entry file, `connector.json`, deps file
|
||||
(`package.json`/`requirements.txt`), `icon_sm.svg`, `icon_lg.svg`,
|
||||
optional `verify.*`.
|
||||
2. Server speaks MCP over stdio (§2); **stdout = JSON-RPC only**.
|
||||
3. `mcp_config.args[0]` names the entry file.
|
||||
4. Correct `type` + `scope` (§3) and `auth.type` (§4).
|
||||
5. For `qr`: implement `login_status` (+ `logout`), persist the session under the
|
||||
connector dir (§4d).
|
||||
6. Deps declared as a file, **not** vendored (§5).
|
||||
7. Add the entry to `connectors.json` with a correct `sha256` for **every** file.
|
||||
8. Bump `version`.
|
||||
```
|
||||
- **Digest mismatch → refused**, all-or-nothing: nothing is written unless every file verifies.
|
||||
- **8 MiB per file.** A `files[].size` above that is rejected before the download starts.
|
||||
- **Path safety.** A `files[].path` or `mcp_config.args[0]` that is absolute, contains `..`,
|
||||
a backslash or a colon is rejected outright.
|
||||
- **A local connector with no `files[]` cannot be installed** ("refusing to install
|
||||
unverifiable code"), and neither can one whose only file is `connector.json`.
|
||||
- **`mcp_config.args[0]` is mandatory** for `mcp_local`.
|
||||
- The feed is fetched as `GET /connectors.json` plus one `GET /<folder>/connector.json` per
|
||||
entry, cached 300 s for browsing; installs always refetch. A manifest that fails to load
|
||||
degrades that one card to whatever the index said — it never fails the listing, which is
|
||||
exactly how a broken manifest hides.
|
||||
|
||||
---
|
||||
|
||||
## 9. Checklist for a new connector
|
||||
|
||||
1. Create `connectors/<id>/` — flat — with: entry file, `connector.json`, `fragment.json`,
|
||||
deps file (`package.json` / `requirements.txt`), `icon_sm.*`, `icon_lg.*`, optional
|
||||
`verify.*`.
|
||||
2. Server speaks MCP over stdio (§2); **stdout = JSON-RPC only**; notifications answered with
|
||||
silence; `title` on every tool.
|
||||
3. `mcp_config.args[0]` names the entry file; `transport` set consistently.
|
||||
4. Correct `type` + `scope` (§3) and `auth.type` (§4); `auth`, `type`, `scope`, `tags`,
|
||||
`requires` and the version trio **identical** in both documents.
|
||||
5. `env[].name` = the real environment variable name (§4b).
|
||||
6. For `qr`: implement `login_status` + `logout`, persist the session under the connector dir
|
||||
(§4d).
|
||||
7. Deps declared as a file, **not** vendored (§5).
|
||||
8. `docs[0].llm_short_description` says what the connector *does*, not which tools it has.
|
||||
9. Add `"<id>"` to `connectors/index.json`.
|
||||
10. Bump `version` (+1), `version_string`, `version_release_date` in **both** documents.
|
||||
11. Run `python3 scripts/compile.py`, then `python3 scripts/compile.py --verify`.
|
||||
12. Record the change in `CHANGELOG.md` in the same commit, then commit and deploy.
|
||||
|
||||
Reference in New Issue
Block a user