dguiducci/skald-connectors
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ssh: fix sudo failing with "password required" (v6 / 1.1.0)

Browse Source 
The connector asked for a sudo password on every privileged call and
failed whenever nobody answered it, which is every unattended run.

- Always probe `sudo -n` first, even for aliases set to sudo="prompt".
  `_sudo_prefix` used to elicit unconditionally, so a host granting this
  user NOPASSWD still opened an Agent Inbox prompt; with no human there
  it hit the client's 300s ELICITATION_DEADLINE, got back `cancel`, and
  surfaced as "sudo password required (user declined or timed out)".
  sudo refuses before running anything when it wants a password, so the
  probe is side-effect free.
- Strip a leading `sudo` from `command` and turn it into sudo=true.
  Agents write `exec(command="sudo systemctl restart x")`: with
  sudo=false that ran a tty-less sudo, with sudo=true it nested
  `sudo -S ... sudo ...` whose inner prompt had no tty either. Handles
  -u/-n/-S/-E/-H/-i/-k/-p/--; an unknown flag leaves the command alone.
  sudo_user now implies sudo=true.
- Run privileged commands as `sh -c '<command>'`, so `&&`, pipes and
  redirections are elevated too instead of only the first word.
- Add SSH_MCP_SUDO_PASSWORD (optional, secret) for unattended runs. It
  is consulted only after `sudo -n` proved a password is needed, so on a
  NOPASSWD host it never lands in the command's own stdin.
- Actionable errors for every sudo failure mode, and a `hint` on a
  nested sudo we could not peel off.

General review of the same server:

- Drain stdout and stderr together and make timeout_sec a real
  wall-clock deadline. Both streams share one SSH channel window, so
  reading stdout to EOF first stalled once a chatty stderr filled it.
  Command stdin is now closed after the optional password.
- Queue messages that arrive while awaiting an elicitation reply instead
  of discarding them, so a concurrent tools/call is not lost.
- Record the client's `elicitation` capability at initialize and fail
  fast when it is absent rather than blocking on a prompt nobody can
  answer.
- Tolerate null/string integer arguments (depth, max_results,
  context_lines, timeout_sec).
- Realign the version across both manifests: fragment.json said 5/1.0.4
  while connector.json said 2/1.0.1, so the feed was permanently ahead
  of the installed version and offered an update forever.

Also lands the pending docs work: CONNECTOR_MANIFEST_GUIDE.md as the
single source of truth, docs/connector.manifest_guide.md retired to a
pointer, CLAUDE.md audited against the repo, compile.py docstring fixed,
and an opencode.json config.
This commit is contained in:
Daniele
2026-09-03 23:48:00 +01:00
parent 78c78d6039
commit d84ce13dab
10 changed files with 875 additions and 708 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/connector.manifest_guide.md
+7 -378
View File
@@ -1,380 +1,9 @@
# Skald Connector Authoring Guide
# Skald Connector Authoring Guide — moved
Instructions for generating a **correct connector** for the Skald marketplace
(`https://connectors.skaldagent.net`). Give this file to the agent that produces
new connectors.
This copy has been retired. The authoring spec now lives, in a single place, at
[`CONNECTOR_MANIFEST_GUIDE.md`](../CONNECTOR_MANIFEST_GUIDE.md) in the repo root — it absorbed
everything this file documented (the `fragment.json` + `scripts/compile.py` build pipeline)
plus the client behaviour verified against `~/projects/skald-circle`.
A connector is a folder served by the marketplace. Skald installs it, verifies
every file against a SHA-256 pinned in the index, then either runs it on the host
(global connector) or copies it into the user's container and runs it there
(per-user connector, blueprint §6/§7).
---
## 1. The two documents (+ one build script)
### 1a. The compiled root index — `connectors.json`
**This file is auto-generated.** Do not edit it by hand. It is produced by
[`scripts/compile.py`](../scripts/compile.py), which reads `index.json` + the
`fragment.json` in each connector folder, scans the physical files for SHA-256
digests, and writes the final index.
The output schema looks like this:
```jsonc
{
"version": 1,
"connectors": [
{
"id": "whatsapp", // unique slug = folder name
"name": "WhatsApp",
"version": 1, // INTEGER build number — the update key (§7)
"version_string": "2.0.1", // semver, display only
"version_release_date": "2026-07-19", // ISO date, display only
"type": "mcp_local", // mcp_local | mcp_remote (see §3)
"scope": "user", // user | global (see §3)
"icon_small": "whatsapp/icon_sm.svg",
"icon_large": "whatsapp/icon_lg.svg",
"user_description": "Send and read WhatsApp messages from your linked account.",
"requires": ["NODE"], // human hint: NODE | PYTHON | OAUTH | API_KEY
"tags": ["messaging", "mcp", "local", "whatsapp", "qr"],
"auth": { "type": "qr" }, // may be repeated here and in the manifest
"folder": "whatsapp", // defaults to id
"files": [
{ "path": "index.js", "sha256": "…", "size": 21258 },
{ "path": "package.json", "sha256": "…", "size": 302 },
{ "path": "connector.json", "sha256": "…", "size": 620 },
{ "path": "icon_sm.svg", "sha256": "…", "size": 306 },
{ "path": "icon_lg.svg", "sha256": "…", "size": 308 }
]
}
]
}
```
**Rules**
- `files[].path` is relative to the connector folder. List **every** file the
connector ships (server code, `package.json`/`requirements.txt`, icons, and the
`connector.json` itself). A missing or mismatched digest fails the install.
- SHA-256 digests and file sizes are **computed automatically** by `compile.py`.
Never write them by hand.
- The excluded files (`fragment.json`, `connectors.json`, `index.json`,
`.DS_Store`) are handled by `compile.py` — you don't need to think about them.
- Do **not** list `node_modules/` or any generated deps — those are installed on
the box, not shipped (see §5).
- `size` is optional but the compiler includes it.
#### How the index is built
Instead of editing `connectors.json` directly, you maintain **two lightweight
source files** and run one command:
1. **`connectors/index.json`** — a flat JSON array of folder ids in display
order: `["gmail", "gcal", "myconn", …]`
2. **`connectors/<id>/fragment.json`** — the connector's index entry with
**every field except `files[]`** (same schema as above minus that array).
3. **`python3 scripts/compile.py`** — reads `index.json`, loads each
`fragment.json`, scans the folder for real files, computes SHA-256, and
writes `connectors.json`.
The `tools[]` block (friendly UI names, §2a) goes into `fragment.json`.
### 1b. The per-connector manifest — `<folder>/connector.json`
The richer document. Fetched per connector and mapped into Skald's catalog.
```jsonc
{
"id": "whatsapp",
"name": "WhatsApp",
"version": 1, // INTEGER build number — the update key (§7)
"version_string": "2.0.1", // semver, display only
"version_release_date": "2026-07-19", // ISO date, display only
"type": "mcp_local",
"scope": "user",
"auth": { "type": "qr" }, // none | api_key | oauth2 | qr (see §4)
"mcp_config": {
"command": "node", // interpreter (local) …
"args": ["index.js"], // … args[0] MUST name the entry file
"transport": "stdio" // stdio (local) | streamable-http (remote)
},
"docs": [{
"lang": "en",
"description": "Human blurb shown in the UI.",
"llm_short_description": "One line the model reads to decide whether to use this connector."
}],
"env": [], // form fields the user fills (see §4b)
"tools": [ // OPTIONAL — friendly UI names per tool (§2a)
{ "name": "send_message", "display_name": "Send Message" }
],
"homepage": "https://…",
"icon_small": "icon_sm.svg", // relative to the folder here
"icon_large": "icon_lg.svg",
"tags": ["messaging", "mcp", "local", "whatsapp", "qr"]
}
```
**`mcp_config.args[0]` is load-bearing for a local connector:** it is how Skald
learns which file to run. At activation Skald rewrites it to the file's path
inside the user's container (`/root/.skald/mcp/<name>/<entry>`), so keep it a
plain relative filename (`index.js`, `server.py`, `pkg/server.py`).
**`llm_short_description` is model-facing:** it's the one-liner injected into
the LLM's system prompt so the model knows what this connector does. Keep it
short and functional ("Weather — current conditions, 16-day forecast, and AQI
data for any location"), **not** a list of tools (the model discovers tools
after `activate_tools`).
---
## 2. Server contract (MCP over stdio)
A **local** connector is a program speaking JSON-RPC 2.0 over stdin/stdout. It
MUST handle:
- `initialize` → `{ protocolVersion, capabilities: { tools: {} }, serverInfo }`
- `notifications/initialized` → no response
- `tools/list` → `{ tools: [ { name, description, inputSchema } ] }`
- `tools/call` → `{ content: [ { type: "text", text } ], isError? }`
**stdout is reserved for JSON-RPC only.** Send all logs/diagnostics to **stderr**.
Anything a library prints to stdout (a logger, a banner) corrupts the protocol —
silence it (e.g. Baileys/pino → a silent logger; Python → `print(…, file=sys.stderr)`).
A **remote** connector is an HTTP MCP endpoint (`mcp_config.url` +
`transport: "streamable-http"`); no code runs on the box.
### 2a. Friendly tool names (`tools[]`) — optional
Raw MCP tool names are ugly in the chat UI (`search_files`, `send_message`). The
optional top-level `tools[]` block gives each one a human title shown as the tool
card's heading:
```jsonc
"tools": [
{ "name": "send_message", "display_name": "Send Message" },
{ "name": "list_chats", "display_name": "List Chats" },
{ "name": "download_media", "display_name": "Download Media" }
]
```
- `name` — the **raw** tool name exactly as your server returns it from `tools/list`.
- `display_name` — the friendly card title (English only; not internationalized).
**Resolution order** for a tool's card title is **`tools[].display_name` → the MCP
`title` field → a prettified raw name**. So you have two ways to set a friendly
name, and can skip `tools[]` entirely:
1. **This block** — the authoritative override, curated in the manifest.
2. **The MCP `title` field** — if your `tools/list` entries already carry a
`title` (MCP 2025-06-18+), Skald uses it automatically; no manifest change
needed. `tools[]` wins if both are present.
3. If neither is set, Skald title-cases the raw name (`send_message` → "Send
Message").
**Icons are per connector, not per tool.** Every tool of a connector shows that
connector's own `icon_small`; there is no per-tool icon field. Only list a tool in
`tools[]` when its prettified name isn't good enough — partial lists are fine
(unlisted tools fall through to steps 2–3).
---
## 3. Placement & risk vocabulary (what the words mean)
| Manifest | Meaning |
| --- | --- |
| `scope: "user"` | runs **once per user**, inside their container. Personal creds. |
| `scope: "global"` | runs **once for the household**, on the host. Shared, stateless. Admin enables it with a key. |
| `type: "mcp_local"` | ships code that will **execute on the box** — installing needs the admin `mcp.register_local_script` capability (RCE-bearing act, §14). |
| `type: "mcp_remote"` | just an HTTP URL; no local code. |
Pick the narrowest: a personal messaging/email/calendar connector is
`scope: "user"`; a shared search API is `scope: "global"`.
---
## 4. Authentication (`auth.type`)
| `auth.type` | Flow | Ships |
| --- | --- | --- |
| `none` | nothing to sign in | — |
| `api_key` | user pastes a key/secret into a form | an `env[]` schema (§4b) |
| `oauth2` | browser consent → paste code back | `auth.provider` + `auth.scopes` + `auth.deliver` (§4c) |
| `qr` | server shows a QR, user scans with a phone | a `login_status` tool (§4d) |
### 4b. `api_key` — the `env[]` schema
Each entry drives one form field **and** is injected as an env var / URL token to
the server:
```jsonc
"env": [{
"name": "tavilyApiKey",
"label": "Tavily API key",
"description": "Create one at https://app.tavily.com.",
"required": true,
"secret": true, // rendered masked, stored encrypted
"example": "tvly-xxxxxxxx"
}]
```
The server reads each value from `process.env.<name>` (or `os.environ`). For a
**remote** connector that wants the key in the URL, use a placeholder:
`"url": "https://mcp.example.com/?key={SECRET:tavilyApiKey}"`.
### 4c. `oauth2` — provider consent
```jsonc
"auth": {
"type": "oauth2",
"provider": "google", // slug into the admin's sign-in providers
"scopes": ["https://www.googleapis.com/auth/gmail.modify"],
"deliver": { "as": "env", "format": "google_authorized_user", "env": "GMAIL_CREDS_JSON" }
}
```
The manifest names **only** the provider slug, scopes, and how the obtained token
is delivered — never client secrets or endpoint URLs (those are admin-entered,
kept off the public feed). Skald handles PKCE + code exchange and injects the
credential as the named env var. `format`: `google_authorized_user` (Google) or
`refresh_token`. Today only `as: "env"` is wired.
### 4d. `qr` / interactive device login — the generic contract
For a connector whose credential is produced by **scanning/pairing** (WhatsApp
today), there is no code to paste. The rule:
> **Expose one extra tool, `login_status`, returning a JSON object** (as the
> `text` of a normal text result). Skald calls it directly (never the agent) and a
> login panel polls it.
```jsonc
// login_status result text (a JSON string):
{
"state": "connecting" | "need_scan" | "ready" | "logged_out",
"qr": "data:image/png;base64,…", // present ONLY while state == need_scan
"message": "human-readable line"
}
```
- `activate` on a `qr` connector inserts a **pending** row and **starts the
server** (so it can produce the QR), then hands off to the login panel.
- The panel polls `POST /api/mcp/login/status`; when `state == "ready"` the
connector is marked ready and starts automatically on later logins.
- Also expose a `logout` tool (clears the session, forces a fresh QR) — the panel
calls it via `POST /api/mcp/login/reset` to re-link a different phone.
- The **credential is the on-disk session**, not a token. Persist it **inside the
connector's own directory** (e.g. `./auth/` next to the entry file). That folder
lives under the bind-mounted home, so it survives container recreates and
connector updates. Never store it under a shared/global path.
Skald resolves `auth.type: "qr"` the same way whether it appears in the index
entry or the manifest.
---
## 5. Dependencies (node & python) — how they get installed
**Do not ship `node_modules/` or vendored wheels.** Declare deps as a standard
manifest **file** and Skald installs them inside the container:
- **node:** ship a `package.json` with a `dependencies` map. Skald runs
`npm ci --omit=dev` (falling back to `npm install --omit=dev`) in the connector
dir. `node_modules/` resolves automatically beside the entry file.
- **python:** ship a `requirements.txt`. Skald installs it with
`pip install --target .pydeps` and puts `.pydeps` on the server's `PYTHONPATH`.
This runs at activation **and** on every startup, guarded by a **content hash** of
the connector's source files:
- first activation / a brand-new container → full install,
- a connector **update** (any shipped file changed) → re-copy + re-install,
- unchanged → skipped in microseconds.
So you never write install steps into the manifest — just ship the dep file, list
it in the index with its SHA-256, and set `requires: ["NODE"]` / `["PYTHON"]` as a
human hint. Pin versions in `package.json` / `requirements.txt` for reproducible
installs. Keep the dep tree lean (containers are slim; avoid native-heavy
packages where a pure alternative exists — e.g. Baileys instead of a browser).
---
## 6. Verify-before-save (optional but recommended)
Ship a `verify.py` / verify snippet and reference it:
```jsonc
"verify": { "command": "python3 verify.py", "timeout_secs": 15 }
```
It runs with the collected env/secret injected and must print **one JSON object**
on stdout: `{"ok": bool, "message": string, "details"?: object}`, exit 0 on
success. Used for `api_key`/`none` connectors to test creds before activating.
(A `qr` connector needs no verify — its `login_status` is the live check.)
---
## 7. Versioning & updates
Three fields, in **both** the index entry and the `connector.json`, kept identical:
| field | type | role |
| --- | --- | --- |
| `version` | **integer** | monotonic build number, **per connector** — the machine comparison key |
| `version_string` | string (semver) | display only |
| `version_release_date` | ISO date `YYYY-MM-DD` | display only |
- `version` is a **number, not a string** (`1`, not `"1"` or `"2.0.1"`). Start at
`1` for the first release under this scheme; **`+1` on every change** to any
shipped file **or to any manifest metadata** (description, icons, `version_string`).
Never reuse or decrement.
- Skald stores the installed `version` and compares it to the feed's: a strictly
greater feed `version` shows **"update available"** in the marketplace, and the
Install button becomes **Update**. Clicking it re-downloads the files and rewrites
the catalog row.
- **The integer is the *only* "is there an update?" signal** — it is compared
strictly (`feed > installed`). `version_string` (semver), icons and
`llm_short_description` are **never** compared, so a change to any of them that
does not also bump the integer is **invisible**: no "update available" badge
appears. This is the common trap — a "content-only" edit (e.g. a better
`llm_short_description`) that forgets the integer.
- **Two propagation paths, do not conflate them:**
- *Per-user code + deps* (the scripts, `package.json`/`requirements.txt`) reconcile
on a **content-hash** of the source files (§5), so new code lands at each user's
next login even without a reinstall.
- *Catalog metadata* (`llm_short_description` → the model's prompt, icons, friendly
name) is **not** in that hash — it lives in the catalog row and is rewritten only
by an explicit **reinstall/Update**. On reinstall Skald re-pulls the current feed
(never the browse cache) and pushes the new description live: enabled global
servers restart with it, and every logged-in user who activated the connector has
it restarted with the fresh `llm_short_description` — no re-login needed.
- So: to ship a new `llm_short_description`, **bump the integer** (so the admin sees
"update available") and the admin clicks **Update**. Nothing auto-propagates a
description change.
- `version_string` and `version_release_date` are display metadata only — never
compared. (Migration note: replace any legacy string `"version": "2.0.1"` with
the integer `version` + `version_string`.)
---
## 8. Checklist for a new connector
1. Folder `myconn/` with: entry file, `connector.json`, deps file
(`package.json`/`requirements.txt`), `icon_sm.svg`, `icon_lg.svg`,
optional `verify.*`.
2. Create `fragment.json` inside the folder (same schema as the index entry
without `files[]` — include `tools[]` if needed).
3. Add `"myconn"` to `connectors/index.json`.
4. Run `python3 scripts/compile.py` (auto-generates `connectors.json` with
fresh SHA-256 digests).
5. Server speaks MCP over stdio (§2); **stdout = JSON-RPC only**.
6. `mcp_config.args[0]` names the entry file.
7. Correct `type` + `scope` (§3) and `auth.type` (§4).
8. For `qr`: implement `login_status` (+ `logout`), persist the session under the
connector dir (§4d).
9. Deps declared as a file, **not** vendored (§5).
10. Bump `version`.
11. Run `python3 scripts/compile.py --verify` to confirm the index is fresh,
then commit.
Keeping two copies is what let them drift in the first place. Edit the root file only; the
history of this one is in git.