dguiducci/skald-connectors
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ssh: fix sudo failing with "password required" (v6 / 1.1.0)

Browse Source 
The connector asked for a sudo password on every privileged call and
failed whenever nobody answered it, which is every unattended run.

- Always probe `sudo -n` first, even for aliases set to sudo="prompt".
  `_sudo_prefix` used to elicit unconditionally, so a host granting this
  user NOPASSWD still opened an Agent Inbox prompt; with no human there
  it hit the client's 300s ELICITATION_DEADLINE, got back `cancel`, and
  surfaced as "sudo password required (user declined or timed out)".
  sudo refuses before running anything when it wants a password, so the
  probe is side-effect free.
- Strip a leading `sudo` from `command` and turn it into sudo=true.
  Agents write `exec(command="sudo systemctl restart x")`: with
  sudo=false that ran a tty-less sudo, with sudo=true it nested
  `sudo -S ... sudo ...` whose inner prompt had no tty either. Handles
  -u/-n/-S/-E/-H/-i/-k/-p/--; an unknown flag leaves the command alone.
  sudo_user now implies sudo=true.
- Run privileged commands as `sh -c '<command>'`, so `&&`, pipes and
  redirections are elevated too instead of only the first word.
- Add SSH_MCP_SUDO_PASSWORD (optional, secret) for unattended runs. It
  is consulted only after `sudo -n` proved a password is needed, so on a
  NOPASSWD host it never lands in the command's own stdin.
- Actionable errors for every sudo failure mode, and a `hint` on a
  nested sudo we could not peel off.

General review of the same server:

- Drain stdout and stderr together and make timeout_sec a real
  wall-clock deadline. Both streams share one SSH channel window, so
  reading stdout to EOF first stalled once a chatty stderr filled it.
  Command stdin is now closed after the optional password.
- Queue messages that arrive while awaiting an elicitation reply instead
  of discarding them, so a concurrent tools/call is not lost.
- Record the client's `elicitation` capability at initialize and fail
  fast when it is absent rather than blocking on a prompt nobody can
  answer.
- Tolerate null/string integer arguments (depth, max_results,
  context_lines, timeout_sec).
- Realign the version across both manifests: fragment.json said 5/1.0.4
  while connector.json said 2/1.0.1, so the feed was permanently ahead
  of the installed version and offered an update forever.

Also lands the pending docs work: CONNECTOR_MANIFEST_GUIDE.md as the
single source of truth, docs/connector.manifest_guide.md retired to a
pointer, CLAUDE.md audited against the repo, compile.py docstring fixed,
and an opencode.json config.
This commit is contained in:
Daniele
2026-09-03 23:48:00 +01:00
parent 78c78d6039
commit d84ce13dab
10 changed files with 875 additions and 708 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+86 -37
View File
@@ -21,21 +21,39 @@ The only tooling is `scripts/compile.py`, which regenerates the index.
`main` is the **release** branch — only production-ready code lands here. Development/alpha versions will
live on separate branches in the future.
### The consuming project
The application that consumes this marketplace — the code that installs, verifies, and runs these
connectors — lives in a **separate repository at `~/projects/skald-circle`** (Rust workspace: `crates/`,
`Cargo.toml`, `blueprint/`, `agents/`). It is the client for everything documented here: it fetches
`connectors.json`, checks each file against its pinned SHA-256, resolves `auth` / `env` / `verify`, and
launches the MCP servers. The parts worth reading when a manifest question comes up:
| Question | File in `~/projects/skald-circle` |
|----------|-----------------------------------|
| How the feed is parsed, hashed, installed | `src/frontend/api/marketplace.rs` |
| Activation, the env form, OAuth/QR handoff | `src/frontend/api/mcp.rs` |
| Server spec, URL placeholders, transports | `crates/skald-core/src/mcp/mod.rs` |
| Folder layout, container copy, deps install | `crates/skald-core/src/mcp/install.rs` |
| Verify runner + `{ENV:}`/`{SECRET:}` engine | `crates/skald-core/src/mcp/verify.rs` |
It **no longer keeps a copy** of the spec: it references
[CONNECTOR_MANIFEST_GUIDE.md](CONNECTOR_MANIFEST_GUIDE.md) in this repo, which is the single source of
truth for the connector format. Edit the format here and nowhere else.
### The spec
The marketplace spec ships **inside this project**: [CONNECTOR_MANIFEST_GUIDE.md](CONNECTOR_MANIFEST_GUIDE.md)
at the repo root is the authoritative, step-by-step specification for producing a correct connector —
the two documents, the MCP-over-stdio server contract, friendly tool names, placement & risk vocabulary,
the `auth.type` variants (`api_key` / `oauth2` / `qr`), dependency installation, verify-before-save,
versioning, and the new-connector checklist. Give this file to any agent that generates new connectors,
and update it whenever the connector format changes.
the folder layout, the three source documents + the compiler, which document the client reads each field
from, the MCP-over-stdio server contract, friendly tool names, placement & risk vocabulary, the
`auth.type` variants, the `env[]` form and placeholder engines, dependency installation,
verify-before-save, versioning, the limits the client enforces, and the new-connector checklist. Give
this file to any agent that generates new connectors, and update it whenever the connector format
changes — it is the **only** copy, referenced by `~/projects/skald-circle` rather than duplicated there.
Consumers outside this repo (e.g. `~/projects/skald-circle`) read that same file as the format reference.
[docs/connector.manifest_guide.md](docs/connector.manifest_guide.md) is an earlier copy of the same guide;
it is the only one that documents the `fragment.json` + `scripts/compile.py` build pipeline, which
`CONNECTOR_MANIFEST_GUIDE.md` does not yet cover. Until the root guide absorbs that section, the compile
workflow is described below in § Architecture and § Local workflow.
[docs/connector.manifest_guide.md](docs/connector.manifest_guide.md) is a retired copy, now a pointer to
the root guide. Do not edit it.
## Architecture
@@ -157,7 +175,7 @@ connectors/
| `version` | ✅ | Per-connector integer, +1 on every file change |
| `version_string` | ✅ | Semver (display only) |
| `version_release_date` | ✅ | ISO 8601 date (display only) |
| `tools` | ✅ | Array of `{name, display_name}` for friendly UI names |
| `tools` | optional | Array of `{name, display_name}` for friendly UI names. **Documentary here** — Skald reads `tools[]` from `connector.json` (see § Friendly tool names) |
| `auth` | optional | Authentication configuration (if other than `"none"`) |
The `files[]` array is added **automatically** by `compile.py` from the files present in the folder — it
@@ -294,11 +312,15 @@ Structure describing how the connector handles authentication:
{"type": "oauth2", "provider": "google", "scopes": ["…"],
"deliver": {"as": "env", "format": "google_authorized_user", "env": "GMAIL_CREDS_JSON"}}
// OAuth2 with file-based deliver (legacy)
// OAuth2 with file-based deliver — ❌ NOT IMPLEMENTED: Skald rejects `as: "file"` at
// activation with an explicit error. Never ship it.
{"type": "oauth2", "provider": "google", "scopes": ["…"],
"deliver": {"as": "file", "format": "google_authorized_user", "path": "{secrets}/gmail_creds.json"}}
// Password / app-password provided via environment variables
// Password / app-password provided via environment variables — ⚠️ `password` is NOT one of
// the values Skald recognizes (`none`/`api_key`/`oauth2`/`qr`/`ssh_key`); it normalizes to
// `none`. `email` works only because its credentials travel through `env[]`. Prefer
// `api_key` or `none` for new connectors.
{"type": "password", "delivery": "env"}
// QR-code pairing at runtime (WhatsApp)
@@ -314,9 +336,9 @@ Declares **how** Skald delivers the obtained OAuth credential to the MCP server
| Field | Required | Description |
|-------|----------|-------------|
| `as` | ✅ | `"file"` (on disk) or `"env"` (environment variable) |
| `as` | ✅ | `"env"` (environment variable). `"file"` parses but is **rejected at activation** — unimplemented |
| `format` | ✅ | Name of the serialization — e.g. `"google_authorized_user"` (Google JSON that `from_authorized_user_file` reads), `"refresh_token"`, `"access_token"` |
| `path` | `as=file` only | Path with the `{secrets}` placeholder (Skald expands it to a per-user dir at runtime). MUST match the path in `mcp_config.env`. |
| `path` | `as=file` only | Unused while file delivery is unimplemented |
| `env` | `as=env` only | Name of the environment variable into which Skald injects the entire authorized_user JSON. **Must not be declared in `mcp_config.env`** — Skald injects it at runtime. |
The feed NEVER contains `client_id`, `client_secret`, the endpoint URL, or `redirect_uri`. These are
@@ -364,8 +386,7 @@ The `email` connector is the reference example. `tavily`, `gmaps`, and `linkedin
## Placeholder syntax (unified)
Every value skald must fill at runtime with user-provided data uses **one of two tokens**, wherever it
appears (URL, `mcp_config.env`, `verify.command`):
Every value skald must fill at runtime with user-provided data uses **one of two tokens**:
| Token | Meaning | Example |
|-------|---------|---------|
@@ -373,17 +394,27 @@ appears (URL, `mcp_config.env`, `verify.command`):
| `{SECRET:NAME}` | Sensitive variable (password, API key, token) | `{SECRET:EMAIL_PASSWORD}` |
`NAME` is the `name` field declared in the `env[]` array. skald collects the values via a form (masking
`{SECRET:}` fields), injects them as environment into the MCP server / verify process, and substitutes
the tokens in the manifest.
`{SECRET:}` fields) and injects them as environment into the MCP server / verify process.
**Substitution happens in exactly two places** — there is no general template engine:
| Where | Engine | Unknown `NAME` resolves to |
|-------|--------|----------------------------|
| `mcp_config.url` (remote connectors) | `skald-core/src/mcp/mod.rs` | `{SECRET:x}` falls back to the connector's api_key; otherwise the token is **left in the URL literally** |
| `verify.command` | `skald-core/src/mcp/verify.rs` | the **empty string** |
⚠️ **`mcp_config.env` is NOT substituted.** What reaches a local server's environment is the form's
`env[]` values keyed by their `name`, verbatim — so `env[].name` must be exactly the variable name the
script reads. The `mcp_config.env` map survives only as a legacy fallback for the form schema (bare key
names); if it is ever the sole env source its `{ENV:…}` strings are injected literally. `email`, `ssh`
and `linkedin` still carry such a map — harmless today, but do not rely on it to rename a variable.
Rules:
- Unrecognized tokens (`{secrets}/…`, legacy `{key}`, `{env:NAME}`) are **deprecated**: skald does not
substitute them and the manifest must be updated.
- Unrecognized tokens (`{secrets}/…`, `{env:NAME}`) are **deprecated**: skald does not substitute them
and the manifest must be updated. Legacy `{key}` still resolves to the api_key in a URL.
- `{SECRET:<auth.param>}` is reserved for the primary key when `auth.type = "api_key"` (e.g. Tavily:
`?tavilyApiKey={SECRET:tavilyApiKey}`). skald also treats that value as the API key for bearer/header
routing.
- A `{ENV:X}` or `{SECRET:X}` token whose `X` is not in the manifest's `env[]` is substituted with an
empty string (the host cannot guess it).
routing, and stops sending it as a bearer header once it has been spent on the URL.
### Deprecations
@@ -408,7 +439,7 @@ the activation, to confirm the credentials just entered actually work.
| Field | Required | Description |
|-------|----------|-------------|
| `command` | ✅ | Shell command. Runs in the same sandbox as the server: container `skald-{userid}` for `mcp_local` user, host for `mcp_remote` global. The declared env/secrets are injected |
| `timeout_secs` | optional | Default 15. skald kills the process at expiry |
| `timeout_secs` | optional | Declared for the record, but **not yet plumbed through**: the runtime applies a fixed 20 s to every verify (`VERIFY_TIMEOUT_SECS` in `src/frontend/api/mcp.rs`). Keep the probe well under that |
### Output convention
@@ -431,10 +462,12 @@ fails). **Never print credentials** in `message`/`details`.
### Where to put the script
If `command` references a file (e.g. `verify.py`), the file must be saved in the connector folder
(`<id>/verify.py`); `compile.py` then picks it up into `files[]` with its SHA-256 and size. skald
downloads it, verifies the hash against the index, and makes it available at the same path as the main
server (container for `mcp_local`, `./scripts/<id>/` on the host for `mcp_remote`). A `verify` command
may also be fully inline (see `firecrawl`, which uses a `node -e "…"` one-liner).
(`<id>/verify.py`); `compile.py` then picks it up into `files[]` with its SHA-256 and size. skald finds
the script by matching a **basename from `files[]`** against the command string, downloads it, verifies
the hash against the index, and runs it in the connector's own directory — inside the container
`skald-{userid}` (`~/.skald/mcp/<name>/`) for a per-user connector, on the host in `./connectors/<id>/`
for a global one. A `verify` command may also be fully inline (see `firecrawl`, which uses a
`node -e "…"` one-liner) — with no basename match, nothing extra is fetched.
### Without `verify`
@@ -448,15 +481,20 @@ Every MCP tool must expose a friendly name for the Skald UI. Two ways, in order
1. **Via the MCP script (preferred)** — add `"title": "Friendly Name"` to each tool definition returned by
`tools/list`. Works for all local scripts (Python/Node) that we control.
2. **Via the manifest (fallback)** — add `"tools": [{"name": "…", "display_name": "…"}]` in
`fragment.json` **and** in `connector.json`. Used only for remote connectors or external packages
(e.g. `npx -y firecrawl-mcp`).
2. **Via the manifest (fallback)** — add `"tools": [{"name": "…", "display_name": "…"}]` to
**`connector.json`**. Used only for remote connectors or external packages (e.g.
`npx -y firecrawl-mcp`).
**Resolution order** used by Skald:
1. `tools[].display_name` from the manifest
2. `title` from the MCP server's `tools/list`
3. Automatic prettify of the raw name (`send_message` → "Send Message")
⚠️ **Skald reads `tools[]` from `connector.json` only** — the index entry carries no `tools` field in
the client's parser, so a block that lives only in `fragment.json` is inert (that is the state of
`google-trends` today, which is harmless because its script also sets `title`). Mirroring the block into
`fragment.json` is optional and purely documentary.
As of 2026-07-21 all marketplace connectors carry `title` in the script or `tools[]` in the manifest.
## MCP server conventions
@@ -486,9 +524,14 @@ Connector-specific notes:
- **Tavily** — `mcp_remote`/`global`; API key declared as an `env[]` secret and templated into the URL as
`{SECRET:tavilyApiKey}`.
- **SSH** — `mcp_local`/`user`; stores aliases in `~/.ssh_aliases.json` (auto-managed, 0600). Auth
per-alias: key/agent (default) or elicited password. Sudo via `nopasswd` or elicited password
(`sudo -S`). No setup-time credentials: `auth.type: "none"` with no `verify`. All `SSH_MCP_*` env vars
are optional with defaults.
per-alias: key/agent (default) or elicited password. Sudo always probes `sudo -n` first (free on a
NOPASSWD host, and side-effect free elsewhere since sudo refuses before running the command), then
falls back to `sudo -S` with a password from `SSH_MCP_SUDO_PASSWORD` or MCP elicitation; a leading
`sudo` inside `command` is stripped and turned into `sudo=true`, and under sudo the command runs as
`sh -c '…'` so pipes/redirections are privileged too. Elicitation needs a human in the Agent Inbox
within 300 s (`ELICITATION_DEADLINE` in `skald-core/src/elicitation/mod.rs`) — unattended runs must
rely on NOPASSWD or `SSH_MCP_SUDO_PASSWORD`. No setup-time credentials: `auth.type: "none"` with no
`verify`. All `SSH_MCP_*` env vars are optional with defaults.
## Current connectors
@@ -539,9 +582,15 @@ A connector without `verify` is activated without any test — see § Without ve
- **Never edit `connectors.json` by hand.** It is generated. Edit `fragment.json` / `index.json` /
the connector files, then run `python3 scripts/compile.py`.
- **Bump `version` (+1) and `version_string` on every file change** to a connector, in both
`fragment.json` and `connector.json`, and keep `version_release_date` current.
`fragment.json` and `connector.json`, and keep `version_release_date` current. When the two disagree
**the manifest wins** and Skald only logs a `marketplace feed version desync` warning — and if the
index carries the *lower* number, the strict `feed > installed` comparison can never fire again and
the connector silently stops offering updates. `compile.py` does not check this.
- **Keep `id`, `name`, `type`, `scope`, `tags`, `requires`, and `auth` consistent** between
`fragment.json` and `connector.json`.
`fragment.json` and `connector.json`. Only the manifest's `auth` is parsed by Skald; the index's is
documentary, with `requires` as the sole coarse fallback (`OAUTH` → oauth, `API_KEY` → api_key).
- **Keep the connector folder flat.** `compile.py` scans only the folder's top level, so any file in a
subdirectory is silently absent from `files[]` — never downloaded, never installed.
- **Adding a `verify` script means shipping the file in the connector folder** and recompiling — skald
refuses to run a script whose SHA-256 is not pinned in the index.
- **Always run `python3 scripts/compile.py` before deploying**; a stale hash breaks integrity