Files

T

dguiducci c151739021 docs: replace connector manifest guide with coding agent version, adapted for compile.py workflow

2026-07-22 23:26:52 +01:00

17 KiB

Raw Blame History

Skald Connector Authoring Guide

Instructions for generating a correct connector for the Skald marketplace (https://connectors.skaldagent.net). Give this file to the agent that produces new connectors.

A connector is a folder served by the marketplace. Skald installs it, verifies every file against a SHA-256 pinned in the index, then either runs it on the host (global connector) or copies it into the user's container and runs it there (per-user connector, blueprint §6/§7).

1. The two documents (+ one build script)

1a. The compiled root index — `connectors.json`

This file is auto-generated. Do not edit it by hand. It is produced by scripts/compile.py, which reads index.json + the fragment.json in each connector folder, scans the physical files for SHA-256 digests, and writes the final index.

The output schema looks like this:

{
  "version": 1,
  "connectors": [
    {
      "id": "whatsapp",                 // unique slug = folder name
      "name": "WhatsApp",
      "version": 1,                     // INTEGER build number — the update key (§7)
      "version_string": "2.0.1",        // semver, display only
      "version_release_date": "2026-07-19", // ISO date, display only
      "type": "mcp_local",              // mcp_local | mcp_remote  (see §3)
      "scope": "user",                  // user | global           (see §3)
      "icon_small": "whatsapp/icon_sm.svg",
      "icon_large": "whatsapp/icon_lg.svg",
      "user_description": "Send and read WhatsApp messages from your linked account.",
      "requires": ["NODE"],             // human hint: NODE | PYTHON | OAUTH | API_KEY
      "tags": ["messaging", "mcp", "local", "whatsapp", "qr"],
      "auth": { "type": "qr" },         // may be repeated here and in the manifest
      "folder": "whatsapp",             // defaults to id
      "files": [
        { "path": "index.js",        "sha256": "…", "size": 21258 },
        { "path": "package.json",    "sha256": "…", "size": 302   },
        { "path": "connector.json",  "sha256": "…", "size": 620   },
        { "path": "icon_sm.svg",     "sha256": "…", "size": 306   },
        { "path": "icon_lg.svg",     "sha256": "…", "size": 308   }
      ]
    }
  ]
}

Rules

files[].path is relative to the connector folder. List every file the connector ships (server code, package.json/requirements.txt, icons, and the connector.json itself). A missing or mismatched digest fails the install.
SHA-256 digests and file sizes are computed automatically by compile.py. Never write them by hand.
The excluded files (fragment.json, connectors.json, index.json, .DS_Store) are handled by compile.py — you don't need to think about them.
Do not list node_modules/ or any generated deps — those are installed on the box, not shipped (see §5).
size is optional but the compiler includes it.

How the index is built

Instead of editing connectors.json directly, you maintain two lightweight source files and run one command:

connectors/index.json — a flat JSON array of folder ids in display order: ["gmail", "gcal", "myconn", …]
connectors/<id>/fragment.json — the connector's index entry with every field except files[] (same schema as above minus that array).
python3 scripts/compile.py — reads index.json, loads each fragment.json, scans the folder for real files, computes SHA-256, and writes connectors.json.

The tools[] block (friendly UI names, §2a) goes into fragment.json.

1b. The per-connector manifest — `<folder>/connector.json`

The richer document. Fetched per connector and mapped into Skald's catalog.

{
  "id": "whatsapp",
  "name": "WhatsApp",
  "version": 1,                         // INTEGER build number — the update key (§7)
  "version_string": "2.0.1",            // semver, display only
  "version_release_date": "2026-07-19", // ISO date, display only
  "type": "mcp_local",
  "scope": "user",
  "auth": { "type": "qr" },             // none | api_key | oauth2 | qr  (see §4)
  "mcp_config": {
    "command": "node",                  // interpreter (local) …
    "args": ["index.js"],               // … args[0] MUST name the entry file
    "transport": "stdio"                // stdio (local) | streamable-http (remote)
  },
  "docs": [{
    "lang": "en",
    "description": "Human blurb shown in the UI.",
    "llm_short_description": "One line the model reads to decide whether to use this connector."
  }],
  "env": [],                            // form fields the user fills (see §4b)
  "tools": [                            // OPTIONAL — friendly UI names per tool (§2a)
    { "name": "send_message", "display_name": "Send Message" }
  ],
  "homepage": "https://…",
  "icon_small": "icon_sm.svg",          // relative to the folder here
  "icon_large": "icon_lg.svg",
  "tags": ["messaging", "mcp", "local", "whatsapp", "qr"]
}

mcp_config.args[0] is load-bearing for a local connector: it is how Skald learns which file to run. At activation Skald rewrites it to the file's path inside the user's container (/root/.skald/mcp/<name>/<entry>), so keep it a plain relative filename (index.js, server.py, pkg/server.py).

llm_short_description is model-facing: it's the one-liner injected into the LLM's system prompt so the model knows what this connector does. Keep it short and functional ("Weather — current conditions, 16-day forecast, and AQI data for any location"), not a list of tools (the model discovers tools after activate_tools).

2. Server contract (MCP over stdio)

A local connector is a program speaking JSON-RPC 2.0 over stdin/stdout. It MUST handle:

initialize → { protocolVersion, capabilities: { tools: {} }, serverInfo }
notifications/initialized → no response
tools/list → { tools: [ { name, description, inputSchema } ] }
tools/call → { content: [ { type: "text", text } ], isError? }

stdout is reserved for JSON-RPC only. Send all logs/diagnostics to stderr. Anything a library prints to stdout (a logger, a banner) corrupts the protocol — silence it (e.g. Baileys/pino → a silent logger; Python → print(…, file=sys.stderr)).

A remote connector is an HTTP MCP endpoint (mcp_config.url + transport: "streamable-http"); no code runs on the box.

2a. Friendly tool names (`tools[]`) — optional

Raw MCP tool names are ugly in the chat UI (search_files, send_message). The optional top-level tools[] block gives each one a human title shown as the tool card's heading:

"tools": [
  { "name": "send_message",   "display_name": "Send Message" },
  { "name": "list_chats",     "display_name": "List Chats" },
  { "name": "download_media", "display_name": "Download Media" }
]

name — the raw tool name exactly as your server returns it from tools/list.
display_name — the friendly card title (English only; not internationalized).

Resolution order for a tool's card title is tools[].display_name → the MCP title field → a prettified raw name. So you have two ways to set a friendly name, and can skip tools[] entirely:

This block — the authoritative override, curated in the manifest.
The MCP title field — if your tools/list entries already carry a title (MCP 2025-06-18+), Skald uses it automatically; no manifest change needed. tools[] wins if both are present.
If neither is set, Skald title-cases the raw name (send_message → "Send Message").

Icons are per connector, not per tool. Every tool of a connector shows that connector's own icon_small; there is no per-tool icon field. Only list a tool in tools[] when its prettified name isn't good enough — partial lists are fine (unlisted tools fall through to steps 2–3).

3. Placement & risk vocabulary (what the words mean)

Manifest	Meaning
`scope: "user"`	runs once per user, inside their container. Personal creds.
`scope: "global"`	runs once for the household, on the host. Shared, stateless. Admin enables it with a key.
`type: "mcp_local"`	ships code that will execute on the box — installing needs the admin `mcp.register_local_script` capability (RCE-bearing act, §14).
`type: "mcp_remote"`	just an HTTP URL; no local code.

Pick the narrowest: a personal messaging/email/calendar connector is scope: "user"; a shared search API is scope: "global".

4. Authentication (`auth.type`)

`auth.type`	Flow	Ships
`none`	nothing to sign in	—
`api_key`	user pastes a key/secret into a form	an `env[]` schema (§4b)
`oauth2`	browser consent → paste code back	`auth.provider` + `auth.scopes` + `auth.deliver` (§4c)
`qr`	server shows a QR, user scans with a phone	a `login_status` tool (§4d)

4b. `api_key` — the `env[]` schema

Each entry drives one form field and is injected as an env var / URL token to the server:

"env": [{
  "name": "tavilyApiKey",
  "label": "Tavily API key",
  "description": "Create one at https://app.tavily.com.",
  "required": true,
  "secret": true,                       // rendered masked, stored encrypted
  "example": "tvly-xxxxxxxx"
}]

The server reads each value from process.env.<name> (or os.environ). For a remote connector that wants the key in the URL, use a placeholder: "url": "https://mcp.example.com/?key={SECRET:tavilyApiKey}".

4c. `oauth2` — provider consent

"auth": {
  "type": "oauth2",
  "provider": "google",                 // slug into the admin's sign-in providers
  "scopes": ["https://www.googleapis.com/auth/gmail.modify"],
  "deliver": { "as": "env", "format": "google_authorized_user", "env": "GMAIL_CREDS_JSON" }
}

The manifest names only the provider slug, scopes, and how the obtained token is delivered — never client secrets or endpoint URLs (those are admin-entered, kept off the public feed). Skald handles PKCE + code exchange and injects the credential as the named env var. format: google_authorized_user (Google) or refresh_token. Today only as: "env" is wired.

4d. `qr` / interactive device login — the generic contract

For a connector whose credential is produced by scanning/pairing (WhatsApp today), there is no code to paste. The rule:

Expose one extra tool, login_status, returning a JSON object (as the text of a normal text result). Skald calls it directly (never the agent) and a login panel polls it.

// login_status result text (a JSON string):
{
  "state": "connecting" | "need_scan" | "ready" | "logged_out",
  "qr": "data:image/png;base64,…",   // present ONLY while state == need_scan
  "message": "human-readable line"
}

activate on a qr connector inserts a pending row and starts the server (so it can produce the QR), then hands off to the login panel.
The panel polls POST /api/mcp/login/status; when state == "ready" the connector is marked ready and starts automatically on later logins.
Also expose a logout tool (clears the session, forces a fresh QR) — the panel calls it via POST /api/mcp/login/reset to re-link a different phone.
The credential is the on-disk session, not a token. Persist it inside the connector's own directory (e.g. ./auth/ next to the entry file). That folder lives under the bind-mounted home, so it survives container recreates and connector updates. Never store it under a shared/global path.

Skald resolves auth.type: "qr" the same way whether it appears in the index entry or the manifest.

5. Dependencies (node & python) — how they get installed

Do not ship node_modules/ or vendored wheels. Declare deps as a standard manifest file and Skald installs them inside the container:

node: ship a package.json with a dependencies map. Skald runs npm ci --omit=dev (falling back to npm install --omit=dev) in the connector dir. node_modules/ resolves automatically beside the entry file.
python: ship a requirements.txt. Skald installs it with pip install --target .pydeps and puts .pydeps on the server's PYTHONPATH.

This runs at activation and on every startup, guarded by a content hash of the connector's source files:

first activation / a brand-new container → full install,
a connector update (any shipped file changed) → re-copy + re-install,
unchanged → skipped in microseconds.

So you never write install steps into the manifest — just ship the dep file, list it in the index with its SHA-256, and set requires: ["NODE"] / ["PYTHON"] as a human hint. Pin versions in package.json / requirements.txt for reproducible installs. Keep the dep tree lean (containers are slim; avoid native-heavy packages where a pure alternative exists — e.g. Baileys instead of a browser).

6. Verify-before-save (optional but recommended)

Ship a verify.py / verify snippet and reference it:

"verify": { "command": "python3 verify.py", "timeout_secs": 15 }

It runs with the collected env/secret injected and must print one JSON object on stdout: {"ok": bool, "message": string, "details"?: object}, exit 0 on success. Used for api_key/none connectors to test creds before activating. (A qr connector needs no verify — its login_status is the live check.)

7. Versioning & updates

Three fields, in both the index entry and the connector.json, kept identical:

field	type	role
`version`	integer	monotonic build number, per connector — the machine comparison key
`version_string`	string (semver)	display only
`version_release_date`	ISO date `YYYY-MM-DD`	display only

version is a number, not a string (1, not "1" or "2.0.1"). Start at 1 for the first release under this scheme; +1 on every change to any shipped file or to any manifest metadata (description, icons, version_string). Never reuse or decrement.
Skald stores the installed version and compares it to the feed's: a strictly greater feed version shows "update available" in the marketplace, and the Install button becomes Update. Clicking it re-downloads the files and rewrites the catalog row.
The integer is the only "is there an update?" signal — it is compared strictly (feed > installed). version_string (semver), icons and llm_short_description are never compared, so a change to any of them that does not also bump the integer is invisible: no "update available" badge appears. This is the common trap — a "content-only" edit (e.g. a better llm_short_description) that forgets the integer.
Two propagation paths, do not conflate them:
- Per-user code + deps (the scripts, package.json/requirements.txt) reconcile on a content-hash of the source files (§5), so new code lands at each user's next login even without a reinstall.
- Catalog metadata (llm_short_description → the model's prompt, icons, friendly name) is not in that hash — it lives in the catalog row and is rewritten only by an explicit reinstall/Update. On reinstall Skald re-pulls the current feed (never the browse cache) and pushes the new description live: enabled global servers restart with it, and every logged-in user who activated the connector has it restarted with the fresh llm_short_description — no re-login needed.
So: to ship a new llm_short_description, bump the integer (so the admin sees "update available") and the admin clicks Update. Nothing auto-propagates a description change.
version_string and version_release_date are display metadata only — never compared. (Migration note: replace any legacy string "version": "2.0.1" with the integer version + version_string.)

8. Checklist for a new connector

Folder myconn/ with: entry file, connector.json, deps file (package.json/requirements.txt), icon_sm.svg, icon_lg.svg, optional verify.*.
Create fragment.json inside the folder (same schema as the index entry without files[] — include tools[] if needed).
Add "myconn" to connectors/index.json.
Run python3 scripts/compile.py (auto-generates connectors.json with fresh SHA-256 digests).
Server speaks MCP over stdio (§2); stdout = JSON-RPC only.
mcp_config.args[0] names the entry file.
Correct type + scope (§3) and auth.type (§4).
For qr: implement login_status (+ logout), persist the session under the connector dir (§4d).
Deps declared as a file, not vendored (§5).
Bump version.
Run python3 scripts/compile.py --verify to confirm the index is fresh, then commit.

17 KiB Raw Blame History Unescape Escape