chore: delete scripts/ and cut requirements.txt down to its real consumers
Nightly Build / build (push) Successful in 7m13s
Nightly Build / build (push) Successful in 7m13s
scripts/ held the pre-marketplace MCP servers (gmail, gcal, gmaps, ssh, weather, google_trends, whatsapp, serpapi_flights). Nothing referenced them any more: connectors are admin-curated and installed into connectors/ from the marketplace, and ci/package.sh never shipped scripts/ in the first place — so on every installed box requirements.txt was pulling google-auth, googlemaps, paramiko, trendspyg and friends for files that did not exist there. requirements.txt now states what it is actually for: the two TTS plugins, which spawn a bare `python3` on an embedded server script and so have no dependency reconciler of their own. A connector's deps stay with the connector — `ensure_installed` puts them in .pydeps/node_modules inside the user's container, `ensure_installed_host` beside the files for a global one. The venv itself stays load-bearing for those two plugins and for the host pip that installs a global connector's deps, so the run/install/update scripts keep creating it — but their "Python MCP servers will be unavailable" warning was naming the one thing that no longer depends on it, and now says what really breaks. CONNECTOR_MANIFEST_GUIDE.md moves to the repo root: it was the one thing in scripts/ still referenced (CLAUDE.md), and being under a gitignored directory it had never been committed at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -193,7 +193,7 @@ MCP servers are surfaced to users as **"Connectors"** (UI naming; `mcp`/schema s
|
||||
|
||||
**Endpoints** (`src/frontend/api/mcp.rs`, mounted in `api/mod.rs`) — admin: `/mcp/catalog` (GET/POST/DELETE), `/mcp/global` (list/enable/delete + `/{id}/access` GET/PUT), `/mcp/providers` (GET/POST + DELETE `/{name}` — OAuth provider creds, secret never returned to the browser). User: `/mcp/available`, `/mcp/activate`, `/mcp/activated` (+ DELETE `/{id}` to deactivate), `/mcp/oauth/start` + `/mcp/oauth/complete` (the §15 OAuth login), `/mcp/login/status` + `/mcp/login/reset` (the §15 QR/device login — see below). `connectors.js` (`<connectors-page>`) is the **single** Connectors surface — a row list, one row per connector (there is no separate catalog page): the user view (activate/deactivate + granted globals) always, plus the admin affordances when `role_id === 'admin'` — the **Add connector** dropdown (from the Marketplace, or manually via the `#connectors/new` sub-page), per-row removal from the catalog, and the **Sign-in providers** modal. The Marketplace stays its own page (`marketplace.js`), reached from that dropdown and linking back to `#connectors`. `connector-detail.js` (`<connector-detail-page>`) is a connector's own page and hosts both the OAuth login panel and the QR login panel.
|
||||
|
||||
**Dependency reconciler (`mcp::install::ensure_installed`).** Copying a local-script connector's files into a container never installed its deps. `ensure_installed` closes that: a **content-hash reconciler** keyed on the connector's *source* files (not a version string) that, when the hash changed, re-copies the files and installs deps inside the container — `npm ci --omit=dev` (node, from `package.json`) and/or `pip install --target .pydeps` (python, from `requirements.txt`, put on the server's `PYTHONPATH` by `user_row_spec`). Runs at activation **and** on every per-user startup path (`UserContext` build, remount) via `mcp::prepare_local_connector`, so a fresh container installs from scratch, an updated connector re-installs, and an unchanged one is a hash-match no-op. Deps are therefore **never vendored** — connectors ship `package.json`/`requirements.txt`, not `node_modules/`. Authoring contract for connectors lives in `scripts/CONNECTOR_MANIFEST_GUIDE.md`.
|
||||
**Dependency reconciler (`mcp::install::ensure_installed`).** Copying a local-script connector's files into a container never installed its deps. `ensure_installed` closes that: a **content-hash reconciler** keyed on the connector's *source* files (not a version string) that, when the hash changed, re-copies the files and installs deps inside the container — `npm ci --omit=dev` (node, from `package.json`) and/or `pip install --target .pydeps` (python, from `requirements.txt`, put on the server's `PYTHONPATH` by `user_row_spec`). Runs at activation **and** on every per-user startup path (`UserContext` build, remount) via `mcp::prepare_local_connector`, so a fresh container installs from scratch, an updated connector re-installs, and an unchanged one is a hash-match no-op. Deps are therefore **never vendored** — connectors ship `package.json`/`requirements.txt`, not `node_modules/`. Authoring contract for connectors lives in `CONNECTOR_MANIFEST_GUIDE.md` (repo root).
|
||||
|
||||
**Connector versioning.** `mcp_catalog` carries `version` (INTEGER — the update-comparison key), `version_string` (semver, display) and `version_release_date` (ISO, display), snapshotted from the feed on install. The marketplace list computes `update_available` = feed `version` > installed `version` (strict) and surfaces it as an "Update" button (`marketplace.js`). The integer is the UI signal; the actual re-install trigger is the reconciler's content-hash.
|
||||
|
||||
@@ -367,11 +367,13 @@ Copy `default.config.yaml` → `config.yml`. Never commit `config.yml` (contains
|
||||
|
||||
## Python environment
|
||||
|
||||
All Python scripts (MCP servers, setup scripts) use a local virtualenv at `.venv/` in the project root.
|
||||
Host-side Python runs from a local virtualenv at `.venv/` in the project root. `run.sh` creates it on first launch (using `uv` if available, otherwise `python3 -m venv`), installs `requirements.txt`, and prepends `.venv/bin` to `PATH` before starting the app, so every child process resolves `python3` to the venv. No manual activation needed.
|
||||
|
||||
`run.sh` creates it automatically on first launch (using `uv` if available, otherwise `python3 -m venv`) and installs `requirements.txt`. It then prepends `.venv/bin` to `PATH` before starting the app, so every child process — MCP server launches, `execute_cmd` shell calls — resolves `python3` to the venv automatically. No manual activation needed. **Python is optional**: if neither `uv` nor `python3` is found, the app starts normally and only Python-based MCP servers will be unavailable.
|
||||
**`requirements.txt` is for the two TTS plugins, and nothing else.** `plugin-tts-kokoro` and `plugin-tts-orpheus-3b` write an embedded server script to disk and spawn a bare `python3` on it — they have no dependency reconciler of their own, so their imports must be satisfied in the venv. The GPU/ML half of Orpheus (torch, transformers, snac, bitsandbytes, huggingface_hub) is split into `requirements-optional.txt`, installed by hand.
|
||||
|
||||
To add a Python dependency: add it to `requirements.txt`. It will be installed on the next `./run.sh` invocation if `.venv` does not yet exist — or run `uv pip install -r requirements.txt` manually.
|
||||
**A connector's deps never go in `requirements.txt`.** A connector ships its own `requirements.txt`/`package.json` and `mcp::install::ensure_installed` installs it into `.pydeps`/`node_modules` — inside the user's container for a per-user connector, beside the connector's files on the host for a global one (`ensure_installed_host`). Putting them in the root file would install them on every box for a connector nobody activated; this is what the file used to do for the since-deleted `scripts/` MCP servers.
|
||||
|
||||
**Python is optional**: with neither `uv` nor `python3` present the app starts normally; the TTS plugins fail to start and a host-run global connector has no interpreter to install its deps with. Per-user connectors are unaffected — they run in the container, which ships its own Python.
|
||||
|
||||
## Frontend components (`web/components/`)
|
||||
|
||||
|
||||
@@ -0,0 +1,348 @@
|
||||
# Skald Connector Authoring Guide
|
||||
|
||||
Instructions for generating a **correct connector** for the Skald marketplace
|
||||
(`https://connectors.skaldagent.net`). Give this file to the agent that produces
|
||||
new connectors.
|
||||
|
||||
A connector is a folder served by the marketplace. Skald installs it, verifies
|
||||
every file against a SHA-256 pinned in the index, then either runs it on the host
|
||||
(global connector) or copies it into the user's container and runs it there
|
||||
(per-user connector, blueprint §6/§7).
|
||||
|
||||
---
|
||||
|
||||
## 1. The two documents
|
||||
|
||||
### 1a. The root index — `connectors.json`
|
||||
|
||||
One array of entries, each pointing at a connector folder. **The index is the
|
||||
signable root: it is the only place that lists a connector's files and their
|
||||
SHA-256 digests.** Skald refuses any file whose bytes do not match.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"version": 1,
|
||||
"connectors": [
|
||||
{
|
||||
"id": "whatsapp", // unique slug = folder name
|
||||
"name": "WhatsApp",
|
||||
"version": 1, // INTEGER build number — the update key (§7)
|
||||
"version_string": "2.0.1", // semver, display only
|
||||
"version_release_date": "2026-07-19", // ISO date, display only
|
||||
"type": "mcp_local", // mcp_local | mcp_remote (see §3)
|
||||
"scope": "user", // user | global (see §3)
|
||||
"icon_small": "whatsapp/icon_sm.svg",
|
||||
"icon_large": "whatsapp/icon_lg.svg",
|
||||
"user_description": "Send and read WhatsApp messages from your linked account.",
|
||||
"requires": ["NODE"], // human hint: NODE | PYTHON | OAUTH | API_KEY
|
||||
"tags": ["messaging", "mcp", "local", "whatsapp", "qr"],
|
||||
"auth": { "type": "qr" }, // may be repeated here and in the manifest
|
||||
"folder": "whatsapp", // defaults to id
|
||||
"files": [
|
||||
{ "path": "index.js", "sha256": "…", "size": 21258 },
|
||||
{ "path": "package.json", "sha256": "…", "size": 302 },
|
||||
{ "path": "connector.json", "sha256": "…", "size": 620 },
|
||||
{ "path": "icon_sm.svg", "sha256": "…", "size": 306 },
|
||||
{ "path": "icon_lg.svg", "sha256": "…", "size": 308 }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Rules**
|
||||
|
||||
- `files[].path` is relative to the connector folder. List **every** file the
|
||||
connector ships (server code, `package.json`/`requirements.txt`, icons, and the
|
||||
`connector.json` itself). A missing or mismatched digest fails the install.
|
||||
- Compute `sha256` over the exact bytes served: `sha256sum <file>`.
|
||||
- Do **not** list `node_modules/` or any generated deps — those are installed on
|
||||
the box, not shipped (see §5).
|
||||
- `size` is optional but recommended.
|
||||
|
||||
### 1b. The per-connector manifest — `<folder>/connector.json`
|
||||
|
||||
The richer document. Fetched per connector and mapped into Skald's catalog.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"id": "whatsapp",
|
||||
"name": "WhatsApp",
|
||||
"version": 1, // INTEGER build number — the update key (§7)
|
||||
"version_string": "2.0.1", // semver, display only
|
||||
"version_release_date": "2026-07-19", // ISO date, display only
|
||||
"type": "mcp_local",
|
||||
"scope": "user",
|
||||
"auth": { "type": "qr" }, // none | api_key | oauth2 | qr (see §4)
|
||||
"mcp_config": {
|
||||
"command": "node", // interpreter (local) …
|
||||
"args": ["index.js"], // … args[0] MUST name the entry file
|
||||
"transport": "stdio" // stdio (local) | streamable-http (remote)
|
||||
},
|
||||
"docs": [{
|
||||
"lang": "en",
|
||||
"description": "Human blurb shown in the UI.",
|
||||
"llm_short_description": "One line the model reads to decide whether to use this connector."
|
||||
}],
|
||||
"env": [], // form fields the user fills (see §4b)
|
||||
"tools": [ // OPTIONAL — friendly UI names per tool (§2a)
|
||||
{ "name": "send_message", "display_name": "Send Message" }
|
||||
],
|
||||
"homepage": "https://…",
|
||||
"icon_small": "icon_sm.svg", // relative to the folder here
|
||||
"icon_large": "icon_lg.svg",
|
||||
"tags": ["messaging", "mcp", "local", "whatsapp", "qr"]
|
||||
}
|
||||
```
|
||||
|
||||
**`mcp_config.args[0]` is load-bearing for a local connector:** it is how Skald
|
||||
learns which file to run. At activation Skald rewrites it to the file's path
|
||||
inside the user's container (`/root/.skald/mcp/<name>/<entry>`), so keep it a
|
||||
plain relative filename (`index.js`, `server.py`, `pkg/server.py`).
|
||||
|
||||
---
|
||||
|
||||
## 2. Server contract (MCP over stdio)
|
||||
|
||||
A **local** connector is a program speaking JSON-RPC 2.0 over stdin/stdout. It
|
||||
MUST handle:
|
||||
|
||||
- `initialize` → `{ protocolVersion, capabilities: { tools: {} }, serverInfo }`
|
||||
- `notifications/initialized` → no response
|
||||
- `tools/list` → `{ tools: [ { name, description, inputSchema } ] }`
|
||||
- `tools/call` → `{ content: [ { type: "text", text } ], isError? }`
|
||||
|
||||
**stdout is reserved for JSON-RPC only.** Send all logs/diagnostics to **stderr**.
|
||||
Anything a library prints to stdout (a logger, a banner) corrupts the protocol —
|
||||
silence it (e.g. Baileys/pino → a silent logger; Python → `print(…, file=sys.stderr)`).
|
||||
|
||||
A **remote** connector is an HTTP MCP endpoint (`mcp_config.url` +
|
||||
`transport: "streamable-http"`); no code runs on the box.
|
||||
|
||||
### 2a. Friendly tool names (`tools[]`) — optional
|
||||
|
||||
Raw MCP tool names are ugly in the chat UI (`search_files`, `send_message`). The
|
||||
optional top-level `tools[]` block gives each one a human title shown as the tool
|
||||
card's heading:
|
||||
|
||||
```jsonc
|
||||
"tools": [
|
||||
{ "name": "send_message", "display_name": "Send Message" },
|
||||
{ "name": "list_chats", "display_name": "List Chats" },
|
||||
{ "name": "download_media", "display_name": "Download Media" }
|
||||
]
|
||||
```
|
||||
|
||||
- `name` — the **raw** tool name exactly as your server returns it from `tools/list`.
|
||||
- `display_name` — the friendly card title (English only; not internationalized).
|
||||
|
||||
**Resolution order** for a tool's card title is **`tools[].display_name` → the MCP
|
||||
`title` field → a prettified raw name**. So you have two ways to set a friendly
|
||||
name, and can skip `tools[]` entirely:
|
||||
|
||||
1. **This block** — the authoritative override, curated in the manifest.
|
||||
2. **The MCP `title` field** — if your `tools/list` entries already carry a
|
||||
`title` (MCP 2025-06-18+), Skald uses it automatically; no manifest change
|
||||
needed. `tools[]` wins if both are present.
|
||||
3. If neither is set, Skald title-cases the raw name (`send_message` → "Send
|
||||
Message").
|
||||
|
||||
**Icons are per connector, not per tool.** Every tool of a connector shows that
|
||||
connector's own `icon_small`; there is no per-tool icon field. Only list a tool in
|
||||
`tools[]` when its prettified name isn't good enough — partial lists are fine
|
||||
(unlisted tools fall through to steps 2–3).
|
||||
|
||||
---
|
||||
|
||||
## 3. Placement & risk vocabulary (what the words mean)
|
||||
|
||||
| Manifest | Meaning |
|
||||
| --- | --- |
|
||||
| `scope: "user"` | runs **once per user**, inside their container. Personal creds. |
|
||||
| `scope: "global"` | runs **once for the household**, on the host. Shared, stateless. Admin enables it with a key. |
|
||||
| `type: "mcp_local"` | ships code that will **execute on the box** — installing needs the admin `mcp.register_local_script` capability (RCE-bearing act, §14). |
|
||||
| `type: "mcp_remote"` | just an HTTP URL; no local code. |
|
||||
|
||||
Pick the narrowest: a personal messaging/email/calendar connector is
|
||||
`scope: "user"`; a shared search API is `scope: "global"`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Authentication (`auth.type`)
|
||||
|
||||
| `auth.type` | Flow | Ships |
|
||||
| --- | --- | --- |
|
||||
| `none` | nothing to sign in | — |
|
||||
| `api_key` | user pastes a key/secret into a form | an `env[]` schema (§4b) |
|
||||
| `oauth2` | browser consent → paste code back | `auth.provider` + `auth.scopes` + `auth.deliver` (§4c) |
|
||||
| `qr` | server shows a QR, user scans with a phone | a `login_status` tool (§4d) |
|
||||
|
||||
### 4b. `api_key` — the `env[]` schema
|
||||
|
||||
Each entry drives one form field **and** is injected as an env var / URL token to
|
||||
the server:
|
||||
|
||||
```jsonc
|
||||
"env": [{
|
||||
"name": "tavilyApiKey",
|
||||
"label": "Tavily API key",
|
||||
"description": "Create one at https://app.tavily.com.",
|
||||
"required": true,
|
||||
"secret": true, // rendered masked, stored encrypted
|
||||
"example": "tvly-xxxxxxxx"
|
||||
}]
|
||||
```
|
||||
|
||||
The server reads each value from `process.env.<name>` (or `os.environ`). For a
|
||||
**remote** connector that wants the key in the URL, use a placeholder:
|
||||
`"url": "https://mcp.example.com/?key={SECRET:tavilyApiKey}"`.
|
||||
|
||||
### 4c. `oauth2` — provider consent
|
||||
|
||||
```jsonc
|
||||
"auth": {
|
||||
"type": "oauth2",
|
||||
"provider": "google", // slug into the admin's sign-in providers
|
||||
"scopes": ["https://www.googleapis.com/auth/gmail.modify"],
|
||||
"deliver": { "as": "env", "format": "google_authorized_user", "env": "GMAIL_CREDS_JSON" }
|
||||
}
|
||||
```
|
||||
|
||||
The manifest names **only** the provider slug, scopes, and how the obtained token
|
||||
is delivered — never client secrets or endpoint URLs (those are admin-entered,
|
||||
kept off the public feed). Skald handles PKCE + code exchange and injects the
|
||||
credential as the named env var. `format`: `google_authorized_user` (Google) or
|
||||
`refresh_token`. Today only `as: "env"` is wired.
|
||||
|
||||
### 4d. `qr` / interactive device login — the generic contract
|
||||
|
||||
For a connector whose credential is produced by **scanning/pairing** (WhatsApp
|
||||
today), there is no code to paste. The rule:
|
||||
|
||||
> **Expose one extra tool, `login_status`, returning a JSON object** (as the
|
||||
> `text` of a normal text result). Skald calls it directly (never the agent) and a
|
||||
> login panel polls it.
|
||||
|
||||
```jsonc
|
||||
// login_status result text (a JSON string):
|
||||
{
|
||||
"state": "connecting" | "need_scan" | "ready" | "logged_out",
|
||||
"qr": "data:image/png;base64,…", // present ONLY while state == need_scan
|
||||
"message": "human-readable line"
|
||||
}
|
||||
```
|
||||
|
||||
- `activate` on a `qr` connector inserts a **pending** row and **starts the
|
||||
server** (so it can produce the QR), then hands off to the login panel.
|
||||
- The panel polls `POST /api/mcp/login/status`; when `state == "ready"` the
|
||||
connector is marked ready and starts automatically on later logins.
|
||||
- Also expose a `logout` tool (clears the session, forces a fresh QR) — the panel
|
||||
calls it via `POST /api/mcp/login/reset` to re-link a different phone.
|
||||
- The **credential is the on-disk session**, not a token. Persist it **inside the
|
||||
connector's own directory** (e.g. `./auth/` next to the entry file). That folder
|
||||
lives under the bind-mounted home, so it survives container recreates and
|
||||
connector updates. Never store it under a shared/global path.
|
||||
|
||||
Skald resolves `auth.type: "qr"` the same way whether it appears in the index
|
||||
entry or the manifest.
|
||||
|
||||
---
|
||||
|
||||
## 5. Dependencies (node & python) — how they get installed
|
||||
|
||||
**Do not ship `node_modules/` or vendored wheels.** Declare deps as a standard
|
||||
manifest **file** and Skald installs them inside the container:
|
||||
|
||||
- **node:** ship a `package.json` with a `dependencies` map. Skald runs
|
||||
`npm ci --omit=dev` (falling back to `npm install --omit=dev`) in the connector
|
||||
dir. `node_modules/` resolves automatically beside the entry file.
|
||||
- **python:** ship a `requirements.txt`. Skald installs it with
|
||||
`pip install --target .pydeps` and puts `.pydeps` on the server's `PYTHONPATH`.
|
||||
|
||||
This runs at activation **and** on every startup, guarded by a **content hash** of
|
||||
the connector's source files:
|
||||
|
||||
- first activation / a brand-new container → full install,
|
||||
- a connector **update** (any shipped file changed) → re-copy + re-install,
|
||||
- unchanged → skipped in microseconds.
|
||||
|
||||
So you never write install steps into the manifest — just ship the dep file, list
|
||||
it in the index with its SHA-256, and set `requires: ["NODE"]` / `["PYTHON"]` as a
|
||||
human hint. Pin versions in `package.json` / `requirements.txt` for reproducible
|
||||
installs. Keep the dep tree lean (containers are slim; avoid native-heavy
|
||||
packages where a pure alternative exists — e.g. Baileys instead of a browser).
|
||||
|
||||
---
|
||||
|
||||
## 6. Verify-before-save (optional but recommended)
|
||||
|
||||
Ship a `verify.py` / verify snippet and reference it:
|
||||
|
||||
```jsonc
|
||||
"verify": { "command": "python3 verify.py", "timeout_secs": 15 }
|
||||
```
|
||||
|
||||
It runs with the collected env/secret injected and must print **one JSON object**
|
||||
on stdout: `{"ok": bool, "message": string, "details"?: object}`, exit 0 on
|
||||
success. Used for `api_key`/`none` connectors to test creds before activating.
|
||||
(A `qr` connector needs no verify — its `login_status` is the live check.)
|
||||
|
||||
---
|
||||
|
||||
## 7. Versioning & updates
|
||||
|
||||
Three fields, in **both** the index entry and the `connector.json`, kept identical:
|
||||
|
||||
| field | type | role |
|
||||
| --- | --- | --- |
|
||||
| `version` | **integer** | monotonic build number, **per connector** — the machine comparison key |
|
||||
| `version_string` | string (semver) | display only |
|
||||
| `version_release_date` | ISO date `YYYY-MM-DD` | display only |
|
||||
|
||||
- `version` is a **number, not a string** (`1`, not `"1"` or `"2.0.1"`). Start at
|
||||
`1` for the first release under this scheme; **`+1` on every change** to any
|
||||
shipped file **or to any manifest metadata** (description, icons, `version_string`).
|
||||
Never reuse or decrement.
|
||||
- Skald stores the installed `version` and compares it to the feed's: a strictly
|
||||
greater feed `version` shows **"update available"** in the marketplace, and the
|
||||
Install button becomes **Update**. Clicking it re-downloads the files and rewrites
|
||||
the catalog row.
|
||||
- **The integer is the *only* "is there an update?" signal** — it is compared
|
||||
strictly (`feed > installed`). `version_string` (semver), icons and
|
||||
`llm_short_description` are **never** compared, so a change to any of them that
|
||||
does not also bump the integer is **invisible**: no "update available" badge
|
||||
appears. This is the common trap — a "content-only" edit (e.g. a better
|
||||
`llm_short_description`) that forgets the integer.
|
||||
- **Two propagation paths, do not conflate them:**
|
||||
- *Per-user code + deps* (the scripts, `package.json`/`requirements.txt`) reconcile
|
||||
on a **content-hash** of the source files (§5), so new code lands at each user's
|
||||
next login even without a reinstall.
|
||||
- *Catalog metadata* (`llm_short_description` → the model's prompt, icons, friendly
|
||||
name) is **not** in that hash — it lives in the catalog row and is rewritten only
|
||||
by an explicit **reinstall/Update**. On reinstall Skald re-pulls the current feed
|
||||
(never the browse cache) and pushes the new description live: enabled global
|
||||
servers restart with it, and every logged-in user who activated the connector has
|
||||
it restarted with the fresh `llm_short_description` — no re-login needed.
|
||||
- So: to ship a new `llm_short_description`, **bump the integer** (so the admin sees
|
||||
"update available") and the admin clicks **Update**. Nothing auto-propagates a
|
||||
description change.
|
||||
- `version_string` and `version_release_date` are display metadata only — never
|
||||
compared. (Migration note: replace any legacy string `"version": "2.0.1"` with
|
||||
the integer `version` + `version_string`.)
|
||||
|
||||
---
|
||||
|
||||
## 8. Checklist for a new connector
|
||||
|
||||
1. Folder `myconn/` with: entry file, `connector.json`, deps file
|
||||
(`package.json`/`requirements.txt`), `icon_sm.svg`, `icon_lg.svg`,
|
||||
optional `verify.*`.
|
||||
2. Server speaks MCP over stdio (§2); **stdout = JSON-RPC only**.
|
||||
3. `mcp_config.args[0]` names the entry file.
|
||||
4. Correct `type` + `scope` (§3) and `auth.type` (§4).
|
||||
5. For `qr`: implement `login_status` (+ `logout`), persist the session under the
|
||||
connector dir (§4d).
|
||||
6. Deps declared as a file, **not** vendored (§5).
|
||||
7. Add the entry to `connectors.json` with a correct `sha256` for **every** file.
|
||||
8. Bump `version`.
|
||||
```
|
||||
@@ -161,7 +161,7 @@ Automatic build on NiPoGi with Gitea Actions (native runner v2.1.0):
|
||||
|
||||
### Technical notes
|
||||
|
||||
- `scripts/` in `.gitignore` — CI scripts moved to `ci/` (tracked by git)
|
||||
- `scripts/` removed — CI scripts live in `ci/` (tracked by git); the legacy MCP servers it held are superseded by marketplace connectors
|
||||
- Build without `whisper-local` on Linux (`--no-default-features`)
|
||||
- `aarch64-linux-gnu-strip` for ARM64 binaries
|
||||
- `actions/checkout@v4` works (native runner has Node.js)
|
||||
|
||||
@@ -86,7 +86,6 @@ When working on **Skald itself** (the project you are in), follow these addition
|
||||
- Agent prompts: `agents/`
|
||||
- Extracted crates: `crates/`
|
||||
- Web app (Lit components): `web/`
|
||||
- Python MCP scripts: `scripts/`
|
||||
- Config: `config.yml` (copy from `default.config.yaml`)
|
||||
- Docs: `docs/`
|
||||
- Database: `database.db` (unless overridden in `config.yml`)
|
||||
|
||||
@@ -116,7 +116,6 @@ When working on **Skald itself** (the project you are in), follow these addition
|
||||
- Agent prompts: `agents/`
|
||||
- Extracted crates: `crates/`
|
||||
- Web app (Lit components): `web/`
|
||||
- Python MCP scripts: `scripts/`
|
||||
- Config: `config.yml`
|
||||
- Docs: `docs/`
|
||||
- Database: `database.db`
|
||||
|
||||
@@ -70,7 +70,7 @@ impl VerifyReport {
|
||||
/// runs inside the user's container via `docker exec`.
|
||||
pub enum VerifyTarget<'a> {
|
||||
/// Run on the Skald host process. `workdir` is an absolute host path
|
||||
/// (typically `<data_root>/scripts/<id>/`).
|
||||
/// (typically `connectors/<folder>/`).
|
||||
Host { workdir: &'a Path },
|
||||
/// Run inside the user's sandbox container. `workdir` is an absolute path
|
||||
/// *inside* the container (e.g. `/root/.skald/mcp/<name>`).
|
||||
|
||||
+4
-4
@@ -192,7 +192,7 @@ check_optional_deps() {
|
||||
if command -v python3 >/dev/null 2>&1; then
|
||||
info "✔ Python 3 found ($(python3 --version 2>&1 | head -1))"
|
||||
else
|
||||
warn "Python 3 not found — Python MCP servers (Gmail, GCal, GMaps, ...) will not work."
|
||||
warn "Python 3 not found — the TTS plugins and host-run connectors will not work."
|
||||
echo " Install it from https://www.python.org/downloads/"
|
||||
echo ""
|
||||
fi
|
||||
@@ -327,13 +327,13 @@ if [ ! -f "$VENV_DIR/bin/python3" ] || ! "$VENV_DIR/bin/python3" -m pip --versio
|
||||
if command -v uv >/dev/null 2>&1; then
|
||||
uv venv --seed "$VENV_DIR" && uv pip install -r "$REQUIREMENTS" \
|
||||
&& info "✔ Python venv ready (uv)" \
|
||||
|| warn "Python venv setup failed — Python MCP servers will be unavailable."
|
||||
|| warn "Python venv setup failed — the TTS plugins and host-run connectors will be unavailable."
|
||||
elif command -v python3 >/dev/null 2>&1; then
|
||||
python3 -m venv "$VENV_DIR" && "$VENV_DIR/bin/pip" install -r "$REQUIREMENTS" \
|
||||
&& info "✔ Python venv ready (pip)" \
|
||||
|| warn "Python venv setup failed — Python MCP servers will be unavailable."
|
||||
|| warn "Python venv setup failed — the TTS plugins and host-run connectors will be unavailable."
|
||||
else
|
||||
warn "python3 not found — Python MCP servers will be unavailable."
|
||||
warn "python3 not found — the TTS plugins and host-run connectors will be unavailable."
|
||||
fi
|
||||
else
|
||||
info "✔ Python venv already exists"
|
||||
|
||||
+4
-4
@@ -199,7 +199,7 @@ check_optional_deps() {
|
||||
if command -v python3 >/dev/null 2>&1; then
|
||||
info "✔ Python 3 found ($(python3 --version 2>&1 | head -1))"
|
||||
else
|
||||
warn "Python 3 not found — Python MCP servers (Gmail, GCal, GMaps, ...) will not work."
|
||||
warn "Python 3 not found — the TTS plugins and host-run connectors will not work."
|
||||
echo " Install it from https://www.python.org/downloads/"
|
||||
echo ""
|
||||
fi
|
||||
@@ -332,13 +332,13 @@ if [ ! -f "$VENV_DIR/bin/python3" ] || ! "$VENV_DIR/bin/python3" -m pip --versio
|
||||
if command -v uv >/dev/null 2>&1; then
|
||||
uv venv --seed "$VENV_DIR" && uv pip install -r "$REQUIREMENTS" \
|
||||
&& info "✔ Python venv ready (uv)" \
|
||||
|| warn "Python venv setup failed — Python MCP servers will be unavailable."
|
||||
|| warn "Python venv setup failed — the TTS plugins and host-run connectors will be unavailable."
|
||||
elif command -v python3 >/dev/null 2>&1; then
|
||||
python3 -m venv "$VENV_DIR" && "$VENV_DIR/bin/pip" install -r "$REQUIREMENTS" \
|
||||
&& info "✔ Python venv ready (pip)" \
|
||||
|| warn "Python venv setup failed — Python MCP servers will be unavailable."
|
||||
|| warn "Python venv setup failed — the TTS plugins and host-run connectors will be unavailable."
|
||||
else
|
||||
warn "python3 not found — Python MCP servers will be unavailable."
|
||||
warn "python3 not found — the TTS plugins and host-run connectors will be unavailable."
|
||||
fi
|
||||
else
|
||||
info "✔ Python venv already exists"
|
||||
|
||||
+17
-16
@@ -1,25 +1,26 @@
|
||||
google-auth
|
||||
google-auth-oauthlib
|
||||
google-api-python-client
|
||||
googlemaps
|
||||
requests
|
||||
httpx
|
||||
|
||||
# SSH MCP server (scripts/ssh_mcp_server.py)
|
||||
paramiko>=3.4
|
||||
|
||||
# Google Trends MCP server (scripts/google_trends_mcp.py)
|
||||
mcp
|
||||
trendspyg>=0.7.0
|
||||
# Host Python dependencies.
|
||||
#
|
||||
# These exist for the two TTS plugins ONLY. Both spawn a bare `python3` (resolved
|
||||
# from PATH, which run.sh points at .venv/bin) to run an embedded server script,
|
||||
# so their imports must be satisfied here — unlike MCP connectors, they have no
|
||||
# dependency reconciler of their own.
|
||||
#
|
||||
# MCP connectors do NOT belong in this file: one ships its own requirements.txt /
|
||||
# package.json, and `mcp::install::ensure_installed` installs it into `.pydeps` /
|
||||
# `node_modules` inside the user's container (or beside the connector on the host
|
||||
# for a global one). Adding a connector's deps here would install them on every
|
||||
# box for a connector nobody activated.
|
||||
|
||||
# Kokoro ONNX TTS (plugin-tts-kokoro)
|
||||
kokoro-onnx
|
||||
soundfile
|
||||
|
||||
# Orpheus TTS 3B (plugin-tts-orpheus-3b) — heavy deps moved to requirements-optional.txt
|
||||
# See requirements-optional.txt if you need the Orpheus TTS plugin.
|
||||
# Orpheus TTS 3B (plugin-tts-orpheus-3b) — the light half; the GPU/ML deps
|
||||
# (torch, transformers, snac, …) live in requirements-optional.txt
|
||||
scipy
|
||||
|
||||
# Shared by both TTS servers
|
||||
fastapi
|
||||
uvicorn
|
||||
scipy
|
||||
numpy
|
||||
pydantic
|
||||
|
||||
+2
-2
@@ -20,12 +20,12 @@ if [ -f "$REQUIREMENTS" ] && { [ ! -f "$VENV_DIR/bin/python3" ] || ! "$VENV_DIR/
|
||||
echo "[run-docker.sh] Setting up Python venv with uv …"
|
||||
uv venv --seed "$VENV_DIR" && uv pip install -r "$REQUIREMENTS" \
|
||||
&& echo "[run-docker.sh] Python venv ready." \
|
||||
|| echo "[run-docker.sh] Warning: Python venv setup failed — Python MCP servers will be unavailable."
|
||||
|| echo "[run-docker.sh] Warning: Python venv setup failed — the TTS plugins and host-run connectors will be unavailable."
|
||||
elif command -v python3 >/dev/null 2>&1; then
|
||||
echo "[run-docker.sh] Setting up Python venv …"
|
||||
python3 -m venv "$VENV_DIR" && "$VENV_DIR/bin/pip" install -r "$REQUIREMENTS" \
|
||||
&& echo "[run-docker.sh] Python venv ready." \
|
||||
|| echo "[run-docker.sh] Warning: Python venv setup failed — Python MCP servers will be unavailable."
|
||||
|| echo "[run-docker.sh] Warning: Python venv setup failed — the TTS plugins and host-run connectors will be unavailable."
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
@@ -27,7 +27,7 @@ if not exist "%VENV_DIR%\Scripts\python3.exe" (
|
||||
call python3 -m venv "%VENV_DIR%" && call "%VENV_DIR%\Scripts\pip" install -r "%REQUIREMENTS%"
|
||||
if !ERRORLEVEL! equ 0 ( echo [run.bat] Python venv ready. ) else ( echo [run.bat] Warning: Python venv setup failed )
|
||||
) else (
|
||||
echo [run.bat] Warning: python3 not found -- Python MCP servers will be unavailable.
|
||||
echo [run.bat] Warning: python3 not found -- the TTS plugins and host-run connectors will be unavailable.
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
@@ -46,8 +46,10 @@ fi
|
||||
|
||||
# ── Python venv setup (optional) ─────────────────────────────────────────────
|
||||
# Creates .venv/ and installs requirements.txt if Python is available.
|
||||
# If Python is not installed, the app starts normally but Python-based MCP
|
||||
# servers (e.g. Gmail, Google Calendar) will fail to connect.
|
||||
# If Python is not installed, the app starts normally but the TTS plugins (which
|
||||
# spawn `python3` directly) will fail to start, and a host-run global connector
|
||||
# will have no interpreter to install its own deps with. Per-user connectors are
|
||||
# unaffected: they run inside the user's container, which ships its own Python.
|
||||
VENV_DIR=".venv"
|
||||
REQUIREMENTS="requirements.txt"
|
||||
|
||||
@@ -60,14 +62,14 @@ if [ ! -f "$VENV_DIR/bin/python3" ] || ! "$VENV_DIR/bin/python3" -m pip --versio
|
||||
echo "[run.sh] Setting up Python venv with uv …"
|
||||
uv venv --seed "$VENV_DIR" && uv pip install -r "$REQUIREMENTS" \
|
||||
&& echo "[run.sh] Python venv ready." \
|
||||
|| echo "[run.sh] Warning: Python venv setup failed — Python MCP servers will be unavailable."
|
||||
|| echo "[run.sh] Warning: Python venv setup failed — the TTS plugins and host-run connectors will be unavailable."
|
||||
elif command -v python3 >/dev/null 2>&1; then
|
||||
echo "[run.sh] Setting up Python venv …"
|
||||
python3 -m venv "$VENV_DIR" && "$VENV_DIR/bin/pip" install -r "$REQUIREMENTS" \
|
||||
&& echo "[run.sh] Python venv ready." \
|
||||
|| echo "[run.sh] Warning: Python venv setup failed — Python MCP servers will be unavailable."
|
||||
|| echo "[run.sh] Warning: Python venv setup failed — the TTS plugins and host-run connectors will be unavailable."
|
||||
else
|
||||
echo "[run.sh] Warning: python3 not found — Python MCP servers will be unavailable."
|
||||
echo "[run.sh] Warning: python3 not found — the TTS plugins and host-run connectors will be unavailable."
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
@@ -269,13 +269,13 @@ main() {
|
||||
if command -v uv >/dev/null 2>&1; then
|
||||
uv venv --seed "$VENV_DIR" && uv pip install -r "$REQUIREMENTS" \
|
||||
&& info "✔ Python venv ready (uv)" \
|
||||
|| warn "Python venv setup failed — Python MCP servers will be unavailable."
|
||||
|| warn "Python venv setup failed — the TTS plugins and host-run connectors will be unavailable."
|
||||
elif command -v python3 >/dev/null 2>&1; then
|
||||
python3 -m venv "$VENV_DIR" && "$VENV_DIR/bin/pip" install -r "$REQUIREMENTS" \
|
||||
&& info "✔ Python venv ready (pip)" \
|
||||
|| warn "Python venv setup failed — Python MCP servers will be unavailable."
|
||||
|| warn "Python venv setup failed — the TTS plugins and host-run connectors will be unavailable."
|
||||
else
|
||||
warn "python3 not found — Python MCP servers will be unavailable."
|
||||
warn "python3 not found — the TTS plugins and host-run connectors will be unavailable."
|
||||
fi
|
||||
|
||||
# ── Restart ────────────────────────────────────────────────────────────────
|
||||
|
||||
Reference in New Issue
Block a user