From 8bcf09a67eacc2de62d68f7b3c0adc8e8607d8ec Mon Sep 17 00:00:00 2001 From: xavix-yo Date: Tue, 28 Jul 2026 22:19:01 +0100 Subject: [PATCH] chore: delete scripts/ and cut requirements.txt down to its real consumers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit scripts/ held the pre-marketplace MCP servers (gmail, gcal, gmaps, ssh, weather, google_trends, whatsapp, serpapi_flights). Nothing referenced them any more: connectors are admin-curated and installed into connectors/ from the marketplace, and ci/package.sh never shipped scripts/ in the first place — so on every installed box requirements.txt was pulling google-auth, googlemaps, paramiko, trendspyg and friends for files that did not exist there. requirements.txt now states what it is actually for: the two TTS plugins, which spawn a bare `python3` on an embedded server script and so have no dependency reconciler of their own. A connector's deps stay with the connector — `ensure_installed` puts them in .pydeps/node_modules inside the user's container, `ensure_installed_host` beside the files for a global one. The venv itself stays load-bearing for those two plugins and for the host pip that installs a global connector's deps, so the run/install/update scripts keep creating it — but their "Python MCP servers will be unavailable" warning was naming the one thing that no longer depends on it, and now says what really breaks. CONNECTOR_MANIFEST_GUIDE.md moves to the repo root: it was the one thing in scripts/ still referenced (CLAUDE.md), and being under a gitignored directory it had never been committed at all. Co-Authored-By: Claude Opus 5 --- CLAUDE.md | 10 +- CONNECTOR_MANIFEST_GUIDE.md | 348 ++++++++++++++++++++++++++++ SKALD.md | 2 +- agents/software-architect/AGENT.md | 1 - agents/software-engineer/AGENT.md | 1 - crates/skald-core/src/mcp/verify.rs | 2 +- install-nightly.sh | 8 +- install.sh | 8 +- requirements.txt | 33 +-- run-docker.sh | 4 +- run.bat | 2 +- run.sh | 12 +- update.sh | 6 +- 13 files changed, 394 insertions(+), 43 deletions(-) create mode 100644 CONNECTOR_MANIFEST_GUIDE.md diff --git a/CLAUDE.md b/CLAUDE.md index 9d378e7..4efe978 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -193,7 +193,7 @@ MCP servers are surfaced to users as **"Connectors"** (UI naming; `mcp`/schema s **Endpoints** (`src/frontend/api/mcp.rs`, mounted in `api/mod.rs`) — admin: `/mcp/catalog` (GET/POST/DELETE), `/mcp/global` (list/enable/delete + `/{id}/access` GET/PUT), `/mcp/providers` (GET/POST + DELETE `/{name}` — OAuth provider creds, secret never returned to the browser). User: `/mcp/available`, `/mcp/activate`, `/mcp/activated` (+ DELETE `/{id}` to deactivate), `/mcp/oauth/start` + `/mcp/oauth/complete` (the §15 OAuth login), `/mcp/login/status` + `/mcp/login/reset` (the §15 QR/device login — see below). `connectors.js` (``) is the **single** Connectors surface — a row list, one row per connector (there is no separate catalog page): the user view (activate/deactivate + granted globals) always, plus the admin affordances when `role_id === 'admin'` — the **Add connector** dropdown (from the Marketplace, or manually via the `#connectors/new` sub-page), per-row removal from the catalog, and the **Sign-in providers** modal. The Marketplace stays its own page (`marketplace.js`), reached from that dropdown and linking back to `#connectors`. `connector-detail.js` (``) is a connector's own page and hosts both the OAuth login panel and the QR login panel. -**Dependency reconciler (`mcp::install::ensure_installed`).** Copying a local-script connector's files into a container never installed its deps. `ensure_installed` closes that: a **content-hash reconciler** keyed on the connector's *source* files (not a version string) that, when the hash changed, re-copies the files and installs deps inside the container — `npm ci --omit=dev` (node, from `package.json`) and/or `pip install --target .pydeps` (python, from `requirements.txt`, put on the server's `PYTHONPATH` by `user_row_spec`). Runs at activation **and** on every per-user startup path (`UserContext` build, remount) via `mcp::prepare_local_connector`, so a fresh container installs from scratch, an updated connector re-installs, and an unchanged one is a hash-match no-op. Deps are therefore **never vendored** — connectors ship `package.json`/`requirements.txt`, not `node_modules/`. Authoring contract for connectors lives in `scripts/CONNECTOR_MANIFEST_GUIDE.md`. +**Dependency reconciler (`mcp::install::ensure_installed`).** Copying a local-script connector's files into a container never installed its deps. `ensure_installed` closes that: a **content-hash reconciler** keyed on the connector's *source* files (not a version string) that, when the hash changed, re-copies the files and installs deps inside the container — `npm ci --omit=dev` (node, from `package.json`) and/or `pip install --target .pydeps` (python, from `requirements.txt`, put on the server's `PYTHONPATH` by `user_row_spec`). Runs at activation **and** on every per-user startup path (`UserContext` build, remount) via `mcp::prepare_local_connector`, so a fresh container installs from scratch, an updated connector re-installs, and an unchanged one is a hash-match no-op. Deps are therefore **never vendored** — connectors ship `package.json`/`requirements.txt`, not `node_modules/`. Authoring contract for connectors lives in `CONNECTOR_MANIFEST_GUIDE.md` (repo root). **Connector versioning.** `mcp_catalog` carries `version` (INTEGER — the update-comparison key), `version_string` (semver, display) and `version_release_date` (ISO, display), snapshotted from the feed on install. The marketplace list computes `update_available` = feed `version` > installed `version` (strict) and surfaces it as an "Update" button (`marketplace.js`). The integer is the UI signal; the actual re-install trigger is the reconciler's content-hash. @@ -367,11 +367,13 @@ Copy `default.config.yaml` → `config.yml`. Never commit `config.yml` (contains ## Python environment -All Python scripts (MCP servers, setup scripts) use a local virtualenv at `.venv/` in the project root. +Host-side Python runs from a local virtualenv at `.venv/` in the project root. `run.sh` creates it on first launch (using `uv` if available, otherwise `python3 -m venv`), installs `requirements.txt`, and prepends `.venv/bin` to `PATH` before starting the app, so every child process resolves `python3` to the venv. No manual activation needed. -`run.sh` creates it automatically on first launch (using `uv` if available, otherwise `python3 -m venv`) and installs `requirements.txt`. It then prepends `.venv/bin` to `PATH` before starting the app, so every child process — MCP server launches, `execute_cmd` shell calls — resolves `python3` to the venv automatically. No manual activation needed. **Python is optional**: if neither `uv` nor `python3` is found, the app starts normally and only Python-based MCP servers will be unavailable. +**`requirements.txt` is for the two TTS plugins, and nothing else.** `plugin-tts-kokoro` and `plugin-tts-orpheus-3b` write an embedded server script to disk and spawn a bare `python3` on it — they have no dependency reconciler of their own, so their imports must be satisfied in the venv. The GPU/ML half of Orpheus (torch, transformers, snac, bitsandbytes, huggingface_hub) is split into `requirements-optional.txt`, installed by hand. -To add a Python dependency: add it to `requirements.txt`. It will be installed on the next `./run.sh` invocation if `.venv` does not yet exist — or run `uv pip install -r requirements.txt` manually. +**A connector's deps never go in `requirements.txt`.** A connector ships its own `requirements.txt`/`package.json` and `mcp::install::ensure_installed` installs it into `.pydeps`/`node_modules` — inside the user's container for a per-user connector, beside the connector's files on the host for a global one (`ensure_installed_host`). Putting them in the root file would install them on every box for a connector nobody activated; this is what the file used to do for the since-deleted `scripts/` MCP servers. + +**Python is optional**: with neither `uv` nor `python3` present the app starts normally; the TTS plugins fail to start and a host-run global connector has no interpreter to install its deps with. Per-user connectors are unaffected — they run in the container, which ships its own Python. ## Frontend components (`web/components/`) diff --git a/CONNECTOR_MANIFEST_GUIDE.md b/CONNECTOR_MANIFEST_GUIDE.md new file mode 100644 index 0000000..0dbcdc8 --- /dev/null +++ b/CONNECTOR_MANIFEST_GUIDE.md @@ -0,0 +1,348 @@ +# Skald Connector Authoring Guide + +Instructions for generating a **correct connector** for the Skald marketplace +(`https://connectors.skaldagent.net`). Give this file to the agent that produces +new connectors. + +A connector is a folder served by the marketplace. Skald installs it, verifies +every file against a SHA-256 pinned in the index, then either runs it on the host +(global connector) or copies it into the user's container and runs it there +(per-user connector, blueprint §6/§7). + +--- + +## 1. The two documents + +### 1a. The root index — `connectors.json` + +One array of entries, each pointing at a connector folder. **The index is the +signable root: it is the only place that lists a connector's files and their +SHA-256 digests.** Skald refuses any file whose bytes do not match. + +```jsonc +{ + "version": 1, + "connectors": [ + { + "id": "whatsapp", // unique slug = folder name + "name": "WhatsApp", + "version": 1, // INTEGER build number — the update key (§7) + "version_string": "2.0.1", // semver, display only + "version_release_date": "2026-07-19", // ISO date, display only + "type": "mcp_local", // mcp_local | mcp_remote (see §3) + "scope": "user", // user | global (see §3) + "icon_small": "whatsapp/icon_sm.svg", + "icon_large": "whatsapp/icon_lg.svg", + "user_description": "Send and read WhatsApp messages from your linked account.", + "requires": ["NODE"], // human hint: NODE | PYTHON | OAUTH | API_KEY + "tags": ["messaging", "mcp", "local", "whatsapp", "qr"], + "auth": { "type": "qr" }, // may be repeated here and in the manifest + "folder": "whatsapp", // defaults to id + "files": [ + { "path": "index.js", "sha256": "…", "size": 21258 }, + { "path": "package.json", "sha256": "…", "size": 302 }, + { "path": "connector.json", "sha256": "…", "size": 620 }, + { "path": "icon_sm.svg", "sha256": "…", "size": 306 }, + { "path": "icon_lg.svg", "sha256": "…", "size": 308 } + ] + } + ] +} +``` + +**Rules** + +- `files[].path` is relative to the connector folder. List **every** file the + connector ships (server code, `package.json`/`requirements.txt`, icons, and the + `connector.json` itself). A missing or mismatched digest fails the install. +- Compute `sha256` over the exact bytes served: `sha256sum `. +- Do **not** list `node_modules/` or any generated deps — those are installed on + the box, not shipped (see §5). +- `size` is optional but recommended. + +### 1b. The per-connector manifest — `/connector.json` + +The richer document. Fetched per connector and mapped into Skald's catalog. + +```jsonc +{ + "id": "whatsapp", + "name": "WhatsApp", + "version": 1, // INTEGER build number — the update key (§7) + "version_string": "2.0.1", // semver, display only + "version_release_date": "2026-07-19", // ISO date, display only + "type": "mcp_local", + "scope": "user", + "auth": { "type": "qr" }, // none | api_key | oauth2 | qr (see §4) + "mcp_config": { + "command": "node", // interpreter (local) … + "args": ["index.js"], // … args[0] MUST name the entry file + "transport": "stdio" // stdio (local) | streamable-http (remote) + }, + "docs": [{ + "lang": "en", + "description": "Human blurb shown in the UI.", + "llm_short_description": "One line the model reads to decide whether to use this connector." + }], + "env": [], // form fields the user fills (see §4b) + "tools": [ // OPTIONAL — friendly UI names per tool (§2a) + { "name": "send_message", "display_name": "Send Message" } + ], + "homepage": "https://…", + "icon_small": "icon_sm.svg", // relative to the folder here + "icon_large": "icon_lg.svg", + "tags": ["messaging", "mcp", "local", "whatsapp", "qr"] +} +``` + +**`mcp_config.args[0]` is load-bearing for a local connector:** it is how Skald +learns which file to run. At activation Skald rewrites it to the file's path +inside the user's container (`/root/.skald/mcp//`), so keep it a +plain relative filename (`index.js`, `server.py`, `pkg/server.py`). + +--- + +## 2. Server contract (MCP over stdio) + +A **local** connector is a program speaking JSON-RPC 2.0 over stdin/stdout. It +MUST handle: + +- `initialize` → `{ protocolVersion, capabilities: { tools: {} }, serverInfo }` +- `notifications/initialized` → no response +- `tools/list` → `{ tools: [ { name, description, inputSchema } ] }` +- `tools/call` → `{ content: [ { type: "text", text } ], isError? }` + +**stdout is reserved for JSON-RPC only.** Send all logs/diagnostics to **stderr**. +Anything a library prints to stdout (a logger, a banner) corrupts the protocol — +silence it (e.g. Baileys/pino → a silent logger; Python → `print(…, file=sys.stderr)`). + +A **remote** connector is an HTTP MCP endpoint (`mcp_config.url` + +`transport: "streamable-http"`); no code runs on the box. + +### 2a. Friendly tool names (`tools[]`) — optional + +Raw MCP tool names are ugly in the chat UI (`search_files`, `send_message`). The +optional top-level `tools[]` block gives each one a human title shown as the tool +card's heading: + +```jsonc +"tools": [ + { "name": "send_message", "display_name": "Send Message" }, + { "name": "list_chats", "display_name": "List Chats" }, + { "name": "download_media", "display_name": "Download Media" } +] +``` + +- `name` — the **raw** tool name exactly as your server returns it from `tools/list`. +- `display_name` — the friendly card title (English only; not internationalized). + +**Resolution order** for a tool's card title is **`tools[].display_name` → the MCP +`title` field → a prettified raw name**. So you have two ways to set a friendly +name, and can skip `tools[]` entirely: + +1. **This block** — the authoritative override, curated in the manifest. +2. **The MCP `title` field** — if your `tools/list` entries already carry a + `title` (MCP 2025-06-18+), Skald uses it automatically; no manifest change + needed. `tools[]` wins if both are present. +3. If neither is set, Skald title-cases the raw name (`send_message` → "Send + Message"). + +**Icons are per connector, not per tool.** Every tool of a connector shows that +connector's own `icon_small`; there is no per-tool icon field. Only list a tool in +`tools[]` when its prettified name isn't good enough — partial lists are fine +(unlisted tools fall through to steps 2–3). + +--- + +## 3. Placement & risk vocabulary (what the words mean) + +| Manifest | Meaning | +| --- | --- | +| `scope: "user"` | runs **once per user**, inside their container. Personal creds. | +| `scope: "global"` | runs **once for the household**, on the host. Shared, stateless. Admin enables it with a key. | +| `type: "mcp_local"` | ships code that will **execute on the box** — installing needs the admin `mcp.register_local_script` capability (RCE-bearing act, §14). | +| `type: "mcp_remote"` | just an HTTP URL; no local code. | + +Pick the narrowest: a personal messaging/email/calendar connector is +`scope: "user"`; a shared search API is `scope: "global"`. + +--- + +## 4. Authentication (`auth.type`) + +| `auth.type` | Flow | Ships | +| --- | --- | --- | +| `none` | nothing to sign in | — | +| `api_key` | user pastes a key/secret into a form | an `env[]` schema (§4b) | +| `oauth2` | browser consent → paste code back | `auth.provider` + `auth.scopes` + `auth.deliver` (§4c) | +| `qr` | server shows a QR, user scans with a phone | a `login_status` tool (§4d) | + +### 4b. `api_key` — the `env[]` schema + +Each entry drives one form field **and** is injected as an env var / URL token to +the server: + +```jsonc +"env": [{ + "name": "tavilyApiKey", + "label": "Tavily API key", + "description": "Create one at https://app.tavily.com.", + "required": true, + "secret": true, // rendered masked, stored encrypted + "example": "tvly-xxxxxxxx" +}] +``` + +The server reads each value from `process.env.` (or `os.environ`). For a +**remote** connector that wants the key in the URL, use a placeholder: +`"url": "https://mcp.example.com/?key={SECRET:tavilyApiKey}"`. + +### 4c. `oauth2` — provider consent + +```jsonc +"auth": { + "type": "oauth2", + "provider": "google", // slug into the admin's sign-in providers + "scopes": ["https://www.googleapis.com/auth/gmail.modify"], + "deliver": { "as": "env", "format": "google_authorized_user", "env": "GMAIL_CREDS_JSON" } +} +``` + +The manifest names **only** the provider slug, scopes, and how the obtained token +is delivered — never client secrets or endpoint URLs (those are admin-entered, +kept off the public feed). Skald handles PKCE + code exchange and injects the +credential as the named env var. `format`: `google_authorized_user` (Google) or +`refresh_token`. Today only `as: "env"` is wired. + +### 4d. `qr` / interactive device login — the generic contract + +For a connector whose credential is produced by **scanning/pairing** (WhatsApp +today), there is no code to paste. The rule: + +> **Expose one extra tool, `login_status`, returning a JSON object** (as the +> `text` of a normal text result). Skald calls it directly (never the agent) and a +> login panel polls it. + +```jsonc +// login_status result text (a JSON string): +{ + "state": "connecting" | "need_scan" | "ready" | "logged_out", + "qr": "data:image/png;base64,…", // present ONLY while state == need_scan + "message": "human-readable line" +} +``` + +- `activate` on a `qr` connector inserts a **pending** row and **starts the + server** (so it can produce the QR), then hands off to the login panel. +- The panel polls `POST /api/mcp/login/status`; when `state == "ready"` the + connector is marked ready and starts automatically on later logins. +- Also expose a `logout` tool (clears the session, forces a fresh QR) — the panel + calls it via `POST /api/mcp/login/reset` to re-link a different phone. +- The **credential is the on-disk session**, not a token. Persist it **inside the + connector's own directory** (e.g. `./auth/` next to the entry file). That folder + lives under the bind-mounted home, so it survives container recreates and + connector updates. Never store it under a shared/global path. + +Skald resolves `auth.type: "qr"` the same way whether it appears in the index +entry or the manifest. + +--- + +## 5. Dependencies (node & python) — how they get installed + +**Do not ship `node_modules/` or vendored wheels.** Declare deps as a standard +manifest **file** and Skald installs them inside the container: + +- **node:** ship a `package.json` with a `dependencies` map. Skald runs + `npm ci --omit=dev` (falling back to `npm install --omit=dev`) in the connector + dir. `node_modules/` resolves automatically beside the entry file. +- **python:** ship a `requirements.txt`. Skald installs it with + `pip install --target .pydeps` and puts `.pydeps` on the server's `PYTHONPATH`. + +This runs at activation **and** on every startup, guarded by a **content hash** of +the connector's source files: + +- first activation / a brand-new container → full install, +- a connector **update** (any shipped file changed) → re-copy + re-install, +- unchanged → skipped in microseconds. + +So you never write install steps into the manifest — just ship the dep file, list +it in the index with its SHA-256, and set `requires: ["NODE"]` / `["PYTHON"]` as a +human hint. Pin versions in `package.json` / `requirements.txt` for reproducible +installs. Keep the dep tree lean (containers are slim; avoid native-heavy +packages where a pure alternative exists — e.g. Baileys instead of a browser). + +--- + +## 6. Verify-before-save (optional but recommended) + +Ship a `verify.py` / verify snippet and reference it: + +```jsonc +"verify": { "command": "python3 verify.py", "timeout_secs": 15 } +``` + +It runs with the collected env/secret injected and must print **one JSON object** +on stdout: `{"ok": bool, "message": string, "details"?: object}`, exit 0 on +success. Used for `api_key`/`none` connectors to test creds before activating. +(A `qr` connector needs no verify — its `login_status` is the live check.) + +--- + +## 7. Versioning & updates + +Three fields, in **both** the index entry and the `connector.json`, kept identical: + +| field | type | role | +| --- | --- | --- | +| `version` | **integer** | monotonic build number, **per connector** — the machine comparison key | +| `version_string` | string (semver) | display only | +| `version_release_date` | ISO date `YYYY-MM-DD` | display only | + +- `version` is a **number, not a string** (`1`, not `"1"` or `"2.0.1"`). Start at + `1` for the first release under this scheme; **`+1` on every change** to any + shipped file **or to any manifest metadata** (description, icons, `version_string`). + Never reuse or decrement. +- Skald stores the installed `version` and compares it to the feed's: a strictly + greater feed `version` shows **"update available"** in the marketplace, and the + Install button becomes **Update**. Clicking it re-downloads the files and rewrites + the catalog row. +- **The integer is the *only* "is there an update?" signal** — it is compared + strictly (`feed > installed`). `version_string` (semver), icons and + `llm_short_description` are **never** compared, so a change to any of them that + does not also bump the integer is **invisible**: no "update available" badge + appears. This is the common trap — a "content-only" edit (e.g. a better + `llm_short_description`) that forgets the integer. +- **Two propagation paths, do not conflate them:** + - *Per-user code + deps* (the scripts, `package.json`/`requirements.txt`) reconcile + on a **content-hash** of the source files (§5), so new code lands at each user's + next login even without a reinstall. + - *Catalog metadata* (`llm_short_description` → the model's prompt, icons, friendly + name) is **not** in that hash — it lives in the catalog row and is rewritten only + by an explicit **reinstall/Update**. On reinstall Skald re-pulls the current feed + (never the browse cache) and pushes the new description live: enabled global + servers restart with it, and every logged-in user who activated the connector has + it restarted with the fresh `llm_short_description` — no re-login needed. +- So: to ship a new `llm_short_description`, **bump the integer** (so the admin sees + "update available") and the admin clicks **Update**. Nothing auto-propagates a + description change. +- `version_string` and `version_release_date` are display metadata only — never + compared. (Migration note: replace any legacy string `"version": "2.0.1"` with + the integer `version` + `version_string`.) + +--- + +## 8. Checklist for a new connector + +1. Folder `myconn/` with: entry file, `connector.json`, deps file + (`package.json`/`requirements.txt`), `icon_sm.svg`, `icon_lg.svg`, + optional `verify.*`. +2. Server speaks MCP over stdio (§2); **stdout = JSON-RPC only**. +3. `mcp_config.args[0]` names the entry file. +4. Correct `type` + `scope` (§3) and `auth.type` (§4). +5. For `qr`: implement `login_status` (+ `logout`), persist the session under the + connector dir (§4d). +6. Deps declared as a file, **not** vendored (§5). +7. Add the entry to `connectors.json` with a correct `sha256` for **every** file. +8. Bump `version`. +``` diff --git a/SKALD.md b/SKALD.md index 82120f8..9293e6f 100644 --- a/SKALD.md +++ b/SKALD.md @@ -161,7 +161,7 @@ Automatic build on NiPoGi with Gitea Actions (native runner v2.1.0): ### Technical notes -- `scripts/` in `.gitignore` — CI scripts moved to `ci/` (tracked by git) +- `scripts/` removed — CI scripts live in `ci/` (tracked by git); the legacy MCP servers it held are superseded by marketplace connectors - Build without `whisper-local` on Linux (`--no-default-features`) - `aarch64-linux-gnu-strip` for ARM64 binaries - `actions/checkout@v4` works (native runner has Node.js) diff --git a/agents/software-architect/AGENT.md b/agents/software-architect/AGENT.md index 1c1b0a3..e7f10b5 100644 --- a/agents/software-architect/AGENT.md +++ b/agents/software-architect/AGENT.md @@ -86,7 +86,6 @@ When working on **Skald itself** (the project you are in), follow these addition - Agent prompts: `agents/` - Extracted crates: `crates/` - Web app (Lit components): `web/` - - Python MCP scripts: `scripts/` - Config: `config.yml` (copy from `default.config.yaml`) - Docs: `docs/` - Database: `database.db` (unless overridden in `config.yml`) diff --git a/agents/software-engineer/AGENT.md b/agents/software-engineer/AGENT.md index 26caba4..7b91a95 100644 --- a/agents/software-engineer/AGENT.md +++ b/agents/software-engineer/AGENT.md @@ -116,7 +116,6 @@ When working on **Skald itself** (the project you are in), follow these addition - Agent prompts: `agents/` - Extracted crates: `crates/` - Web app (Lit components): `web/` - - Python MCP scripts: `scripts/` - Config: `config.yml` - Docs: `docs/` - Database: `database.db` diff --git a/crates/skald-core/src/mcp/verify.rs b/crates/skald-core/src/mcp/verify.rs index 1b59d82..25f5c0e 100644 --- a/crates/skald-core/src/mcp/verify.rs +++ b/crates/skald-core/src/mcp/verify.rs @@ -70,7 +70,7 @@ impl VerifyReport { /// runs inside the user's container via `docker exec`. pub enum VerifyTarget<'a> { /// Run on the Skald host process. `workdir` is an absolute host path - /// (typically `/scripts//`). + /// (typically `connectors//`). Host { workdir: &'a Path }, /// Run inside the user's sandbox container. `workdir` is an absolute path /// *inside* the container (e.g. `/root/.skald/mcp/`). diff --git a/install-nightly.sh b/install-nightly.sh index 7ef694e..a93c08f 100755 --- a/install-nightly.sh +++ b/install-nightly.sh @@ -192,7 +192,7 @@ check_optional_deps() { if command -v python3 >/dev/null 2>&1; then info "✔ Python 3 found ($(python3 --version 2>&1 | head -1))" else - warn "Python 3 not found — Python MCP servers (Gmail, GCal, GMaps, ...) will not work." + warn "Python 3 not found — the TTS plugins and host-run connectors will not work." echo " Install it from https://www.python.org/downloads/" echo "" fi @@ -327,13 +327,13 @@ if [ ! -f "$VENV_DIR/bin/python3" ] || ! "$VENV_DIR/bin/python3" -m pip --versio if command -v uv >/dev/null 2>&1; then uv venv --seed "$VENV_DIR" && uv pip install -r "$REQUIREMENTS" \ && info "✔ Python venv ready (uv)" \ - || warn "Python venv setup failed — Python MCP servers will be unavailable." + || warn "Python venv setup failed — the TTS plugins and host-run connectors will be unavailable." elif command -v python3 >/dev/null 2>&1; then python3 -m venv "$VENV_DIR" && "$VENV_DIR/bin/pip" install -r "$REQUIREMENTS" \ && info "✔ Python venv ready (pip)" \ - || warn "Python venv setup failed — Python MCP servers will be unavailable." + || warn "Python venv setup failed — the TTS plugins and host-run connectors will be unavailable." else - warn "python3 not found — Python MCP servers will be unavailable." + warn "python3 not found — the TTS plugins and host-run connectors will be unavailable." fi else info "✔ Python venv already exists" diff --git a/install.sh b/install.sh index aee48c0..8a0e54d 100755 --- a/install.sh +++ b/install.sh @@ -199,7 +199,7 @@ check_optional_deps() { if command -v python3 >/dev/null 2>&1; then info "✔ Python 3 found ($(python3 --version 2>&1 | head -1))" else - warn "Python 3 not found — Python MCP servers (Gmail, GCal, GMaps, ...) will not work." + warn "Python 3 not found — the TTS plugins and host-run connectors will not work." echo " Install it from https://www.python.org/downloads/" echo "" fi @@ -332,13 +332,13 @@ if [ ! -f "$VENV_DIR/bin/python3" ] || ! "$VENV_DIR/bin/python3" -m pip --versio if command -v uv >/dev/null 2>&1; then uv venv --seed "$VENV_DIR" && uv pip install -r "$REQUIREMENTS" \ && info "✔ Python venv ready (uv)" \ - || warn "Python venv setup failed — Python MCP servers will be unavailable." + || warn "Python venv setup failed — the TTS plugins and host-run connectors will be unavailable." elif command -v python3 >/dev/null 2>&1; then python3 -m venv "$VENV_DIR" && "$VENV_DIR/bin/pip" install -r "$REQUIREMENTS" \ && info "✔ Python venv ready (pip)" \ - || warn "Python venv setup failed — Python MCP servers will be unavailable." + || warn "Python venv setup failed — the TTS plugins and host-run connectors will be unavailable." else - warn "python3 not found — Python MCP servers will be unavailable." + warn "python3 not found — the TTS plugins and host-run connectors will be unavailable." fi else info "✔ Python venv already exists" diff --git a/requirements.txt b/requirements.txt index 20ad8ff..bfdca45 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,25 +1,26 @@ -google-auth -google-auth-oauthlib -google-api-python-client -googlemaps -requests -httpx - -# SSH MCP server (scripts/ssh_mcp_server.py) -paramiko>=3.4 - -# Google Trends MCP server (scripts/google_trends_mcp.py) -mcp -trendspyg>=0.7.0 +# Host Python dependencies. +# +# These exist for the two TTS plugins ONLY. Both spawn a bare `python3` (resolved +# from PATH, which run.sh points at .venv/bin) to run an embedded server script, +# so their imports must be satisfied here — unlike MCP connectors, they have no +# dependency reconciler of their own. +# +# MCP connectors do NOT belong in this file: one ships its own requirements.txt / +# package.json, and `mcp::install::ensure_installed` installs it into `.pydeps` / +# `node_modules` inside the user's container (or beside the connector on the host +# for a global one). Adding a connector's deps here would install them on every +# box for a connector nobody activated. # Kokoro ONNX TTS (plugin-tts-kokoro) kokoro-onnx soundfile -# Orpheus TTS 3B (plugin-tts-orpheus-3b) — heavy deps moved to requirements-optional.txt -# See requirements-optional.txt if you need the Orpheus TTS plugin. +# Orpheus TTS 3B (plugin-tts-orpheus-3b) — the light half; the GPU/ML deps +# (torch, transformers, snac, …) live in requirements-optional.txt +scipy + +# Shared by both TTS servers fastapi uvicorn -scipy numpy pydantic diff --git a/run-docker.sh b/run-docker.sh index 5dc04f8..3b22c39 100644 --- a/run-docker.sh +++ b/run-docker.sh @@ -20,12 +20,12 @@ if [ -f "$REQUIREMENTS" ] && { [ ! -f "$VENV_DIR/bin/python3" ] || ! "$VENV_DIR/ echo "[run-docker.sh] Setting up Python venv with uv …" uv venv --seed "$VENV_DIR" && uv pip install -r "$REQUIREMENTS" \ && echo "[run-docker.sh] Python venv ready." \ - || echo "[run-docker.sh] Warning: Python venv setup failed — Python MCP servers will be unavailable." + || echo "[run-docker.sh] Warning: Python venv setup failed — the TTS plugins and host-run connectors will be unavailable." elif command -v python3 >/dev/null 2>&1; then echo "[run-docker.sh] Setting up Python venv …" python3 -m venv "$VENV_DIR" && "$VENV_DIR/bin/pip" install -r "$REQUIREMENTS" \ && echo "[run-docker.sh] Python venv ready." \ - || echo "[run-docker.sh] Warning: Python venv setup failed — Python MCP servers will be unavailable." + || echo "[run-docker.sh] Warning: Python venv setup failed — the TTS plugins and host-run connectors will be unavailable." fi fi diff --git a/run.bat b/run.bat index cfd6477..0f9db16 100644 --- a/run.bat +++ b/run.bat @@ -27,7 +27,7 @@ if not exist "%VENV_DIR%\Scripts\python3.exe" ( call python3 -m venv "%VENV_DIR%" && call "%VENV_DIR%\Scripts\pip" install -r "%REQUIREMENTS%" if !ERRORLEVEL! equ 0 ( echo [run.bat] Python venv ready. ) else ( echo [run.bat] Warning: Python venv setup failed ) ) else ( - echo [run.bat] Warning: python3 not found -- Python MCP servers will be unavailable. + echo [run.bat] Warning: python3 not found -- the TTS plugins and host-run connectors will be unavailable. ) ) ) diff --git a/run.sh b/run.sh index 960c703..11e5b3b 100755 --- a/run.sh +++ b/run.sh @@ -46,8 +46,10 @@ fi # ── Python venv setup (optional) ───────────────────────────────────────────── # Creates .venv/ and installs requirements.txt if Python is available. -# If Python is not installed, the app starts normally but Python-based MCP -# servers (e.g. Gmail, Google Calendar) will fail to connect. +# If Python is not installed, the app starts normally but the TTS plugins (which +# spawn `python3` directly) will fail to start, and a host-run global connector +# will have no interpreter to install its own deps with. Per-user connectors are +# unaffected: they run inside the user's container, which ships its own Python. VENV_DIR=".venv" REQUIREMENTS="requirements.txt" @@ -60,14 +62,14 @@ if [ ! -f "$VENV_DIR/bin/python3" ] || ! "$VENV_DIR/bin/python3" -m pip --versio echo "[run.sh] Setting up Python venv with uv …" uv venv --seed "$VENV_DIR" && uv pip install -r "$REQUIREMENTS" \ && echo "[run.sh] Python venv ready." \ - || echo "[run.sh] Warning: Python venv setup failed — Python MCP servers will be unavailable." + || echo "[run.sh] Warning: Python venv setup failed — the TTS plugins and host-run connectors will be unavailable." elif command -v python3 >/dev/null 2>&1; then echo "[run.sh] Setting up Python venv …" python3 -m venv "$VENV_DIR" && "$VENV_DIR/bin/pip" install -r "$REQUIREMENTS" \ && echo "[run.sh] Python venv ready." \ - || echo "[run.sh] Warning: Python venv setup failed — Python MCP servers will be unavailable." + || echo "[run.sh] Warning: Python venv setup failed — the TTS plugins and host-run connectors will be unavailable." else - echo "[run.sh] Warning: python3 not found — Python MCP servers will be unavailable." + echo "[run.sh] Warning: python3 not found — the TTS plugins and host-run connectors will be unavailable." fi fi diff --git a/update.sh b/update.sh index 2c1ec84..232b884 100755 --- a/update.sh +++ b/update.sh @@ -269,13 +269,13 @@ main() { if command -v uv >/dev/null 2>&1; then uv venv --seed "$VENV_DIR" && uv pip install -r "$REQUIREMENTS" \ && info "✔ Python venv ready (uv)" \ - || warn "Python venv setup failed — Python MCP servers will be unavailable." + || warn "Python venv setup failed — the TTS plugins and host-run connectors will be unavailable." elif command -v python3 >/dev/null 2>&1; then python3 -m venv "$VENV_DIR" && "$VENV_DIR/bin/pip" install -r "$REQUIREMENTS" \ && info "✔ Python venv ready (pip)" \ - || warn "Python venv setup failed — Python MCP servers will be unavailable." + || warn "Python venv setup failed — the TTS plugins and host-run connectors will be unavailable." else - warn "python3 not found — Python MCP servers will be unavailable." + warn "python3 not found — the TTS plugins and host-run connectors will be unavailable." fi # ── Restart ────────────────────────────────────────────────────────────────