dguiducci/Skald-Circle
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

First Version

Browse Source
This commit is contained in:
xavix-yo
2026-07-10 15:02:09 +01:00
commit 38494a85a9
562 changed files with 196313 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/agents.md
+183
View File
@@ -0,0 +1,183 @@
# Agents
## Directory Layout
```text
agents/
<id>/
meta.json ← required: metadata, LLM preferences
AGENT.md ← required: system prompt
common/ ← shared include files; skipped by discover()
```
`agents::discover()` scans every subdirectory except `common/`. A directory without both files is skipped with a `WARN` log.
---
## meta.json Schema
| Field | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `name` | string | yes | — | Display name |
| `description` | string | yes | — | **Routing** text for the orchestrator LLM ("when to delegate / what it returns"). Used in `list_items` (type=agents) output and the `AGENTS_LIST` directive. |
| `friendly_description` | string \| null | no | null | Human-facing blurb shown to the **user** on the frontend Agents page. Frontend falls back to `description` when absent. Applies to every agent type. Never sent to the LLM. |
| `instructions` | string \| null | no | null | Note for the **calling LLM** on *how* to invoke the agent well (expected inputs, format, gotchas). Kept short. Surfaced only in `list_items` (type=agents) output — and only meaningful for `task` agents, since that listing already includes task agents only (no extra gating needed). Not in `AGENTS_LIST`. |
| `inject_memory` | `string[]` | no | `[]` | Paths to files injected into the system prompt as `<memory_file>` blocks. Relative paths resolve against Skald's process cwd. The `$WD` placeholder expands to the session's effective working directory (RunContext WD, or process cwd when unset) — e.g. `"$WD/SKALD.md"` loads a project-local file. The path **shown** in the block is relative to the working directory when the file is under it, absolute otherwise — so it always resolves back to the same file under the loop's working-directory injection. Missing files inject a `(file not created yet)` placeholder. |
| `client` | string \| null | no | null | Pin to a specific named LLM model (must exist in DB) |
| `scope` | string \| null | no | null | Task domain for AUTO client selection |
| `strength` | LlmStrength \| null | no | null | Minimum LLM capability for AUTO selection |
| `type` | `"chat"` \| `"task"` \| `"system"` | **yes** | — | The agent's role. `chat`: a user-facing conversational entry-point (e.g. `main`, `project-coordinator`) — not dispatchable, not a task root. `task`: a dispatchable sub-agent **and** valid root of a scheduled/async task. `system`: a hidden background agent wired into the runtime by id (e.g. `tic`) — never listed, never dispatchable from the tool surface. Only `task` agents appear in `list_items` (type=agents) / `AGENTS_LIST` and can be dispatched or run as a task. A `meta.json` without `type` is skipped at discovery (warn) and rejected on direct load. |
| `inject_skills` | bool | no | `true` | When `true` (the default, **including when the key is absent**), the skills registry (`skills/index.md`) is injected into the system prompt as a `<skills_index>` block so the agent can discover installed skills. Path resolution follows the `inject_memory` rule (relative under the working directory, absolute otherwise). Skipped silently if no skills are installed. Set `false` for background agents that don't need them. |
### Three descriptive fields, three audiences
The three descriptive fields exist because one string would have to serve three readers with different needs:
- **`description`** → the **orchestrator LLM** deciding *whether/when* to delegate (routing).
- **`friendly_description`** → the **user** browsing the frontend Agents page.
- **`instructions`** → the **calling LLM** on *how* to drive the agent for the best result.
Keep `instructions` distinct from `AGENT.md`: `AGENT.md` is the agent's *own* system prompt ("who I am, how I think"); `instructions` is outward-facing ("how you, the caller, should ask me, and what I return").
The frontend Agents page (`web/components/agents.js`) groups cards into three sections by `type` — **Chat**, **Task Executors**, and **System** — and shows `friendly_description` (falling back to `description`).
### Tool restriction
Tool restriction is **not** declared in the agent file (the per-agent `allow_tools` whitelist and the `run_context` default were both removed). Tool visibility and execution-time approval are governed uniformly by **permission groups** bound to **run contexts** (see [approval/index.md](approval/index.md)).
A run context is assigned to a **session** at runtime, never in `meta.json`:
- explicitly via the UI / API (`set_session_run_context`),
- via a dedicated config property for system sessions (e.g. TIC's `tic.run_context`),
- per cron job (`run_context_id`).
When a session has no run context it uses the built-in **"default"** group. The visibility filter (hide tools whose effective action for the session's group is `Deny`) runs in `src/core/session/handler/config.rs` (depth 0) and `agent_dispatch.rs` (sub-agents). MCP tools are excluded from this filter — the Approval gate governs them.
---
## AGENT.md Directives
| Directive | Behavior |
| --- | --- |
| `<!-- INCLUDE: path/to/file.md -->` | Replaced with the content of `agents/path/to/file.md` at load time. Supports recursive includes. |
| `<!-- AGENTS_LIST -->` | Replaced with a bullet list of agents where `type == "task"`: `- **id** — description`. Injected only into the five **delegating** agents (`main`, `project-coordinator`, `tech-lead`, `software-architect`, `spec-writer`); the leaf task agents and `tic` omit it. |
| `<!-- MCP_LIST -->` | Replaced at request time (in `build_openai_messages`) with the available MCP servers — active and inactive — so the agent knows what it can activate via `activate_tools`. Resolves inside `INCLUDE`d files: it is the sentinel that ships inside `common/mcp.md`. Present in every agent. |
| `<!-- KEY -->` (any uppercase name) | Runtime substitution sentinel. Replaced at request time via `SendMessageOptions::system_substitutions`. The agent's system prompt contains `__KEY__` which is swapped for the provided value before the LLM call. |
### Shared includes (`agents/common/`)
Reusable prompt fragments pulled in via `<!-- INCLUDE: common/<file>.md -->` (the `common/` directory is skipped by `discover()`):
| File | Content | Included by |
| --- | --- | --- |
| `mcp.md` | MCP activation prose paired with the `<!-- MCP_LIST -->` sentinel — keeps the "how to activate" text and the server list together. | every agent except `tic` (which has its own inline MCP block) |
| `tools.md` | `update_scratchpad` (shared blackboard) vs `write_todos` (private list) guidance. | `main`, `project-coordinator`, `tech-lead`, `software-architect`, `software-engineer`, `spec-writer` |
| `memory.md` | Persistent-memory conventions (`data/memory/`). | `main`, `spec-writer`, `tic` |
| `core_rules.md` | Baseline behavioural rules (read-before-edit, respond in the user's language). | `main` |
---
## Available Agents
The required `type` field declares each agent's role: `chat` (user talks to it directly;
not dispatchable, not a task root), `task` (dispatchable sub-agent **and** valid task/cron
root), or `system` (hidden, runtime-wired only). Only `task` agents appear in
`AGENTS_LIST` / `list_items(type=agents)` and can be dispatched or run as a task.
| id | name | type | scope | strength | description |
| --- | --- | --- | --- | --- | --- |
| `main` | Main Assistant | `chat` | — | — | General-purpose; persists notes in `data/memory/index.md` |
| `project-coordinator` | Project Coordinator | `chat` | `reasoning` | `average` | Conversational coordinator for a single project's interactive chat (source `project-{id}`) — **any kind** of project (software, travel, study, writing, personal goals…), adapting to the injected description. Receives the project context via its session `RunContext` (working dir, description, fs-write grants), does everyday planning/writing itself, and delegates specialized work (research, or code via tech-lead/software-architect/software-engineer) via `execute_task`. Maintains the project's `SKALD.md` diary. |
| `software-architect` | Software Architect | `task` | `reasoning` | `high` | Plans code changes and delegates to software-engineer |
| `software-engineer` | Software Engineer | `task` | `coding` | `high` | Writes and modifies source files across any file type |
| `code-explorer` | Code Explorer | `task` | `reasoning` | `high` | Studies code, investigates bugs, analyses architecture, produces structured Markdown reports in `data/explorer/`. No implementation. |
| `spec-writer` | Spec Writer | `task` | `reasoning` | `very_high` | Transforms project ideas into comprehensive spec documents in `data/`. Never writes code. Saves output path to scratchpad. |
| `tech-lead` | Tech Lead | `task` | `reasoning` | `very_high` | Reads project documentation, breaks scope into implementation tasks, sequences them by dependency, and orchestrates `software-architect`/`software-engineer` sub-agents to deliver end-to-end. Tracks its plan with `write_todos` (private, not `update_scratchpad`) and owns the single authoritative build+test run. |
| `researcher` | Researcher | `task` | `general` | `average` | Multi-step web research; writes structured Markdown reports (default `data/research/`, or wherever the caller specifies) and saves the path to scratchpad |
| `generalist` | Generalist | `task` | `general` | `average` | General-purpose task executor: carries out well-defined work ordered by the calling agent — file edits, shell commands, batch operations. No planning or QA. |
| `business-analyst` | Business Analyst | `task` | `reasoning` | `very_high` | Stress-tests a business idea/plan against provided evidence; finds flaws, proposes fixes, gives a GO/NO-GO/PIVOT verdict. Does no web research — reasons from inputs. |
| `tic` | TIC | `system` | — | — | Background event processor; calls `notify` when something is worth surfacing. Ephemeral. `notify` is injected as an `InterfaceTool` by `TicManager`. Tool access is restricted via the run context set from the `tic.run_context` property. |
### Orchestration model (tech-lead + software-engineer)
Two conventions keep multi-agent builds efficient and avoid context pollution:
- **Private plan, not shared state.** `tech-lead` records its task plan and progress with `write_todos` — a stateless, per-stack list that lives only in its own tool-result history. It must **not** use `update_scratchpad` for the plan: the scratchpad is a shared blackboard injected into every sub-agent, so a plan written there would pollute each `software-engineer`'s context. The scratchpad stays reserved for genuine cross-agent communication (e.g. a discovered path or type).
- **Verify once, at the top.** `software-engineer` runs only a fast compile-check (e.g. `cargo check`) after writing — never the test suite. `tech-lead` owns the single full build + test run in its integration phase, against the merged result. This replaces the old pattern of N engineers each running a full build+test.
---
## Sub-agent Mechanics
A synchronous sub-agent call (`execute_task` mode=sync / `execute_subtask`) is **not** in `ToolRegistry`. It is intercepted in `run_agent_turn` before any registry lookup, then handled by `dispatch_sub_agent` (`src/core/session/handler/agent_dispatch.rs`):
1. Validate `agent_id` and `prompt` args.
2. Reject self-calls.
3. Load target agent's `meta.json` via `load_task_meta` and reject anything that is not a `task` agent — `chat` (e.g. `main`, `project-coordinator`) and `system` (e.g. `tic`) agents are invisible as sub-agents, and unknown ids surface a not-found error, all in one gate.
4. Check depth: `parent_frame.depth + 1 <= MAX_AGENT_DEPTH`.
5. Resolve target client (see below).
6. Create child `chat_sessions_stack` row (`depth = parent + 1`, `parent_tool_call_id` set).
7. Load any existing `stack_mcp_grants` for the child stack (restart recovery) → populate `active_mcp_grants`.
8. Build child `AgentRunConfig` via `for_sub_agent()`. That call inherits the parent's `base_tool_defs` but **strips the per-level augmentations** (`ask_user_clarification`, `execute_subtask`) — they are re-derived below, so stripping prevents the inherited copy from duplicating the freshly added one (the OpenAI-compat APIs reject non-unique tool names with HTTP 400). Then:
- Replace `active_mcp_grants` with the pre-populated arc from step 7.
- Append `sub_agents_only` tools and `ask_user_clarification`.
- Append `execute_subtask` (so the child can dispatch its own sub-agents, e.g. `tech-lead` → `software-architect`/`software-engineer`) — **only** when `depth + 1 < MAX_AGENT_DEPTH`, since at the limit the call would be rejected anyway (and the inherited copy was already stripped, so a max-depth agent advertises no tool it cannot use). `for_sub_agent()` clears all interface tools (including the root's `execute_task`/`execute_subtask` interface tools), so without this re-injection a sub-agent has **no** tool definition to delegate further; the call itself is then intercepted in `run_agent_turn` and routed back through `dispatch_sub_agent`.
- Inject `activate_tools` (stack-scoped, `stack_id = Some(child.id)`) as interface tool.
9. Append prompt as `role = agent` message in child stack.
10. Emit `AgentStart` event.
11. **Run the child inline** — `resume_pending_tools` + `run_agent_turn` on the child stack, awaited recursively **in the same task**, holding the **same** `processing` lock and the **same** `CancellationToken` clone.
12. Delete `stack_mcp_grants` for the child stack; emit `AgentDone`; terminate the child stack frame.
13. Map the child `TurnOutcome` to the return value: `Final{content}` → `Ok(content)`; `Cancelled` → `Ok("…cancelled")` (the shared token also stops the parent at its next check); `Exhausted` → `Ok("…exceeded rounds")`; `Err` → `Err`.
The returned string becomes the parent's tool-call result via the normal `Ok(result)` branch in `run_agent_turn` (which calls `chat_llm_tools::complete` and emits `ToolDone`) — so completion logic lives in exactly one place. There is **no** task spawn, no `WaitingChild` signal, and no resume cascade for the sync path.
### Mutex / token invariant
One user message = one logical critical section: the `processing` lock is acquired once in `handle_message` and held for the whole parent+child recursion. Parent and child share one `CancellationToken` clone, so a `/stop` that cancels a running child stops the parent by construction (its next round/tool check observes `is_cancelled()`).
`resume_turn` and its cascade are retained only for app-restart recovery of an active child stack, async task result injection (`inject_async_result`), and the WS resume message — not for normal sync dispatch.
---
## Client Resolution Order
For a sub-agent dispatch (`execute_task` mode=sync / `execute_subtask`) to agent `X`:
1. `args.client` (explicit override in the tool call)
2. `X/meta.json` → `client` field (pinned model name)
3. AUTO selection using `X/meta.json` → `scope` + `strength`
> **Important:** the parent agent's resolved client is **not** inherited by sub-agents.
> Passing a concrete model name to `resolve()` bypasses scope/strength checks entirely,
> so sub-agents always auto-select unless an explicit override is provided via (1) or (2).
> This ensures `strength: high` in `meta.json` is always respected regardless of which
> model the caller is using.
---
## Depth Limit
**`MAX_AGENT_DEPTH = 5`** (hardcoded in `src/core/session/handler/mod.rs`).
Depth 0 = root `main` session. Each sub-agent dispatch (`execute_task` mode=sync / `execute_subtask`) increments depth by 1. Attempting to exceed the limit returns an error to the LLM without calling the sub-agent.
An agent cannot call itself or the `main` agent.
---
## Adding an Agent
1. Create `agents/<id>/meta.json` with at minimum `name` and `description`.
2. Create `agents/<id>/AGENT.md` with the system prompt.
3. Optionally restrict tools by assigning the agent's sessions a run context (permission group) at runtime — see [approval/index.md](approval/index.md).
4. **No restart required** — agents are discovered on every request.
---
## When to Update This File
- An agent is added, removed, or its meta fields change
- `MAX_AGENT_DEPTH` constant changes
- sub-agent dispatch (`execute_task` / `execute_subtask`) validation logic changes (new restrictions or resolution order)
- `meta.json` schema gains a new field
docs/approval/index.md
+631
View File
@@ -0,0 +1,631 @@
# Approval Gate (Human-in-the-Loop)
## Overview
`ApprovalManager` is a top-level service (in `Skald`) that intercepts every tool call before execution and decides whether to:
- **Allow** — execute freely (no matching rule, or an explicit `allow` rule)
- **Deny** — block immediately (`deny` rule)
- **Require** — suspend and ask the user for confirmation
It is designed to be extensible: multiple notification channels (web, Telegram), granular policies per agent/source/tool, and future support for resuming interrupted sessions.
---
## Architecture
```
llm_loop.rs
└─► ApprovalManager.check(session_id, category, agent_id, source, tool_name, args)
│
├─ GateResult::Allow → execute immediately
├─ GateResult::Deny → fail tool call (not bypassable)
└─ GateResult::Require
├─ (session bypass active?) → GateResult::Allow → execute immediately
└─► ApprovalManager.register(...) → (request_id, rx)
│ emits ServerEvent::PendingWrite or ApprovalRequired
└─► await rx ← resolved by WS/Telegram via resolve(request_id, decision)
```
`ApprovalManager` lives in `src/core/approval/mod.rs` and is independent of `ChatSessionManager`.
---
## Permission Groups and RunContext
Rules are scoped to **permission groups** (`tool_permission_groups` table). A session's active **RunContext** references a group via its `security_group` field; rules in that group take precedence over rules in the `"default"` group.
**For full RunContext documentation** (fields, resolution, API, project integration) — see [../session/run-context.md](../session/run-context.md) (source of truth).
The `"default"` group is seeded automatically at startup and **cannot be deleted**. Its rules can be freely edited.
---
## Rules
Rules are stored in SQLite in the `approval_rules` table and evaluated in `priority ASC` order (lower number = evaluated first). The first matching rule determines the action. If no rule matches, the fallback is `Require` (default-closed policy).
The **Default** group has a seeded final catch-all `require * priority=999999`, so it is **require-by-default**: any tool not matched by a more-specific `allow`/`deny`/`require` rule falls through to it and prompts for human approval. Whitelist (`allow`) and blacklist (`deny`) rules layer *above* the catch-all at lower priority numbers. A handful of benign built-in plumbing/UI tools (`write_todos`, `notify`, `activate_tools`, `show_file_to_user`, `image_generate`) are seeded as `allow` so they don't prompt.
The catch-all is seeded **only when absent** (at DB init, or if the group has no `*` rule), so it is never re-created or overwritten on restart — you can change the general rule from the frontend (edit the `*` row → `allow`/`deny`/`require`) and the choice persists. To prevent the shadowing bug where two `*` catch-alls with different actions coexist (the lower priority number silently wins), `add_rule`/`update_rule` **reject** adding a second `*` catch-all with a different action to a group; edit the existing row instead.
> **Historical note:** earlier builds seeded an `allow * priority=9999` catch-all (permissive-by-default) via `seed_allow_all_default`. That could shadow a user-added `require *` at a higher priority number and let unmatched tools run without approval. It was replaced by `seed_default_catch_all` (require, only-when-absent).
### Table Schema
| Column | Type | Description |
| ------ | ---- | ----------- |
| `id` | INTEGER | PK |
| `agent_id` | TEXT (nullable) | Filter on a specific agent. `NULL` = all |
| `source` | TEXT (nullable) | Filter on source: `web`, `telegram`, `cron`. `NULL` = all |
| `tool_pattern` | TEXT | Exact name, glob with `*` suffix (e.g. `mcp__gmail__*`), or a `@fs_*` filesystem class token (see below) |
| `path_pattern` | TEXT (nullable) | Glob on the normalised file path. `<dir>/*` matches the dir subtree; other patterns use trailing-`*`/exact. `NULL` = no path filter |
| `action` | TEXT | `require` \| `allow` \| `deny` |
| `note` | TEXT (nullable) | Descriptive note |
| `priority` | INTEGER | Evaluation order (default 100; system defaults use 10) |
| `group_id` | TEXT | Permission group this rule belongs to (default: `"default"`) |
### Pattern Matching
| Pattern | Matches |
| ------- | ------- |
| `execute_cmd` | only `execute_cmd` |
| `mcp__gmail__*` | all tools from the `gmail` server |
| `mcp__*` | all MCP tools |
| `*` | any tool |
| `@fs_read` | any file-read tool (`read_file`, `grep_files`, `list_files`, `search_file`, `get_ast_outline`) |
| `@fs_write` | any file-write tool (`write_file`, `edit_file`, `insert_at_line`, `replace_lines`) |
| `@fs_any` | any filesystem tool (read **or** write) |
The `@fs_*` **filesystem class tokens** let a single rule target a whole operation class by
path, regardless of the individual tool name. They are resolved by `is_file_read_tool` /
`is_file_write_tool` in `tool_pattern_matches`, and back the **File System** permission panel
(one path row = one rule row). See [File System category](#file-system-category-ui).
The `path_pattern` field is matched by `path_pattern_matches` against the **normalised** path.
Normalisation canonicalizes `args["path"]` (resolving `..` and symlinks via
`tools::fs::canonicalize_for_policy`) and makes it relative to the process cwd, so
`docs/../secrets/x` or a symlink into `secrets/` cannot evade a rule. A `<dir>/*` pattern is a
**directory subtree**: it matches the directory node itself (`path == "<dir>"`) and everything
under it (`path` starts with `"<dir>/"`), using a `/`-delimited boundary so `memory/*` matches
`memory` and `memory/x` but **not** the sibling `memory-secrets/x`. Any other pattern falls back
to exact / trailing-`*`. If `path_pattern` is set but the tool has no `path` argument, the rule
**does not** match.
### Evaluation Order
`ApprovalManager.check()` runs **first**; the RunContext filesystem fast-path runs **after**
and can only relax a `Require` to `Allow` (never overrides a `Deny`). Inside `check()`:
1. DB rules for the session's group, then `"default"` group as fallback — sorted by `priority ASC, id ASC` within each tier — first matching rule wins. (`memory/` is auto-allowed by a seeded `@fs_any allow memory/*` rule, not a hardcoded exception.)
2. **Session bypass** (in-memory): if the result would be `Require` and an active bypass matches `session_id` + `category`, convert to `Allow`. `Deny` is never bypassed.
3. No matching rule → `Require` (default-closed)
> **Fail-closed on error.** If loading the rules themselves fails (transient DB error),
> `check()` returns `Require`, **never** `Allow` — a rule-load failure must not silently
> let an un-vetted `execute_cmd`/`restart`/write through. This matches the default-closed
> policy; the error is logged at `error` level.
Then, back in `llm_loop.rs`, if the result is still `Require`, the **RunContext filesystem
fast-path** applies: a file-read tool whose path is read-allowed (`rc.is_read_allowed`:
working dir / `docs/` / `skills/` / `allow_fs_reads` / `allow_fs_writes`) or a file-write tool
whose path is write-allowed (`rc.is_write_allowed`) is upgraded to `Allow`. A `Deny` from
step 2 is never reached here, so the `secrets/` deny holds even inside the auto-read working
dir. Paths are canonicalized (`..`/symlinks resolved) before matching.
### Path Whitelist
There are two ways to pre-authorize writes to a directory:
**Option A — RunContext `allow_fs_writes`** (session-scoped, no DB rule needed):
Set `allow_fs_writes` on the session's `RunContext`. The fast-path fires in `llm_loop.rs` after `ApprovalManager` and upgrades a `Require` to `Allow`, so no approval event is emitted (a `Deny` rule still wins).
```json
{
"security_group": "cron_restrictive",
"allow_fs_writes": ["data/output", "/abs/path/to/dir"]
}
```
Matching semantics: exact file OR recursive directory prefix (no wildcards). `"data/output"` matches `data/output/foo.txt`, `data/output/sub/bar.txt`, etc. Entries can be absolute or relative to the session's `working_directory`.
**Option B — approval_rules DB** (persistent, applies to all sessions in the group):
Add a single `@fs_*` `allow` rule at a low priority (e.g. 5, before the generic catch-all).
One row covers every filesystem tool of that class — no need to repeat it per tool:
```sql
-- allow read + write anywhere under data/ (the `/*` also matches the `data` dir node)
INSERT INTO approval_rules (tool_pattern, path_pattern, action, note, priority)
VALUES ('@fs_any', 'data/*', 'allow', 'auto-allow data/', 5);
```
Use `@fs_read` for read-only access, `@fs_write` for write-only, `@fs_any` for both. This is
exactly what the **File System** panel writes; the defaults (`memory/*`, `data/*`, `secrets/*`)
are inserted automatically on first startup by `seed_fs_path_rules()`.
### Default Rules (seeded automatically on first startup with empty DB)
| Tool | Action | Priority |
|------|--------|----------|
| `execute_cmd` | require | 10 |
| `restart` | require | 10 |
| `mobile_start_pairing` | require | 10 |
Default rules are inserted only when the `approval_rules` table is empty. They can be modified or deleted normally. **Filesystem tools are no longer seeded as per-tool `require` rules** — filesystem gating is owned by the File System category (`@fs_*` path rules) backstopped by the `*` catch-all.
### File System path defaults (seeded)
`seed_fs_path_rules()` seeds the default File System rows (priority 5, `default` group), each a
single `@fs_*` row:
| Path | Rule | Effect |
| ------ | ------ | -------- |
| `memory/*` | `@fs_any` allow | LLM manages its own memory autonomously (replaces the former hardcoded `is_memory_path` bypass) |
| `data/*` | `@fs_any` allow | scratch/data workspace, read + write |
| `secrets/*` | `@fs_any` **deny** | no read **or** write access |
Each is inserted independently only when its exact `(tool_pattern, path_pattern)` is absent, so
a row the user deleted is only ever re-created individually.
**Secrets deny.** Reading a secret would leak its value into the LLM context, chat history, the
compactor's summaries and the WS stream — worse than a write — hence `deny` (non-bypassable)
rather than `require`. `@fs_any` denies **writes as well as reads** (the old per-read-tool seed
denied reads only). Since `Deny` is evaluated before the RunContext read fast-path, `secrets/`
stays unreadable even inside the auto-read working dir. The `secrets/*` pattern also matches the
`secrets` dir node, so a recursive `list_files`/`grep_files` rooted at it is covered; the read
tools additionally skip any `secrets` directory during traversal (`SKIP_DIRS`).
> This protects the cwd-relative `secrets/` folder. Tokens read by external MCP server
> processes are unaffected (they read their own files directly, not via these tools).
**Legacy migration.** `migrate_legacy_fs_rules()` runs once at startup (before the seeder) and
removes the superseded legacy filesystem rows (the per-tool write `require` defaults, the old
per-tool `data/*` allows, and the old per-read-tool `secrets` denies), identified by the exact
`note` text the old seeders stamped. User-authored rules carry different notes and are untouched.
### File System category (UI)
In the **Security → \<group\>** page, filesystem tools are **not** shown as individual per-tool
Allow/Require/Deny chips. Instead the [`<approval-rules-page>`](../../web/components/approval-rules.js)
renders a dedicated **File System** panel: one row per path, each with a single selector whose
options map to a `@fs_*` rule row:
| Selector | Rule written |
| ---------- | -------------- |
| Allow read | `@fs_read` allow |
| Allow write | `@fs_any` allow (write implies read) |
| Deny | `@fs_any` deny |
| Require | `@fs_any` require |
A path entered as a directory is stored as `<dir>/*` and given a priority by depth (deeper =
evaluated first). The panel also exposes a settable **Default** row (an `@fs_any` no-path rule at
priority 900); when unset the effective default is the group's `*` catch-all (Require). No new API
is involved — the panel reuses `POST/PUT/DELETE /api/approval/rules`.
---
## Useful Rule Examples
### Require approval for all Gmail tools
```sql
INSERT INTO approval_rules (tool_pattern, action, note, priority)
VALUES ('mcp__gmail__*', 'require', 'Gmail requires approval', 5);
```
### Require approval only for cron jobs (not for web)
```sql
INSERT INTO approval_rules (source, tool_pattern, action, note, priority)
VALUES ('cron', 'mcp__*', 'require', 'All MCP tools from cron require approval', 20);
```
### Always allow a specific tool for a specific agent
```sql
INSERT INTO approval_rules (agent_id, tool_pattern, action, note, priority)
VALUES ('email-assistant', 'mcp__gmail__list_messages', 'allow', 'free read for email-assistant', 1);
```
### Allow free writes to a specific subfolder
```sql
-- For the researcher agent only, allow writes to data/research/ without approval
INSERT INTO approval_rules (agent_id, tool_pattern, path_pattern, action, note, priority)
VALUES ('researcher', 'write_file', 'data/research/*', 'allow', 'researcher writes freely to data/research/', 3);
```
The `researcher` defaults to `data/research/` but accepts a caller-specified output directory (e.g. when orchestrated by a command like `/ideagen`, which tells it to write into a session-scoped `data/ideagen-*/` folder). Add one rule per pattern you want pre-approved:
```sql
-- Also let the researcher write into /ideagen session dirs without approval
INSERT INTO approval_rules (agent_id, tool_pattern, path_pattern, action, note, priority)
VALUES ('researcher', 'write_file', 'data/ideagen-*/*', 'allow', 'researcher writes freely to ideagen session dirs', 3);
```
---
## Session Bypass (Temporary Allow-All)
The human can temporarily suppress approval prompts for a session without modifying DB rules. The bypass is **in-memory only** — it disappears on app restart or when the session ends.
### Activation
The bypass is activated by the **human** (not the LLM) from any of these surfaces:
- **Agent Inbox** page (REST `/api/inbox/approvals/:id/resolve` with `bypass_secs`)
- **Copilot chat** (WebSocket `approve_write`/`approve_tool` with `bypass_secs` field)
- **Telegram bot** inline keyboard (⏱ 15 min / 🔄 Session buttons → `ApprovalApi::approve_with_bypass`)
The LLM has no tools to activate it — giving the LLM the ability to disable its own oversight would defeat the purpose of the gate.
### Scope
Each bypass entry targets a specific `BypassScope`:
| Scope | What it covers |
| ----- | -------------- |
| `All` | Every tool regardless of category |
| `Category(ToolCategory)` | Only tools with the given registered category (e.g. `Filesystem`, `Shell`) |
| `McpServer(String)` | Only tools from the named MCP server (matched by the `mcp__<server>__` prefix in the tool name) |
A bypass entry also has an optional expiry (`expires_at: Option<Instant>`). `None` means indefinite (session-scoped).
### How It Works
`ApprovalManager` holds `session_bypasses: Mutex<HashMap<i64, Vec<ApprovalBypass>>>`. `check()` receives `session_id`, `category`, and `tool_name`. After rule evaluation, if the result is `Require` and a matching active bypass exists, the result is converted to `Allow`. Expired entries are pruned lazily on each `check()` call.
### Invariants
- `Deny` rules are **never** bypassable.
- The bypass state is cleared when `cancel_for_session()` is called (WS disconnect).
- Multiple bypasses can coexist for the same session (e.g. "all categories: 30 min" + "filesystem: indefinite").
- MCP tools match `McpServer` scope; they are also covered by `All` scope.
### Rust API
```rust
approval.bypass_session(session_id).await; // indefinite, all
approval.bypass_session_for(session_id, Duration::from_secs(600)).await; // 10 min, all
approval.bypass_session_for_category(session_id, ToolCategory::Shell, Some(Duration::from_secs(600))).await;
approval.bypass_session_for_mcp(session_id, "gmail".into(), Some(Duration::from_secs(1800))).await;
approval.clear_session_bypass(session_id).await;
```
---
## Session Sources (`source`)
| Value | When |
| ----- | ---- |
| `web` | Chat from the web UI |
| `telegram` | Chat from the Telegram bot |
| `cron` | Trigger from scheduled_jobs |
Headless sessions (cron) have no active interface: approval requests are registered as pending and the agent suspends until a response arrives (via web or Telegram).
---
## Pending Approvals
All pending requests are accessible via `Inbox.list_pending()` (which internally calls `ApprovalManager.list_pending()` and `ClarificationManager.list_pending()`), exposed by the `GET /api/inbox` endpoint, and displayed on the **Agent Inbox** frontend page.
Each entry contains:
| Field | Type | Description |
| ----- | ---- | ----------- |
| `request_id` | i64 | Unique ID for resolution |
| `session_id` | i64 | Session that generated the request |
| `tool_call_id` | i64 | Tool call in the DB |
| `tool_name` | String | Name of the tool to execute |
| `arguments` | JSON | Full arguments |
| `agent_id` | String | Agent that called the tool |
| `source` | String | Session source |
| `context_label` | Option\<String\> | Human-readable origin label (e.g. `"CronJob: Daily Digest"`) |
| `created_at` | String | ISO-8601 timestamp |
| `tool_category` | Option\<String\> | Registered tool category (`filesystem`, `shell`, …); `null` for MCP/unknown tools |
| `mcp_server` | Option\<String\> | MCP server name extracted from the tool name (e.g. `"gmail"`); `null` for non-MCP tools |
`context_label` is set by `ChatSessionHandler::set_context_label()` before the run (e.g. `TaskManager` sets `"CronJob: <title>"`). It is read in `llm_loop.rs` and `resume.rs` and passed to `approval.register()`.
---
## Inbox bus events (`GlobalEvent`)
Inbox lifecycle changes are broadcast on the global `GlobalEvent` bus so any subscriber (Telegram, the mobile-connector plugin) can react without polling. Plugins subscribe via `ctx.chat_hub.events(...)`. Four events cover the full Inbox cycle:
| Event (`ServerEvent`) | Emitted by | When |
| --- | --- | --- |
| `ApprovalRequested { request_id, tool_call_id, tool_name }` | `ApprovalManager::register` | A tool call is gated and enters the Inbox |
| `ApprovalResolved { request_id, tool_call_id, approved }` | `ApprovalManager::resolve` **and** `resolve_for_tool_call` | An approval is approved/rejected (from any surface: Inbox REST, WS, mobile, or the inline copilot card) |
| `ClarificationRequested { request_id, title }` | `ClarificationManager::register` | A clarification question enters the Inbox |
| `ClarificationResolved { request_id }` | `ClarificationManager::resolve` | A clarification is answered |
These are distinct from the per-session WS events `ApprovalRequired` (carries full args for the active client) and `AgentQuestion` (the interactive clarification prompt). The `ClarificationManager` now holds a `broadcast::Sender<GlobalEvent>` injected from `Skald::new` (same `event_tx` the `ApprovalManager` uses), mirroring the approval manager.
---
## Agent Inbox
The **Agent Inbox** is the unified web page for managing all pending requests from background sessions (cron, etc.):
- **Approval requests** — tool calls requiring human confirmation (e.g. `execute_cmd`, `write_file`)
- **Clarification requests** — questions posed by the agent via `ask_user_clarification` when it cannot proceed autonomously
### REST API
| Method | Endpoint | Description |
| ------ | -------- | ----------- |
| `GET` | `/api/inbox` | Returns `{ total, approvals, clarifications }` |
| `POST` | `/api/inbox/approvals/:request_id/resolve` | Resolve an approval by `request_id` (see body below) |
| `POST` | `/api/inbox/clarifications/:request_id/resolve` | Body: `{ answer: string }` |
| `POST` | `/api/tools/:tool_call_id/resolve` | Resolve an **inline chat** approval by `tool_call_id` (source-agnostic); body `{ action, note }`. See [Resolution](#resolution). |
**Resolve approval body:**
```json
{
"action": "approve" | "reject",
"note": "",
"bypass_secs": 900,
"bypass_scope": "category" | "mcp_server" | "all"
}
```
`bypass_secs` and `bypass_scope` are optional. When present (only on `approve`):
- `bypass_secs = 0` → indefinite bypass (until WS disconnect)
- `bypass_secs = N` → bypass expires after N seconds
- `bypass_scope` defaults to `"category"` if `tool_category` is set, `"mcp_server"` if `mcp_server` is set, otherwise `"all"`
The legacy endpoints `/api/approval/pending` and `/api/approval/resolve/:id` remain active for backwards compatibility.
### Frontend
The page is implemented in `web/components/agent-inbox.js` (`<agent-inbox-page>`). Polls every 8 s when open. The red badge in the sidebar (independent polling every 10 s) shows the total pending count.
See [../frontend.md](../frontend.md) for component details.
---
## Resolution
### From WebSocket (web copilot)
The client sends a JSON message:
```json
{ "type": "approve_tool", "request_id": 42 }
{ "type": "reject_tool", "request_id": 42, "note": "optional reason" }
```
**Bypass via WebSocket** — include `bypass_secs` on any approve message:
```json
{ "type": "approve_tool", "request_id": 42, "bypass_secs": 900 } // 15-min bypass
{ "type": "approve_tool", "request_id": 42, "bypass_secs": 0 } // session bypass (indefinite)
```
`bypass_secs = 0` maps to an indefinite bypass (until session ends); positive values are seconds. The scope (category / MCP server / all) is auto-detected from the pending request, same as the REST endpoint.
The types `approve_write`/`reject_write` are aliases for `approve_tool`/`reject_tool` and work identically.
### From the inline chat card (any source — web / mobile / project)
A pending tool call rendered inline in the chat (copilot **or** mobile) is resolved
by its globally-unique `tool_call_id`, **not** by `request_id`:
```http
POST /api/tools/{tool_call_id}/resolve
{ "action": "approve" | "reject", "note": "optional reason" }
```
This path is **source-agnostic**. `resolve_tool` (`src/frontend/api/sessions.rs`)
looks the tool call up by id alone and derives the owning session from the tool
call's own `chat_sessions_stack` row, so an approval raised in a `mobile` /
`telegram` / project session resolves correctly regardless of which client posts
it — there is no "active session" scoping. (Historically this endpoint hardcoded
the `web` source, which made a mobile-created approval fail with
`tool_call_id … not found in current web session`.)
- **Live** (server still up): delegates to `ApprovalManager::resolve_for_tool_call`,
which fires the waiting `oneshot` and unblocks the turn.
- **Post-restart** (no in-memory entry): executes the tool directly on the session
that owns it (`ChatHub::handler_for_session`) and continues the loop.
The client only falls back to this REST path when the inline card lacks a live
`request_id` — i.e. it was rebuilt from history after a reconnect/reload. While the
server is up, `build_items` re-attaches the live `request_id` (via
`ApprovalManager::request_id_for_tool_call`) to history-rebuilt pending cards, so
they resolve through the WebSocket path above with full bypass support. The old
`/api/web/tools/{tool_call_id}/resolve` route remains as a back-compat alias.
### From Telegram
The Telegram plugin uses `ApprovalApi::approve_with_bypass` (defined in `crates/core-api/src/approval.rs`, implemented on `ApprovalManager`). The inline keyboard shows four buttons in two rows:
```text
[✅ Approve] [❌ Reject]
[⏱ 15 min] [🔄 Session]
```
Tapping **⏱ 15 min** → `approve_with_bypass(request_id, Some(900))`.
Tapping **🔄 Session** → `approve_with_bypass(request_id, None)`.
`approve_with_bypass` calls `ApprovalManager::approve()` then registers the appropriate session bypass (auto-detected scope).
---
## Behaviour on Restart
The live `request_id` → `oneshot` registry is in-memory and lost on restart, but the
tool-call intent survives in `chat_llm_tools.status='pending'`. The system reconstructs
around that:
- **Inbox survives restart.** `Inbox::list_pending` unions the in-memory approvals with
`ApprovalManager::list_persisted_pending()` — DB `pending` rows (excluding
`ask_user_clarification`, which shares the status). These carry
`request_id = PERSISTED_REQUEST_ID` (a falsy sentinel, `0`) telling the client to resolve
them by `tool_call_id` (there is no live registry entry / oneshot to address). Both inbox
frontends branch on it: truthy `request_id` → `POST /api/inbox/approvals/{request_id}/resolve`
(with bypass); falsy → `POST /api/tools/{tool_call_id}/resolve` (bypass buttons hidden).
- **The inline chat card** likewise has no live `request_id` after a restart, so it also
resolves via `POST /api/tools/{tool_call_id}/resolve`.
- **`resolve_tool`'s post-restart branch** dispatches correctly per tool kind:
- *Simple tools* (registry / MCP) execute directly on the owning session and return.
- *Sub-agent tools* (`execute_task`, `execute_subtask`, `run_subtask`) cannot run
through the flat `execute_tool` path (they need the recursive dispatcher). The
endpoint marks the call **pre-approved** (`ChatSessionHandler::mark_pre_approved`)
and drives `ChatHub::resume_session`, which re-dispatches the tool through
`execute_tool_call` — the same router as a live turn — so the sub-agent runs instead
of failing with `Unknown tool: execute_task`. The approval gate consumes the
pre-approved flag and skips re-prompting.
- **`resume_pending_tools`** (the recovery path run on a new message or WS reconnect)
routes through `execute_tool_call` too, and `ChatHub::resume`/`resume_session` inject
the `execute_task` interface tool so `mode=async` tasks rebuild via `build_execution`.
So both sync and async `execute_task` recover.
There is **no** separate `request_id` counter: `tool_call_id` (the durable
`chat_llm_tools` rowid) is the identity throughout. Live approvals carry
`request_id == tool_call_id`; DB-rebuilt ones carry the falsy `PERSISTED_REQUEST_ID`.
So `ApprovalManager::resolve` (by request_id) and `resolve_for_tool_call` are the same
lookup, and `request_id_for_tool_call` just returns the id when a live entry exists.
---
## Tool Visibility Filtering
Beyond the execution-time approval gate, tools are filtered at **invitation time** — before being included in the LLM context. This reduces token usage and prevents the LLM from attempting to call tools it cannot execute.
### Semantics
`ApprovalManager.is_tool_visible(rules, tool_name)` checks the pre-loaded rules synchronously:
- If the first matching rule has action `Deny` → tool is hidden from the LLM
- All other cases (Allow, Require, or no match) → tool is visible
Only `tool_pattern` is considered (path/agent/source filters are ignored for visibility — those are execution-time concerns).
### Where it runs
1. **Parent agent** (`src/core/session/handler/config.rs`, `build_agent_config`): rules are loaded once with `list_for_group`, then `base_tool_defs.retain(...)` filters the list before building `AgentRunConfig`.
2. **Sub-agents** (`src/core/session/handler/agent_dispatch.rs`, `dispatch_sub_agent`): same filter applied after sub-agent-only tools are added.
Sub-agents share the parent session's permission group. The execution-time `ApprovalManager.check()` gate remains active as a second enforcement layer.
### Tool Visibility API
```rust
// Sync: applied to pre-loaded rules slice
approval.is_tool_visible(rules: &[ApprovalRule], tool_name: &str) -> bool
// Async: one DB round-trip, returns the matched RuleAction (or None if no rule matches)
approval.check_tool_visibility(group_id: &str, tool_name: &str) -> Option<RuleAction>
// Via RunContextManager (resolves group_id from run_context_id automatically)
run_context_manager.check_tool_visibility(run_context_id: Option<&str>, tool_name: &str) -> Option<RuleAction>
```
---
## Group Duplication
`POST /api/tool-permission-groups/{id}/duplicate`
Body: `{ "id": "<new_group_id>", "name": "<new display name>" }`
Creates a new permission group that is an exact copy of the source group's rules. The operation is atomic: the new group row and all copied rules are inserted in a single SQLite transaction. The new group inherits the source's `description`.
Implemented in `RunContextManager::duplicate_group` (`src/core/run_context/mod.rs`).
---
## AllTools Response (`GET /api/approval/tools`)
The endpoint returns `AllTools`:
```json
{
"built_in": [
{ "name": "read_file", "description": "...", "source": "built-in", "server": null, "category": "filesystem" },
{ "name": "send_voice_message", "description": "...", "source": "built-in", "server": null, "category": "dynamic" }
],
"mcp": [ { "name": "mcp__gmail__list_messages", "description": "...", "source": "mcp", "server": "gmail" } ],
"mcp_servers": {
"gmail": { "friendly_name": "Gmail", "description": "Read and send Gmail messages" }
}
}
```
`mcp_servers` is keyed by the MCP server's internal name (matching `server` fields in `mcp` entries). The frontend uses it to group MCP tools under their server's `friendly_name` and display the server `description` as a section subtitle.
### Making dynamically-injected tools gate-able
The permission grid can only assign allow/require/deny to tools the endpoint enumerates. `ToolCatalog::list_all()` covers registry tools + a small static `synthetic_tools()` list (core interface tools) + live MCP tools. Everything injected outside the registry — plugin tools (Telegram `send_voice_message`), provider tools, memory tools — is surfaced by **runtime discovery** instead of a hand-maintained list:
- [`ToolDiscovery`](../../src/core/tool_discovery.rs) observes the tool array at `AgentRunConfig::all_tool_defs()` (tapped in `llm_loop.rs` each round) and upserts every offered tool into the `known_tools` table. An in-memory seen-set keeps this a no-op after each tool's first sighting; the DB write runs in a spawned task off the turn's critical path.
- `list_tools` (`src/frontend/api/approval.rs`) merges `known_tools` into the response, deduping names already present as built-in/MCP tools, and tags the remainder with `category: "dynamic"` (rendered as its own "Dynamic" group in the grid).
Consequence: a tool appears in the grid once it has been offered to the LLM at least once (in practice, after first use of the interface/provider that injects it); until then the catch-all `* require @999999` gates it safely. This is drift-proof — it can never fall out of sync with what is actually offered — and needs no per-tool or per-plugin registration. See [tools docs](../tools.md#dynamically-injected-tools--discovery).
---
## Module Structure
| File | Role |
| ---- | ---- |
| `crates/core-api/src/approval.rs` | `ApprovalApi` trait — `approve`, `reject`, `approve_with_bypass`; exposed to plugins via `PluginContext` |
| `src/core/pending_registry.rs` | `PendingRegistry<Info, Resolution>` — generic in-memory pending-request store (map + oneshot) shared by the three managers below. Id minting and event emission stay in the managers. |
| `src/core/approval/mod.rs` | `ApprovalManager` (composes a `PendingRegistry` keyed by `tool_call_id` + rules engine + session bypasses), `GateResult`, `ApprovalRule`, `PendingApprovalInfo`, `PERSISTED_REQUEST_ID`, session bypass methods; `is_tool_visible` (sync); `check_tool_visibility` (async); `impl ApprovalApi` |
| `src/core/clarification/mod.rs` | `ClarificationManager` (composes a `PendingRegistry` + its own `request_id` counter), `PendingClarificationInfo` |
| `src/core/elicitation/mod.rs` | `ElicitationManager` (composes a `PendingRegistry` + counter + secret handling + MCP `ElicitationBridge`), `PendingElicitationInfo` |
| `src/core/inbox.rs` | `Inbox`: unified façade for pending approvals + clarifications + elicitations (wraps ApprovalManager, ClarificationManager, ElicitationManager) |
| `src/core/run_context/mod.rs` | `RunContext` domain object: fields `security_group`, `system_prompt`, `allow_fs_writes`, `allow_fs_reads`, `working_directory` + applicative methods `tool_group_id()`, `extra_system_prompt()`, `effective_working_dir()`, `is_write_allowed()`, `is_read_allowed()`. `RunContextManager`: CRUD for permission groups; `duplicate_group` (atomic); `check_tool_visibility`. |
| `src/core/tools/fs/mod.rs` | `canonicalize_for_policy` / `path_under` — path canonicalization shared by the RunContext fast-paths and `approval::normalize_path` |
| `src/core/db/approval_rules.rs` | SQLite queries: list, insert, update, delete |
| `src/core/db/mod.rs` | `approval_rules` table creation |
| `src/core/session/handler/config.rs` | Loads rules once with `list_for_group`, calls `approval.is_tool_visible` to filter `base_tool_defs` for the parent agent |
| `src/core/session/handler/agent_dispatch.rs` | Same visibility filter applied to sub-agent `base_tool_defs` after sub-agent-only tools are added |
| `src/core/session/handler/llm_loop.rs` | Resolves `category` via `ToolRegistry::category_of`, calls `approval.check(session_id, category, ...)` + `approval.register()` |
| `src/core/session/handler/resume.rs` | Same `check()` call as `llm_loop.rs` for pending tool re-gating |
| `src/core/session/handler/mod.rs` | `ChatSessionHandler` holds `Arc<ApprovalManager>`, `Arc<ClarificationManager>`, `context_label: RwLock<Option<String>>` |
| `src/frontend/api/inbox.rs` | `/api/inbox` endpoint + resolve for approval and clarification (uses `skald.inbox`) |
| `src/frontend/api/approval.rs` | Approval rules CRUD + `/api/approval/pending` + `/api/approval/tools` (returns `AllTools` with `mcp_servers` metadata map) |
| `src/frontend/api/run_context.rs` | `POST /api/tool-permission-groups/{id}/duplicate` handler |
| `web/components/approval-groups.js` | Groups list page (`<approval-groups-page>`): create, rename, duplicate, delete groups; fires `approval-navigate` event |
| `web/components/approval-rules.js` | Per-group rules view (`<approval-rules-page>`): rule matrix + override/low-priority panels + default action bar; listens to `approval-navigate` |
| `src/frontend/api/ws.rs` | Handles `approve_tool`/`reject_tool`/`approve_write`/`reject_write`; optional `bypass_secs` field activates `approve_with_bypass` |
| `src/core/events.rs` | `ServerEvent::ApprovalRequired` (generic tools) and `PendingWrite` (files with diff) |
---
## Frontend — Approval Rules page
The UI is split into two Lit components that communicate via the `approval-navigate` custom event (see [frontend.md](frontend.md) for the event protocol).
**`<approval-groups-page>`** (`web/components/approval-groups.js`): lists all `tool_permission_groups`. Each group card shows its name, description, and rule count. Groups can be added, renamed, duplicated, or deleted; the `"default"` group cannot be deleted. Clicking a group fires `approval-navigate` with the group object and hides itself.
**`<approval-rules-page>`** (`web/components/approval-rules.js`): per-group rules view with four panels:
| Panel | Priority range | Purpose |
| --- | --- | --- |
| Overrides | `< 0` | Wildcard/path rules evaluated before any per-tool entry |
| Per-tool matrix | `= 0` | Simple 4-chip toggle (—/Allow/Require/Deny) per tool, grouped by category/MCP server |
| Low Priority | `1…999998` | Wildcard/path rules as a safety net, evaluated after the matrix |
| Default Action | `999999` | Catch-all `*` rule with no filters; inline selector; missing = no catch-all |
MCP tools are grouped under their server's `friendly_name` (from `mcp_servers` in the `GET /api/approval/tools` response). The server `description` is shown as a subtitle.
The **Agent Profiles** page (`web/components/agent-profiles.js`, `<agent-profiles-page>`) is a separate sidebar entry that manages `run_contexts`. Each profile links a session to a permission group via a dropdown. The `"default"` profile cannot be deleted. See [../session.md](../session.md) for the resolution chain.
---
## When to Update This File
- New action types in rules
- New notification channel added (e.g. Telegram)
- Pending approval persistence added to DB
- New fields in `PendingApprovalInfo` or `PendingClarificationInfo`
- New Agent Inbox APIs
docs/architecture.md
+238
View File
@@ -0,0 +1,238 @@
# Architecture
## Two-Layer Design
```
src/
core/ ← headless application core (no HTTP, no Axum)
skald/ ← Skald: Runtime context + 8 domain bundles; staged new() / shutdown()
mod.rs ← Skald struct (aggregate) + new() composition root + shutdown()
runtime.rs ← Runtime: cross-cutting context (db, buses, global_tx, shutdown, supervisor)
bundles.rs ← 8 domain bundles + their build() functions
wiring.rs ← wire() (OnceLock cycle-breakers) + spawn_background()
supervisor.rs ← TaskSupervisor: named background-task handles, joined on shutdown
accessors.rs ← impl Skald: one accessor per manager (the logical API surface)
… ← all domain modules (db, llm, session, cron, plugin, …)
frontend/ ← web presentation layer
mod.rs ← WebFrontend: wires router_factory, starts plugins, runs Axum
server.rs ← WebServer (Axum router, TcpListener)
api/ ← 18 HTTP + WebSocket handlers — State<Arc<Skald>>
core/config.rs ← CoreConfig + DbConfig, LlmConfig, TicConfig, … (core-owned types)
frontend/config.rs ← FrontendConfig + ServerConfig, WebConfig
config.rs ← Config (YAML parse only) + into_split()
main.rs ← thin: tracing → Config → into_split → plugins → Skald::new → WebFrontend::start → shutdown
```
`Skald` knows nothing about Axum or HTTP. It can be started headlessly. `WebFrontend` is the only component that imports Axum and constructs an HTTP server. `Skald` is no longer a God Object: its ~30 managers are grouped into a cross-cutting `Runtime` context plus eight cohesive domain bundles, and every consumer (frontend handlers, plugin context) reaches a manager through an accessor method (`skald.chat_hub()`, `skald.db()`, …) — never a public field. That accessor surface is the logical boundary a future `skald-core` crate would keep.
`Config::into_split()` produces a `CoreConfig` (db, llm, tic, cron, timezone) for `Skald::new()` and a `FrontendConfig` (server, web, timezone) for `WebFrontend::new()`. The YAML file structure is unchanged. `timezone` is cloned into both since it is used by the cron scheduler (core) and optionally by the frontend.
Plugin instances are constructed in `main.rs` as `Vec<Arc<dyn Plugin>>` and injected into `Skald::new()` — the core never depends on concrete plugin crates.
---
## Component Map
| Struct | Created by | Held as | Depends on |
| --- | --- | --- | --- |
| `SqlitePool` | `db::init_pool()` | `Arc<SqlitePool>` | — |
| `LlmManager` | `LlmManager::new()` | `Arc<LlmManager>` | `SqlitePool` |
| `McpManager` | `McpManager::new()` | `Arc<McpManager>` | `SqlitePool` |
| `TaskManager` | `TaskManager::new()` | `Arc<TaskManager>` | `SqlitePool`, `ChatSessionManager` (via OnceLock), `ChatHub` (via OnceLock) |
| `ToolRegistry` | `Tools::build()` | `Arc<ToolRegistry>` | `McpManager`, `TaskManager`, `PluginManager`, `SecretsStore` |
| `ApprovalManager` | `Interaction::build()` | `Arc<ApprovalManager>` | `SqlitePool` |
| `ClarificationManager` | `Interaction::build()` | `Arc<ClarificationManager>` | — |
| `Inbox` | `Interaction::build()` | (owned by `Interaction` bundle) | `ApprovalManager`, `ClarificationManager`, `ElicitationManager`, `ToolRegistry` |
| `ToolCatalog` | `Tools::build()` | (owned by `Tools` bundle) | `ToolRegistry`, `McpManager` |
| `ChatEventBus` | `Runtime::bootstrap()` | `Arc<ChatEventBus>` | — |
| `ContextCompactor` | `Conversation::build()` (when `llm.compaction` configured) | `Option<Arc<ContextCompactor>>` (consumed by `ChatSessionManager`) | `LlmManager`, `ChatEventBus` |
| `ChatSessionManager` | `ChatSessionManager::new()` | `Arc<ChatSessionManager>` | `SqlitePool`, `LlmManager`, `ToolRegistry`, `McpManager`, `ApprovalManager`, `ClarificationManager`, `ChatEventBus`, `ContextCompactor` |
| `ChatHub` | `ChatHub::new()` | `Arc<ChatHub>` | `SqlitePool`, `ChatSessionManager`, `ApprovalManager` |
| `TicManager` | `TicManager::new()` | `Arc<TicManager>` | `SqlitePool`, `ChatHub`, `ChatSessionManager` |
| `Skald` | `Skald::new(&core_cfg, plugins)` | `Arc<Skald>` | all of the above |
| `WebFrontend` | `WebFrontend::new(skald, &frontend_cfg)` | owned by `main` | `Arc<Skald>`, `FrontendConfig` |
### Circular Dependencies
The construction cycles cannot be expressed as a nested value tree, so the managers break them with `std::sync::OnceLock` (or `RwLock<Option<_>>`) setters. All of these setters are called in one place — the `wire()` function in `skald/wiring.rs` — run after every bundle is built and before background tasks start. Bundle structs therefore never hold references to each other; the only cross-bundle links live in the managers' `OnceLock`s.
**`TaskManager` ↔ `ChatSessionManager`**: `TaskManager` needs `ChatSessionManager` to dispatch jobs, but `ChatSessionManager` is built after `ToolRegistry` which holds `Arc<TaskManager>`. `TaskManager` is created first (in `Tasks::build`), `set_session()` is called in `wire()` once `ChatSessionManager` exists.
**`TaskManager` ↔ `ChatHub`**: Same pattern — `set_hub()` is called in `wire()`. The cron tick loop starts 30 s after `start()`, so the hub is always ready by the first real job dispatch.
**`McpManager` ↔ `ElicitationManager`**: `set_elicitation_handler()` is called in `wire()`; MCP `initialize()` is only spawned afterwards (in `spawn_background()`), so stdio servers start with a handler for server-initiated `elicitation/create`.
**`PluginManager` ↔ `Skald`**: the one `Arc<Skald>` back-reference. `PluginManager` is constructed early (in `Integrations::build`, so tools can register), then `set_skald(Arc<Skald>)` is called as the very last step of `new()`, after `Arc::new(Skald { … })`. `set_router_factory(RouterFactory)` is called by `WebFrontend::start()` before `start_enabled()`.
---
## Startup Sequence
### `main.rs`
1. Init tracing (`tracing-appender` daily rolling to `logs/`)
2. `Config::load()` → `config.into_split()` → `(CoreConfig, FrontendConfig)`
3. Build `Vec<Arc<dyn Plugin>>` — all plugin instances constructed here
4. `Skald::new(&core_cfg, plugins)` — see sequence below
5. `WebFrontend::new(skald, &frontend_cfg)` + `.start()` — see sequence below
6. Await `ctrl_c`
7. `skald.shutdown()` + `handle.shutdown()`
### Inside `Skald::new(&core_cfg, plugins)`
`new()` is a **staged composition root**: it builds the `Runtime` context, then each domain bundle in dependency order via its own `Bundle::build(...)`, then resolves cycles and starts background tasks. (`db::init_pool()` runs earlier, in `main.rs`, and the pool is passed in.)
1. `agents::discover()` — scans `agents/*/` for `meta.json` + `AGENT.md` (logged only)
2. `Runtime::bootstrap(pool)` — `GlobalConfigManager`, `SystemEventBus`, `ChatEventBus`, the `GlobalEvent` broadcast channel (`global_tx`), `CancellationToken`, `TaskSupervisor`
3. `Models::build` — `ProviderRegistry` (+6 built-in providers), `LlmManager`, `SecretsStore`, `MemoryManager`
4. `Media::build` — `ImageGeneratorManager`, `TranscribeManager`, `TtsManager`
5. `Integrations::build` — `McpManager` (NOT yet initialized), `PluginManager` (plugins registered, not started)
6. `Tasks::build` — `TaskManager` (scheduler, not started), `ProjectManager`, `ProjectTicketManager` — before `Tools` so cron tools can capture cron
7. `Tools::build` — `ToolRegistry` (all built-in tools, capturing mcp/plugins/cron/secrets + mobile-connector tools), `ToolCatalog`, `LlmCommandManager`
8. `Interaction::build` — `ApprovalManager` (+ 4 non-fatal `seed_*`), `ClarificationManager`, `ElicitationManager`, `Inbox`
9. `Conversation::build` — `RunContextManager` (+ seed), `ContextCompactor` (if configured), `ChatSessionManager`, `ChatHub` (+ `register("web"/"talk")`), `TicManager`
10. `Infra::build` — `LatexCompiler`, `LocationManager`, `remote` slot (`None`)
11. `wire(...)` — all `OnceLock` cycle-breakers in one place (`cron.set_session/set_hub/set_self_arc`, `ticket.set_task_manager`, `chat_hub.set_task_mgr`, `mcp.set_elicitation_handler`)
12. `spawn_background(...)` — every long-lived task, each registered by name with the supervisor: `llm-log-cleanup`, `session-cancel`, `mcp-init` (`mcp.initialize()`), `cron`, `ticket-listener`, `tic`
13. `Arc::new(Skald { rt, models, media, tools, integrations, tasks, conversation, interaction, infra })`
14. `skald.plugin_manager().set_skald(Arc::clone(&skald))` — the one `Arc<Skald>` back-reference, last
### Inside `WebFrontend::start()`
28. `plugin_manager.set_router_factory(factory)` — provides Axum router factory to plugins
29. `plugin_manager.set_web_port(port)` — provides HTTP port to plugins (e.g. Tailscale)
30. `plugin_manager.start_enabled()` — starts Telegram and other enabled plugins
31. `plugin_manager.start_config_watcher(shutdown_token)` — polls DB every 30 s
32. `WebServer::start(addr)` — Axum HTTP+WS server begins listening
---
## Request Lifecycle
1. Client opens WebSocket: `GET /api/ws`
2. `handle_socket()` gets or creates `ChatSessionHandler` via `ChatHub::session_handler("web")`
3. Client sends `ClientMessage` JSON over WS
4. `ChatHub::send_message("web", prompt, opts)` **enqueues** the message on the source's inbox and returns; it does not run the turn inline (see *Per-source inbox* in [session.md](session.md))
5. The source's single consumer task drains the inbox, **coalescing** any messages that piled up during an in-flight turn into one prompt (joined by blank lines), and calls `handler.handle_message(...)`
6. `handle_message` acquires `processing: Mutex<()>` (one at a time per session)
7. `run_agent_turn` is a thin round orchestrator (up to `max_tool_rounds` rounds) that delegates each step to focused collaborators (see [session.md](session.md#run_agent_turn-inner-loop)):
- Build context: `build_openai_messages()` → system prompt + history + tool results
- Apply permission-group visibility filter (hide tools `Deny`d for the session's run-context group)
- `call_llm_round()` — one LLM call + automatic model fallback on retriable errors
- `LlmTurn::Message` → send `Done` event, exit loop
- `LlmTurn::ToolCalls` → for each call, `handle_tool_call()`:
- `effective_args()` (working-dir injection) → `run_approval_gate()` → optional `PendingWrite`, wait for user
- `execute_tool_call()` → send `ToolStart`; `call_agent` recurses via `dispatch_sub_agent`
- `record_tool_outcome()` → send `ToolDone` / `ToolError` / `ToolCancelled`
- All events are emitted through the `TurnEmitter` seam (`emitter.rs`)
8. Main loop sends `Done` event with final content and token counts
---
## Notification Flow (background)
```text
MCP server stdout (JSON-RPC notification, no id field)
→ McpServer reader loop (src/core/mcp/server.rs)
→ notification_tx (mpsc::UnboundedSender)
→ McpManager::notification_consumer
→ db::mcp_events::insert(source, method, payload)
[every tic.interval_secs (default 900 s) — TicManager::run_tick()]
→ mcp_events::pending_limited(tic.batch_size)
→ mcp_events::mark_processed(ids)
→ build_prompt(events)
→ ChatHub::send_message("tic", prompt) ← ephemeral session
→ TIC agent runs, calls notify(briefing)
→ ChatHub::notify_sync → mpsc channel
[ChatHub::notification_consumer]
→ batching window (200 ms)
→ appends synthetic Assistant message to chat_history (reasoning_content + is_synthetic=true)
→ inserts chat_llm_tools row for read_notification (status='done', result=[briefings])
→ calls hub.resume(home_source)
→ resume_turn picks up the synthetic tool call, runs the LLM loop
→ user sees assistant briefing in home conversation
```
---
## Skald Fields (bundle model)
`Skald` holds one cross-cutting `Runtime` context plus eight domain bundles — all private. Consumers reach managers through **accessor methods** named after the manager (`skald.chat_hub()`, `skald.db()`, `skald.approval()`, …), defined in `skald/accessors.rs`. The frontend uses these via `State<Arc<Skald>>`; the plugin context (`PluginManager::build_context`) uses the same surface. High-level facets like `Inbox` and `ToolCatalog` encapsulate several underlying managers behind a simpler API.
| Bundle | Field | Members (accessor → manager) |
| --- | --- | --- |
| `Runtime` | `rt` | `db` → `SqlitePool`, `config` → `GlobalConfigManager`, `config_properties`, `system_bus` → `SystemEventBus`, `event_bus` → `ChatEventBus`, `global_tx` (`GlobalEvent` sender), `shutdown_token`, `supervisor` → `TaskSupervisor` |
| `Models` | `models` | `provider_registry`, `llm_manager`, `secrets`, `memory_manager` |
| `Media` | `media` | `image_generator_manager`, `transcribe_manager`, `tts_manager` |
| `Tools` | `tools` | `tools` → `ToolRegistry`, `catalog` → `ToolCatalog`, `command_manager` → `LlmCommandManager` |
| `Integrations` | `integrations` | `mcp` → `McpManager`, `plugin_manager` → `PluginManager` |
| `Tasks` | `tasks` | `cron` → `TaskManager`, `projects` → `ProjectManager`, `ticket_manager` → `ProjectTicketManager` |
| `Conversation` | `conversation` | `manager` → `ChatSessionManager`, `chat_hub` → `ChatHub`, `run_context_manager` → `RunContextManager`, `tic_manager` → `TicManager` |
| `Interaction` | `interaction` | `approval` → `ApprovalManager`, `inbox` → `Inbox`, `clarification` → `ClarificationManager`, `elicitation` → `ElicitationManager` |
| `Infra` | `infra` | `latex_compiler` → `LatexCompiler`, `location_manager` → `LocationManager`, `remote` → `Arc<RwLock<Option<Arc<dyn RemoteAccess>>>>` |
`TicManager` lives in `Conversation` (not `Tasks`) because it is constructed from and drives the conversation stack (session manager + chat hub + run context); this keeps every bundle a single-shot `build()` with no two-phase init.
---
## Graceful Shutdown
On SIGINT, `main.rs` executes:
1. `skald.shutdown()`:
- `rt.shutdown_token.cancel()` — signals all background loops to exit their `select!`
- `rt.supervisor.join_all(10 s)` — awaits every supervised task against a shared deadline, logging any laggard **by name**
- `plugin_manager.stop_all()`
2. `handle.shutdown()` — drains and closes the Axum HTTP server
Tasks registered with the `TaskSupervisor` (spawned in `spawn_background`, joined on shutdown, listed by their supervisor name):
| Supervisor name | Source |
| --- | --- |
| `cron` | `src/core/cron/mod.rs` (`TaskManager::start`) |
| `ticket-listener` | `src/core/projects/tickets.rs` |
| `tic` | `src/core/tic/mod.rs` |
| `session-cancel` | `src/core/skald/wiring.rs` |
| `mcp-init` | `src/core/skald/wiring.rs` → `McpManager::initialize` |
| `llm-log-cleanup` | `src/core/db/llm_requests/cleanup.rs` |
Other tasks are spawned inside their managers and honor `shutdown_token.cancelled()` but are not (yet) registered with the supervisor — cancellation stops them, but they are not individually joined: `PluginManager` config watcher (`src/core/plugin/mod.rs`), `McpManager` notification consumer (`src/core/mcp/mod.rs`), `ChatHub` notification consumer (`src/core/chat_hub/mod.rs`), `TtsManager`/`TranscribeManager` reload watchers (`src/core/{tts,transcribe}/manager.rs`).
---
## Workspace Crates
The binary depends on several independent library crates in `crates/`. Each crate has no dependency on the main `skald` crate and can be published or reused standalone.
| Crate | Path | Purpose |
| --- | --- | --- |
| `core-api` | `crates/core-api/` | Shared types and traits: `ServerEvent`, `GlobalEvent`, `InterfaceTool`, `SendMessageOptions`, `ChatHubApi` trait |
| `llm-client` | `crates/llm-client/` | LLM client abstraction (OpenAI-compat, Anthropic, Ollama) |
| `mcp-client` | `crates/mcp-client/` | MCP client (JSON-RPC over stdio/SSE) |
| `honcho-client` | `crates/honcho-client/` | Honcho long-term memory HTTP client |
### `core-api` — plugin extraction boundary
`core-api` is the designated contract crate for plugin independence. A plugin that depends only on `core-api` (instead of the full main crate) can be extracted into its own workspace member without circular dependencies.
See [crates/workspace.md](crates/workspace.md) for the full extraction roadmap.
### Future: `skald-core` crate
`src/core/` is designed as a stepping stone toward extracting the headless core into a standalone `crates/skald-core/` crate. The accessor facade in `skald/accessors.rs` is the seam: the frontend and plugin context already consume `Skald` only through those methods, never its fields. When the extraction happens:
- `src/core/` moves to `crates/skald-core/src/`
- the `skald/accessors.rs` methods are promoted to a `pub trait SkaldApi` (they already have that shape)
- the two raw-SQL leaks the frontend still has (`skald.db()` used directly for a handful of `sqlx::query*` calls in `sessions.rs`/`dev.rs`/`stats.rs`, and `skald.system_bus()` for two `.send()` sites) must be closed behind repository/intent methods first
- `src/frontend/` depends on `skald-core` as a path dependency
---
## When to Update This File
- A new manager is added to a `Skald` bundle (add its accessor + update the bundle table)
- A bundle is added, removed, or a manager moves between bundles
- The staged construction in `Skald::new()` / a `Bundle::build()` or `WebFrontend::start()` changes
- The request lifecycle changes (new event type, new loop behavior)
- A new circular dependency and its `wire()` resolution is introduced
- A background task is added to (or removed from) the `TaskSupervisor`
- A new workspace crate is added
docs/build-and-distribution.md
+124
View File
@@ -0,0 +1,124 @@
# Build & Distribution
How Skald is built for local development and how a **single portable binary** is
produced for headless servers (Ubuntu Server, AWS containers, mini-PCs).
## TLS / crypto: rustls + `ring`, no OpenSSL
Skald links **no OpenSSL and no `aws-lc-rs`**. All HTTPS/WSS traffic goes through
[`rustls`](https://crates.io/crates/rustls) with the pure-`ring` crypto backend.
This is the single most important property for a portable binary: there is no
dynamic link to a system `libssl`/`libcrypto`, so the binary does not depend on
the OpenSSL version installed (or missing) on the target machine.
How it is wired:
- Every `reqwest` dependency in the workspace is declared
`default-features = false, features = ["rustls-no-provider", …]`. `reqwest 0.13`
has no bundled-`ring` feature, only `rustls-no-provider` (rustls with **no**
crypto provider selected).
- Because no provider is bundled, exactly one **process-wide** provider is
installed at startup, before any TLS handshake, in `src/main.rs`:
```rust
rustls::crypto::ring::default_provider()
.install_default()
.expect("install rustls ring crypto provider");
```
- `rustls` is pinned as a direct dependency of the root crate **only** to select
the provider: `default-features = false, features = ["ring", "std", "tls12", "logging"]`.
Without this, rustls' default feature would pull `aws-lc-rs` back in.
- `teloxide` (Telegram plugin) is on `default-features = false, features = ["rustls", …]`
so it does not drag in `native-tls`/OpenSSL either.
> **Feature-unification trap.** rustls is a single shared crate across every
> consumer. If *any* dependency enables its `aws_lc_rs` feature, `aws-lc-rs`
> (a cmake/NASM C build) comes back for the whole tree. Consumers that must be
> kept on `ring`: all `reqwest` (done), `tokio-tungstenite` (relay client — it
> declares `tokio-rustls` with `default-features = false`, so it inherits `ring`),
> and the **embedded Tailscale** provider (see below). Verify with:
>
> ```sh
> cargo tree -e no-dev -i aws-lc-rs # must print "did not match any packages"
> cargo tree -e no-dev -i ring # must list rustls consumers
> ```
## Native (development) build
```sh
cargo build # or: cargo run
./run.sh # supervisor loop (rebuilds on exit -1); -d for debug
```
On macOS/dev machines this dynamically links the system libc, which is fine —
the portability concern only applies to the distributed binary.
## Portable static build (musl)
The distribution target is `x86_64-unknown-linux-musl` (or
`aarch64-unknown-linux-musl`), which produces a **fully static** binary: no
`libssl`, no `libc`, no shared libraries at all. Copy the single file to the
server and run it.
Since there is no OpenSSL/aws-lc build, the only native code left to
cross-compile is **SQLite** (bundled via `libsqlite3-sys`), the **tree-sitter C
grammars**, and `ring` — all of which the musl-cross toolchain handles out of the
box. There is **no host toolchain requirement** other than Docker:
```sh
scripts/build-musl.sh # x86_64 static binary
TARGET=aarch64-unknown-linux-musl \
IMAGE=messense/rust-musl-cross:aarch64-musl \
scripts/build-musl.sh # arm64 static binary
```
Output: `target/musl/<target>/release/skald`. Verify it is static:
```sh
file target/musl/x86_64-unknown-linux-musl/release/skald
# → ELF 64-bit LSB executable, x86-64, statically linked, …
```
The script builds `--no-default-features` (see the feature table below) to drop
`whisper-local`; set `FEATURES=""` to include it (needs a C++ cross-compile).
### glibc alternative
If a static musl binary is more than you need, a **glibc** binary built inside an
old base image (e.g. `debian:bullseye` / `ubuntu:22.04`) runs on any server with
a same-or-newer glibc. Now that OpenSSL is gone, this build needs no special
crypto handling — the normal gnu toolchain compiles SQLite/`ring`/tree-sitter
directly. It is not fully static (glibc stays dynamic) but is broadly compatible
and simpler to produce than musl.
## Cargo features that affect the binary
| Feature | Default | Effect | Portability cost |
| --- | --- | --- | --- |
| `whisper-local` | **on** | Local STT via whisper.cpp | Compiles whisper.cpp (**C++**) — heavy; drop for server builds (`--no-default-features`) |
| `embedded-tailscale` | **off** | Pure-Rust embedded Tailscale provider (no system `tailscaled`) | Pulls the `tailscale` crate, which **forces `aws-lc-rs`** (cmake/NASM C build) back into the tree — breaks the ring-only static binary |
The recommended Tailscale provider, `tailscale_sys` (drives the system
`tailscaled`), is always compiled and needs neither feature. `embedded-tailscale`
exists only for a self-contained mesh where installing `tailscaled` is not an
option; enabling it re-introduces the aws-lc-rs C build. See
[plugins/remote.md](plugins/remote.md).
## What the binary needs at runtime
- **Nothing dynamically linked** in the musl build — no OpenSSL, no libc.
- SQLite is **statically bundled** (compiled from source), so the target needs no
system `libsqlite3`.
- Python is **optional** and only required for Python-based MCP servers; the app
starts without it (see `run.sh`).
- The web UI is static assets under `web/` (`web.static_dir`); serve it from the
same binary or point a reverse proxy at it.
## Self-restart on a server
`run.sh` is a dev supervisor: exit `255` → rebuild+restart, exit `0` → stop. On a
server, prefer **platform-native supervision** instead of the shell loop:
`systemd` with `Restart=on-failure` (map the `restart` tool's `exit(-1)` = 255 to
a restart), a container restart policy, or `launchd` on macOS. See
[self-rewriting.md](self-rewriting.md).
docs/chat-event-bus.md
+215
View File
@@ -0,0 +1,215 @@
# Event Buses
Two independent in-process buses:
| Bus | Type | Purpose |
| --- | ---- | ------- |
| **Chat** | `ChatEventBus` | Chat-turn events (user messages, assistant responses, compaction) |
| **System** | `SystemEventBus` | Infrastructure lifecycle events (provider registration, etc.) |
See below for details on each.
---
## Chat Event Bus
`src/core/chat_event_bus.rs` — thin re-export of `crates/core-api/src/bus.rs`.
All types (`ChatEventBus`, `BusEvent`, `ChatEvent`, `CompactionEvent`, etc.) are defined in `core-api` and re-exported here for backward compatibility.
## Purpose
Decouples producers (session handlers, compactor) from consumers (Honcho
memory, future analytics, etc.). Events are published **only after an operation
completes successfully** — messages are already persisted in the DB when the
event fires.
---
## Top-level Type: `BusEvent`
```rust
pub enum BusEvent {
UserMessage(ChatEvent),
AssistantResponse(ChatEvent),
CompactionDone(CompactionEvent),
}
```
Consumers match on the variant to decide what to handle:
```rust
match event {
BusEvent::UserMessage(e) | BusEvent::AssistantResponse(e) => { /* ... */ }
BusEvent::CompactionDone(e) => { /* optional */ }
}
```
---
## Sub-types
### `ChatEvent`
Published once per completed turn — one `UserMessage` and one
`AssistantResponse` in order.
| Field | Type | Description |
| --- | --- | --- |
| `session_id` | `i64` | `chat_sessions.id` |
| `stack_id` | `i64` | `chat_sessions_stack.id` |
| `message_id` | `i64` | `chat_history.id` for this message |
| `role` | `ChatEventRole` | `User`, `Assistant`, or `Agent` |
| `content` | `String` | Message text |
| `is_synthetic` | `bool` | `true` when the *message* is system-generated (TicManager ticks, ChatHub briefings) |
| `is_interactive` | `bool` | `true` when a real user participates (web, Telegram) |
| `is_ephemeral` | `bool` | `true` for short-lived automated sessions (cron, tic) |
| `tool_calls` | `Vec<ToolCallEvent>` | Non-empty only for `Assistant` events |
| `created_at` | `DateTime<Utc>` | Timestamp of publication |
### `ToolCallEvent`
| Field | Type | Description |
| --- | --- | --- |
| `name` | `String` | Tool name |
| `arguments` | `Option<String>` | JSON-serialized arguments |
| `result` | `Option<String>` | Tool output or error message |
| `status` | `String` | `"done"` or `"failed"` |
### `CompactionEvent`
Published by `ContextCompactor` after a summary is persisted to `chat_summaries`.
| Field | Type | Description |
| --- | --- | --- |
| `session_id` | `i64` | `chat_sessions.id` |
| `stack_id` | `i64` | `chat_sessions_stack.id` |
| `summary_id` | `i64` | `chat_summaries.id` of the new row |
| `covers_up_to_message_id` | `i64` | Boundary: messages with `id > this` are loaded raw from now on |
| `triggered_by_tokens` | `u32` | Input token count that triggered this compaction |
---
## ChatEventBus
```rust
// Instantiated once in main.rs, stored in `Skald` and ChatSessionManager.
let bus = Arc::new(ChatEventBus::new()); // default capacity: 256
// Publish (called internally):
bus.user_message(event); // from ChatSessionHandler
bus.assistant_response(event); // from ChatSessionHandler
bus.compaction_done(event); // from ContextCompactor
// Subscribe (call once at startup, loop in tokio::spawn):
let mut rx = bus.subscribe();
tokio::spawn(async move {
loop {
match rx.recv().await {
Ok(BusEvent::UserMessage(e)) => { /* process */ }
Ok(BusEvent::AssistantResponse(e)) => { /* process */ }
Ok(BusEvent::CompactionDone(e)) => { /* optional */ }
Err(RecvError::Lagged(n)) => warn!("lagged by {n} events"),
Err(RecvError::Closed) => break,
}
}
});
```
---
## Publication Rules
### Chat events
| Source | `is_synthetic` | Published? |
| --- | --- | --- |
| User via WebSocket / Telegram | `false` | ✅ on `TurnOutcome::Final` |
| Cron job | `false` | ✅ on `TurnOutcome::Final` |
| TicManager tick | `true` | ✅ on `TurnOutcome::Final` |
| ChatHub notification briefing | `true` (injected via `resume_turn`) | ❌ never (uses `resume_turn`, not `handle_message`) |
| `TurnOutcome::Cancelled` | — | ❌ never |
| `TurnOutcome::Exhausted` | — | ❌ never |
| Sub-agent turns (`dispatch_sub_agent`) | — | ❌ never (only root turns publish) |
Per each successful turn, **two events** are published in order:
1. `BusEvent::UserMessage` — content of the user message
2. `BusEvent::AssistantResponse` — content of the assistant response + tool calls
### Compaction events
`BusEvent::CompactionDone` is published by `ContextCompactor::try_compact()`
whenever a new summary is successfully persisted. It fires **before** the LLM
call that processes the user's message (Opzione C trigger). See
[compaction.md](compaction.md) for the full flow.
---
## Adding a Consumer
1. Call `skald.event_bus.subscribe()` in `main.rs` after `Skald::new()` completes.
2. Spawn a background task with a receive loop.
3. Match on `BusEvent` variants — ignore variants you don't care about.
4. On `RecvError::Lagged`, log and continue — do not panic.
5. Keep the consumer fast; if it does I/O (HTTP calls), buffer or batch internally.
---
## Channel Capacity
Default: **256 events** (`DEFAULT_CAPACITY` in `chat_event_bus.rs`). If a
consumer falls behind by more than 256 events it receives `Lagged` errors and
misses intermediate events. Tune via `ChatEventBus::with_capacity(n)`.
---
## When to Update This File
- New variants added to `BusEvent`
- New fields added to `ChatEvent`, `ToolCallEvent`, or `CompactionEvent`
- Publication rules change (new sources, new conditions)
- A new consumer is wired up in `main.rs`
---
## System Event Bus
`crates/core-api/src/system_bus.rs` — infrastructure lifecycle events, separate from chat-turn events.
### `SystemEvent`
```rust
pub enum SystemEvent {
ApiProviderRegistered { type_id: String },
ApiProviderUnregistered { type_id: String },
}
```
### Wiring
- **Created** early in `main.rs` (no dependencies), stored in `skald.system_bus` and `PluginContext::system_bus`.
- **Producers**: `ProviderRegistry::register_plugin()` / `unregister_plugin()` emit automatically — plugins do not need to touch the bus directly.
- **Consumers**: `TtsManager` and `TranscribeManager` subscribe at construction time and call `reload()` on `ApiProviderRegistered` / `ApiProviderUnregistered`. This ensures DB-backed models whose provider was not yet in the registry at startup are picked up as soon as the plugin starts.
### Adding a consumer
```rust
let mut rx = system_bus.subscribe();
tokio::spawn(async move {
loop {
match rx.recv().await {
Ok(SystemEvent::ApiProviderRegistered { type_id }) => { /* ... */ }
Ok(SystemEvent::ApiProviderUnregistered { type_id }) => { /* ... */ }
Err(RecvError::Lagged(n)) => warn!("system_bus lagged by {n}"),
Err(RecvError::Closed) => break,
}
}
});
```
### Adding a new `SystemEvent` variant
1. Add the variant to `SystemEvent` in `crates/core-api/src/system_bus.rs`.
2. Emit it from the relevant producer.
3. Update consumers that need to react.
4. Update this file.
docs/chat-hub.md
+111
View File
@@ -0,0 +1,111 @@
# ChatHub
`ChatHub` (`src/core/chat_hub/`) is the single entry point for **interactive, user-facing chat sessions** — web, mobile, Telegram, project chats. It owns the `source → session` mapping, serializes incoming user messages per source (injecting them into an in-flight turn where possible), runs each turn through a `ChatSessionHandler`, and bridges every turn's events onto the global broadcast bus that connected clients subscribe to.
## What it is — and is not
ChatHub manages **one live, persistent session per `source`**, addressed by source id through the `sources` table. It is **not** a runner for background / non-interactive agents:
- Cron jobs, TIC ticks, and async sub-agent tasks go through `TaskManager` / `ChatSessionManager` directly. They are not user-facing, have no broadcast audience, and must not appear in the `sources` table.
- The one bridge from background → interactive is **notifications**: a background agent calls `ChatHub::notify(...)`, and the notification consumer delivers aggregated structured notifications to the *home* source (see below).
Keep that boundary: routing a non-interactive agent through ChatHub is a misuse.
## Source → session mapping
The `sources` table (`src/core/db/sources.rs`) maps each `source_id` to its `active_session_id`. `get_or_create_session` looks it up and lazily creates a session on first use; `provision_session(reset=true)` discards the current session and starts a fresh one (emitting a `NewSession` event). `clear(source)` is the thin wrapper used by `/clear` / "new conversation".
## Per-source inbox (serialization + live injection)
Each source gets one **`SourceInbox`** and one **consumer task**, created lazily on the first message (`src/core/chat_hub/inbox.rs`). This sits *in front of* the handler's `processing` mutex and gives two properties the old per-message detached-spawn dispatch lacked:
1. **FIFO ordering** — a single consumer per source means arrival order = execution order. (Previously each message was a detached `tokio::spawn` racing for the `processing` lock, so order was not guaranteed.)
2. **Live injection** — messages that arrive while a turn is running are **injected into the running turn** at its next round boundary (see [mid-turn injection](#mid-turn-injection-live-steering)), rather than waiting for a separate follow-up turn. Messages are kept as **individual** rows; coalescing for the LLM happens later in the `MessageBuilder` (merging consecutive user rows into one `role:user`), not in the inbox.
### Flow
```text
send_message(source, prompt, opts)
→ push QueuedMessage onto the source's inbox, notify, return immediately
(turn errors surface via the Error event on the bus, not the return value)
[per-source consumer task]
→ wait for notify (+ optional debounce window)
→ loop: build_unit(pending) → dispatch_turn(...) until the queue is empty
build_unit pops ONE message to seed a turn (no coalescing)
dispatch_turn = resolve session/handler, bridge events to the global bus,
inject execute_task, build the PendingUserInput handle (real user turns
only), call handler.handle_message (takes the processing lock)
```
The consumer holds the inbox lock **only** while building a unit (never during the turn). While a turn runs, the turn itself drains `pending` at each round boundary via the `PendingUserInput` handle, so new messages are injected live; only messages that arrive *after* the turn's last boundary remain in `pending` and seed the next turn on a following iteration. The consumer and the in-flight turn never touch `pending` concurrently — the consumer is parked awaiting `dispatch_turn`.
**Panic isolation** — each turn runs on its own `tokio::spawn`ed task whose `JoinHandle` the consumer awaits. A panic inside a turn therefore surfaces as a `JoinError` (logged as *"source turn panicked — consumer surviving"*) instead of unwinding the consumer task itself. Without this, a single panicking turn would silently kill the source's consumer: subsequent messages would enqueue and `notify()`, but nothing would ever drain them — the chat would appear frozen with no error. (Panics are also routed to the log file via a `std::panic::set_hook` in `main.rs`; the default hook only writes to stderr, i.e. the `run.sh` terminal.)
### Inbox helpers (`inbox.rs`)
- `build_unit(pending)` — pops a **single** message (no coalescing) to seed a turn. Empty queue → `None`.
- `drain_leading_user(pending)` — drains the leading run of consecutive **non-synthetic** messages, returning them individually; stops at the first **synthetic** message (`opts.is_synthetic` — notifications / TIC), which is left for the notification path. Used by the running turn (via `InboxUserInput: PendingUserInput`) to inject queued user input at a round boundary.
### Idle debounce
`SOURCE_COALESCE_DEBOUNCE_MS` (in `mod.rs`) defaults to **0**: a message to an idle source dispatches immediately. Raising it batches messages sent rapidly to an *idle* source, at the cost of that latency on the first message of a burst.
### `/stop` and `/clear`
- **`cancel(source)`** (`/stop`, stop button) clears the inbox's pending queue *and* cancels the in-flight turn (`handler.cancel()`). A `cancel_epoch` counter guards the tiny window where the consumer drained a unit microseconds before the stop — the stale unit is dropped instead of dispatched.
- **`clear(source)` / `provision_session(reset=true)`** (`/clear`, new conversation) also drops messages queued for the discarded session.
## Event bus
ChatHub owns a single global broadcast channel (`global_tx`, capacity 512). Every turn gets a fresh mpsc sender via `bridge_to_global`, which forwards the handler's `ServerEvent`s onto the global bus wrapped in a `GlobalEvent { source, session_id, event }`. Subscribers (`events(source)`) filter by source themselves — e.g. the WebSocket handler and the Telegram `persistent_forwarder`. `emit(...)` posts a sessionless event directly.
## Notifications (background → home source)
`notify(Notification)` / `notify_sync(Notification)` push a structured `Notification` (`src/core/notification.rs`: `{source, event_type, summary, event_time, refs}`) onto a central mpsc queue. The `notification_consumer` task batches bursts over `NOTIFY_BATCH_WINDOW_MS` (200 ms), then delivers them to the *home* source (`set_home` / `HOME_SOURCE_KEY`, default `web`) by appending a synthetic Assistant message with a pre-completed `read_notification` tool call (result = JSON array of `Notification` objects, `result_type='json'`) and calling `resume(...)`. This path uses `resume`, not `send_message`, so it does not go through the per-source inbox.
## API surface (`ChatHubApi` trait)
Defined in `crates/core-api/src/chat_hub.rs`, implemented on `ChatHub`:
| Method | Purpose |
|--------|---------|
| `send_message` | Enqueue a user message for a source (async; injected into an in-flight turn, or seeds a new one) |
| `register` | Register a source (no-op with the global bus) |
| `clear` | New session for the source, discard the previous one |
| `cancel` | Stop the in-flight turn + clear the queued backlog |
| `resume` | Resume an interrupted turn (pending tools / async result injection) |
| `reset_mcp` | Revoke all session-scoped tool-group grants (MCP servers + `config`); exposed to users as `/resettools` |
| `set_home` | Set which source receives background notifications |
| `context_info` / `cost_info` | Last-turn token usage / total session spend |
| `force_compact` | Force context compaction now |
| `events` | Subscribe to the global event bus |
| `resolve_question` | Answer a pending `ask_user_clarification` |
| `approve` / `reject` | Resolve a pending tool-call approval |
## Mid-turn injection (live steering)
A message sent while a turn is already running is delivered *into* that turn instead of waiting for it to finish. This works because the LLM loop rebuilds history fresh from the DB each round (`llm_loop.rs`), so a `user` row appended at a round boundary is picked up by the next round automatically.
- `dispatch_turn` builds an `InboxUserInput` (an `Arc<dyn PendingUserInput>` wrapping the `SourceInbox`) and passes it into `handle_message` → `run_agent_turn`. It is `Some` only for real user turns (never synthetic), and only the **root** turn receives it — sub-agents, resume, and non-interactive runners pass `None`.
- At the top of each round, `run_agent_turn` calls `drain_user()` and appends each queued message as its own `chat_history` `user` row, then emits a `UserMessage` event carrying the new `message_id` (telnet-style echo — see [frontend.md](frontend.md)). A round boundary is the only clean ordering point: the previous round's assistant message and its tool results are all persisted, so a `user` row appended there is well-ordered (never between an assistant tool-call and its tool results).
- Injection does **not** interrupt the in-flight LLM call or tool, and does **not** reset the round budget (`max_tool_rounds`).
- `MessageBuilder` merges consecutive non-failed `user`/`agent` rows into one `role:user` for the LLM, so several injected messages read as a single clean user turn while the DB keeps them distinct.
- `/stop` clears `pending` (`clear_inbox`): queued-but-not-yet-injected messages are dropped, never persisted, never echoed.
## Relevant files
| Path | Role |
|------|------|
| `src/core/chat_hub/mod.rs` | `ChatHub`: API, dispatch, event bridge, notification consumer, per-source consumer |
| `src/core/chat_hub/inbox.rs` | `SourceInbox`, `QueuedMessage`, `build_unit` (single pop) + `drain_leading_user` (mid-turn injection) + tests |
| `src/core/db/sources.rs` | `source → active_session_id` mapping |
| `crates/core-api/src/chat_hub.rs` | `ChatHubApi` trait + `SendMessageOptions` |
| `src/core/session/handler/` | `ChatSessionHandler` — the turn itself (see [session.md](session.md)) |
## When to update this file
- New `ChatHubApi` methods or changed dispatch flow
- Changes to inbox draining (`build_unit` / `drain_leading_user`) or the debounce constant
- Changes to `/stop` / `/clear` queue semantics
- Changes to mid-turn injection (round-boundary drain, `PendingUserInput`)
docs/comfyui-workflow-format.md
+173
View File
@@ -0,0 +1,173 @@
# ComfyUI Workflow Format — Agent Guide
Guide for receiving a ComfyUI workflow file (e.g. via Telegram attachment),
understanding it, adding the required metadata, and saving it as an image
generation provider.
---
## JSON API Format Structure
When exporting a workflow from ComfyUI with "Save (API Format)", the result is
a JSON where every **numeric** key is a pipeline node:
```json
{
"3": { "class_type": "KSampler", "inputs": { "steps": 20, "cfg": 7, "seed": 42, ... } },
"4": { "class_type": "CheckpointLoaderSimple", "inputs": { "ckpt_name": "v1-5-pruned.safetensors" } },
"6": { "class_type": "CLIPTextEncode", "inputs": { "text": "", "clip": ["4", 1] } },
"7": { "class_type": "CLIPTextEncode", "inputs": { "text": "", "clip": ["4", 1] } },
"8": { "class_type": "EmptyLatentImage", "inputs": { "width": 512, "height": 512, "batch_size": 1 } },
"9": { "class_type": "SaveImage", "inputs": { "filename_prefix": "ComfyUI", "images": ["10", 0] } },
"10": { "class_type": "VAEDecode", "inputs": { "samples": ["3", 0], "vae": ["4", 2] } }
}
```
**Key rules:**
- Numeric keys are node IDs. They need not be consecutive.
- **Non-numeric** keys (e.g. `_personal_agent`) are ignored by ComfyUI
and used only by the plugin.
- Array values in `inputs` (e.g. `["4", 1]`) are links to other nodes:
`[node_id, output_slot]`.
---
## Relevant Nodes
| `class_type` | Field to modify | Description |
|---|---|---|
| `CLIPTextEncode` | `inputs.text` | Text prompt (positive or negative) |
| `CLIPTextEncodeSD3` | `inputs.clip_l`, `inputs.clip_g`, `inputs.t5xxl` | SD3.5 text prompt — all three fields must be populated |
| `KSampler` | `inputs.steps` | Number of diffusion steps |
| `KSampler` | `inputs.cfg` | CFG scale (creativity vs. prompt adherence) |
| `KSampler` | `inputs.seed` | Seed for reproducibility |
| `EmptyLatentImage` | `inputs.width` | Image width in pixels |
| `EmptyLatentImage` | `inputs.height` | Image height in pixels |
| `CheckpointLoaderSimple` | `inputs.ckpt_name` | SD model name to use |
| `LoraLoader` | `inputs.lora_name` | LoRA to apply |
| `SaveImage` | `inputs.filename_prefix` | Output filename prefix |
---
## The `_personal_agent` Block
Add this as a non-numeric key to the JSON to configure the provider.
### Full Schema
```json
"_personal_agent": {
"name": "Workflow Name",
"description": "Description for the agent: style, format, ideal use cases.",
"prompt_node": "6",
"negative_prompt_node": "7",
"prompt_field": "clip_l",
"prompt_field_extra": ["clip_g", "t5xxl"],
"negative_prompt_field": "clip_l",
"negative_prompt_field_extra": ["clip_g", "t5xxl"],
"extra_params": {
"width_node": "8",
"height_node": "8",
"steps_node": "3"
}
}
```
| Field | Required | Notes |
| ----- | :------: | ----- |
| `name` | ✓ | Name shown in the provider listing |
| `description` | — | Free text for the agent: style, default dimensions, use cases |
| `prompt_node` | ✓ | ID of the node for the positive prompt (`CLIPTextEncode` or `CLIPTextEncodeSD3`) |
| `negative_prompt_node` | — | ID of the node for the negative prompt |
| `prompt_field` | — | Input field to inject the prompt into. Default: `"text"`. For SD3.5: `"clip_l"` |
| `prompt_field_extra` | — | Additional input fields to copy the prompt into. For SD3.5: `["clip_g", "t5xxl"]` |
| `negative_prompt_field` | — | Input field for the negative prompt. Default: `"text"` |
| `negative_prompt_field_extra` | — | Additional input fields for the negative prompt |
| `extra_params.width_node` | — | ID of the node with `inputs.width` (usually `EmptyLatentImage`) |
| `extra_params.height_node` | — | ID of the node with `inputs.height` |
| `extra_params.steps_node` | — | ID of the node with `inputs.steps` (usually `KSampler`) |
If `prompt_node` is omitted, the plugin heuristically picks the first
`CLIPTextEncode` or `CLIPTextEncodeSD3` node found (ascending numeric ID order).
---
## Identifying the Correct Nodes
To find the right node IDs by reading the JSON:
1. **Positive prompt** — find nodes with `"class_type": "CLIPTextEncode"` or
`"CLIPTextEncodeSD3"`. There are usually two: one for the positive prompt
(empty or descriptive text) and one for the negative. Conventionally the
positive one has the lower ID.
2. **Dimensions** — find `"class_type": "EmptyLatentImage"`. Read
`inputs.width` and `inputs.height` to know the workflow's default dimensions.
3. **Steps** — find `"class_type": "KSampler"`. The `inputs.steps` field is
the number of diffusion steps.
### Example: reading defaults for `extra_params_schema`
Given the node:
```json
"8": { "class_type": "EmptyLatentImage", "inputs": { "width": 768, "height": 1024 } }
```
The plugin will automatically generate:
```json
"extra_params_schema": {
"properties": {
"width": { "type": "integer", "default": 768 },
"height": { "type": "integer", "default": 1024 }
}
}
```
---
## Recommended Editing Workflow
1. **Receive the file** — Telegram attachment, web upload, or local path.
2. **Read the JSON** and identify:
- The `CLIPTextEncode` node for the positive prompt (lowest ID among those present).
- The `CLIPTextEncode` node for the negative prompt (if present).
- The `EmptyLatentImage` node for dimensions.
- The `KSampler` node for steps.
3. **Add `_personal_agent`** with the discovered node IDs and a meaningful
description. Example for a landscape 1024×512 workflow:
```json
"_personal_agent": {
"name": "Landscape XL",
"description": "Landscapes and horizontal scenes. Default 1024x512. Great for backgrounds and scenery.",
"prompt_node": "6",
"negative_prompt_node": "7",
"extra_params": {
"width_node": "8",
"height_node": "8",
"steps_node": "3"
}
}
```
4. **Save to** `data/comfyui/workflows/<name>.json`.
The filename becomes the provider ID: `landscape-xl.json` →
provider `comfyui-landscape-xl`.
5. **The watcher detects the file within 5 s** and registers the provider.
Verifiable by calling `image_generate_providers_list`.
---
## Notes on Workflows Without `_personal_agent`
If the file does not contain a `_personal_agent` block, the plugin:
- Uses the filename as the provider name.
- Heuristically searches for the first `CLIPTextEncode` or `CLIPTextEncodeSD3` node for the prompt.
- Registers the provider without `description` or `extra_params_schema`.
- If no `CLIPTextEncode` or `CLIPTextEncodeSD3` node is found, **skips the file** with a warning.
Adding `_personal_agent` is always preferred to give the agent the context
needed to pick the right provider.
docs/commands.md
+101
View File
@@ -0,0 +1,101 @@
# Custom slash commands
User-defined `/command` shortcuts that expand a **prompt template** into a normal
user message on the interactive session. They let you package a long, reusable
instruction behind a short command (`/deeploop`, `/review`, …) — the template is
interpolated with whatever the user typed and handed to the model as if the user had
written it out in full.
Because the expansion becomes an ordinary user turn on the `main` session, a command
is **fully interactive**: after it fires the model can ask follow-up questions,
iterate with the user, write files, and dispatch sub-agents — exactly like any other
turn. A command is not a one-off request.
## File layout (mirrors `agents/`)
Each command is a directory under `commands/`:
```
commands/<name>/meta.json # manifest
commands/<name>/COMMAND.md # the prompt template
```
- `<name>` is the directory name and the token typed after `/` (e.g. `commands/echo/`
→ `/echo`). Use lowercase `[a-z0-9_-]`.
- `meta.json`:
- `description` (required) — one line, shown in `/help` and the composer autocomplete.
- `enabled` (optional, default `true`) — set `false` to hide a command without deleting it.
- `COMMAND.md` — the template body. `{{args}}` (or `{{prompt}}`) is replaced with the
user's arguments. If neither placeholder appears, the arguments are appended after a
`---` separator (opencode-like default).
Files are read at **request time**, so edits to `meta.json` / `COMMAND.md` take effect
without a restart (same as `agents/*/AGENT.md`). There is no DB table and no CRUD API —
you create a command by adding files, just like an agent. A missing `commands/`
directory is fine (no custom commands).
## Precedence
Hard-coded **system** commands (`/help`, `/clear`, `/new`, `/model`, `/models`,
`/context`, `/cost`, `/compact`, `/resettools`, `/sethome`, `/stop`) always win: they
are matched first in each surface's command handler (the WS handler for web/mobile,
the Telegram plugin handler), so a custom command with a colliding name is
unreachable and is filtered out of `/help` and the autocomplete. An unrecognised `/…`
is rejected ("Unknown command") and never forwarded to the LLM.
## Dual view (typed command vs. expanded template)
The history row's `content` stores the **expanded template** (what the model replays);
the UI shows the **typed command** (`/echo ciao`), never the template. This is carried
by `MessageMetadata.command` (`CommandRef { name, display }`), persisted on the user
turn next to `attachments`:
- The backend substitutes `content = command.display` when emitting the `user_message`
echo (`emitter.user_message`, both call sites in `llm_loop.rs` / `session/handler/mod.rs`)
and when serialising history (`api/sessions.rs`).
- The frontend is **telnet-style**: custom commands are *not* echoed optimistically on
send — the bubble appears only when the backend re-emits the `user_message` carrying
the typed form. Only system commands (which reply with a `Done` and never echo back)
are rendered optimistically. See `SYSTEM_SLASH_COMMANDS` in `web/lib/chat-session.js`.
## Composer autocomplete
Typing `/` in the copilot composer opens a dropdown: built-in system commands first,
then custom commands fetched from `GET /api/commands`. Filter by prefix, navigate with
`↑`/`↓`, accept with `Enter`/`Tab` (inserts `/name `), dismiss with `Esc`.
(`web/components/copilot.js`; styles in `web/css/copilot-input.css`.)
## Implementation map
| Concern | Location |
| --- | --- |
| Capability trait + DTOs + expansion | `crates/core-api/src/command.rs` (`CommandApi`, `CommandInfo`, `ResolvedCommand`, `expand_template`) |
| Discovery / resolve / expand | `src/core/command/mod.rs` (`LlmCommandManager: CommandApi`, owned by `Skald` as `Arc<…>`) |
| Metadata dual-view type | `crates/core-api/src/message_meta.rs` (`CommandRef`, `MessageMetadata.command`) |
| Matching + dynamic `/help` (web + mobile) | `src/frontend/api/ws.rs` (`dynamic_help`, custom-command interposition) |
| Matching + dynamic `/help` (Telegram) | `crates/plugin-telegram-bot/src/handlers.rs` (`help_text`, fallback-command arm; resolves via `ctx.command`) |
| Plugin wiring | `PluginContext.command` (`crates/core-api/src/plugin.rs`) → `TgShared.command` |
| Echo `content = display` | `src/core/session/handler/{llm_loop.rs, mod.rs}`, `src/frontend/api/sessions.rs` (applies to every surface: web, mobile, Telegram) |
| Listing endpoint | `GET /api/commands` → `src/frontend/api/commands.rs` |
| Autocomplete + telnet gate | `web/components/copilot.js`, `web/lib/chat-session.js` |
## Example
`commands/echo/meta.json`:
```json
{ "description": "Test command — echoes your input", "enabled": true }
```
`commands/echo/COMMAND.md`:
```
Repeat the user's input back verbatim.
User input:
{{args}}
```
Typing `/echo hello world` runs a turn where the model receives
`Repeat the user's input back verbatim.\n\nUser input:\nhello world`, while the chat
bubble shows `/echo hello world`.
docs/compaction.md
+297
View File
@@ -0,0 +1,297 @@
# Context Compaction
## Overview
As a conversation grows, the LLM context window fills up with old messages that
are no longer directly relevant. This increases latency and cost, and eventually
hits the model's token limit.
**Context compaction** solves this by periodically summarising the older portion
of the history into a dense text block. Only the summary and the most recent raw
messages are sent to the LLM on subsequent turns.
The feature is **opt-in** via `config.yml` and is **disabled by default**.
---
## Architecture
```
┌──────────────────────────────────────────────────────────────────┐
│ handle_message() │
│ 1. Check last_input_tokens > threshold │
│ 2. Call ContextCompactor::try_compact() ←── new │
│ 3. Run normal LLM loop (build_openai_messages injects summary)│
│ 4. Store input_tokens in last_input_tokens ←── new │
└──────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ ContextCompactor src/compactor.rs │
│ ────────────────────────────────────────────────────────────── │
│ • Stateless service, shared via Arc across all sessions │
│ • System prompt hard-coded (not an AGENT.md agent) │
│ • Uses LlmManager::resolve(strength) for AUTO model selection │
│ • Persists result to chat_summaries DB table │
│ • Publishes BusEvent::CompactionDone to ChatEventBus │
└──────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ DB: chat_summaries src/db/chat_summaries │
│ id │ stack_id │ content │ covers_up_to_message_id │ created_at │
└──────────────────────────────────────────────────────────────────┘
│ read by
▼
┌──────────────────────────────────────────────────────────────────┐
│ build_openai_messages() src/session/handler/ │
│ messages.rs │
│ if summary exists: │
│ • inject <conversation_summary> after system prompt │
│ • load only messages with id > covers_up_to_message_id │
│ else: load all (current behaviour) │
│ if compaction disabled: apply max_history_messages as fallback │
└──────────────────────────────────────────────────────────────────┘
│ publishes
▼
┌──────────────────────────────────────────────────────────────────┐
│ ChatEventBus: broadcast<BusEvent> src/chat_event_bus.rs │
│ BusEvent::UserMessage(ChatEvent) │
│ BusEvent::AssistantResponse(ChatEvent) │
│ BusEvent::CompactionDone(CompactionEvent) ←── new │
└──────────────────────────────────────────────────────────────────┘
│ consumed by
├── HonchoPlugin (UserMessage, AssistantResponse only)
└── (future consumers: CompactionDone)
```
---
## Trigger Strategy — Opzione C
Compaction is checked **at the start of each `handle_message` call**, using
the `input_tokens` value from the **previous** turn (stored in
`last_input_tokens: AtomicU32` on `ChatSessionHandler`).
This means:
- Turn N uses many tokens → `last_input_tokens` is stored after turn N.
- Turn N+1 starts → compaction is triggered **before** the LLM runs.
- The user waits for compaction + the new turn, but sees a single response.
No background task, no lock contention, no concurrency hazard.
### Skipped cases
| Condition | Behaviour |
|---|---|
| `compaction` absent from config | Feature disabled entirely |
| `is_ephemeral = true` (cron, tic) | Skipped — sessions are short-lived |
| `last_input_tokens == 0` (first turn or no usage data) | Character estimate used as fallback |
| Fewer messages than `keep_recent` past the summary boundary | Nothing to summarise, skipped |
| LLM returns empty summary | Skipped, warning logged |
### Manual trigger
Compaction can also be triggered **manually** via `ChatSessionHandler::force_compact()` or
`ChatHub::force_compact(source_id)`. The manual path (`force_compact` on
`ContextCompactor`) skips the threshold check entirely and uses a character-based
token estimate, but still respects the ephemeral guard.
A Telegram `/compact` command is available as a user-facing interface; see
[docs/telegram.md](telegram.md).
---
## Compaction Flow
```
try_compact(pool, session_id, stack_id, last_input_tokens, is_ephemeral)
│
├─ guard: is_ephemeral → Ok(false)
├─ resolve effective_tokens:
│ if last_input_tokens > 0 → use it
│ else → estimate_tokens_for_stack (sum of chars / 4)
├─ guard: effective_tokens < threshold → Ok(false)
│
├─► do_compact(pool, session_id, stack_id, effective_tokens)
│
│ (see below)
│
└─ Ok(true/false)
force_compact(pool, session_id, stack_id, is_ephemeral)
│
├─ guard: is_ephemeral → Ok(false)
├─ effective_tokens = estimate_tokens_for_stack() ← no threshold check
│
├─► do_compact(pool, session_id, stack_id, effective_tokens)
│
└─ Ok(true/false)
do_compact(pool, session_id, stack_id, effective_tokens)
│
├─ latest_summary = chat_summaries::latest_for_stack()
├─ messages = if latest_summary:
│ for_stack_since(covers_up_to_message_id)
│ else:
│ for_stack()
│
├─ guard: messages.len() <= keep_recent → Ok(false)
│
├─ to_summarise = messages[0 .. len - keep_recent]
├─ last_covered_id = to_summarise.last().id
│
├─ full_prompt = format_for_summary(
│ messages = to_summarise,
│ prior_summary = latest_summary.content (if any)
│ )
│ → returns a single string (SUMMARIZER_PREAMBLE + transcript + SUMMARY_TEMPLATE)
│ → if prior summary exists: iterative-update path ("PREVIOUS SUMMARY: …")
│ → if first compaction: "Create a structured checkpoint summary…"
│ → transcript uses Hermes-style labels:
│ [USER]: …
│ [ASSISTANT]: … [Tool calls: name(args)]
│ [TOOL RESULT tc_N]: …
│ with head+tail truncation (6 000+1 500 for messages, 4 000+1 500 for results)
│
├─ (client_name, llm) = LlmManager::resolve(None, None, config.strength)
├─ call llm.chat_with_tools([{role: "user", content: full_prompt}], tools=[], options)
│ options.temperature = 0.3 (faithful, low-creativity)
│ (Hermes-style: everything in a single user message, no separate system message)
│
├─ summary_id = chat_summaries::save(stack_id, summary_text, last_covered_id)
├─ event_bus.compaction_done(CompactionEvent { ... })
└─ Ok(true)
```
---
## Database Schema
```sql
CREATE TABLE chat_summaries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
stack_id INTEGER NOT NULL REFERENCES chat_sessions_stack(id),
content TEXT NOT NULL,
-- All chat_history rows with id <= this value are covered.
-- build_openai_messages loads only rows with id > this value.
covers_up_to_message_id INTEGER NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX idx_chat_summaries_stack ON chat_summaries (stack_id);
```
**One active summary per stack** — multiple rows can exist (each compaction
creates a new one), and the most recent is always used. There are no nested
summaries: when a second compaction runs, the prior summary body is fed to the
LLM under `PREVIOUS SUMMARY:` in the iterative-update path, and the new row
supersedes it.
---
## LLM Selection
The compactor uses `LlmManager::resolve(None, None, config.strength)` — the
same AUTO selection used for agents. `strength` maps to the existing tier
system:
| Strength | Typical use |
|---|---|
| `very_low` / `low` | Fastest, cheapest local model |
| `average` | Recommended for summaries |
| `high` / `very_high` | Overkill for summarisation |
The compaction LLM is **not** a registered agent — its prompt constants are
hard-coded in `src/core/compactor.rs` and are not configurable from `agents/` or
AGENT.md files:
| Constant | Role |
| --- | --- |
| `pub const SUMMARY_PREFIX` | Handoff header prepended to the summary when injected as context. Tells the LLM "reference only — resume from `## Active Task`". Also used in `messages.rs`. |
| `SUMMARIZER_PREAMBLE` | Opening of the compaction prompt. Plain wording to avoid content-filter false positives on Azure/OpenAI-compatible providers. |
| `SUMMARY_TEMPLATE` | 13-section structured template the LLM must fill. Ported from [Hermes agent](https://github.com/NousResearch/hermes-agent). |
---
## ChatEventBus Extension
The `ChatEventBus` was extended from `broadcast<ChatEvent>` to
`broadcast<BusEvent>`, a new enum:
```rust
pub enum BusEvent {
UserMessage(ChatEvent),
AssistantResponse(ChatEvent),
CompactionDone(CompactionEvent), // new
}
```
Existing consumers (`honcho` plugin) were updated to match on
`BusEvent::UserMessage` / `BusEvent::AssistantResponse` and ignore
`CompactionDone`. Future consumers can subscribe and react to compaction
events (e.g. to flush external memory, reset embeddings, etc.).
---
## Configuration
```yaml
# config.yml (under the llm: section)
llm:
# ... existing settings ...
compaction:
# Trigger compaction when the previous turn used more than this many
# input tokens. Required.
threshold_tokens: 30000
# Number of recent raw messages to keep outside the summary. Default: 6.
keep_recent: 6
# Minimum LLM strength for summary generation (AUTO selection).
# Summaries are simple writing tasks — low or average is sufficient.
# Omit to use whatever AUTO picks.
strength: low
```
### Recommended values by use case
| Scenario | `threshold_tokens` | Notes |
|---|---|---|
| Local 8k model (LM Studio) | 5 000 – 6 000 | Compact early and often |
| Local 32k model | 20 000 – 25 000 | Leave room for the summary itself |
| Claude Sonnet (200k) | 100 000 – 150 000 | Or omit compaction entirely |
| Claude Haiku (200k) | 80 000 | Cheaper per call; compact less often |
### Provider without token usage (e.g. some LM Studio setups)
If the LLM provider does not return `input_tokens` in the response, the
compactor falls back to estimating token usage as `total_chars / 4`. Set a
lower `threshold_tokens` when relying on this estimate since it underestimates
non-ASCII content.
---
## Known Limitations
- **Sub-agent stacks**: compaction applies only to the root session stack
(`depth = 0`). Sub-agent stacks are typically short-lived and do not benefit
from compaction.
- **Tool results in summary**: the serialiser uses a head+tail strategy —
message bodies are truncated to 6 000+1 500 chars, tool results to
4 000+1 500 chars, tool-call arguments to 1 200 chars. Very long outputs
are preserved at both ends; only the middle is replaced with `...[truncated]...`.
- **Tool result pruning in live context**: `maybe_hide_tool_result` replaces
oversized previous-turn results with an informative 1-line summary
(e.g. `[execute_cmd] ran \`cargo build\` → exit 0, 47 lines output`) rather
than a generic "hidden" placeholder. No LLM call — pure string parsing.
- **Frontend visibility**: the compaction step is transparent to the user.
No `Compacting` event is sent to the WebSocket. The `BusEvent::CompactionDone`
event is available on the internal bus for future subscribers.
- **Cold restart**: `last_input_tokens` is stored in memory (`AtomicU32`), not
in the DB. After a restart, the first turn of a session will not trigger
compaction even if the history is already long. The second turn will trigger
it correctly once the LLM reports usage.
docs/crates/index.md
+9
View File
@@ -0,0 +1,9 @@
# Workspace Crates
Independent library crates in the workspace. Extracted plugins depend only on `core-api` and external crates.
## Files
- [workspace.md](workspace.md) — Workspace structure, crate catalogue, `core-api` module reference, plugin extraction roadmap
See [../index.md#infrastructure--security](../index.md#infrastructure--security) for navigation.
docs/crates/workspace.md
+306
View File
@@ -0,0 +1,306 @@
# Workspace Crates
Independent library crates in `crates/`. None depend on the main `skald` binary crate.
---
## `core-api` — `crates/core-api/`
Shared contract types and traits used by both the main crate and future independent plugin crates.
### Modules
| Module | Contents |
| --- | --- |
| `core_api::chatbot` | `ChatbotClient` trait, `Message`, `Role`, `ChatOptions`, `ChatResponse`, `LlmTurn`, `ToolCall`, `LlmRawMeta` |
| `core_api::provider` | `ApiProvider` trait, `ApiProviderRegistry` trait, `ProviderUiMeta`, `ProviderField`, `ServiceType`, `BuiltLlmClient`; DB record types: `LlmProviderRecord`, `LlmModelRecord`, `LlmStrength`, `RemoteLlmModelInfo` |
| `core_api::tts` | `TextToSpeech` trait, `TtsProvider`, `TtsRegistry`; `TtsModelRecord`, `RemoteTtsModelInfo` |
| `core_api::transcribe` | `Transcribe` trait, `TranscribeProvider`, `TranscribeRegistry`; `TranscribeModelRecord`, `RemoteTranscribeModelInfo` |
| `core_api::image_generate` | `ImageGenerate` trait, `ImageGenerateRegistry`; `ImageGenerateModelRecord` |
| `core_api::events` | `ServerEvent`, `GlobalEvent`, `ClientMessage`, `InboundDataMessage` |
| `core_api::bus` | `ChatEventBus` — in-process broadcast for completed turns |
| `core_api::interface_tool` | `InterfaceTool`, `ToolFuture` — LLM-callable tools injected by interfaces |
| `core_api::chat_hub` | `SendMessageOptions`, `ChatHubApi` trait |
| `core_api::location` | `GpsCoord`, `LocationEntry`, `LocationManager`; `LocationUpdater` trait |
| `core_api::tool` | `Tool` trait, `ToolCategory`, `ToolDescriptionLength`, `truncate_label` |
| `core_api::memory` | `Memory` trait — pluggable long-term memory backend contract |
| `core_api::remote` | `RemoteAccess` trait — mesh/remote-connectivity provider contract |
| `core_api::approval` | `ApprovalApi` trait — resolve pending tool-call approvals |
| `core_api::inbox` | `InboxApi` trait + `InboxSnapshot`/`InboxApprovalItem`/`InboxClarificationItem` — unified approvals + clarifications façade |
| `core_api::plugin` | `Plugin` trait, `PluginContext`, `RouterFactory` — plugin lifecycle contract and dependency bag |
### `PluginContext` fields (dependency bag)
`PluginContext` (`crates/core-api/src/plugin.rs`) carries the deps a plugin may need. Notable fields:
| Field | Type | Purpose |
| --- | --- | --- |
| `chat_hub` | `Arc<dyn ChatHubApi>` | Send messages; `events()` subscribes to the global `GlobalEvent` bus |
| `approval` | `Arc<dyn ApprovalApi>` | Resolve tool-call approvals |
| `inbox` | `Arc<dyn InboxApi>` | Unified approvals + clarifications: `list_pending`, `approve`, `reject`, `answer` (idempotent by `request_id`) |
| `db` | `Arc<sqlx::SqlitePool>` | Skald's shared SQLite pool — plugins create/use their own tables (e.g. `relay_*`) here |
| `event_bus` / `system_bus` | `Arc<ChatEventBus>` / `Arc<SystemEventBus>` | In-process buses |
| `web_port` / `remote_slot` / `router_factory` | — | Networking deps (mesh plugins) |
### Plugin HTTP routes (`Plugin::http_router`)
A plugin may contribute one `axum::Router` by overriding `fn http_router(&self) -> Option<axum::Router>` (default `None`, so existing plugins are unaffected). After `start_enabled()`, `WebFrontend::start` calls `PluginManager::collect_plugin_routers()` and nests each enabled plugin's router under `/api/plugin/<id>/`, behind Skald's normal auth. The router must close over the plugin's own state (it receives no axum `State`). Only routes are supported — no nav entries, JS assets, or Lit components (see plugin.md §12.3). The mesh-facing router built by `router_factory` does **not** include plugin routes.
### `ChatHubApi` trait
Defines the surface a plugin needs to interact with the agent system:
```rust
#[async_trait]
pub trait ChatHubApi: Send + Sync {
async fn register(&self, source_id: &str);
async fn send_message(&self, source_id: &str, prompt: &str, opts: SendMessageOptions) -> anyhow::Result<()>;
async fn clear(&self, source_id: &str) -> anyhow::Result<i64>;
fn events(&self, source_id: &str) -> broadcast::Receiver<GlobalEvent>;
async fn set_home(&self, source_id: &str) -> anyhow::Result<()>;
async fn context_info(&self, source_id: &str) -> anyhow::Result<(Option<i64>, Option<i64>)>;
async fn force_compact(&self, source_id: &str) -> anyhow::Result<bool>;
async fn resume(&self, source_id: &str) -> anyhow::Result<()>;
async fn approve(&self, request_id: i64);
async fn reject(&self, request_id: i64, note: String);
async fn resolve_question(&self, source_id: &str, request_id: i64, answer: String);
}
```
`ChatHub` in `src/core/chat_hub/mod.rs` implements this trait. To call trait methods on `Arc<ChatHub>`, import the trait: `use crate::chat_hub::ChatHubApi as _;`.
### `InterfaceTool`
```rust
pub struct InterfaceTool {
pub definition: Value, // OpenAI tool definition
pub handler: Arc<dyn Fn(Value) -> ToolFuture + Send + Sync>,
}
```
Interface tools are injected per-turn via `SendMessageOptions::interface_tools`. They are only visible to the root agent — sub-agents do not inherit them (except `activate_tools` which is re-injected explicitly).
---
## Plugin Extraction Roadmap
The goal is to allow plugins to live in their own workspace crates without depending on the full main binary. All plugins depend only on `core-api` and external crates.
### Extracted plugins
| Plugin | Crate | Doc |
| --- | --- | --- |
| `honcho` | `crates/plugin-honcho/` | [honcho.md](honcho.md) |
| `remote_connectivity` | `crates/plugin-tailscale-remote/` | [remote.md](remote.md) |
| `whisper_local` | `crates/plugin-transcribe-whisper-local/` | [whisper-local.md](whisper-local.md) |
| `telegram` | `crates/plugin-telegram-bot/` | [telegram.md](telegram.md) |
| `orpheus_tts_3b` | `crates/plugin-tts-orpheus-3b/` | [tts-providers.md](tts-providers.md) |
| `kokoro_tts` | `crates/plugin-tts-kokoro/` | [tts-providers.md](tts-providers.md) |
| `elevenlabs` | `crates/plugin-elevenlabs/` | [tts-providers.md](tts-providers.md) |
| `mobile-connector` | `crates/plugin-mobile-connector/` | [mobile-connector.md](mobile-connector.md) |
### Remaining in main crate
All plugins have been extracted to independent workspace crates. ElevenLabs (TTS + transcription) was extracted into `crates/plugin-elevenlabs/` — it registers itself as an `ApiProvider` so the existing `llm_providers` + `tts_models` / `transcribe_models` UI continues to work unchanged.
### All `core-api` contracts needed by plugins
| Dependency | Status |
| --- | --- |
| `core_api::chatbot::ChatbotClient` (+ associated types) | ✅ In `core-api` |
| `core_api::provider::{ApiProvider, ApiProviderRegistry, LlmProviderRecord, …}` | ✅ In `core-api` |
| `core_api::tts::{TextToSpeech, TtsProvider, TtsRegistry, TtsModelRecord, …}` | ✅ In `core-api` |
| `core_api::transcribe::{Transcribe, TranscribeProvider, TranscribeRegistry, TranscribeModelRecord, …}` | ✅ In `core-api` |
| `core_api::image_generate::{ImageGenerate, ImageGenerateRegistry, ImageGenerateModelRecord}` | ✅ In `core-api` |
| `core_api::events::{ServerEvent, GlobalEvent}` | ✅ In `core-api` |
| `core_api::interface_tool::InterfaceTool` | ✅ In `core-api` |
| `core_api::chat_hub::{ChatHubApi, SendMessageOptions}` | ✅ In `core-api` |
| `core_api::location::{GpsCoord, LocationManager, LocationUpdater}` | ✅ In `core-api` |
| `core_api::remote::RemoteAccess` | ✅ In `core-api` |
| `core_api::plugin::{Plugin, PluginContext, RouterFactory}` | ✅ In `core-api` |
| `core_api::bus::{BusEvent, ChatEvent, ChatEventRole, RecvError}` | ✅ In `core-api` |
| `core_api::memory::Memory` | ✅ In `core-api` |
| `core_api::tool::{Tool, ToolCategory}` | ✅ In `core-api` |
| `core_api::approval::ApprovalApi` | ✅ In `core-api` |
| `core_api::inbox::{InboxApi, InboxSnapshot, …}` | ✅ In `core-api` |
---
## Decoupling Pattern — OnceLock extraction
When a plugin cannot receive its typed deps at construction time (because `Skald` is built after plugin registration), use `std::sync::OnceLock` to extract and name the deps on first `start()`:
```rust
pub struct MyPlugin {
// named, typed deps — no Arc<Skald>
chat_hub: OnceLock<Arc<dyn ChatHubApi>>,
some_config: OnceLock<u16>,
}
fn extract_deps(&self, ctx: &PluginContext) {
let _ = self.chat_hub.set(Arc::clone(&ctx.chat_hub));
let _ = self.some_config.set(ctx.web_port);
}
async fn start(&self, ctx: PluginContext) -> Result<()> {
self.extract_deps(&ctx);
self.do_start().await // no Skald needed here
}
```
`OnceLock::set` is idempotent — safe across multiple `reload()` calls. The values must be stable for the process lifetime (config values, `Arc` handles to singletons).
`RemotePlugin` (`crates/plugin-tailscale-remote/src/lib.rs`) uses this pattern with three deps: `port`, `remote_slot`, and `router_factory` — all sourced from `PluginContext`.
---
## `llm-client` — `crates/llm-client/`
Concrete LLM provider implementations: OpenAI-compatible, native Anthropic, Ollama, LmStudio.
Depends on `core-api` — `ChatbotClient` and all associated types (`Message`, `Role`, `ChatOptions`, `ChatResponse`, `LlmTurn`, `ToolCall`, `LlmRawMeta`) are defined there and re-exported from `llm-client` for backward compatibility.
Utility functions that depend on `reqwest` (`headers_to_json`, `redact_key`) remain in `llm-client` and are not part of `core-api`.
---
## `mcp-client` — `crates/mcp-client/`
MCP (Model Context Protocol) client over stdio and SSE transports. Used by `McpManager`.
---
## `honcho-client` — `crates/honcho-client/`
HTTP client for the Honcho long-term memory service. Used by `crates/plugin-honcho/`.
---
## `plugin-honcho` — `crates/plugin-honcho/`
Independent plugin crate for the Honcho long-term memory integration. Depends only on `core-api` and `honcho-client`. See [honcho.md](honcho.md).
---
## `plugin-tailscale-remote` — `crates/plugin-tailscale-remote/`
Independent plugin crate that exposes the web app on a Tailscale mesh network. Depends only on `core-api` and external crates (`tailscale`, `axum`, `tokio`, …). See [remote.md](remote.md).
Contains three modules:
| Module | Contents |
| --- | --- |
| `lib.rs` | `RemotePlugin` — plugin lifecycle, provider selection |
| `tailscale_sys.rs` | `TailscaleSystemProvider` — reads IP from system `tailscaled` daemon |
| `tailscale.rs` | `TailscaleEmbeddedProvider` — embedded netstack via `tailscale-rs` (feature-gated) |
Feature flags (in `crates/plugin-tailscale-remote/Cargo.toml`):
```toml
[features]
default = ["remote-tailscale"]
remote-tailscale = ["dep:tailscale"]
```
---
## `plugin-transcribe-whisper-local` — `crates/plugin-transcribe-whisper-local/`
Independent plugin crate providing local Speech-to-Text via whisper.cpp (Metal-accelerated on Apple Silicon). Depends only on `core-api`, `whisper-rs`, and `hound`. See [whisper-local.md](whisper-local.md).
`whisper-rs` and `hound` live exclusively in this crate — the main binary no longer depends on them directly.
### Key types
| Type | Role |
| --- | --- |
| `WhisperLocalPlugin` | `Plugin` impl — manages model lifecycle and registers/deregisters `WhisperLocalTranscriber` |
| `WhisperLocalTranscriber` | `Transcribe` impl — lightweight handle passed to `TranscribeManager` at `start()` |
Audio is converted to 16 kHz mono WAV via `ffmpeg` before being fed to whisper.cpp. Model must be a GGML `.bin` file; path is configured via the plugins REST API.
---
## `plugin-telegram-bot` — `crates/plugin-telegram-bot/`
Independent plugin crate for the private Telegram bot interface. Depends only on `core-api`, `teloxide`, and supporting crates (`tokio-util`, `chrono`, `rand`, `regex`). See [telegram.md](telegram.md).
`teloxide` and `tokio-util` live exclusively in this crate — the main binary no longer depends on them directly. The name `plugin-telegram-bot` distinguishes a bot-account integration from a potential future userbot (personal account) plugin.
### Source modules
| Module | Contents |
| --- | --- |
| `lib.rs` | `TelegramPlugin` — plugin lifecycle, bot startup, dispatcher wiring; `TgShared` holds `Arc<dyn TtsProvider>` |
| `events.rs` | `persistent_forwarder` — subscribes to ChatHub events and forwards to Telegram; `callback_handler` — inline keyboard button presses |
| `handlers.rs` | `message_handler`, `edited_message_handler` — incoming message classification and dispatch |
| `auth.rs` | `WhitelistFile`, pairing flow, `whitelist_watchdog` |
| `attachments.rs` | `TelegramAttachment` — download and describe documents, photos, locations |
| `helpers.rs` | `escape_html`, `label_to_html`, `send_long`, Markdown→HTML sanitizer |
| `tools.rs` | `interface_tools` (async) — `send_attachment` always present (sends images/videos inline by default, other types as a document, `as_document=true` to force a file); `send_voice_message` injected only when at least one TTS provider is active |
`send_voice_message` calls `TtsProvider::get()` at message time, synthesises text via the highest-priority active provider, and sends the result with `bot.send_voice()`. The tool's description automatically includes the provider's `instructions()` field so the LLM knows how to format text for that specific voice engine.
---
## `skald-relay-common` — `crates/skald-relay-common/`
Shared building blocks for the Skald Remote Control relay **and** the mobile-connector plugin, so both ends stay byte-identical on the wire and against the interop vectors (`data/ios-app/test-vectors.md`). Lightweight: **no** axum/tokio/Skald dependency.
Implements two transport versions:
| Module | Contents |
| --- | --- |
| `crypto` | Domain constants (`AUTH_DOMAIN`, `NS_DOMAIN`, KDF/session salts, direction prefixes), `decode_hex`, `namespace_id`, `challenge_message`, `sign_challenge`/`verify_challenge`, `ct_eq`; E2E suite: `derive_keys`, `ecdh`, `derive_aes_key`, `build_nonce`, `build_aad`, `seal`/`open` (AES-256-GCM) |
| `frames` | serde control-frame types for the **legacy v1** JSON wire protocol: `Incoming`/`Outgoing`, `AuthFrame`, `MessageIn`, `AuthorizeFrame`, `PairingStartFrame`, and the `codes` module (historical, no longer used by deployed code) |
| `proto` | **Active:** Protobuf-generated types for the **v2** binary wire protocol (`data/ios-app/v2/relay-protocol.md` §1-4). Compiled from `proto/skald/relay/v2/relay_frame.proto` by `build.rs` (prost-build). Exposed as `skald_relay_common::proto::v2` so future versions can sit alongside. `bytes` fields come through as `::prost::bytes::Bytes` (prost 0.13 default — wire-compatible with `Vec<u8>`). Every WebSocket binary frame post-auth is **exactly one** `RelayFrame` message. |
| `bin/gen-vectors` | Reference generator for the crypto interop vectors + protobuf encoding. Run with `cargo run -p skald-relay-common --bin gen-vectors`; a thin driver over the `crypto` library functions. Produces both v1 (JSON) and v2 (protobuf) test vectors. |
The relay only uses the verify/namespace subset of `crypto`; the full E2E suite is end-to-end between agent and client and used by the plugin + `gen-vectors`. See [plugin.md §1.1](../data/ios-app/plugin.md) and [relay.md §2](../data/ios-app/relay.md).
**Transport versions:**
- **v1** (JSON-text): frames in `frames` module; no app deployed
- **v2** (protobuf-binary): active; adds presence + live channel (route-or-fail). Documented in `data/ios-app/v2/` ([index.md](../data/ios-app/v2/index.md), [relay-protocol.md](../data/ios-app/v2/relay-protocol.md), [framing.md](../data/ios-app/v2/framing.md))
---
## `skald-relay-server` — `crates/skald-relay-server/`
Zero-trust store-and-forward relay + push bridge for the iOS/Android remote-control feature. Depends on `skald-relay-common` for **protobuf types** (`proto::v2`) and the verify/namespace crypto subset. Implements the **v2 binary wire protocol** (`data/ios-app/v2/relay-protocol.md`): every post-auth WebSocket binary frame is exactly one `RelayFrame` protobuf message. Includes presence tracking and a live channel (route-or-fail) for state pulls that don't queue. No `gen-vectors` binary here anymore (moved to the common crate).
### Source modules
| Module | Contents |
| --- | --- |
| `main.rs` | Thin binary entry: tracing init → `Config::from_env` → `AppState::build` → `axum::serve` with graceful shutdown |
| `lib.rs` | `AppState` (shared state, pusher wiring), `router`, `spawn_gc`, `shutdown_signal`. Conditionally swaps `LogPusher` for the live `ApnsPusher` when the `push-live` feature is on and APNs creds are present |
| `config.rs` | Env-driven `Config` (`bind`, `db_path`); `ApnsConfig` (team/key/PEM/bundle/sandbox) loaded from `APNS_KEY_PATH` + `APNS_BUNDLE_ID` + `APNS_SANDBOX` (gated by `push-live`) |
| `push.rs` | `Pusher` trait + `LogPusher` (default, redacted log only), `PushItem` + `apns_payload()`/`fcm_payload()` builders, the **content-in-push vs wake** decision (3500 B threshold, always compiled and unit-tested). Behind `push-live`: `ApnsPusher` (ES256 JWT provider auth, HTTP/2 over TLS via reqwest) and `build_pusher` factory |
| `store.rs` | `sqlx`/`sqlite` persistence: namespaces, clients, queue, pairing |
| `routing.rs` | `Registry` — in-memory `namespace_id → agent + clients` connection map; presence tracking per namespace |
| `auth.rs` | Re-export of `skald-relay-common::auth` (challenge verify, `namespace_id` derivation) |
| `types.rs` | Re-export of `skald-relay-common::proto::v2` (protobuf control-frame types for v2 binary transport) |
| `limits.rs` | Content-push byte cap, TTLs, rate-limits (`MAX_FRAME_BYTES` 64 KiB, `MAX_LIVE_FRAME_BYTES` 512 KiB per relay-protocol.md §5) |
| `ws.rs` | `handle_socket` — v2 transport driver (relay-protocol.md §1). Challenge → protobuf `Auth` decode → role dispatch → forward loop. Presence events, live (`Message.live=true`) route-or-fail dispatch, and store-and-forward queue. WS-level Ping/Pong keepalive (not protobuf messages). |
### Push bridge
The normative decision (content-in-push vs wake-only) and the JSON payload builders (`apns_payload` / `fcm_payload`) are always compiled and unit-tested — no Apple/Google credentials needed.
The **live senders** are behind the `push-live` cargo feature (default: off):
| Sender | Status | Notes |
| --- | --- | --- |
| `ApnsPusher` | ✅ implemented | ES256 JWT (refresh at 30 min, TTL 60 min) + HTTP/2 via reqwest ALPN. Headers: `apns-topic`, `apns-push-type` (`alert`/`background`), `apns-id` (UUID v4), `authorization: bearer <jwt>`. Body = `item.apns_payload()`. Sandbox vs prod selected by `ApnsConfig::sandbox`. Android notifications ignored (no FCM sender yet) |
| `FcmPusher` | ⬜ not implemented | FCM HTTP v1, OAuth2 service account, data-only message. Not in scope yet — until then, Android pushes are dropped at the platform check inside `ApnsPusher` |
Credentials: read at boot from `config/apns-key.json` (`{team_id, key_id, private_key}`) plus `APNS_BUNDLE_ID` / `APNS_SANDBOX` env vars. The PEM is already newline-decoded by `serde_json` and passed straight to `jsonwebtoken`. Logs never include the full `device_token` or any payload content — only `short()`-truncated identifiers and `apns-id` for correlation.
**Empty device tokens.** A client may connect (or pair) before its APNs/FCM registration has produced a token, sending an empty `device_token`. The relay treats an empty token as "none right now": `update_client_device_token` / `upsert_pending_client` keep any previously stored token instead of clobbering it (SQL `CASE WHEN ?n = '' THEN <existing> ELSE ?n END`), `forward_message` skips the push when the stored token is empty, and `ApnsPusher::notify` early-returns on an empty token. Without this, an empty token would build `/3/device/` and Apple returns `400 MissingDeviceToken`, silently breaking push routing for that device.
---
## `plugin-mobile-connector` — `crates/plugin-mobile-connector/`
The **agent** end of the relay protocol: bridges Skald's Inbox to mobile apps over a single permanent WebSocket, E2E encrypted. Depends on `skald-relay-common` (all of `crypto` + `proto::v2`) and `core-api` (`Plugin`, `InboxApi`, `ChatHubApi`, `SqlitePool`). Owns the `relay_clients` table and the `data/relay/seed` file. Implements the **v2 binary transport client** (`data/ios-app/v2/relay-protocol.md`): encodes/decodes `RelayFrame` protobuf, sends presence events, handles the live channel for `inbox_request` pulls. Implements `Plugin` (lifecycle + `http_router` for the QR endpoint) and `RelayAgent` (control surface). Exports `mobile_tools(agent)` so the main crate registers its three LLM tools. See [mobile-connector.md](mobile-connector.md).
docs/cron.md
+283
View File
@@ -0,0 +1,283 @@
# Cron Jobs & Background Tasks
## TaskManager
`TaskManager` manages scheduled cron jobs and on-demand background tasks (sync and async). It uses `std::sync::OnceLock` to hold late-injected dependencies, breaking circular chains that would arise if they were required at construction time:
| Dependency | Injected via | Needed for |
|---|---|---|
| `ChatSessionManager` | `set_session()` | Creating ephemeral sessions per job run |
| `ChatHub` | `set_hub()` | Sending completion/failure notifications; injecting `execute_task` InterfaceTool |
| `Arc<TaskManager>` (self) | `set_self_arc()` | Passing self-reference into `run_job` for sub-task tools |
`TaskManager` also holds an `Arc<SystemEventBus>` (passed at construction time, not via OnceLock) used to publish `SystemEvent::JobCompleted` when a job finishes. `ProjectTicketManager` subscribes independently; `TaskManager` has no direct reference to it.
In `main.rs`:
1. `TaskManager::new(pool, tz, system_bus)` — created first, OnceLocks empty
2. `ChatSessionManager::new(...)` — created second
3. `cron.set_session(Arc::clone(&manager))` — fills first OnceLock
4. `cron.start()` — background tasks begin (tick every 30 s)
5. `ChatHub::new()` — created after cron starts
6. `cron.set_hub(Arc::clone(&chat_hub))` — fills second OnceLock
7. `cron.set_self_arc(Arc::clone(&cron))` — fills third OnceLock
8. `chat_hub.set_task_mgr(Arc::clone(&cron))` — hub holds ref for InterfaceTool injection
The cron tick loop first fires 30 s after `start()`, so all OnceLocks are guaranteed to be filled before any job dispatch.
---
## Background Tasks
`start()` spawns two independent background tasks:
### Scheduler Loop
- Ticks every **30 seconds**
- Calls `db::scheduled_jobs::list_due(pool, &Utc::now().to_rfc3339())`
- Any cron job with `enabled=1`, `next_run_at <= now`, and `running_session_id IS NULL` is returned
- Each due job is spawned as an independent `tokio::task` via `run_job()`
### Cleanup Loop
- Waits 15 s at startup, then runs hourly
- Calls `cleanup_expired_single_runs(pool)` for jobs that are single-run, disabled, and older than 7 days. To satisfy the FK constraints, it clears the dependents first: nulls `project_tickets.job_id` (whose FK has no `ON DELETE` action, so a ticket still pointing at the job would block the delete), then deletes the job's `job_runs` rows, then deletes the `scheduled_jobs` rows. Tickets keep their result/error — only the GC'd job pointer is dropped. The manual `scheduled_jobs::delete()` performs the same cascade.
---
## 7-Field Cron Expression Format
**Format**: `sec min hour dom month dow year`
This is the format of the [`cron`](https://crates.io/crates/cron) crate — **not** standard Unix crontab (which uses 5 fields without seconds or year).
| Field | Values |
|---|---|
| sec | 0–59 |
| min | 0–59 |
| hour | 0–23 |
| dom | 1–31 or `*` |
| month | 1–12 or `*` |
| dow | 0–6 (Sun=0) or `*` |
| year | 4-digit year or `*` |
Examples:
| Expression | Meaning |
|---|---|
| `0 0 9 * * * *` | Every day at 09:00:00 |
| `0 */30 * * * * *` | Every 30 minutes |
| `0 0 8 * * 1 *` | Every Monday at 08:00 |
The `execute_task` tool (mode=cron) validates the expression with `Schedule::from_str()` before saving.
---
## Timezone
Cron expressions are evaluated in the timezone configured under `timezone` in `config.yml` (top-level IANA name, e.g. `Europe/Rome`). When omitted, the server's system local timezone is used as fallback. The same setting also controls the timestamp injected into the LLM context each turn.
The timezone is loaded at startup, logged at `INFO` level, and passed into `TaskManager`. All three points where `next_run_at` is computed (`add_job`, `toggle_job`, `run_job`) use the same `next_fire(schedule, tz)` helper which converts the result to UTC before storing.
---
## `next_run_at` (pre-computed fire time)
Rather than a sliding look-back window, the scheduler uses a **pre-computed `next_run_at` timestamp** stored in the DB:
- Set at job creation (first upcoming fire time after now, in the configured timezone)
- Advanced to the next fire time after each successful run
- Cleared when a job is disabled
- Recalculated from the cron expression when `toggle_item` (kind=cron) re-enables a job
This means: a tick simply does `WHERE next_run_at <= now` — no expression evaluation in the hot path. A missed tick is automatically covered because `next_run_at` stays in the past until the job actually runs.
---
## `kind` Column (three modes)
`scheduled_jobs` has a `kind` column with three values:
| `kind` | Behavior |
| ------ | -------- |
| `cron` | Scheduled job with a 7-field cron expression. Picked up by the tick loop when `next_run_at` is due. Result notified via `ChatHub::notify` (home conversation). |
| `sync` | Runs immediately on creation. No cron expression, no `next_run_at`. `single_run` is always true. Caller blocks until the agent finishes and receives the result inline. |
| `async` | Runs immediately in the background. Returns `task_id` immediately. When the agent finishes, the result is injected into the parent session as a synthetic message (see [Async Result Delivery](#async-result-delivery)). |
The `list_due()` query filters by `kind = 'cron'`, so sync/async tasks are never picked up by the scheduler tick loop. Recovery (`list_interrupted()`) applies to all kinds.
---
## `single_run` (one-shot jobs)
If `single_run=true`, after the first execution `finish_run()` receives `next_run_at=None`, which sets `enabled=0` (disabling the job) rather than advancing the schedule. The job stays in the DB as a disabled record and is purged after 7 days by the cleanup loop.
**Auto-detection for cron mode**: `add_job()` calls `next_fire_and_single()` which advances the cron iterator twice. If there is no second fire time — i.e. the expression can only ever match one point in time (e.g. `0 30 9 15 6 * 2026`) — `single_run` is forced to `true` regardless of what the caller passed. The LLM does not need to set `single_run` explicitly for specific-datetime expressions.
For `sync` and `async` modes, `single_run` is always `true` (they run once and are done).
---
## Job Lifecycle
### `cron` mode
1. LLM calls `execute_task(mode="cron", title, cron, prompt, agent_id)` → inserted in DB with `enabled=1`, `next_run_at` set to first upcoming fire time
2. Scheduler tick → `list_due()` returns the job
3. `run_job()` spawned (see below)
4. On completion: `hub.notify(...)` emits a structured completion notification (`source="cron"`) to the home conversation
### `sync` mode
1. LLM calls `execute_task(mode="sync", title, prompt, agent_id)` → LLM tool call blocks
2. `add_job_sync()` creates DB record and calls `run_job()` inside `block_in_place`
3. Agent runs to completion; final assistant message returned inline to the LLM tool call
4. Job marked disabled (single_run)
### `async` mode
1. LLM calls `execute_task(mode="async", title, prompt, agent_id)` → returns `task_id` immediately
2. `add_job_async()` creates DB record with `parent_session_id` set to the calling session and `run_context` (JSON blob) inherited from the parent
3. Agent spawned in background; LLM continues
4. On completion: `inject_async_result()` sends a synthetic message to the parent session
---
## `run_job` — execution core
`run_job(pool, session_mgr, task_mgr, hub, job, tz)` handles all three kinds:
1. New ephemeral session created (`is_ephemeral=1, is_interactive=0, source="cron"`)
2. `set_running(pool, job.id, session_id)` — marks job in-flight
3. If `job.run_context` is `Some`, stamps the RunContext JSON blob onto the new `chat_sessions` row directly before `get_or_create_handler()` loads it
4. `handler.set_context_label("CronJob: <title>")` — used for Agent Inbox labels
5. Job context injected via `extra_system_dynamic_override`
6. `execute_subtask` InterfaceTool injected, carrying the same `run_context` so nested subtasks also inherit it (see [Background Tool Restrictions](#background-tool-restrictions))
7. `tokio::spawn(handler.handle_message(...))` + concurrent drain of the event channel (prevents deadlock when the buffer fills)
8. After completion: delivery branch on `job.kind` (notify / return inline / inject_async_result)
9. `record_job_run()` writes to `job_runs` audit trail
10. `finish_run()` advances `next_run_at` for cron jobs; disables single-run jobs
11. Publishes `SystemEvent::JobCompleted { job_id, origin_ref, result, error }` on `system_bus`; `ProjectTicketManager` receives this event via its `start_listener()` background task and updates the ticket state when `origin_ref` starts with `"PROJECT_TASK:"`
On failure: error logged, job_runs row recorded with status `"failed"`, `hub.notify(...)` sends an error notification.
### Deadlock prevention
`handle_message` sends `ServerEvent` values into a bounded channel. If the caller drains only _after_ `handle_message` returns, the channel buffer can fill (especially with long agent chains) causing a deadlock. Fix: `handle_message` is spawned via `tokio::spawn`, and the calling task drains the channel concurrently. The `JoinHandle` is awaited after the channel closes.
---
## Async Result Delivery
When a `kind="async"` job completes, `inject_async_result()` follows the same pattern as the notification system:
1. Resolves the `source_id` via `chat_sessions::find_by_id(pool, parent_session_id)` → `session.source`
2. Gets the active stack for the parent session via `chat_sessions_stack::active_for_session`
3. Writes a synthetic **assistant message** (reasoning trace) directly to `chat_history`
4. Writes a completed **`task_completed` tool call** to `chat_llm_tools`, carrying `{task_id, title, result}` as the tool result payload
5. Calls `hub.resume(source_id)` — this bridges events to the global WebSocket bus and runs the LLM loop, which sees the completed tool call and responds
The delivery happens inside `run_job` (not in the `add_job_async` spawn closure) so the recovery path also calls it correctly.
**Note:** if the parent session has been cleared (`/clear`) the result is still injected — the session's history starts fresh but the notification arrives. This is intentional: the parent session ID is the correct semantic target even after a clear.
---
## Background Tool Restrictions
Background sessions (`kind="cron"` or `kind="async"`) **cannot** call `execute_task`. They receive `execute_subtask` instead:
| Tool | Available in | Notes |
| ---- | ------------ | ----- |
| `execute_task` | Interactive sessions only | Injected as InterfaceTool by `ChatHub::send_message`; `session_id` and `run_context` (JSON blob) captured in closure at tool-build time |
| `execute_subtask` | Background sessions only | Sync-only; no `mode` field; calls `add_job_sync()` internally; `run_context` propagated from the parent job |
This rule eliminates the complexity of tracking nested async/cron task lifecycles. Background tasks can spawn synchronous sub-work (via `execute_subtask` or via `call_agent`) but cannot launch new fire-and-forget or cron tasks.
---
## `running_session_id` (restart recovery)
`scheduled_jobs.running_session_id` is non-null while a job is in-flight. On restart:
1. `recover_interrupted()` runs once, before the first tick
2. Queries `list_interrupted()` — all jobs where `running_session_id IS NOT NULL`
3. For each interrupted job, `run_job()` is spawned again (creates a fresh session — the old one is abandoned)
4. For async jobs, `inject_async_result()` is called when the re-run completes
`list_due()` excludes rows with `running_session_id IS NOT NULL`, preventing double-runs.
---
## Session Handling
Each run always creates a **new ephemeral session**:
| Property | Value |
|---|---|
| `source` | `"cron"` |
| `is_interactive` | `0` |
| `is_ephemeral` | `1` |
| `agent_id` | job's `agent_id` (required at creation; must be a `task` agent) |
| `run_context` | inherited from `scheduled_jobs.run_context` JSON blob (may be null → falls back to the implicit `"default"` group) |
Sessions are not reused across runs. Each run gets a fresh context.
---
## RunContext Inheritance
Every task inherits the RunContext of the session that created it. This controls which tool-permission group the task runs under (tool visibility, approval rules).
**Inheritance chain:**
1. The parent interactive session has a `run_context` JSON blob (set by the user via the API; `None` otherwise)
2. `ChatHub::send_message` reads `handler.run_context_json()` **before** building the `execute_task` InterfaceTool and captures the value in the closure
3. `execute_with_session()` passes `run_context` to `add_job / add_job_sync / add_job_async`, which store it in `scheduled_jobs.run_context`
4. `run_job()` stamps the JSON blob onto the ephemeral child session before `get_or_create_handler()` loads the session — the manager's resolution path (`session.run_context` → `RunContext::from_db`) picks it up automatically
5. `execute_subtask` also captures `run_context`, so nested synchronous sub-tasks inherit it transitively
For **project tickets**, `ProjectTicketManager.start()` resolves the `run_context` itself (ticket override → project default) and passes it to `TaskManager.spawn_async_job()`.
**Override via Tasks UI:** the `PATCH /api/cron/jobs/{id}/run-context` endpoint allows overriding `scheduled_jobs.run_context` after creation. The dropdown in the Tasks page calls this endpoint.
---
## Agent Interaction
Jobs run via the `task` agent named in `agent_id` (required — no default; see [agents.md](agents.md)). `TaskManager` rejects an empty or non-`task` agent at creation. A typical task agent:
- Executes the task described in the cron prompt
- Delegates complex work to sub-agents (software-engineer, researcher, software-architect) via `execute_subtask`
- Calls `ask_user_clarification` when genuinely uncertain — this creates a pending entry in the `ClarificationManager` (visible in Agent Inbox) rather than blocking
- Its final assistant message is captured for delivery (notification / inline result / async injection)
---
## `job_runs` (audit trail)
Every execution is recorded in `db::job_runs`. Schema: see [database.md](database.md).
---
## LLM Tools for Tasks
| Tool | Availability | Action |
| ---- | ------------ | ------ |
| `execute_task` | Interactive sessions (web, telegram) | Create and run a task — cron/sync/async modes; validates cron expression; auto-detects single_run |
| `execute_subtask` | Background sessions only | Run a sync sub-task; blocks until complete; returns result inline |
| `read_agent_result` | Interactive sessions | Poll stub — always returns `not_ready`; real delivery is via synthetic message |
| `list_items` (type=cron) | All sessions | Returns JSON array of all tasks (id, title, cron, enabled, kind, next_run_at, single_run, last_run_at) |
| `delete_cron_job` | All sessions | Permanently deletes task by id |
| `toggle_item` (kind=cron) | All sessions | Enables or disables a task; recalculates next_run_at when re-enabling |
---
## When to Update This File
- Scheduler tick interval changes
- `next_run_at` / `list_due` logic changes
- `run_job` session-handling logic changes
- New task-related tools are added
- Recovery or cleanup loop logic changes
- Async delivery mechanism changes
docs/database.md
+546
View File
@@ -0,0 +1,546 @@
# Database
## Connection and Init
`db::init_pool(path)` opens a SQLite connection pool with `create_if_missing=true`, then calls `create_tables()`. All `CREATE TABLE IF NOT EXISTS` statements are idempotent — safe to run on every startup.
The pool is opened in **WAL journal mode** (`synchronous=NORMAL`) with a **5 s `busy_timeout`**. The DB is written concurrently (chat loop, cron, and plugins such as the mobile-connector persisting its E2E `send_counter` before each send). Without `busy_timeout`, a contended write returns `SQLITE_BUSY` ("database is locked") immediately and the operation is lost — which silently dropped outbound mobile messages (the `inbox_update` never reached the device). WAL lets readers run alongside the single writer; `busy_timeout` makes a writer wait for the lock instead of failing.
---
## Table Schemas
### chat_sessions
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `title` | TEXT | nullable |
| `source` | TEXT | NOT NULL DEFAULT `'web'` |
| `agent_id` | TEXT | NOT NULL DEFAULT `'main'` (added via ALTER) |
| `is_interactive` | INTEGER | NOT NULL DEFAULT `1` (added via ALTER) |
| `is_ephemeral` | INTEGER | NOT NULL DEFAULT `0` (added via ALTER) |
| `run_context` | TEXT | nullable — `RunContext` JSON blob (e.g. `{"security_group":"admin"}`); formerly `run_context_id` (renamed in migration 10) |
| `created_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
**`is_interactive`** — `1` when a real user is actively participating in the session (web, Telegram). `0` for fully automated sessions where all user-role messages are generated by the application (cron, tic).
**`is_ephemeral`** — `1` for short-lived task sessions (cron, tic) with no long-term conversational value. Memory sinks (e.g. Honcho) should skip ephemeral sessions.
Index: `idx_stack_session ON chat_sessions_stack(session_id)`
---
### chat_sessions_stack
Represents one agent call frame. Root frames have `depth=0`; sub-agent frames increment depth.
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `session_id` | INTEGER | NOT NULL REFERENCES `chat_sessions(id)` |
| `agent_id` | TEXT | NOT NULL DEFAULT `'main'` |
| `agent_prompt` | TEXT | nullable (the `call_agent` prompt) |
| `depth` | INTEGER | NOT NULL DEFAULT 0 |
| `parent_tool_call_id` | INTEGER | nullable (links to `chat_llm_tools.id`) |
| `terminated_at` | TEXT | nullable; set when `dispatch_sub_agent` finishes |
| `created_at` | TEXT | NOT NULL |
---
### chat_history
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `session_stack_id` | INTEGER | NOT NULL REFERENCES `chat_sessions_stack(id)` |
| `role` | TEXT | NOT NULL CHECK(`user` \| `assistant` \| `agent`) |
| `content` | TEXT | NOT NULL DEFAULT `''` |
| `status` | TEXT | NOT NULL DEFAULT `ok` CHECK(`ok` \| `failed`) |
| `input_tokens` | INTEGER | nullable |
| `output_tokens` | INTEGER | nullable |
| `duration_ms` | INTEGER | nullable |
| `model_db_id` | INTEGER | nullable REFERENCES `llm_models(id)` |
| `is_synthetic` | INTEGER | NOT NULL DEFAULT `0` — `1` when the message was system-generated |
| `reasoning_content` | TEXT | nullable — chain-of-thought from reasoning models |
| `cost` | REAL | nullable — turn cost in USD, when the provider reports it (OpenRouter `usage.cost`) |
| `metadata` | TEXT | nullable — generic JSON metadata bag (added v17); currently `{ "attachments": [...] }` |
| `created_at` | TEXT | NOT NULL |
- `role = 'agent'` is a sub-agent prompt message; sent as `user` to the LLM but hidden in the UI
- `status = 'failed'` rows are excluded from `for_stack()` (not sent to LLM)
- `model_db_id` is set only on `role = 'assistant'` rows; `null` for user/agent messages and for rows created before this column was added
- `is_synthetic = 1` marks messages injected by the system (ChatHub notification assistant messages with tool-call injection, or legacy synthetic user turns). They are included in the LLM context but excluded from the UI history (not shown on page reload). Currently used for `role = 'assistant'` rows that carry the synthetic `read_notification` tool call and reasoning trace.
- `reasoning_content` is populated only for `role = 'assistant'` rows from providers that return chain-of-thought (currently DeepSeek thinking mode); echoed back in the assistant message on subsequent turns
- `cost` is set on `role = 'assistant'` rows only when the provider returns a per-request price (OpenRouter exposes it under `usage.cost`); `null` for providers that don't bill per-call. Extracted via the `ChatbotClient::extract_cost` trait method (see [llm-clients.md](llm-clients.md))
- `metadata` is a generic JSON bag (type `MessageMetadata` in `core-api`), set on `role = 'user'` rows that carry file attachments. The raw `[SYSTEM INFO]` block the LLM sees is **never** stored here — it is generated on the fly from `metadata.attachments` by the message builder, while the UI renders the same list as chips. `content` always stays the clean typed text. See [frontend.md](frontend.md) (attachments) and [session.md](session.md).
Index: `idx_history_stack ON chat_history(session_stack_id)`
---
### chat_summaries
Persisted conversation summaries generated by the **context compactor** (see
[compaction.md](compaction.md)). Each row covers all `chat_history` messages up
to and including `covers_up_to_message_id`. `build_openai_messages` loads only
messages *after* this boundary and injects the summary as context.
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `stack_id` | INTEGER | NOT NULL REFERENCES `chat_sessions_stack(id)` |
| `content` | TEXT | NOT NULL — the summary text |
| `covers_up_to_message_id` | INTEGER | NOT NULL — boundary: messages with `id > this` are loaded raw |
| `created_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
Index: `idx_chat_summaries_stack ON chat_summaries(stack_id)`
Multiple rows can exist per stack (one per compaction cycle). Only the most
recent row is used; older rows are retained for historical reference.
---
### chat_llm_tools
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `message_id` | INTEGER | NOT NULL REFERENCES `chat_history(id)` |
| `name` | TEXT | NOT NULL |
| `arguments` | TEXT | nullable (JSON string) |
| `result` | TEXT | nullable |
| `status` | TEXT | NOT NULL DEFAULT `running` CHECK(`running` \| `pending` \| `done` \| `failed` \| `cancelled` \| `rejected`) |
| `created_at` | TEXT | NOT NULL |
- `running` → tool is executing (or app crashed mid-execution — re-run on resume)
- `pending` → blocked on a human approval / clarification (re-gate / re-ask on resume)
- `done` → result is in `result` column
- `failed` → tool/runtime error message is in `result` column
- `cancelled` → stopped by the user via `/stop` (terminal, **not** re-run on resume) — distinct from `failed`
- `rejected` → denied by an approval policy or a human (terminal) — distinct from `failed`
The richer in-memory `ToolExecutionState` maps onto these strings (see [tools.md](tools.md#tool-execution-lifecycle)). The table is created directly with the full 6-value `CHECK` constraint; `cancelled`/`rejected` (distinct from `failed`) were historically added by a table rebuild in schema versions 15–16, now folded into the baseline (see [Migration Pattern](#migration-pattern)).
Index: `idx_tools_message ON chat_llm_tools(message_id)`
---
### mcp_servers
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `name` | TEXT | NOT NULL UNIQUE |
| `transport` | TEXT | NOT NULL DEFAULT `'stdio'` |
| `command` | TEXT | nullable |
| `args_json` | TEXT | nullable (JSON array) |
| `env_json` | TEXT | nullable (JSON object) |
| `url` | TEXT | nullable |
| `api_key` | TEXT | nullable |
| `enabled` | INTEGER | NOT NULL DEFAULT 1 |
| `created_at` | TEXT | NOT NULL |
---
### llm_providers
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `name` | TEXT | NOT NULL UNIQUE |
| `type` | TEXT | NOT NULL (`lm_studio`, `ollama`, `open_ai`, `anthropic`) |
| `api_key` | TEXT | nullable |
| `base_url` | TEXT | nullable |
| `description` | TEXT | nullable |
| `removed_at` | TEXT | nullable; set on soft-delete (api_key is also cleared) |
| `created_at` | TEXT | NOT NULL |
Providers are **never hard-deleted**. `DELETE /api/llm/providers/{id}` sets `removed_at` and clears `api_key`, and cascades soft-delete to all models belonging to that provider. Rows with `removed_at IS NOT NULL` are excluded from all queries.
---
### llm_models
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `provider_id` | INTEGER | NOT NULL REFERENCES `llm_providers(id)` ON DELETE CASCADE |
| `model_id` | TEXT | NOT NULL (provider's internal model name) |
| `name` | TEXT | NOT NULL UNIQUE (display name, used as client key) |
| `strength` | TEXT | nullable (`very_low` \| `low` \| `average` \| `high` \| `very_high`) |
| `scope` | TEXT | NOT NULL DEFAULT `'[]'` (JSON array of scope strings) |
| `is_default` | INTEGER | NOT NULL DEFAULT 0 |
| `priority` | INTEGER | NOT NULL DEFAULT 100 (lower = tried first by AUTO) |
| `extra_params` | TEXT | nullable (JSON object, merged into request body at call time) |
| `context_length` / `max_output_tokens` / `knowledge_cutoff` / `capabilities` | INTEGER / INTEGER / TEXT / TEXT | nullable catalog metadata (see `docs/llm-clients.md`) |
| `reasoning` | TEXT | nullable; selected reasoning value (JSON string for a `ValueSet` mode, JSON number for a `Range` mode; NULL = off) — see *Reasoning* in `docs/llm-clients.md` |
| `removed_at` | TEXT | nullable; set on soft-delete |
| `created_at` | TEXT | NOT NULL |
The only uniqueness constraint is `name` (also the resolution key). There is **no** `UNIQUE(provider_id, model_id)` — the same underlying model may be registered several times under one provider with different aliases/reasoning (e.g. `glm-4.6` vs `glm-4.6-thinking`). Migration **v20** dropped the historical `(provider_id, model_id)` constraint by rebuilding the table (preserving `id` values, which are FK targets in `llm_requests.model_db_id` / `chat_history.model_db_id`) and added the `reasoning` column.
Models are **never hard-deleted**. `DELETE /api/llm/models/{id}` sets `removed_at`. Rows with `removed_at IS NOT NULL` are excluded from all queries, including the AUTO selector. The `id` column remains a valid FK target for `chat_history.model_db_id` even after removal. Re-adding a model with the same `name` **revives** the soft-deleted row (`insert_model` upserts on `name`).
---
### transcribe_models
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `provider_id` | INTEGER | NOT NULL REFERENCES `llm_providers(id)` |
| `model_id` | TEXT | NOT NULL (provider's internal model name, e.g. `openai/whisper-1`) |
| `name` | TEXT | NOT NULL UNIQUE (display name) |
| `language` | TEXT | nullable; BCP-47 code (e.g. `"it"`, `"en"`) or NULL for auto-detect |
| `priority` | INTEGER | NOT NULL DEFAULT 100 (lower = tried first by `TranscribeManager`) |
| `removed_at` | TEXT | nullable; soft-delete |
| `created_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
Only providers that declare `ServiceType::Transcribe` in `ApiProvider::supported_types()` should have rows here (OpenAI, OpenRouter). See [providers/transcribe.md](providers/transcribe.md).
---
### image_generate_models
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `provider_id` | INTEGER | NOT NULL REFERENCES `llm_providers(id)` |
| `model_id` | TEXT | NOT NULL (provider's internal model name, e.g. `x-ai/grok-2-vision`) |
| `name` | TEXT | NOT NULL UNIQUE (display name, also used as `provider_id` in the `image_generate` LLM tool) |
| `priority` | INTEGER | NOT NULL DEFAULT 100 (lower = tried first) |
| `removed_at` | TEXT | nullable; soft-delete |
| `created_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
Only providers that declare `ServiceType::ImageGenerate` in `ApiProvider::supported_types()` should have rows here (OpenRouter, OpenAI). See [providers/image.md](providers/image.md).
---
### secrets
| Column | Type | Constraints |
| --- | --- | --- |
| `key` | TEXT | PRIMARY KEY (uppercase by convention, e.g. `HUGGINGFACE_TOKEN`) |
| `value` | TEXT | NOT NULL — stored in plain text, never log |
| `created_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
| `updated_at` | TEXT | NOT NULL DEFAULT `datetime('now')` — updated on upsert |
Managed via `SecretsStore` (`src/core/secrets.rs`). See [secrets.md](secrets.md).
---
### tts_models
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `provider_id` | INTEGER | NOT NULL REFERENCES `llm_providers(id)` |
| `model_id` | TEXT | NOT NULL (generation model, e.g. `tts-1-hd`, `eleven_multilingual_v2`) |
| `voice_id` | TEXT | nullable; speaker voice ID — required for ElevenLabs, NULL for OpenAI (added in schema v2) |
| `name` | TEXT | NOT NULL UNIQUE (display name, also used as the synthesiser `id()`) |
| `description` | TEXT | nullable; human-readable description shown in UI |
| `instructions` | TEXT | nullable; default voice style / tone / speed (overridable per call) |
| `response_format` | TEXT | nullable; audio format `mp3`/`opus`/`aac`/`flac`/`wav`/`pcm` sent to the provider — NULL ⇒ `mp3` (added in schema v18) |
| `priority` | INTEGER | NOT NULL DEFAULT 100 (lower = tried first by `TtsManager`) |
| `removed_at` | TEXT | nullable; soft-delete |
| `created_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
See [providers/tts.md](providers/tts.md).
---
### scheduled_jobs
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `title` | TEXT | NOT NULL |
| `description` | TEXT | NOT NULL DEFAULT `''` |
| `cron` | TEXT | NOT NULL (7-field format) |
| `prompt` | TEXT | NOT NULL |
| `agent_id` | TEXT | NOT NULL DEFAULT `'main'` (column default; in practice always set explicitly — `agent_id` is required at task creation) |
| `enabled` | INTEGER | NOT NULL DEFAULT 1 |
| `last_run_at` | TEXT | nullable (RFC3339 datetime) |
| `next_run_at` | TEXT | nullable (RFC3339); pre-computed next fire time; used by `list_due()` (added via ALTER) |
| `single_run` | INTEGER | NOT NULL DEFAULT 0; if 1 the job is disabled after its first execution (added via ALTER) |
| `running_session_id` | INTEGER | nullable; non-null while a run is in-flight; cleared by `finish_run()`; used by `list_interrupted()` for restart recovery (added via ALTER) |
| `kind` | TEXT | NOT NULL DEFAULT `'cron'` — `'cron'` scheduled, `'sync'` run-now-blocking, `'async'` run-now fire-and-forget |
| `parent_session_id` | INTEGER | nullable FK → `chat_sessions(id)`; set for `kind='async'` tasks to route the result back to the calling session |
| `run_context` | TEXT | nullable; `RunContext` JSON blob inherited from the creating session (or overridden via the Tasks UI); applied to the ephemeral child session at run time. Formerly `run_context_id` (renamed in migration 10). |
| `origin_ref` | TEXT | nullable; free-form reference to the entity that originated this job (e.g. `"PROJECT_TASK:42"`); used by `TaskManager` for post-completion callbacks (added in migration 11) |
| `created_at` | TEXT | NOT NULL |
**`next_run_at`** is set at job creation (first upcoming fire time after now), advanced after each successful run, cleared on disable, and recalculated by `toggle_item` (kind=cron) when re-enabling. `list_due(pool, now)` queries `WHERE kind='cron' AND enabled=1 AND next_run_at <= now AND running_session_id IS NULL`. Sync and async tasks run immediately on creation and are never picked up by the scheduler tick.
**`origin_ref`** is set only for jobs created by `ProjectTicketManager.start()`. When the job completes, `run_job()` checks this field and dispatches `ticket_manager.on_job_completed()` to update the ticket's status and result.
**`running_session_id`** is set by `set_running()` before `handle_message()` starts and cleared by `finish_run()` after the run completes. At startup, `recover_interrupted()` queries `list_interrupted()` and re-spawns any jobs still in-flight from a crash.
---
### job_runs
Audit trail of every cron job and immediate task execution. One row per run, written after each execution by `TaskManager::run_job()`.
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `job_id` | INTEGER | NOT NULL REFERENCES `scheduled_jobs(id)` |
| `session_id` | INTEGER | nullable — the ephemeral session that ran the job |
| `started_at` | TEXT | NOT NULL (RFC3339) |
| `completed_at` | TEXT | nullable (RFC3339) |
| `duration_ms` | INTEGER | nullable |
| `status` | TEXT | NOT NULL (`success` \| `failed`) |
| `final_response` | TEXT | nullable — last assistant message from the session |
| `error` | TEXT | nullable — error message on failure |
| `created_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
Index: `idx_job_runs_job_id ON job_runs(job_id)`
DAO in `src/core/db/job_runs.rs`: `insert(pool, ...)`, `list_for_job(pool, job_id, limit)`.
---
### sources
One row per source (e.g. `"web"`, `"telegram"`). Tracks the active session for each source.
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | TEXT | PRIMARY KEY |
| `active_session_id` | INTEGER | nullable REFERENCES `chat_sessions(id)` |
| `updated_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
`ChatHub::get_or_create_session()` reads this table; `sources::upsert()` writes it.
---
### relay_clients
Created and owned by the `mobile-connector` plugin (`crates/plugin-mobile-connector/src/db.rs`), not the main crate. One row per paired mobile device. Counters MUST persist across restarts to prevent AES-GCM nonce reuse / replay (see [mobile-connector.md](mobile-connector.md)).
| Column | Type | Constraints |
| --- | --- | --- |
| `ed25519_pub` | BLOB | PRIMARY KEY (32 B) |
| `x25519_pub` | BLOB | NOT NULL (32 B, for ECDH) |
| `state` | TEXT | NOT NULL (`pending` \| `authorized`) |
| `platform` | TEXT | nullable (`ios` \| `android`) |
| `device_info` | TEXT | nullable JSON (from the E2E `hello`) |
| `send_counter` | INTEGER | NOT NULL DEFAULT 0 (dir agent→client) |
| `recv_counter` | INTEGER | NOT NULL DEFAULT 0 (last seen, dir client→agent) |
| `authorized_at` | INTEGER | nullable unix ms |
| `last_seen` | INTEGER | nullable unix ms |
---
### config
Global key/value store for runtime configuration.
| Column | Type | Constraints |
| --- | --- | --- |
| `key` | TEXT | PRIMARY KEY |
| `value` | TEXT | NOT NULL |
| `updated_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
Currently used keys:
| Key | Owner | Description |
| --- | --- | --- |
| `source_home` | `ChatHub` | Source ID that receives background agent notifications. Default: `"web"` |
| `tic.run_context` | `TicManager` | Run context ID applied to each TIC session tick. Empty = no run context. |
| `tic.interval_minutes` | `TicManager` | Override tick interval in minutes. Empty = use `tic.interval_secs` from `config.yml`. |
Keys are managed via `GET /api/config` + `PUT /api/config/:key`. Registered properties are declared by each subsystem via `core_api::ConfigProperty` and collected in `Skald::config_properties`.
---
### mcp_events
Persistent store for push notifications received from MCP servers via JSON-RPC notifications (stdout, no `id` field). Written by `McpManager::notification_consumer`; consumed by `TicManager` at a configurable interval (`tic.interval_secs` in `config.yml`, default 900 s).
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `source` | TEXT | NOT NULL — MCP server name (e.g. `"whatsapp"`, `"gmail"`, `"gcal"`) |
| `method` | TEXT | NOT NULL — notification method (e.g. `"event/new_email"`) |
| `payload` | TEXT | NOT NULL — JSON string of the notification `params` field |
| `processed` | INTEGER | NOT NULL DEFAULT 0 — set to 1 by `TicManager` before running the agent |
| `processed_at` | TEXT | nullable — timestamp set when `processed` is flipped |
| `created_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
Index: `idx_mcp_events_pending ON mcp_events(id) WHERE processed = 0` — partial index for efficient pending-event queries.
DAO in `src/core/db/mcp_events.rs`: `insert`, `pending_limited(limit)`, `pending`, `mark_processed(ids)`, `all_recent(limit)`.
---
### session_mcp_grants
**Root-agent** record of which tool groups have been activated via `activate_tools` — MCP server names and/or the reserved keyword `config` (which unlocks the built-in `Config`-category tools). Grants survive restarts and persist for the whole session lifetime. Tools for granted groups are included in the LLM's tool list on every subsequent turn via the dynamic `all_tool_defs()` call.
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `session_id` | INTEGER | NOT NULL (FK to `chat_sessions.id`) |
| `mcp_name` | TEXT | NOT NULL — MCP server name (e.g. `"gmail"`) |
| `granted_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
Unique constraint: `(session_id, mcp_name)` — granting the same server twice is a no-op (INSERT OR IGNORE).
DAO in `src/core/db/session_mcp_grants.rs`: `grant(pool, session_id, mcp_name)`, `list_for_session(pool, session_id)`.
> **Sub-agents do not write here.** They use `stack_mcp_grants` instead.
---
### stack_mcp_grants
**Sub-agent** record of which MCP servers have been activated by a specific stack frame. Grants are:
- **Isolated** from the parent session — they do not affect `session_mcp_grants`.
- **Persistent** across restarts — if a sub-agent is interrupted, `dispatch_sub_agent` re-loads these grants from DB when execution resumes.
- **Cleaned up** automatically when the stack frame terminates: `dispatch_sub_agent` calls `delete_for_stack(pool, stack_id)` before returning.
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `stack_id` | INTEGER | NOT NULL (FK to `chat_sessions_stack.id`) |
| `mcp_name` | TEXT | NOT NULL — MCP server name |
| `granted_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
Unique constraint: `(stack_id, mcp_name)` — granting the same server twice is a no-op (INSERT OR IGNORE).
DAO in `src/core/db/stack_mcp_grants.rs`: `grant(pool, stack_id, mcp_name)`, `list_for_stack(pool, stack_id)`, `delete_for_stack(pool, stack_id)`.
---
### llm_requests
Debug log of every `chat_with_tools` call. Populated by [`LoggingChatbotClient`](../src/chatbot/logging.rs) (fire-and-forget, invisible to the LLM loop). Enabled via `config.yml` — see `llm.request_log`.
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `session_id` | INTEGER | nullable — `chat_sessions.id` of the calling session |
| `stack_id` | INTEGER | nullable — `chat_sessions_stack.id` of the calling frame |
| `model_name` | TEXT | NOT NULL — display name of the model (e.g. `"claude"`) |
| `request_json` | TEXT | NOT NULL — full HTTP request body sent to the provider (compact JSON) |
| `request_headers` | TEXT | nullable — HTTP request headers as JSON object (api-key redacted) |
| `response_json` | TEXT | nullable — full HTTP response body (compact JSON); NULL on error |
| `response_headers` | TEXT | nullable — HTTP response headers as JSON object |
| `error_text` | TEXT | nullable — error message when the HTTP call failed |
| `input_tokens` | INTEGER | nullable |
| `output_tokens` | INTEGER | nullable |
| `duration_ms` | INTEGER | NOT NULL DEFAULT 0 — wall-clock round-trip time |
| `created_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
Index: `idx_llm_requests_created ON llm_requests(created_at)` — used by the retention cleanup.
Rows are purged automatically by a background task (runs at boot + every hour) via `db::llm_requests::cleanup(pool, retention_days)`. Default retention: 14 days.
DAO in `src/core/db/llm_requests.rs`: `insert(pool, LlmRequestRow)`, `cleanup(pool, retention_days)`.
**Config** (`config.yml`):
```yaml
llm:
request_log:
enabled: true # default false
retention_days: 14 # default 14
```
---
### session_scratchpad
Per-session key-value store used by the `update_scratchpad` tool. Shared by all agents in the same chat session (including async sub-tasks, which are transparently redirected to the parent session's scratchpad at runtime); not persisted across sessions (tied to `chat_sessions.id`).
| Column | Type | Constraints |
| --- | --- | --- |
| `session_id` | INTEGER | NOT NULL REFERENCES `chat_sessions(id)` |
| `key` | TEXT | NOT NULL |
| `value` | TEXT | NOT NULL |
Primary key: `(session_id, key)`.
`upsert()` uses `INSERT ... ON CONFLICT DO UPDATE` — writing the same key twice overwrites the previous value. Contents are injected into every agent's system context via `build_openai_messages`.
---
## Schema Creation
The database has no migration system. `create_tables()` builds every table directly at its final shape on first boot (`CREATE TABLE IF NOT EXISTS`). There is no `schema_version` tracking, no incremental migration logic — the schema is a single flat baseline.
To change the schema, edit the relevant `CREATE TABLE` in `create_tables()` (`src/core/db/mod.rs`). Since the system has no existing users, schema changes simply start from a fresh DB (delete the old `data/skald.db` if needed).
---
### projects
Filesystem-linked projects, each with an independent Kanban board of tickets. `updated_at` is refreshed on every project or ticket operation (via `db::projects::touch`) so the UI can order by recency.
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `name` | TEXT | NOT NULL |
| `path` | TEXT | NOT NULL — absolute filesystem path for the project folder |
| `description` | TEXT | NOT NULL DEFAULT `''` |
| `run_context` | TEXT | nullable — `RunContext` JSON blob applied to tickets when they have no own run_context |
| `created_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
| `updated_at` | TEXT | NOT NULL DEFAULT `datetime('now')` — refreshed on every project/ticket operation |
DAO in `src/core/db/projects.rs`: `list(pool)` (ORDER BY updated_at DESC), `get(pool, id)`, `create(...)`, `update(...)`, `touch(pool, id)`, `delete(pool, id)` (CASCADE deletes tickets).
---
### project_tickets
Individual work items belonging to a project. Each ticket can be executed by a background agent.
**Lifecycle**: `todo` → `pending` (start() called) → `in_progress` (job starts) → `done` / `failed`
| Column | Type | Constraints |
| --- | --- | --- |
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT |
| `project_id` | INTEGER | NOT NULL REFERENCES `projects(id)` ON DELETE CASCADE |
| `title` | TEXT | NOT NULL |
| `description` | TEXT | NOT NULL DEFAULT `''` |
| `status` | TEXT | NOT NULL DEFAULT `'todo'` CHECK(`todo` \| `pending` \| `in_progress` \| `done` \| `failed`) |
| `agent_id` | TEXT | NOT NULL DEFAULT `'main'` |
| `run_context` | TEXT | nullable — `RunContext` JSON blob; stores only static config set at creation time (e.g. `security_group`). Runtime fields (`working_directory`, `allow_fs_writes`, `system_prompt`) are computed by `ProjectTicketManager.start()` and never persisted here. Falls back to `projects.run_context` if NULL. |
| `job_id` | INTEGER | nullable FK → `scheduled_jobs(id)` (no `ON DELETE` action); set when `start()` creates the job. Callers that delete a `scheduled_jobs` row (`scheduled_jobs::delete()`, cron's `cleanup_expired_single_runs`) must null this first, or the delete fails with a FK constraint error. |
| `result` | TEXT | nullable — final agent output (populated when status = `done`) |
| `error` | TEXT | nullable — error message (populated when status = `failed`) |
| `created_at` | TEXT | NOT NULL DEFAULT `datetime('now')` |
| `started_at` | TEXT | nullable — set when the job begins execution |
| `completed_at` | TEXT | nullable — set when the job finishes |
DAO in `src/core/db/project_tickets.rs`: `list_for_project(pool, project_id)`, `get(pool, id)`, `create(...)`, `delete(pool, id)`, `set_status(pool, id, status)`, `start(pool, id, job_id)`, `complete(pool, id, result, error)`, `reset(pool, id)`.
---
## Soft-Failure Pattern
On `handle_message` start, two recovery operations run:
1. **Orphaned user message check** — if the last `chat_history` row for the stack has `role='user'` or `role='agent'` (no following assistant message), it is marked `status='failed'`. This prevents sending a user→user sequence to strict LLM APIs (e.g. OpenRouter).
2. **`resume_pending_tools(stack_id, config, tx)`** — finds all `chat_llm_tools` rows still in `status='pending'` for the stack. For each one:
- Re-runs it through the `ApprovalManager` gate (rules may have changed since the interruption).
- `Deny` → marks the tool call `failed` immediately.
- `Require` → emits an `ApprovalRequired`/`PendingWrite` event and waits for human input, just like a live turn. If the WS is disconnected, returns early leaving the calls still pending.
- `Allow` → executes the tool directly; marks it `done` or `failed` depending on outcome.
After all pending calls are resolved, `run_agent_turn` runs normally and the LLM sees a complete history.
> **Note:** `fail_pending_for_stack()` exists in `db/chat_llm_tools.rs` but is not called from the session handler. Pending tool calls are re-executed (not hard-failed) so the LLM receives real results rather than synthetic errors.
---
## When to Update This File
- Any table schema changes (new column, new table, removed table)
- Migration approach changes
- A new soft-failure recovery step is added to `handle_message`
docs/desktop.md
+282
View File
@@ -0,0 +1,282 @@
# Desktop bundle (Tauri)
Skald ships in **two shapes** from the same source tree, selected by the
`desktop` cargo feature:
| Shape | How it runs | How the user reaches it | Typical host |
| --- | --- | --- | --- |
| **Headless server** (default, no `desktop` feature) | The binary blocks on its own tokio runtime, as before. | Browser on `http://<host>:<port>` (LAN-friendly, binds `0.0.0.0`). | Linux server, dev workstation, Docker container. |
| **Desktop bundle** (`--features desktop`) | A Tauri event loop wraps the same headless backend; the backend runs as a task on Tauri's shared tokio runtime. A system-tray icon exposes `Open` / `Quit`; the webview loads `http://127.0.0.1:<port>`. | Double-click the `.app` / `.exe` / `.AppImage`. | End-user macOS / Windows / Linux machine. |
The `desktop` feature is **opt-in and default-off**: every existing build path
(`cargo build`, `run.sh`, `Dockerfile`) is unchanged because `--no-default-features`
already excludes it.
---
## Build
### Dev mode (debug, no bundle)
```sh
cargo run --features desktop
```
A real Tauri window + tray icon appears. The webview loads
`http://127.0.0.1:<config-port>` from the running dev binary; the binary reads
`config.yml`, `agents/`, `skills/`, `web/` from the crate root exactly as the
headless build does (the cwd is **not** relocated in dev mode — see
*Data directory* below).
### Distributable bundle (release, packaged)
```sh
cargo tauri build --features desktop
```
This requires the `tauri-cli`:
```sh
cargo install tauri-cli --version "^2"
```
Produces native installers under `target/release/bundle/`:
| OS | Artifact |
| --- | --- |
| macOS | `Skald.app`, `Skald.dmg` |
| Windows | `Skald.msi`, `Skald.exe` (NSIS) |
| Linux | `Skald.deb`, `Skald.AppImage` |
> **macOS deployment target.** The default `whisper-local` feature compiles
> `whisper.cpp` (C++), whose `ggml-backend-reg.cpp` uses
> `std::filesystem::path` — introduced in macOS 10.15. `cargo tauri build`
> injects `MACOSX_DEPLOYMENT_TARGET` from
> `tauri.conf.json > bundle.macOS.minimumSystemVersion` (Tauri's default is
> `"10.13"`), on which `std::filesystem::path` is marked *unavailable*, so the
> ggml build fails with ~20 `'path' is unavailable` errors.
>
> The fix has **three coordinated pieces**, all pinned to `"10.15"` (the exact
> floor that unblocks `std::filesystem`):
>
> 1. **`.cargo/config.toml`** sets **two** forced env vars. Both are needed
> because the C++ compile otherwise ends up with two conflicting
> `-mmacosx-version-min` flags and clang lets the *last* one win:
> - `MACOSX_DEPLOYMENT_TARGET` → the `cc` crate turns this into the CFLAGS'
> `-mmacosx-version-min`.
> - `CMAKE_OSX_DEPLOYMENT_TARGET` → `whisper-rs-sys`'s `build.rs` forwards any
> `CMAKE_*` env var to cmake as `-DCMAKE_OSX_DEPLOYMENT_TARGET=…`, which
> sets CMake's *own* `-mmacosx-version-min` **and overrides a value cached
> in a stale `CMakeCache.txt`**. Without this, CMake's cached 10.13 wins.
>
> `force = true` makes cargo override whatever the Tauri CLI injects
> (`MACOSX_DEPLOYMENT_TARGET=10.13`), so the C/C++ build is
> **deterministically** 10.15 regardless of Tauri's env-propagation. It also
> gives the headless binary a 10.15 portability floor.
> 2. **`tauri.conf.json > bundle.macOS.minimumSystemVersion = "10.15"`** — the
> value baked into the bundle's `Info.plist` as `LSMinimumSystemVersion`.
> 3. If a build still fails after a target change, **wipe the stale CMake
> caches**: `rm -rf target/*/build/whisper-rs-sys-*` (`cargo clean -p
> whisper-rs-sys` does not always clear the cmake `out/` dir). Piece 1's
> explicit `-D` flag makes this rarely necessary, but it is the escape hatch.
>
> Keep pieces 1 and 2 in sync. Raise all to `"11.0"` / `"12.0"` only if you want
> an Apple-Silicon-only bundle.
For cross-platform CI builds use a GitHub Actions matrix (the
[`tauri-apps/tauri-action`](https://github.com/tauri-apps/tauri-action) is the
canonical setup).
---
## Architecture
```
┌────────────────── main.rs ──────────────────┐
│ install rustls ring provider │
│ init_logging() │
│ │
│ ┌── cfg(feature = "desktop") ──┐ │
│ │ desktop::run() │ │
│ │ tauri::Builder │ │
│ │ .setup(spawn backend) │ │
│ │ .on_window_event(hide) │ │
│ │ .run(ExitRequested→ │ │
│ │ shutdown_backend) │ │
│ └───────────────────────────────┘ │
│ ┌── cfg(not(feature = "desktop")) ─┐ │
│ │ tokio runtime → async_main() │ │
│ │ run_backend() │ │
│ │ wait_for_shutdown_signal() │ │
│ │ shutdown_backend() │ │
│ └───────────────────────────────────┘ │
└──────────────────────────────────────────────┘
```
### Module map
| Path | Role |
| --- | --- |
| `src/main.rs` | Dispatcher. Installs the rustls provider, sets up tracing, then branches on the `desktop` feature. Exposes `run_backend()` and `shutdown_backend()` shared by both entry points. |
| `src/desktop/mod.rs` | Tauri shell. Builds the system tray + menu, creates the main `WebviewWindow` (URL derived from `config.yml`'s `server.port`), spawns the backend on Tauri's shared tokio runtime, and handles graceful shutdown. Compiled only under `--features desktop`. |
| `src/config.rs` | `bootstrap_data_dir()` relocates the cwd to a per-user data dir when running inside a `.app` bundle (no-op in dev mode and headless mode). |
| `src/core/tools/restart.rs` | Branches on the feature: under `desktop`, restarts the Tauri process (cleanup + respawn the same binary, **no rebuild** — the bundle is read-only); otherwise `libc::_exit(-1)` so `run.sh` rebuilds. |
| `build.rs` | Calls `tauri_build::build()` under the `desktop` feature; no-op otherwise. |
| `tauri.conf.json` | Tauri bundle config: identifier, icons, security. The main window is **not** declared here — it is built programmatically in the setup hook so the URL can read the backend port from `config.yml`. |
| `capabilities/default.json` | Tauri v2 capability set for the `main` window (window show/hide/focus permissions). The frontend never invokes Tauri's JS API — it is the regular Skald web app served by Axum. |
| `icons/` | App icon (`icon.png`, `32x32.png`, `128x128.png`, `128x128@2x.png`, `icon.icns`, `icon.ico`) and a monochrome tray template (`tray-template.png`). See *Icons* below. |
### Why a single binary, not a launcher
`skald` is a binary crate (`src/main.rs`, no `src/lib.rs`). Putting the Tauri
shell in `src/desktop/` behind `#[cfg(feature = "desktop")]` — instead of a
separate crate — keeps everything private (no `pub` leakage) and lets
`tauri::generate_context!()` find `tauri.conf.json` in `CARGO_MANIFEST_DIR`.
### Why Tauri's runtime, not a fresh tokio
`tauri::async_runtime::spawn` lands the task on Tauri's internal tokio
runtime. One runtime, one process, no IPC bridge — the webview talks to the
backend over plain HTTP/WS on `127.0.0.1`, exactly like a browser would.
---
## System tray + window policy
* A single tray icon is built in `build_tray()` with two menu items: **Open**
and **Quit**.
* **Left-click** the tray icon toggles the main window (show+focus or hide).
* **Open** menu item shows + focuses the main window.
* The window's close button (traffic-light red on macOS, X on Windows/Linux)
is intercepted in `on_window_event` → `WindowEvent::CloseRequested` and
turned into a `hide()`. The app keeps running in the tray.
* **Quit** (and Cmd+Q / system shutdown) triggers `RunEvent::ExitRequested`.
The handler spawns `shutdown_backend()` on the async runtime, waits for it
to drain the HTTP server / Skald managers / DB pool, then calls
`app.exit(0)`. An `AtomicBool` re-entrancy guard prevents the second
`ExitRequested` (emitted by the `app.exit(0)` itself) from looping.
---
## Data directory
| Mode | Working directory at startup |
| --- | --- |
| Headless (`cargo run`, `run.sh`, Docker) | Whatever the user launched from (today, the crate root). Unchanged. |
| Desktop dev (`cargo run --features desktop`) | The crate root — same as headless, so `config.yml`, `agents/`, `skills/`, `web/` resolve from the source tree. |
| Desktop bundle (inside `Skald.app/Contents/MacOS/`) | Relocated to the OS per-user data dir, see table below. |
When packaged as a `.app` (or Windows / Linux equivalents), the process cwd is
typically `/`. `bootstrap_data_dir()` detects this (via
`running_from_bundle()` — a `.app/` heuristic on macOS, currently always-false
on Windows/Linux until those targets land) and `set_current_dir`s to the
per-user data dir:
| OS | Location |
| --- | --- |
| macOS | `~/Library/Application Support/Skald` |
| Windows | `%APPDATA%\Skald` (= `C:\Users\<u>\AppData\Roaming\Skald`) |
| Linux | `~/.local/share/Skald` |
This makes every relative path in `config.yml` (`db.path`, `web.static_dir`,
`data/`, `secrets/`, `models/`, …) resolve under the user's data dir without
touching the source tree.
If `config.yml` is missing on first launch, it is seeded from
`DEFAULT_CONFIG_EMBEDDED` (the `default.config.yaml` baked into the binary via
`include_str!`).
**Logs.** `init_logging()` runs *before* the cwd is relocated, so a relative
`logs/` would land against the launch cwd (`/` for a Finder-launched `.app`) and
silently fail to write. `config::resolved_log_dir()` returns an **absolute**
path under the data dir (`~/Library/Application Support/Skald/logs`) in a bundle,
so logs always land in a writable place; headless and desktop-dev keep the
relative `logs/`.
### Read-only bundled assets
The relocated data dir holds only **mutable** state (db, config, logs, secrets,
models, uploads). The **read-only** assets the backend needs — `agents/`,
`web/`, `skills/`, `commands/` — are packaged into the bundle's `Resources/`
dir via `tauri.conf.json > bundle > resources`, and made reachable from the
relocated cwd by `link_bundled_assets()` (in `src/config.rs`), which runs right
after the relocation:
* For each asset name it creates a **symlink** in the data dir pointing at the
copy inside `…/Skald.app/Contents/Resources/<name>`. Symlinking (not copying)
keeps the assets in lock-step with the installed app version.
* A pre-existing **real** directory in the data dir is treated as a user
override and left untouched; only stale symlinks are refreshed each launch.
Without this step `Skald::new` fails on launch with *"Failed to read agents
directory 'agents'"* and the app exits immediately — the assets live next to the
binary, not in the freshly-relocated cwd.
> **Note (App Translocation).** An unsigned, quarantined `.app` launched from
> `~/Downloads` or a mounted `.dmg` is run from an ephemeral read-only path by
> Gatekeeper, so the symlink targets change per launch (they are recreated each
> time, so this is harmless). Moving the app to `/Applications` clears the
> quarantine and stabilises the paths.
---
## `restart` tool behaviour
`src/core/tools/restart.rs` branches on the feature flag:
| Mode | Behaviour |
| --- | --- |
| Headless | `libc::_exit(-1)` (= exit code 255). `run.sh` sees 255, runs `cargo build`, relaunches. **Rebuilds** the binary — used after the agent edits source code. |
| Desktop | `AppHandle` is fetched from `desktop::app_handle()` (a `OnceLock` populated in the setup hook); then `handle.cleanup_before_exit()` + `std::process::Command::new(current_exe).spawn()` + `std::process::exit(0)`. **No rebuild** — the bundled binary is read-only. Useful for picking up `config.yml` / DB changes that are only read at startup. |
See also [self-rewriting.md](self-rewriting.md).
---
## Icons
Two families live under `icons/`:
| Family | Files | Purpose |
| --- | --- | --- |
| App icon (colour) | `icon.png` (1024×1024 source), `32x32.png`, `128x128.png`, `128x128@2x.png`, `icon.icns` (macOS), `icon.ico` (Windows) | Window icon, bundle icon. |
| Tray template (monochrome) | `tray-template.png` (32×32, black on transparent) | macOS menu-bar template image (auto-recoloured by the system for dark/light). On Windows/Linux a coloured icon would be more visible — currently `default_window_icon()` is used as a fallback everywhere until a Tauri 2.x image-loading API is wired in. |
The current subject is a stylised amber feather on a dark rounded square
(Skald = Norse *bard*). To regenerate from a new design, edit
`/tmp/gen_all_icons.py` (Pillow) and re-run `iconutil` for `.icns` and `magick`
for `.ico`. Sources live with the icons; the script is not yet committed.
---
## Known limitations / next steps
* **`LSUIElement` (menu-bar-only on macOS).** Removed from `tauri.conf.json`
because Tauri v2 dropped the `bundle.macOS.infoPlist` key (v1) — the v2 way
is a custom `Info.plist` file under `src-tauri/`. Until the crate layout
grows a `src-tauri/` directory, the app shows both a Dock icon and a tray
icon. Add the `Info.plist` to make it menu-bar-only.
* **Tray template icon.** The bundled `tray-template.png` is not yet wired up
(Tauri 2.11.5's `Image::from_path` / `from_bytes` API surface differs from
later versions). The tray currently reuses `app.default_window_icon()`, which
is the colour app icon — visible on macOS but not template-styled.
* **Bundled assets — done.** `agents/`, `web/`, `skills/`, `commands/` are
packaged via `bundle.resources` and symlinked into the data dir by
`link_bundled_assets()` (see *Read-only bundled assets* above). Remaining
polish: user-agent overrides beyond the symlink escape-hatch, and refreshing
the copy-on-write story for signed/notarized distribution.
* **Windows/Linux bundle.** `running_from_bundle()` currently only detects
`.app/` on macOS; Windows (`Program Files`) and Linux
(`/usr/share`/`/opt`) detection is TBD.
* **CI matrix.** No GitHub Actions workflow yet for cross-platform bundle
builds.
---
## When to update this file
* The `desktop` feature or its dependencies change.
* The tray menu, window policy, or shutdown path change.
* The data-directory relocation heuristic changes (new platform, new path).
* The `restart` tool's desktop-mode behaviour changes.
* The packaging story is completed (`bundle.resources`, `Info.plist`, CI).
docs/frontend.md
+648
View File
@@ -0,0 +1,648 @@
# Frontend
## HTTP Server
Axum router assembled in `src/frontend/server.rs` (`WebServer::build_router_with_plugins`):
- `/api/*` — the app's HTTP handlers (`State<Arc<Skald>>`), plus per-plugin routers nested under `/api/plugin/<id>/`.
- `/data/*` — the on-disk `data/` directory (served via `tower_http::services::ServeDir`, so e.g. `/data/gmail_attachments/...` resolves to a file URL).
- Static fallback — the `web.static_dir` directory (`ServeDir`), i.e. the SPA assets.
**Compression.** A global `tower_http::compression::CompressionLayer` (gzip + brotli, enabled via the `compression-gzip` / `compression-br` features) wraps the whole router. Encoding is negotiated from the client's `Accept-Encoding` (no-op for clients that don't advertise one and for already-compressed media). The main motivation is the **mobile relay path**: the native WebView's HTTP traffic is reverse-proxied byte-for-byte over a relay pipe (`http-local-proxy`), so shrinking text assets (JS/CSS/HTML) ~70-90% means far fewer bytes cross the slow link. Desktop browsers benefit the same way.
**Caching.** Static responses (SPA assets + `/data/*`) carry `Cache-Control: no-cache` (applied via a `tower_http::set_header::SetResponseHeaderLayer` on the two `ServeDir`s). The browser may store the asset but **must revalidate before use** — so after a self-rewrite/restart the client never serves a stale asset (no heuristic caching), and revalidation yields cheap `304`s (`ETag`/`Last-Modified` from `ServeDir`). `/api/*` is deliberately left without the header (dynamic data, not cached). Note: because the mobile loopback proxy listens on an OS-assigned port, WKWebView's URLCache is keyed by a port that changes across app/tab restarts — so cross-session cache hits depend on that port being stable (tracked as a separate, app-side follow-up).
## WebSocket Endpoint
`GET /api/ws?source=<string>`
`source` identifies the conversation: `web` (default, desktop copilot), `mobile` (mobile chat page), or `project-{id}` (a project's interactive chat — see below). The same endpoint serves all; ChatHub maintains one independent, persistent session per source.
One connection per source. The connection is upgraded by Axum's WS handler in `src/frontend/api/ws.rs`. The client sends one `ClientMessage`, receives a stream of `ServerEvent`s, then can send additional messages (cancel, approval) while events are in flight.
History for a source: `GET /api/<source>/messages` (or the legacy alias `/api/web/messages`).
### Project chats
A project's chat is a persistent session bound to source `project-{id}` and driven by the
`project-coordinator` agent. `POST /api/projects/{id}/session` provisions (or resumes) it,
seeding the session's `RunContext` with the project's working directory, fs-write grants, and
context — then returns `{ source, session_id }`. The frontend connects the WS to that source.
Because the session is **not** ephemeral and ChatHub reuses the existing session for a source,
the conversation persists and is resumed on reopen. Resetting it (`POST /api/sessions?source=project-{id}`)
recreates it with the coordinator agent, not `main` (the handler resolves agent + RunContext per
source via `provisioning_for_source`).
In the desktop copilot these appear as browser-style **tabs**: `General` (the `web` source, always
present, not closable) plus one tab per open project chat. The board's **Open Chat** button
dispatches a `project-chat-open` window event (`{source, label}`); the copilot adds/focuses the tab
and switches the live connection via `ChatSession._switchSource(source)`. Closing a project tab is
UI-only — the session persists and can be reopened from the board.
---
## ClientMessage Fields
| Field | Type | Description |
|---|---|---|
| `content` | `String` | The user's prompt text |
| `client` | `Option<String>` | Named LLM model override (or `"auto"`) |
| `attachments` | `Attachment[]` | Files uploaded beforehand via `POST /api/{source}/uploads`; each `{ path, name, mimetype?, filesize? }`. See [Attachments](#attachments) |
---
## ServerEvent Types
All events are JSON objects with a `"type"` tag (snake_case).
| type | Key fields | When emitted |
|---|---|---|
| `tool_start` | `tool_call_id`, `message_id`, `name`, `arguments`, `label_short`, `label_full`, `path?` | Tool call recorded, about to execute. `path` (optional) is the viewable file the call targets — rendered as a clickable link to the file viewer |
| `tool_done` | `tool_call_id`, `result` | Tool executed successfully |
| `tool_error` | `tool_call_id`, `error` | Tool execution failed |
| `agent_start` | `stack_id`, `parent_tool_call_id`, `agent_id`, `depth` | Sub-agent stack frame opened |
| `agent_done` | `stack_id` | Sub-agent stack frame closed |
| `thinking` | `message_id`, `content`, `input_tokens`, `output_tokens` | LLM produced text before tool calls |
| `pending_write` | `request_id`, `tool_call_id`, `path`, `old_content`, `new_content` | Approval required for write/command |
| `agent_question` | `request_id`, `tool_call_id`, `title`, `question`, `suggested_answers` | Sub-agent needs user clarification |
| `file_changed` | `path` | A tool wrote to a file |
| `open_file` | `path` | Agent-driven file open: the file viewer supports Markdown, source code, plain text, raster images (PNG/JPG/GIF/WebP/…), SVG, PDF, LaTeX (`.tex` / `.latex` — compiled to PDF automatically on the server), and HTML (`.html` / `.htm` — rendered live in an origin-isolated `<iframe srcdoc sandbox="allow-scripts">`, toggleable to source). For a LaTeX document, open the `.tex` source rather than a pre-built `.pdf`: only the `.tex` path is compiled, cached dependency-aware, and watched for dependency changes — a raw `.pdf` is served statically and stays stale when its sources change. Emitted by the `show_file_to_user` interface tool (SPA-only, injected in `ws.rs`; `path` normalised relative to the project root) |
| `done` | `message_id`, `stack_id`, `content`, `input_tokens`, `output_tokens` | Turn complete, final response |
| `truncated` | `output_tokens` | LLM hit token limit (`finish_reason=length`) |
| `error` | `message` | Fatal error (session handler failed) |
| `model_fallback` | `from`, `to`, `reason` | Active model swapped to fallback automatically |
| `llm_failed` | `tried`, `last_error` | All LLM fallback attempts exhausted |
| `approval_required` | `request_id`, `tool_call_id`, `tool_name`, `arguments` | Non-file tool call requires user approval |
| `approval_resolved` | `request_id`, `approved` | Approval resolved (any source); all clients update their UI |
| `user_message` | `message_id`, `content`, `attachments?` | A user message persisted to history, echoed to **every** client of the source (the sender included). Emitted at save time — at turn start, or at a round boundary for mid-turn injection — so the bubble lands in its correct position. Carries the typed text + structured attachments (never the `[SYSTEM INFO]` block) |
| `new_session` | `session_id` | Session was cleared (`/new`, `/clear`); clients reset their message list |
| `turn_running` | `running` | Sent to a client on (re)connect: whether a turn is in flight for its session, so a reloaded page restores the SEND→STOP button |
| `client_selected` | `client` | The pinned LLM client for the source changed (`/model` command or dropdown change). Clients update their dropdown/select to match — the backend is the single source of truth |
---
## Attachments
The desktop copilot and the mobile chat page let the user attach files to a message.
Files are added with the paperclip button, **drag & drop** onto the composer, or **paste**
(`Ctrl+V`) — all handled by `ChatSession._addFiles` / `_onDrop` / `_onPaste` in
`web/lib/chat-session.js`. Text is required: a message is never sent with attachments alone.
Flow:
1. On selection, each file is uploaded immediately to `POST /api/{source}/uploads`
(multipart). The handler (`src/frontend/api/uploads.rs`) **streams** each part straight
to `data/uploads/{session_id}/` (`field.chunk()` → file, never buffered in RAM) and the
route disables the default body-size limit. It returns the saved `Attachment`s
(`{ path, name, mimetype, filesize }`, `path` project-root-relative so `/data/…` serves it).
2. The pending attachments render as **chips above the textarea** (`renderAttachmentChips` in
`copilot-render.js`, `removable: true`, with a spinner while the upload is in flight).
3. On send, the client posts `{ content, attachments }` over the WebSocket. `content` is the
clean typed text; `attachments` are the uploaded objects.
4. Server-side, the message is persisted as a user `chat_history` row, and a `user_message`
event (carrying its `message_id` + `attachments`) is broadcast **at save time** — at turn
start, or at a round boundary for messages injected mid-turn (see *Telnet-style echo* below).
The attachments are stored in the generic `metadata` JSON column. **The `[SYSTEM INFO]` block
the LLM sees is generated on the fly** by the message builder from `metadata.attachments`
(path-only — the agent reads the files with its own tools), so `content` and the UI stay
clean. On reload, `build_items`
surfaces `attachments` again so the chips reappear (clickable → file viewer).
The Telegram plugin reuses the same `MessageMetadata`/`Attachment` types for Document/Photo
uploads, so those render as chips too when viewing the `telegram` source — see
[plugins/telegram.md](plugins/telegram.md).
## Sending messages: telnet-style echo + mid-turn injection
The client does **not** render the user's message optimistically. `_send()` clears the
composer and posts `{ content, attachments }` over the WebSocket; the bubble appears only when
the backend persists the message and echoes it back as a `user_message` event (with its real
`message_id`). This "telnet" model makes the backend the single source of truth — no
client-generated id, no content-based dedup, and every client (the sender included) renders the
same echo. The `user_message` handler in `chat-session.js` therefore just pushes a bubble; the
old dedup against a local optimistic push is gone.
- **Sending while a turn is running is allowed.** The composer is no longer disabled on
`_waiting`, and the send button is shown **alongside** the STOP button during a turn (Enter
still sends on desktop). The message is queued and [injected into the running turn](session.md#mid-turn-injection)
at its next round boundary; the bubble appears (via echo) at that moment — i.e. after the
current round's tools, where the agent actually sees it. With a long-running tool the echo is
delayed until the round ends.
- **Slash commands are the exception:** they are handled server-side and never persisted/echoed,
so `_send()` renders them optimistically (`content.startsWith('/')`).
- A `/stop` before the next round boundary drops the queued message: no echo, no bubble.
## Slash Commands (Web Copilot)
The web copilot supports the following slash commands, intercepted server-side in
`src/frontend/api/ws.rs` before reaching the LLM:
| Command | Effect |
|---|---|---|
| `/new` | Create a new chat session (handled client-side, clears context) |
| `/help` | Show available commands |
| `/models` | List available LLM models ordered by priority (numbered `0..N`, index 0 is `auto`) |
| `/model <N\|name\|auto>` | Pin the model for this chat by index, name (substring allowed), or reset to `auto`. The web dropdown and the `/model` command share the same backend state (`ChatHub.selected_clients[source]`); changes from either mutate the SOT and broadcast `ClientSelected`, so all open tabs/mobile update in sync. Cleared on server restart |
| `/context` | Show last turn's token usage (`↑X tok · ↓Y tok`) |
| `/cost` | Show total spend for this session in USD (sync sub-agents included; async tasks excluded). `None` → "no cost recorded" when the provider does not report pricing |
| `/compact` | Force context compaction (bypasses the token threshold) |
| `/resettools` | Remove all activated tool groups (MCP servers + `config`) from the session |
| `/sethome` | Set web as the home source for background notifications |
Any other message starting with `/` is treated as an unknown command: the server
replies with an "Unknown command" notice followed by the help list, and never
forwards it to the LLM.
---
## Tool Call Status Lifecycle
Tool calls in `chat_llm_tools` progress through these states:
| DB status | Meaning | Frontend `build_items` |
|------------|---------|------------------------|
| `running` | Tool executing — no user action required | `status: 'error', error: 'Interrupted.'` (shown after page refresh/restart) |
| `pending` | Blocked on explicit user input (approval gate `Require`, or `ask_user_clarification`) | `status: 'pending'` → shows approval/clarification form |
| `done` | Completed successfully | `status: 'done'` |
| `failed` | Terminated with error | `status: 'error'` |
On **page refresh** or **app restart**, the frontend detects pending/interrupted tools in history (`_hasPendingTools` flag set in `_loadHistory`). On `ws.onopen` it sends `{"type":"resume"}`, which triggers `resume_turn()` → `resume_pending_tools()`:
- `running` tools → re-executed through the approval gate
- `pending` tools (approval) → approval channel re-registered, `approval_required` re-emitted with new `request_id`
- `pending` tools (`ask_user_clarification`) → question re-asked via `dispatch_ask_user_clarification`
- `call_agent` tools → skipped here; child stack is resumed by `resume_turn()` cascade (see below)
`resume_turn()` also cascades upward when a sub-agent stack completes: it terminates the child, marks the parent's `call_agent` tool as `done`, then continues running the parent stack until the root emits `Done`.
---
## Approval Flow
1. Server emits `pending_write` with `request_id`, `path`, `old_content`, `new_content`.
2. Frontend shows a diff and prompts the user.
3. User approves → client sends: `{"type":"approve_write","request_id":<N>}`
4. User rejects → client sends: `{"type":"reject_write","request_id":<N>,"note":"<optional reason>"}`
5. Server receives the message via `handle_approval_msg()`, calls `handler.resolve_approval(request_id, decision)`.
6. The `oneshot` channel unblocks in `run_agent_turn`, execution proceeds or is skipped.
Before blocking on the approval channel, the server sets `status='pending'` in `chat_llm_tools` via `set_approval_pending()`. This is what distinguishes "waiting for user" from "tool was executing when the session was interrupted" (`running`).
## Clarification Flow
### Interactive sessions (web / Telegram)
1. An agent (root or sub-agent) calls `ask_user_clarification(title, question, suggested_answers?)`.
2. Server sets `status='pending'` for the tool call, registers it with `ClarificationManager` (so it also appears in the Agent Inbox), then sends `agent_question` with `request_id`, `title`, `question`, and optional `suggested_answers`.
3. Frontend shows the question and collects a free-text answer (suggested answers shown as clickable chips).
The `question` body is rendered as **sanitised Markdown** via the shared `renderMarkdown()` (`web/lib/base.js`, marked + DOMPurify), so the agent may use `**bold**`, lists, `code`, etc. The `title` and the suggested-answer chips remain plain text.
4. Client sends: `{"type":"answer_question","request_id":<N>,"answer":"<user text>"}`
5. Server calls `handler.resolve_question(request_id, answer)`.
6. The answer is returned as the tool result and the agent continues.
On WS disconnect while waiting, `cancel_pending_questions()` drops all channels, causing the awaiting tool call to fail with an error. On reconnect, auto-resume re-asks the question.
### Background sessions (cron / tic)
1. The agent (root or sub-agent) calls `ask_user_clarification(title, question, suggested_answers?)`.
2. `dispatch_ask_user_clarification` sets `status='pending'` then registers with `ClarificationManager` (in-memory, in-process).
3. The entry appears in `GET /api/inbox` under `clarifications`.
4. User answers via the Agent Inbox page → `POST /api/inbox/clarifications/:request_id/resolve`.
5. The `oneshot` channel unblocks, answer is returned as tool result, agent continues.
Cancel message (abort current turn): `{"type":"cancel"}`
---
## Elicitation Flow (MCP server-initiated input)
An MCP server can request input *during* a tool call (e.g. a sudo password). Unlike
clarification, this is driven by the **server**, not an agent tool call, and supports
`accept`/`decline`/`cancel` with a masked input. See `docs/mcp.md` for the protocol.
1. The MCP server sends `elicitation/create`; the `mcp-client` read-loop bridges it to
`ElicitationManager::register` (in-memory, no session binding).
2. The entry appears in `GET /api/inbox` under `elicitations` and as a global
`elicitation_requested` event (so the Inbox badge / mobile push update).
3. The Agent Inbox renders a **"Secrets"** card (`_renderElicitationCard` in
`web/lib/inbox-mixin.js`): a masked `<input type="password">` when `sensitive`, a
plain input otherwise, or just Confirm/Reject when `is_confirmation`.
4. User confirms or rejects → `POST /api/inbox/elicitations/:request_id/resolve` with
`{action, content?}`. On `accept` the value is packed as `{ [field_name]: value }`.
5. `ElicitationManager::resolve` unblocks the bridge, which writes the JSON-RPC reply to
the server's stdin. The value is **never** logged, broadcast, or persisted.
---
## Lit Component Inventory
| File | Element | Responsibility |
|---|---|---|---|
| `web/lib/chat-session.js` | `ChatSession` (base) | Shared WS logic, message state, all approval/LLM event handling, **voice recording + transcription** (`_checkTranscribe`, `_startRecording`, `_stopRecording`, `_toggleRecording`, `_submitAudio`; renders a mic button when `/api/transcribe/has` returns 204), and textarea helpers (`_inputEl` hook, `_autoResize`). Subclasses override `_wsSource`, `_inputEl`, `_getInputContent`/`_clearInput` (defaults now driven by `_inputEl`), `_scrollToBottom`, `_onMessagePushed`. Effective source is `_source` (`_activeSource ?? _wsSource`); `_switchSource(source)` tears down the WS, reloads history, and reconnects to switch sessions live. **Attachments** (`_attachments` state, `_addFiles`/`_removeAttachment`/`_onDrop`/`_onPaste`): upload on selection, send with the next message — see [Attachments](#attachments) |
| `web/components/copilot.js` | `<app-copilot>` | Desktop copilot panel (`_wsSource='web'`); resize, composer input with model pill and auto-resize textarea. **Voice recording is inherited from `ChatSession`**; only the desktop-only Ctrl+Space push-to-talk shortcut (`_onKeydown`/`_onKeyup`) lives here. Browser-style tabs (`General` + project chats); listens for the `project-chat-open` window event to add/focus a project tab |
| `web/components/shared/chat-page.js` | `<chat-page>` | Mobile chat page; extends `ChatSession` with a mobile-specific layout. Composer mirrors the desktop copilot: a single unified box (`.chat-page-composer`) wrapping an auto-resizing textarea with a toolbar below — toolbar-left holds a native `<select>` model pill (`auto` + providers, opens the OS picker), toolbar-right holds the mic button (inherited recording) and the send/stop button. **Enter inserts a newline** (no Shift+Enter on mobile) — only the send button submits. The `source` prop (default `mobile`) re-points the chat: when it changes the component calls `_switchSource` to bind to a project's `project-{id}` session; it also honours `source` on the first connect (cold deep link from the native shell); inside a project the header shows the project `label` + a back button that emits `project-exit` |
| `web/components/shared/projects-page.js` | `<projects-page>` | Mobile project list. Loads `GET /api/projects`; tapping a project `POST`s `/api/projects/{id}/session` and emits a `project-open` event (`{source, label}`) so `<mobile-app>` re-points the chat |
| `web/components/copilot-render.js` | (helpers) | Renders messages, tool call blocks, diffs — shared by copilot and chat-page. Tool labels and diff headers render the call's `path` (when present) as a clickable link via `renderLabel`/`renderPath` → `openFile(path)` |
| `web/components/sidebar.js` | `<app-sidebar>` | Navigation sidebar; polls `/api/inbox` every 10 s for badge count |
| `web/components/topbar.js` | `<app-topbar>` | Top navigation bar |
| `web/components/editor.js` | (removed) | The legacy `<app-main>` editor panel was removed. Use `<file-viewer>` (see [File Viewer](#file-viewer)) instead |
| `web/components/shared/file-viewer-base.js` | `FileViewerBase` (base) | Shared file-viewer engine: fetch, kind detection (image/pdf/svg/latex/html/text/binary), markdown asset rewriting, LaTeX compile + error formatting, HTML preview⇄source toggle, live watcher, and `_renderBody`. Navigation-agnostic — driven by `_show(path)` / `_hide()`; subclasses supply the chrome |
| `web/components/file-viewer-page.js` | `<file-viewer-page>` | Desktop subclass of `FileViewerBase`. Self-routes off the hash (`#file_viewer?path=...`) via the `llm-page-change` + `hashchange` events; renders in the main workspace. Opened by `window.openFile(path)`. Preview only — editor + watcher tabs planned |
| `web/components/shared/file-viewer-mobile.js` | `<mobile-file-viewer-page>` | Mobile subclass of `FileViewerBase`. Prop-driven (`visible` / `path`, set by `<mobile-app>`'s hash router); full-screen with a mobile header + back button |
| `web/components/cron-jobs.js` | `<cron-jobs-page>` | Cron job management UI — columns: Title (+ one-shot badge), Cron, Agent, Last run, Next run, Enabled, Actions |
| `web/components/agent-inbox.js` | `<agent-inbox-page>` | Unified inbox for pending approvals, clarifications, and MCP elicitations ("Secrets"); polls `/api/inbox` every 8 s when open |
| `web/components/models-hub.js` | `<models-hub-page>` | Models hub — 3-card landing page (LLM / Transcription / Image Generation) with live model counts; internal navigation to sub-sections |
| `web/components/models-llm.js` | `<models-llm-section>` | LLM model management: drag-and-drop priority, catalog picker (OpenRouter/Ollama/…), add/edit/delete |
| `web/components/models-transcribe.js` | `<models-transcribe-section>` | Transcription model CRUD; filters providers by `supported_types.includes('transcribe')` |
| `web/components/models-image.js` | `<models-image-section>` | Image generation model CRUD; filters providers by `supported_types.includes('image_generate')` |
| `web/components/llm-providers.js` | `<llm-providers-page>` | LLM provider management |
| `web/components/agents.js` | `<agents-page>` | Agent discovery and configuration |
| `web/components/approval-groups.js` | `<approval-groups-page>` | Groups list: create, rename, duplicate, delete permission groups; navigates to rules view via `approval-navigate` event |
| `web/components/approval-rules.js` | `<approval-rules-page>` | Per-group rules view: rule matrix, override/low-priority panels, default action bar; shows when `approval-navigate` fires with a non-null group |
| `web/components/llm-requests.js` | `<llm-requests-page>` | LLM request log viewer with filterable table, pagination, clickable rows that drill into detail view (`#llm-requests/<id>`) |
| `web/components/llm-request-detail.js` | `<llm-request-detail>` | LLM request detail: stat bar, system prompt, conversation messages, tool definitions, response — with collapsible sections |
| `web/components/session-detail.js` | `<session-detail-page>` | Read-only debug view of any session. Navigate to `#session/{id}` to load. Shows full message tree with tool calls, sub-agent frames, synthetic user messages, and collapsible reasoning blocks. Not linked from the sidebar — accessed by typing the hash directly. |
All components extend `LightElement` from `web/lib/base.js` (Lit-based).
### Markdown rendering & link behavior
`renderMarkdown(text)` (in `web/lib/base.js`) is the single entry point for rendering assistant/file markdown: it runs `marked.parse` then `DOMPurify.sanitize`. **External links** (http/https whose origin differs from the page) get `target="_blank" rel="noopener noreferrer"` via a DOMPurify `uponSanitizeElement` hook, so they open in a new tab instead of navigating away from the app. Relative paths, hash anchors (e.g. the app's `#file_viewer?...` routing), and non-http schemes (`mailto:`, `tel:`) are left untouched, preserving in-app navigation and native handlers.
### Approval Rules navigation protocol
`<approval-groups-page>` and `<approval-rules-page>` communicate via a custom DOM event instead of shared state:
| Event | Detail | Who fires | Who handles |
| --- | --- | --- | --- |
| `approval-navigate` | `{ group: ToolPermissionGroup \| null }` | groups page (navigate to rules) | rules page (show with group) |
| `approval-navigate` | `{ group: null }` | rules page (`← Back` button) | groups page (show again) |
Hash persistence: `window.location.hash` is set to `#approval/{group_id}` when navigating to a rules view. On page load, the groups page reads the hash and re-fires the event so deep-links and page reloads restore the correct sub-view.
### Agent Inbox page
Approval cards have a yellow left border; clarification cards have a blue left border. Clarification cards show suggested-answer chips (click pre-populates the input) and a free-text input — submit with Enter or the Send button.
Approval cards have Approve / Reject buttons and a timed bypass menu (15 min / 1 hour / Session) scoped to the tool's category or MCP server. The bypass scope auto-detects from the pending approval's metadata: `tool_category` for category-scoped, `mcp_server` for MCP server-scoped, otherwise `all`. The REST API also supports `bypass_secs` and `bypass_scope` fields in the resolve body.
---
## Mobile App & Native Shell
The mobile UI (`web/mobile.html` → `<mobile-app>`) is the same SPA the desktop uses, laid out for touch. It is also what the **native iOS shell** renders in a WKWebView over the relay (see [relay/pipe.md](relay/pipe.md)).
### Hash routing
`<mobile-app>` drives its active section from the URL hash — the same model as the desktop sidebar (`web/components/sidebar.js`), so the URL is always the source of truth. `_readHash()` / `_applyHash()` react to `hashchange` and `popstate`; `_nav(section)` sets `location.hash`. This gives deep links, working back/refresh, and — for the native shell — a single observable signal for menu sync.
| Hash | Section | Notes |
|---|---|---|
| `#inbox` | Inbox | Pending approvals + clarifications |
| `#projects` | Projects | Project list |
| `#chat` | Chat | Main mobile session (source `mobile`) |
| `#chat/project-<id>` | Chat | Bound to a project's session (source `project-<id>`). The header label resolves from `/api/projects` (cached in `<mobile-app>`), so a cold deep link still shows the name. Back/refresh keep the user inside the project |
| `#file_viewer?path=<enc>` | File viewer | Full-screen file preview, reached from content via `openFile(path)` (e.g. a clickable tool path) — **not** a bottom-nav tab. The back button returns to the previous section via history |
| `#notifications`, `#settings` | (coming soon) | Placeholder |
`<chat-page>` (`web/components/shared/chat-page.js`) honours its `source` prop on the **first** connect (via `_activeSource`), so a cold `#chat/project-<id>` deep link connects straight to that session instead of opening the `mobile` session and switching a tick later.
`_applyHash()` **skips** the `skaldNav` notify for the `file_viewer` section: the viewer is a transient overlay reached from content (not a tab), so opening a file from the chat must not deselect the native "chat" tab.
### Native shell mode (`?native=true`)
When loaded with `?native=true`, `<mobile-app>` sets a `data-native` attribute and **hides its HTML bottom nav** — the native tab bar is the chrome. `web/css/mobile.css` drops the web-side safe-area insets under `mobile-app[data-native]` (the native chrome owns the status-bar + home-indicator insets). Everything else is identical to the mobile-browser path.
### Native ↔ Web contract
The native tab bar and the web router stay in sync over one mechanism each direction:
| Direction | Mechanism | Payload / call |
|---|---|---|
| Native → Web | set the hash | `webView.evaluateJavaScript("location.hash='#projects'")` — the web `hashchange` handler switches section. A project deep link works too (`#chat/project-<id>`). Same code path the browser uses. |
| Web → Native | `WKScriptMessageHandler` named **`skaldNav`** | on every section change `<mobile-app>` calls `window.webkit.messageHandlers.skaldNav.postMessage({ section, project })`, where `project` is `null` or `'project-<id>'`. The shell updates its tab highlight. No-op when the handler is absent (mobile browser). |
The `skaldNav` bridge is the reliable sync: relying solely on observing the WKWebView URL is fragile, because same-document (hash-only) navigations don't reliably fire `WKNavigationDelegate` callbacks across iOS versions.
---
## File-Change Watcher (live reload)
`<file-viewer-page>` automatically reloads when the file it is showing changes
on disk — whether the change comes from a Skald tool, an external editor
(VSCode, vim, …), or any other process. The mechanism is a dedicated
WebSocket.
### Endpoint
`GET /api/file/watch` — upgrades to a long-lived WebSocket. Client → server
commands:
| Command | Effect |
|---|---|
| `{"op":"subscribe","path":"docs/index.md"}` | Start watching `path` (relative or absolute — same path model as `GET /api/file`) |
| `{"op":"unsubscribe","path":"docs/index.md"}` | Stop watching `path` |
Server → client messages:
| Message | Meaning |
|---|---|
| `{"type":"subscribed","path":"..."}` | Ack — watch installed successfully |
| `{"type":"unsubscribed","path":"..."}` | Ack — watch removed |
| `{"type":"changed","path":"..."}` | The file at `path` changed on disk (any event kind: create/modify/remove) |
| `{"type":"error","path":"...","error":"..."}` | Watch install failed (e.g. path does not exist, permission denied) |
| `{"type":"error","error":"..."}` | Malformed client message or unknown op (no `path`) |
The `path` field always round-trips the original user-supplied string, so the
client can match it against the path it asked to watch.
### Implementation notes
- **Backend** (`src/frontend/api/file_watch.rs`): one `notify::RecommendedWatcher`
per watched file per connection (one watcher per file — for a LaTeX source
that means one per dependency, see below). On disconnect every watcher is
dropped and OS resources are released automatically. Path resolution uses
`fs_tools::resolve` (same as `GET /api/file`), so absolute paths are used
as-is and relative paths resolve against Skald's process CWD.
- **LaTeX dependency watching**: when subscribing to a `.tex` / `.latex`
source, the server expands the single path into the full dependency set
discovered via `LatexCompiler::watch_paths_for()` (which reads the cached
`.fls` recorder file — see [LaTeX](#latex) below). One OS watcher is
installed per dependency; any change to any of them is forwarded to the
client as a `changed` event for the original `.tex` path. The dependency
set is re-synced on every change event (watchers dropped and re-installed
with the fresh `.fls`), so newly-added `\input`s are picked up
automatically. On the very first subscribe, when no compile has happened
yet, only the main `.tex` itself is watched; once the viewer's first
compile writes the `.fls`, the next change event triggers the re-sync.
- **Frontend** (`web/lib/file-watcher.js`): singleton `fileWatcher` with a
single persistent connection, ref-counting per path (multiple consumers of
the same path share one OS watcher and one subscribe message),
auto-reconnect on close with 2 s backoff, and automatic re-subscribe of all
active paths on reconnect. Consumers call `fileWatcher.watch(path, cb)` and
get back an `unsub()` function.
- **`<file-viewer-page>`** subscribes when it opens (or when the path changes
via hash navigation) and unsubscribes when it closes or navigates away.
Change notifications are debounced (300 ms) and trigger a **silent reload**
(no spinner, no flicker — image previews swap the object URL only after the
new blob is ready, text previews replace the content atomically).
- **Cross-platform:** uses `notify`'s recommended backend (FSEvents on macOS,
inotify on Linux, ReadDirectoryChangesW on Windows).
- **Dirty-buffer conflict handling** is not implemented yet — there is no
editor tab yet. When the CodeMirror editor lands (roadmap), a changed event
arriving while the buffer has unsaved edits will show an "Overwrite / Discard
/ Ignore" banner instead of auto-reloading.
---
## File Viewer
`<file-viewer-page>` is a top-level page that previews files from disk in the
main workspace, so users (and agents, in future phases) can read
markdown/code/images without leaving the UI. It is registered in
`web/app.js` and lives once in the DOM at `index.html` (default
`display:none`, shown via the standard page router — see below).
The fetch / kind-detection / markdown-asset / LaTeX-compile / live-watch logic
and `_renderBody` live in `FileViewerBase` (`web/components/shared/file-viewer-base.js`),
shared with the mobile viewer. The base is navigation-agnostic — driven only by
`_show(path)` / `_hide()`; each subclass supplies its own chrome and decides when
to call them: the desktop `<file-viewer-page>` from the hash, the mobile
`<mobile-file-viewer-page>` from props (see [Mobile](#mobile) below).
### Opening files
The global helper `openFile(path)` is the single entry point — defined in
`web/lib/open-file.js`:
```js
openFile('data/memory/index.md');
// or
window.openFile('docs/frontend.md');
```
`openFile(path)` sets `location.hash` to `#file_viewer?path=<enc>`, which the
sidebar hash router (`web/components/sidebar.js:_pageFromHash`) resolves to
the `file_viewer` page. This means:
- **Back/forward browser navigation** works naturally.
- **Deep-linkable** — the URL can be shared or bookmarked.
- **The chat and everything else stay usable** while the file is open
(clicking any other sidebar entry just changes the hash and the page
switches out).
Both surfaces (`openFile(...)` and setting the hash directly) are equivalent.
Components that want to open a file should call `openFile(...)` so the URL
format lives in one place.
**Callers of `openFile`:**
- **Tool-call cards & write diffs** in the copilot/chat transcript. When a tool
call targets a single viewable file, the backend reports the path in
`ServerEvent::ToolStart.path` (via `Tool::target_path` — see
[tools.md](tools.md#clickable-target-path)); `copilot-render.js` renders it as
a clickable link. Falls back gracefully: tools without a `path` render the
plain label, unsupported file types show the viewer's "preview not available"
state, and unreadable paths show its error state.
- **The `show_file_to_user` tool**, via the `OpenFile` WebSocket event
(`open_file`, handled in `chat-session.js`). It is an `InterfaceTool` injected
only for SPA clients (`web` + `mobile`) in `src/frontend/api/ws.rs`, so the
assistant can proactively open a file — see
[tools.md](tools.md#registration-pattern). Every kind — HTML included — routes
through `openFile` and renders inside the viewer.
### Routing
`file_viewer` is registered in the sidebar's segment whitelist
(`sidebar.js:_pageFromHash`) but has **no sidebar menu entry** — like
`#session/{id}`, it's an "accessory" page reachable only via link or
`openFile`. The page follows the standard pattern: it listens for
`llm-page-change` (shows/hides itself) and `hashchange` (re-reads the path
from the URL when navigating between files while staying on the page).
Path resolution is delegated to the backend via `GET /api/file?path=<enc>`
(`src/frontend/api/files.rs`), which calls `fs_tools::resolve`:
- **Relative paths** resolve against Skald's process CWD (the data root).
- **Absolute paths** are used as-is — required when opening files that live
outside the data root, e.g. inside a project's custom working directory.
`get_file` serves **raw bytes** with a `Content-Type` derived from the extension
(`content_type_for`), not `read_to_string` — so binary formats (images, PDFs)
work. The viewer reads text kinds via `res.text()` and binary kinds via
`res.blob()` → object URL.
A query parameter `?force_download=true` makes the handler add
`Content-Disposition: attachment` so the browser **saves** the file instead of
rendering it inline. The attachment filename is the path's basename — or, when
combined with `?compile-latex=true` on a `.tex` source, `<stem>.pdf`. The
filename is sanitised to visible ASCII (header-value constraint). This backs the
header's **download button** (see below).
A query parameter `?compile-latex=true` switches the handler into LaTeX mode
when `path` is a `.tex` / `.latex` file: instead of returning the raw source,
the server runs `latexmk -xelatex` (via `LatexCompiler` in
`src/core/latex/`) and returns the resulting PDF (`application/pdf`). Compiled
PDFs are cached under `<tmp>/skald-latex/` in a **dependency-aware** way:
- `<path-hash>.fls` — the `.fls` recorder file from the last compile of that
source (keyed by SHA-256 of the source's absolute path). Lists every file TeX
actually read.
- `<deps-hash>.pdf` — the compiled PDF (keyed by SHA-256 of every
user-controlled input's contents, sorted by path).
On each request the compiler first re-derives `deps-hash` from the cached
`.fls` and serves the matching PDF without invoking `latexmk` if it exists.
This means a change to any `\input`'ed fragment, custom `.sty` / `.cls`
package, `.bib`, or `\includegraphics` target invalidates the cache correctly
even when the main `.tex` file is unchanged. Inputs under system TeX
distribution paths (`/usr/local/texlive`, `/Library/TeX`, …) and TeX
auxiliary outputs (`.aux`, `.log`, `.fls`, `.synctex.gz`, …) are filtered out
of the dependency set — they only change on a distro upgrade, which is rare
and easy to handle by clearing the cache. Failures produce a non-2xx response
with the textual `latexmk` log in the body:
| Outcome | Status | Body |
|---|---|---|
| Compiled (or cache hit) | `200` | PDF bytes (`application/pdf`) |
| Compilation error | `422` | `latexmk` log (`text/plain`) |
| `latexmk` not installed | `501` | Explanatory message |
| Compile timeout (> 30s) | `504` | Explanatory message |
### Header chrome
Both chromes render a header with the file name and a **download button**
(`bi-download`, right-aligned). The button calls `_download()` in
`FileViewerBase`, which builds `/api/file?path=…&force_download=true` (adding
`compile-latex=true` for LaTeX kinds, so a `.tex` always downloads its compiled
PDF) and clicks a transient `<a download>` — the server's `Content-Disposition`
supplies the saved filename.
The file name uses **tail-truncation**: when a path is too long the ellipsis is
placed at the *start* (`…/dir/report.tex`) so the filename stays visible, via
`direction: rtl; text-align: left` with the path wrapped in `<bdi>` (to keep its
characters left-to-right). The full path remains on the `title` attribute. The
desktop chrome shows the full path; the mobile chrome shows only the basename.
### Supported kinds
| Kind | Extensions | Rendering |
|---|---|---|
| **Markdown** | `.md`, `.markdown` | `renderMarkdown()` (marked + DOMPurify) via `unsafeHTML`. Relative `<img>` sources are rewritten (`rewriteMarkdownAssets`) to `/api/file?path=<dir-of-md + src>` so images referenced relative to the file load from disk; external/`data:`/root-relative URLs are left untouched |
| **Image** | `.png .jpg .jpeg .gif .webp .bmp .ico .avif` | `<img>` loaded as a Blob from `/api/file` (object URL) |
| **SVG** | `.svg` | `<iframe sandbox="allow-same-origin">` to a Blob object URL. Rendered in a sandboxed iframe (not `<img>`): the SVG root fills the iframe viewport so viewBox-only files scale correctly, and `allow-scripts` is withheld so any embedded `<script>` cannot execute. `allow-same-origin` is required for the iframe to read the blob: URL |
| **PDF** | `.pdf` | `<iframe>` to a Blob object URL — the browser's native PDF viewer |
| **HTML** | `.html`, `.htm` | Fetched as text, then rendered live in `<iframe srcdoc sandbox="allow-scripts allow-forms allow-modals allow-popups">`. `srcdoc` (not a blob `src`) gives the frame a **unique opaque origin**, so its JS runs but is fully isolated from the app origin — it cannot read cookies/localStorage or reach `/api/*`. `allow-same-origin` is deliberately withheld: combined with `allow-scripts` it would let the frame strip its own sandbox. A header toggle (`_renderModeToggle`) flips between the live preview and the raw source (`<pre><code>`). Limitation: relative asset paths inside the HTML don't resolve (opaque origin has no disk base) — self-contained HTML and absolute/CDN URLs work |
| **LaTeX** | `.tex`, `.latex` | On open, the viewer first requests `?compile-latex=true`; on `200` it renders the resulting PDF in an `<iframe>` (same path as a native `.pdf`). On any non-2xx response (compilation error, missing `latexmk`, timeout) it falls back to showing the raw source as a `<pre><code>` block, with the extracted compilation error in a collapsible banner — `formatLatexError` distils the full latexmk log down to the actionable `path:line: …` / `! …` lines plus context (the leading banner + package preamble are dropped), so the shown text is what a user pastes into an agent. The file watcher installs one OS watcher per dependency discovered via the `.fls` recorder file (`\input`'ed fragments, custom `.sty` / `.cls`, `.bib`, images, etc.) — so saving any of them triggers an automatic recompile. Requires `latexmk` with `xelatex` on the server's `PATH` (e.g. MacTeX / TeX Live). See the [LaTeX compile & cache](#latex-compile--cache) section below for the full dependency-aware algorithm. |
| **Text/code** | `.txt .rs .js .ts .py .json .yml .toml .sh .sql .go .css .vue ...` (see `TEXT_EXTS` in the source) | `<pre><code>` block, monospace, horizontal scroll |
| **Binary/unknown** | anything else | Placeholder: "Preview not available for this file type." |
### LaTeX compile & cache
The `.tex` kind is special: it is the only kind where the server produces a
derived artefact (PDF) on demand rather than serving the raw file. The
`LatexCompiler` in `src/core/latex/` orchestrates `latexmk -xelatex` and
maintains a dependency-aware cache so:
- saving any `\input`'ed fragment, custom `.sty` / `.cls`, `.bib`, or
`\includegraphics` target invalidates the cache correctly (even when the
main `.tex` is unchanged);
- unchanged inputs are served without recompiling.
Two artefacts live under `<tmp>/skald-latex/`:
| Artefact | Key | Purpose |
|-------------------|------------------------------------------------|--------------------------------------------------|
| `<path-hash>.fls` | SHA-256 of the `.tex` absolute path | Last-known input list for that source |
| `<deps-hash>.pdf` | SHA-256 of every user-controlled input's bytes | The compiled PDF for that exact content state |
Per request:
1. Read `<path-hash>.fls` (the recorder file produced by the last compile).
Missing → fresh compile.
2. Filter out system TeX paths (`/usr/local/texlive`, `/Library/TeX`, …) and
auxiliary artefacts (`.aux`, `.log`, `.fls`, `.synctex.gz`, …).
3. Hash every remaining input's contents, derive `<deps-hash>`.
4. If `<deps-hash>.pdf` exists → cache hit, serve it.
5. Otherwise → run `latexmk` in a per-compile scratch directory with
`-output-directory=<tmp>/skald-latex/<path-key>-<pid>-<ns>/`, capture the
new `.fls`, overwrite the `<path-hash>.fls` sidecar, save the PDF as
`<deps-hash>.pdf`, serve.
The file watcher (`/api/file/watch`) re-syncs its OS watchers on every change
event for a `.tex` source — dropping the per-dependency watchers and
re-installing them with the fresh `.fls`, so newly-added `\input`s are picked
up automatically. On the very first subscribe (no `.fls` yet), only the main
`.tex` is watched; once the first compile writes the sidecar, the next change
event triggers the re-sync.
Limitations: system TeX distribution files are excluded from the dependency
hash (they only change on a distro upgrade — clear the cache directory to
force a rebuild); shell-escape inputs (`\input{|"command"}`) are not tracked.
### Mobile
The mobile app renders its own `<mobile-file-viewer-page>` (`web/components/shared/file-viewer-mobile.js`), a thin subclass of `FileViewerBase` that shares all of the desktop viewer's fetch/render/watch logic and only swaps the chrome (full-screen page, mobile header + back button). `<mobile-app>`'s hash router (see [Mobile App & Native Shell](#mobile-app--native-shell)) routes `#file_viewer?path=...` to a non-tab `file_viewer` section and binds the component's `visible` / `path` props; the same `openFile(path)` used in the desktop transcript (a clickable tool path) therefore works unchanged on mobile. The back button returns to the previous section via history, and `_applyHash` skips the `skaldNav` notify so the native tab highlight stays put.
### Roadmap
The page is the foundation for several follow-up phases (tracked separately):
1. **Tab Editor (CodeMirror 6)** — second tab in the page with syntax-highlighted
editable buffer; saves via `PUT /api/file`. Bypasses the approval gate (user is
editing manually, not via an agent tool). Will introduce the "Overwrite /
Discard / Ignore" banner for dirty-buffer conflicts.
2. **Agent-driven opening** — new `ServerEvent::OpenFile { path }` emitted by a
`show_file_to_user(path)` interface tool; `chat-session.js` sets the hash from
the WS payload, so both manual and agent-driven paths funnel into the same
`<file-viewer-page>`.
3. **More media** — video (`<video>`), audio (`<audio>`), PDF (`<iframe>`).
### Files
| File | Purpose |
|---|---|
| `web/lib/open-file.js` | Defines and registers `window.openFile`; sets `location.hash` |
| `web/components/shared/file-viewer-base.js` | `FileViewerBase` — shared engine: fetch, kind detection, markdown assets, LaTeX compile, watcher, `_renderBody`. Driven by `_show`/`_hide` |
| `web/components/file-viewer-page.js` | `<file-viewer-page>` desktop subclass — hash routing (`llm-page-change` + `hashchange`) + desktop chrome |
| `web/components/shared/file-viewer-mobile.js` | `<mobile-file-viewer-page>` mobile subclass — prop-driven (`visible`/`path`) + mobile chrome (full-screen, back button) |
| `web/lib/file-watcher.js` | Singleton client for `/api/file/watch` — ref-counting, auto-reconnect, re-subscribe |
| `web/css/file-viewer.css` | Page + content styling (markdown, code, image, LaTeX compile-error banner, state). Loaded by both `index.html` and `mobile.html` |
| `src/frontend/api/file_watch.rs` | `/api/file/watch` WS handler — `notify::RecommendedWatcher` per watched file (one per LaTeX dependency for `.tex` sources) |
| `src/core/latex/mod.rs`, `src/core/latex/compiler.rs` | `LatexCompiler` — `latexmk -xelatex` invocation, SHA-256 content cache, error mapping. Called by `get_file` when `?compile-latex=true` |
---
## Adding a New ServerEvent
1. Add the variant to `ServerEvent` enum in `src/core/events.rs`.
2. Add the `type_name()` match arm in `src/core/events.rs`.
3. Emit it at the appropriate point (session handler, ChatHub, or ws.rs).
4. Handle it in `web/lib/chat-session.js` `_handleServerMsg()` — all clients inherit the handler automatically.
5. Update the ServerEvent Types table above.
---
## Debug Mode
A persistent flag stored in the `config` DB table under key `DEBUG_MODE` (`"true"` / `"false"`). The API is in `src/frontend/api/dev.rs`.
| Method | Path | Body | Response |
| --- | --- | --- | --- |
| `GET` | `/api/dev/debug_mode` | — | `{ "enabled": bool }` |
| `POST` / `PUT` | `/api/dev/debug_mode` | `{ "enabled": bool }` | `{ "enabled": bool }` |
| `GET` | `/api/dev/llm-requests` | query: `?page=1&per_page=20&agent_id=&source=&from=&to=` | `{ items: LlmRequest[], total: int }` |
| `GET` | `/api/dev/llm-requests/{id}` | — | Full request/response payload with system prompt, messages, tool definitions, and response |
The frontend reads this flag at startup and uses it to show or hide sections in the sidebar menu that are otherwise invisible in production.
---
## When to Update This File
- A `ServerEvent` variant is added, removed, or its fields change
- `ClientMessage` gains or loses a field
- A new Lit component is added
- The approval message format changes
- The debug-mode endpoint changes
- The file viewer gains a new phase (editor tab, agent-driven opening) or a new supported kind
- The `/api/file/watch` protocol (commands, messages) changes
docs/index.md
+228
View File
@@ -0,0 +1,228 @@
# Skald — Documentation
## Documentation Rule (MANDATORY)
**Every source code change — made by a human or by an LLM — must be accompanied by an update to the relevant doc file(s). No exception.**
This includes:
- Adding or removing a tool → update [tools.md](tools.md)
- Changing a table schema → update [database.md](database.md)
- Modifying the approval gate or tool loop → update [session.md](session.md)
- Adding a new agent → update [agents.md](agents.md)
- Any change to the WS protocol → update [frontend.md](frontend.md)
- Changing the project/ticket lifecycle or project chats → update [projects.md](projects.md)
- Changing ChatHub dispatch, the per-source inbox/coalescing, or `/stop`/`/clear` semantics → update [chat-hub.md](chat-hub.md)
- Adding or changing custom slash commands, the command manager, or the composer autocomplete → update [commands.md](commands.md)
- Changing the `desktop` feature, Tauri wiring, tray/menu, window policy, or `bootstrap_data_dir` → update [desktop.md](desktop.md)
---
## Key paths (agent: read this first)
| Resource | Default path | Override |
| --- | --- | --- |
| **SQLite database** | `./database.db` | `db.path` in `config.yml` |
| **Config file** | `./config.yml` | — (copy from `default.config.yaml`) |
| **Secrets folder** | `./secrets/` | — |
| **Model cache** | `./models/` | — |
| **Log files** | `./logs/` | — |
| **Static web assets** | `./web/` | `web.static_dir` in `config.yml` |
When looking for the database, **always use `./database.db`** unless `config.yml` says otherwise.
---
## Project Summary
A local chat server (Axum + Tokio + SQLite) where an LLM handles user queries via tool calls. The app can rewrite and restart its own source code autonomously. Multiple specialized agents collaborate via a recursive sub-agent system. External tools are integrated via MCP (Model Context Protocol). Entry point: `run.sh`.
## Workspace Structure
The project is a Cargo workspace. Extracted crates live in `crates/`:
| Crate | Path | Notes |
| --- | --- | --- |
| `skald` | `.` (root) | Main application binary |
| `core-api` | `crates/core-api/` | Shared types and traits: `ServerEvent`, `GlobalEvent`, `ChatHubApi`, `ApprovalApi`, `InboxApi`, `Tool`, `Memory`, `ChatbotClient`, `Transcribe`, `TextToSpeech`, `ImageGenerate`, `SecretsApi`, `LocationManager`, `InterfaceTool`, `Plugin`, `PluginContext`, `ApiProvider`, `ApiProviderRegistry`, `RemoteAccess`. Also owns all DB record types: `LlmProviderRecord`, `LlmModelRecord`, `TtsModelRecord`, `TranscribeModelRecord`, `ImageGenerateModelRecord`. |
| `llm-client` | `crates/llm-client/` | OpenAI-compatible, Anthropic, Ollama, LmStudio implementations of `ChatbotClient`. Depends on `core-api` and re-exports the trait and associated types for backward compatibility. |
| `mcp-client` | `crates/mcp-client/` | MCP protocol layer: `McpServer` (stdio), `McpHttpServer`, `McpServerClient` trait, config types |
| `honcho-client` | `crates/honcho-client/` | Honcho v3 REST API client — zero dependencies on the main crate |
| `plugin-honcho` | `crates/plugin-honcho/` | Honcho memory sink plugin |
| `plugin-tailscale-remote` | `crates/plugin-tailscale-remote/` | Remote connectivity via Tailscale mesh |
| `plugin-transcribe-whisper-local` | `crates/plugin-transcribe-whisper-local/` | Local STT via whisper.cpp (Metal-accelerated) |
| `plugin-telegram-bot` | `crates/plugin-telegram-bot/` | Private Telegram bot interface |
| `plugin-tts-orpheus-3b` | `crates/plugin-tts-orpheus-3b/` | Local TTS via Orpheus 3B (Python subprocess) |
| `plugin-tts-kokoro` | `crates/plugin-tts-kokoro/` | Local TTS via Kokoro ONNX (lightweight, multilingual) |
| `skald-relay-common` | `crates/skald-relay-common/` | Shared crypto + v2 protobuf frame types for the relay and mobile-connector plugin; owns the `gen-vectors` reference generator. Implements v2 binary transport (presence, live channel). No axum/tokio/Skald deps. |
| `skald-relay-server` | `crates/skald-relay-server/` | Zero-trust store-and-forward relay + push bridge (iOS/Android remote control). v2 binary transport (protobuf), presence tracking, live channel route-or-fail. Depends on `skald-relay-common`. Live APNs sender behind the `push-live` cargo feature (see [crates/workspace.md](crates/workspace.md)). |
| `plugin-mobile-connector` | `crates/plugin-mobile-connector/` | Agent end of the v2 relay protocol: bridges the Inbox to mobile apps over WebSocket, E2E encrypted. Handles presence, live inbox pulls, pairing. See [plugins/mobile-connector.md](plugins/mobile-connector.md). |
To add a new extracted crate: create `crates/<name>/`, add it to the `[workspace].members` list in the root `Cargo.toml`, then add a `path` dependency in `[dependencies]`.
---
## Module Map
| Source path | Role | Doc |
| --- | --- | --- |
| `src/main.rs` | Thin entry point: tracing → `Config` → `into_split` → plugins → `Skald::new` → `WebFrontend::start` → shutdown | [architecture.md](architecture.md) |
| `src/desktop/mod.rs` | Tauri shell — only compiled under `--features desktop`. Wraps `run_backend()` in a Tauri event loop with a system-tray icon (`Open` / `Quit`), creates the main `WebviewWindow` from `config.yml`'s `server.port`, and handles graceful shutdown | [desktop.md](desktop.md) |
| `src/core/skald.rs` | `Skald` — headless application core; owns all managers; `new(cfg, plugins)` / `shutdown()` | [architecture.md](architecture.md) |
| `src/core/config.rs` | `CoreConfig` + core config types (`DbConfig`, `LlmConfig`, `TicConfig`, `CompactionConfig`, …) | [architecture.md](architecture.md) |
| `src/frontend/config.rs` | `FrontendConfig` (`ServerConfig`, `WebConfig`, `timezone`) | [architecture.md](architecture.md) |
| `src/core/session/handler/` | Core LLM loop, tool dispatch, approval | [session.md](session.md) |
| `src/core/session/handler/message_builder.rs` | `MessageBuilder` — pure service for building OpenAI message arrays, testable in isolation | [session.md](session.md) |
| `src/core/session/manager.rs` | Session factory | [session.md](session.md) |
| `src/core/agents.rs` | Agent discovery, prompt loading | [agents.md](agents.md) |
| `src/core/command/mod.rs` | `LlmCommandManager` — file-based custom slash commands (`commands/<name>/`) | [commands.md](commands.md) |
| `src/core/tools/` | Built-in tool registry | [tools.md](tools.md) |
| `src/core/tools/tool_names.rs` | Centralised tool name constants (`CALL_AGENT`, `RESTART`, …) | [tools.md](tools.md) |
| `src/core/tool_catalog.rs` | `ToolCatalog`: unified listing façade for built-in + MCP tools (wraps ToolRegistry + McpManager); `AllTools` response includes `mcp_servers: HashMap<String, McpServerMeta>` (friendly name + description per MCP server) | [tools.md](tools.md) |
| `src/core/provider/` | `ProviderRegistry` (implements `ApiProviderRegistry`) — thin wrapper around `core-api::provider`. All types re-exported for internal use. | [llm-clients.md](llm-clients.md) |
| `src/core/service_manager.rs` | `ServiceManager` trait — lightweight umbrella for all model managers | [llm-clients.md](llm-clients.md) |
| `src/core/chatbot/` | LLM provider clients | [llm-clients.md](llm-clients.md) |
| `src/core/llm/manager.rs` | LLM selection, health tracking | [llm-clients.md](llm-clients.md) |
| `src/core/chat_event_bus.rs` | In-process broadcast bus for chat turns and compaction events | [chat-event-bus.md](chat-event-bus.md) |
| `src/core/compactor.rs` | Context compaction — summarises old history to reduce token usage | [compaction.md](compaction.md) |
| `src/core/memory/` | Pluggable long-term memory layer (trait + manager) | [memory.md](memory.md) |
| `src/core/chat_hub/` | Central chat orchestrator for **interactive, user-facing sessions only** (web, mobile, project chats — one persistent session per source via the `sources` table); per-source coalescing inbox; notification pipeline. `provision_session(source, agent_id, rc, reset)` is the single source→session entry point; `clear()` is a thin `main`-agent wrapper over it. **Not** for background agents (cron/TIC/sub-agents → `TaskManager`/`ChatSessionManager`) | [chat-hub.md](chat-hub.md) |
| `src/core/tic/` | Background MCP event processor (TicManager) | [architecture.md](architecture.md) |
| `src/core/mcp/` | MCP server management, push notification ingestion | [mcp.md](mcp.md) |
| `src/core/cron/` | Scheduled job scheduler | [cron.md](cron.md) |
| `src/core/plugin/` | Plugin system (PluginManager) | [plugins.md](plugins.md) |
| `src/core/secrets.rs` | SecretsStore — centralised token/key store over SQLite | [secrets.md](secrets.md) |
| `src/core/transcribe/` | TranscribeManager, OpenAiAudioTranscriber, ElevenLabsTranscriber. Traits and record types re-exported from `core-api`. | [providers/transcribe.md](providers/transcribe.md) |
| `src/core/tts/` | TtsManager (DB-backed + plugin slots), OpenAiTtsSynthesiser, ElevenLabsTtsSynthesiser. Traits and record types re-exported from `core-api`. | [providers/tts.md](providers/tts.md) |
| `src/core/image_generate/` | ImageGenerate trait, ImageGeneratorManager (DB-backed + plugin slots), OpenRouterImageGenerator | [providers/image.md](providers/image.md) |
| `src/core/run_context/mod.rs` | `RunContext` domain object: fields `security_group`, `system_prompt`, `allow_fs_writes`, `working_directory` + applicative methods `tool_group_id()`, `extra_system_prompt()`, `effective_working_dir()`, `is_write_allowed()`. `RunContextManager`: permission group CRUD; `set_session_run_context`; `duplicate_group`; `check_tool_visibility`. | [approval/index.md](approval/index.md) |
| `src/core/projects/mod.rs` | `ProjectManager` — CRUD for projects (filesystem-linked, ordered by `updated_at`). Free fn `build_runtime_run_context(project, base)` layers project-runtime fields (`working_directory = project.path`, `allow_fs_writes` for the project tree + `{skald_cwd}/data`, project-context system prompt fragments) over an optional base RC — shared by ticket jobs and interactive project chats | [projects.md](projects.md) |
| `src/core/projects/tickets.rs` | `ProjectTicketManager` — CRUD + lifecycle for project tickets (`start`, `on_job_completed`, `reset`); `start()` resolves the base `RunContext` (ticket override → project static config) and delegates to `projects::build_runtime_run_context` for the runtime fields | [projects.md](projects.md) |
| `src/core/inbox.rs` | `Inbox`: unified façade for pending approvals + clarifications (wraps ApprovalManager, ClarificationManager, ChatHub) | [approval/index.md](approval/index.md) |
| `src/core/db/` | SQLite schema and queries | [database.md](database.md) |
| `src/core/events.rs` | WS protocol types | [frontend.md](frontend.md) |
| `src/frontend/mod.rs` | `WebFrontend` — wires `router_factory`, starts plugins, runs Axum | [architecture.md](architecture.md) |
| `src/frontend/server.rs` | `WebServer` — Axum router, TcpListener, `WebServerHandle` | [architecture.md](architecture.md) |
| `src/frontend/api/` | HTTP + WebSocket handlers — `State<Arc<Skald>>` | [frontend.md](frontend.md) |
| `src/frontend/api/projects.rs` | REST CRUD for projects and tickets — `GET/POST /api/projects`, `GET/PUT/DELETE /api/projects/{id}`, tickets sub-routes, `start`/`reset` lifecycle. `POST /api/projects/{id}/session` opens/resumes the project chat (source `project-{id}`, agent `project-coordinator`). `provisioning_for_source(skald, source)` maps a source → (agent, RunContext) and is reused by `POST /api/sessions` so project resets recreate with the coordinator | [projects.md](projects.md) |
| `src/config.rs` | `Config` (YAML aggregate: `ServerConfig`, `WebConfig` + re-exports from `core::config`) + `Config::into_split()` + `bootstrap_data_dir()` (cwd relocation under the `desktop` feature) | [logging-config.md](logging-config.md), [desktop.md](desktop.md) |
| `crates/plugin-honcho/` | Honcho memory sink (standalone crate) | [honcho.md](honcho.md) |
| `crates/plugin-tailscale-remote/` | Remote connectivity via Tailscale mesh (standalone crate) | [remote.md](remote.md) |
| `crates/plugin-transcribe-whisper-local/` | Local STT via whisper.cpp (standalone crate) | [whisper-local.md](whisper-local.md) |
| `crates/plugin-telegram-bot/` | Private Telegram bot (standalone crate) | [plugins/telegram.md](plugins/telegram.md) |
| `crates/plugin-tts-orpheus-3b/` | Orpheus TTS 3B — local TTS via Python subprocess (standalone crate) | [providers/tts.md](providers/tts.md) |
| `crates/plugin-tts-kokoro/` | Kokoro ONNX — lightweight local TTS, multilingual (standalone crate) | [providers/tts.md](providers/tts.md) |
| `crates/honcho-client/` | Honcho v3 REST API client (standalone crate) | [honcho.md](honcho.md) |
| `web/components/` | Lit frontend components | [frontend.md](frontend.md) |
| `run.sh` | Supervisor loop | [self-rewriting.md](self-rewriting.md) |
---
## Critical Constants
| Constant | Value | Location |
| --- | --- | --- |
| `MAX_AGENT_DEPTH` | **5** | `src/core/session/handler/mod.rs` |
| `DEFAULT_MAX_TOOL_ROUNDS` | **20** | `src/core/session/handler/mod.rs` |
| `FAILURE_DEGRADED` | **3** consecutive failures | `src/core/llm/manager.rs` |
| `FAILURE_DOWN` | **5** consecutive failures | `src/core/llm/manager.rs` |
| Cron scheduler tick | **30 s** | `src/core/cron/mod.rs` |
| Cron fire-check window | **90 s** | `src/core/cron/mod.rs` |
| MCP startup timeout | **120 s** | `src/core/mcp/mod.rs` |
| TIC tick interval | **900 s** default | `config.yml` → `tic.interval_secs`; overridable at runtime via `tic.interval_minutes` DB key |
| TIC batch size | **50 events** default | `config.yml` → `tic.batch_size` |
| Notification batch window | **200 ms** | `src/core/chat_hub/mod.rs` |
---
## Navigation
### Core Architecture
- [architecture.md](architecture.md) — component wiring, startup sequence, request lifecycle
- [desktop.md](desktop.md) — Tauri desktop bundle: system tray, window policy, two-deployment-shape model, build instructions
- [build-and-distribution.md](build-and-distribution.md) — portable static binary: rustls+ring (no OpenSSL/aws-lc), musl build recipe, cargo features, runtime deps
- [self-rewriting.md](self-rewriting.md) — restart mechanism, safe self-modification workflow
- [database.md](database.md) — SQLite schema, migration pattern
- [logging-config.md](logging-config.md) — log levels, config.yml full reference
### Session & LLM Loop
- [session.md](session.md) — ChatSessionHandler, message flow, approval gate integration
- [session/run-context.md](session/run-context.md) — RunContext: permissions, system prompt, file authorization, working directory
- [llm-clients.md](llm-clients.md) — ChatbotClient trait, LlmManager, ApiProvider, ProviderRegistry, AUTO selection
- [compaction.md](compaction.md) — context compaction: trigger, summarisation flow, DB schema, config
- [memory.md](memory.md) — Memory trait, MemoryManager, integration in the LLM loop
### Approval & Permissions
- [approval/index.md](approval/index.md) — ApprovalManager: human-in-the-loop, rules, pending approvals, session bypass; tool visibility filtering; group duplication
- [session/run-context.md](session/run-context.md) — RunContext fields and integration (single source of truth)
### Tools & Agents
- [agents.md](agents.md) — agent discovery, meta.json, call_agent, depth limit
- [commands.md](commands.md) — custom `/command` shortcuts: file layout, `{{args}}`, dual-view, autocomplete
- [tools.md](tools.md) — Tool trait, ToolRegistry, built-in catalogue
- [chat-event-bus.md](chat-event-bus.md) — ChatEventBus, event types, publication rules, adding consumers
- [cron.md](cron.md) — TaskManager, task kinds (cron/sync/async), 7-field cron syntax, job lifecycle, async result delivery
### Model Providers
- [llm-clients.md](llm-clients.md) — LLM client trait and selection
- [providers/tts.md](providers/tts.md) — Text-to-Speech: trait, manager, provider catalogue, tts_models DB table
- [providers/transcribe.md](providers/transcribe.md) — Cloud STT via OpenAI-compatible audio API, transcribe_models DB table
- [providers/image.md](providers/image.md) — Image generation: trait, manager, async task system, LLM tools, REST endpoint
### MCP (Model Context Protocol)
- [mcp.md](mcp.md) — McpManager, transports, naming convention, enable/disable, integration
- [mcp/specs/index.md](mcp/specs/index.md) — Specification reference: one file per official MCP spec revision (2024-11-05 Legacy → 2025-06-18 Stable → 2025-11-25 Latest Stable → 2026-07-28 draft), with a comparative feature matrix
- [mcp/servers/gmail.md](mcp/servers/gmail.md) — Gmail read+modify MCP server (custom Python)
- [mcp/servers/gcal.md](mcp/servers/gcal.md) — Google Calendar read-only MCP server (custom Python)
- [mcp/servers/gmaps.md](mcp/servers/gmaps.md) — Google Maps transit/directions MCP server (custom Python)
- [mcp/servers/whatsapp.md](mcp/servers/whatsapp.md) — WhatsApp read+send MCP server (custom Node.js)
### Plugin System
- [plugins.md](plugins.md) — Plugin trait, PluginManager, HTTP router integration
- [plugins/honcho.md](plugins/honcho.md) — Honcho memory plugin: setup, config, filtering, lifecycle
- [plugins/mobile-connector.md](plugins/mobile-connector.md) — Mobile app relay bridge, E2E encryption, Inbox synchronization
- [plugins/telegram.md](plugins/telegram.md) — Telegram bot setup, pairing, whitelist, HITL approval
- [plugins/whisper-local.md](plugins/whisper-local.md) — Local STT via whisper.cpp, model setup, TranscribeManager integration
- [plugins/remote.md](plugins/remote.md) — Tailscale mesh remote connectivity
### Relay Protocol (Mobile Remote Control)
- [relay/index.md](relay/index.md) — Architecture, actors, threat model, encoding conventions
- [relay/crypto.md](relay/crypto.md) — Crypto contract: seed, key derivation, ECDH, HKDF, AES-256-GCM, anti-replay
- [relay/relay-protocol.md](relay/relay-protocol.md) — WebSocket protocol: protobuf transport, auth, pairing, live channel, presence
- [relay/framing.md](relay/framing.md) — E2E plaintext framing: version byte + optional zlib compression
- [relay/payloads.md](relay/payloads.md) — E2E payload schemas: inbox_update, approval_response, clarification_response, …
- [relay/describe-and-push.md](relay/describe-and-push.md) — Approval rendering: summary + structured blocks, push delivery model
- [relay/server.md](relay/server.md) — Relay server implementation: zero-trust, store-and-forward, APNs/FCM bridge, deploy
- [relay/test-vectors.md](relay/test-vectors.md) — Crypto test vectors + reference generator for byte-for-byte interop
### Projects & Tickets
- [projects.md](projects.md) — Projects subsystem: kanban tickets, lifecycle, `build_runtime_run_context`, interactive project chats
### Frontend & Notifications
- [frontend.md](frontend.md) — WebSocket protocol, ServerEvent types, Lit components
- [notifications.md](notifications.md) — Notification system: `read_notification` tool, synthetic injection flow, `data/notifications.md` format
### Infrastructure & Security
- [secrets.md](secrets.md) — SecretsApi trait, SecretsStore, well-known keys, security notes
- [crates/workspace.md](crates/workspace.md) — Workspace crate catalogue, `core-api` module reference, plugin extraction roadmap
### Miscellaneous
- [skills.md](skills.md) — Skills system: reusable Python capability packages
## When to Update This File
- A new source module is added or removed
- A critical constant changes
- A new doc file is added to `docs/`
docs/llm-clients.md
+433
View File
@@ -0,0 +1,433 @@
# LLM Clients
## Workspace Location
The `ChatbotClient` trait and all provider implementations live in the standalone crate `crates/llm-client` (no dependencies on the main crate). `src/core/chatbot/mod.rs` is a thin re-export layer. `LoggingChatbotClient` (`src/core/chatbot/logging.rs`) remains in the main crate because it depends on `sqlx`.
---
## ChatbotClient Trait
```rust
#[async_trait]
pub trait ChatbotClient: Send + Sync {
async fn chat(&self, messages: &[Message], options: &ChatOptions) -> Result<ChatResponse>;
async fn chat_with_tools(&self, messages: &[Value], tools: &[Value], options: &ChatOptions) -> Result<LlmTurn>;
async fn chat_with_tools_raw(&self, messages: &[Value], tools: &[Value], options: &ChatOptions) -> Result<(LlmTurn, Option<LlmRawMeta>)>;
}
```
Only `AnthropicClient` and `OpenAiClient` implement native tool support (`chat_with_tools`). Other clients have a default fallback that strips tool definitions and calls `chat()` instead.
`chat_with_tools_raw` is used by the logging wrapper: it returns the same `LlmTurn` plus raw HTTP request/response metadata (`LlmRawMeta`). `AnthropicClient`, `OpenAiClient`, and `LmStudioClient` override it; all others fall back to calling `chat_with_tools` with no metadata.
`ChatOptions` carries two optional fields — `session_id` and `stack_id` — set by the LLM loop for correlation. Providers ignore them; only `LoggingChatbotClient` reads them.
---
## Transparent Request Logging
`LoggingChatbotClient` (`src/core/chatbot/logging.rs`) is a `ChatbotClient` wrapper that intercepts every `chat_with_tools` call:
1. Calls `inner.chat_with_tools_raw(...)` to capture the HTTP wire data.
2. Spawns a **fire-and-forget** `tokio::spawn` to insert a row into `llm_requests`.
3. Returns the `LlmTurn` to the caller unchanged.
The LLM loop is fully unaware — it holds an `Arc<dyn ChatbotClient>` and calls `chat_with_tools` as usual. The wrapper is applied in `LlmManager::build_entry` when `request_log_enabled = true` (set from `config.yml → llm.request_log.enabled`).
What is logged per row: full request body (provider-specific format), request headers (api-key redacted), full response body, response headers, token counts, round-trip duration, session/stack ID.
A background task (boot + every hour) deletes rows older than `retention_days` (default 14).
`LlmTurn` return variants:
- `Message(ChatResponse)` — final text answer
- `ToolCalls { content, calls, input_tokens, output_tokens, reasoning_content, cost }` — one or more tool calls requested
Both variants carry an optional `reasoning_content: Option<String>`. Populated only by providers that return chain-of-thought (currently DeepSeek thinking mode). Saved to `chat_history.reasoning_content` and echoed back on subsequent turns — see *Reasoning Content / DeepSeek Thinking Mode* below.
Both variants also carry an optional `cost: Option<f64>` — the request price in USD. Populated via the `ChatbotClient::extract_cost(&self, response: &Value)` trait method, whose default reads `usage.cost` from the raw JSON response (OpenRouter and other OpenAI-compatible gateways report it there). Providers that don't bill per-request leave it `None`. `llm_loop` persists it to `chat_history.cost`. Providers with a different response shape can override `extract_cost`.
---
## Provider Registry
Providers are no longer identified by a hard-coded enum. Instead, each provider is a struct implementing the `ApiProvider` trait (`src/core/provider/mod.rs`), registered at startup in `main.rs` via `ProviderRegistry::register_builtin()`. The DB column `llm_providers.type` stores the provider's `type_id` string.
```rust
// src/provider/mod.rs
#[async_trait]
pub trait ApiProvider: Send + Sync {
fn type_id(&self) -> &'static str; // e.g. "open_ai", "anthropic"
fn display_name(&self) -> &'static str;
fn supported_types(&self) -> &'static [ServiceType];
// ── Remote model catalogs (default: Ok(None)) ─────────────────────────────
async fn list_llm_models(&self, record: &LlmProviderRecord) -> Result<Option<Vec<RemoteLlmModelInfo>>>;
async fn llm_model_info(&self, record: &LlmProviderRecord, model_id: &str) -> Result<Option<RemoteLlmModelInfo>>;
async fn list_tts_models(&self, record: &LlmProviderRecord) -> Result<Option<Vec<RemoteTtsModelInfo>>>;
async fn list_transcribe_models(&self, record: &LlmProviderRecord) -> Result<Option<Vec<RemoteTranscribeModelInfo>>>;
// ── Factories (default: None) ─────────────────────────────────────────────
fn build_llm(&self, record: &LlmProviderRecord, model: &LlmModelRecord) -> Option<Result<BuiltLlmClient>>;
fn build_tts(&self, record: &LlmProviderRecord, model: &TtsModelRecord) -> Option<Result<Arc<dyn TextToSpeech>>>;
fn build_transcriber(&self, record: &LlmProviderRecord, model: &TranscribeModelRecord) -> Option<Result<Arc<dyn Transcribe>>>;
fn build_image_generator(&self, record: &LlmProviderRecord, model: &ImageGenerateModelRecord) -> Option<Result<Arc<dyn ImageGenerate>>>;
fn ui_meta(&self) -> ProviderUiMeta; // served to frontend via GET /api/llm/providers/types
}
```
`list_tts_models` and `list_transcribe_models` have a default implementation returning `Ok(None)` — providers that don't support listing do not need to implement them. Only `ElevenLabsProvider` currently overrides both, calling `GET https://api.elevenlabs.io/v1/models` and filtering by capability flag.
`BuiltLlmClient` bundles the constructed `Arc<dyn ChatbotClient>` with a `prompt_cache: bool` flag that controls whether Anthropic KV-cache headers are injected by the session loop.
**`ProviderRegistry`** (`src/core/provider/mod.rs`) holds built-in and plugin providers separately. Plugin providers shadow built-in ones with the same `type_id`. Plugins can call `registry.register_plugin()` / `registry.unregister_plugin()` at any time after startup.
`LlmManager`, `TranscribeManager`, `TtsManager`, and `ImageGeneratorManager` all receive an `Arc<ProviderRegistry>` at construction and use it to build clients and look up `supported_types`.
### Built-in Providers
| `type_id` | Client struct | api_key required | Default base_url | Prompt cache |
| ------------- | ----------------- | ---------------- | ---------------------------------- | ------------------------------ |
| `lm_studio` | `LmStudioClient` | No | `http://localhost:1234/v1` | ❌ |
| `ollama` | `OllamaClient` | No | `http://localhost:11434` | ❌ |
| `open_ai` | `OpenAiClient` | **Yes** | `https://api.openai.com/v1` | ❌ |
| `openrouter` | `OpenAiClient` | **Yes** | `https://openrouter.ai/api/v1` | ✅ `anthropic/*` models only |
| `anthropic` | `AnthropicClient` | **Yes** | `https://api.anthropic.com` | ❌ (planned) |
| `deepseek` | `OpenAiClient` | **Yes** | `https://api.deepseek.com/v1` | ✅ automatic (see below) |
| `zai` | `OpenAiClient` | **Yes** | `https://api.z.ai/api/paas/v4` | ❌ |
| `elevenlabs` | — | **Yes** | `https://api.elevenlabs.io` | ❌ (TTS + Transcribe only) |
`openrouter`, `deepseek`, and `zai` reuse `OpenAiClient` with different base URLs. `elevenlabs` does not support LLM chat — `build_llm()` returns `None`.
`zai` (Z.AI / Zhipu AI GLM) has no `GET /models` endpoint, so `list_llm_models()` returns a **curated static catalog** of published GLM models (`glm-5.2`, `glm-5.1`, `glm-5`, `glm-5-turbo`, `glm-4.7`, `glm-4.6`, `glm-4.5`, `glm-4-32b-0414-128k`) with heuristic context lengths — the same UI "model picker" flow as `deepseek`/`openrouter`. Note the base URL ends in `/paas/v4` because `OpenAiClient` appends `/chat/completions`.
---
## Prompt Caching (KV Cache)
When `LlmEntry.prompt_cache = true` (currently set only for `OpenRouter`), the agent enables Anthropic-compatible KV caching on every request:
### What is sent
1. **`anthropic-beta: prompt-caching-2024-07-31` HTTP header** — tells OpenRouter/Anthropic to activate the caching feature.
2. **Static system message tagged for caching** — `build_openai_messages` emits the first system message (AGENT.md + memory files + `extra_system_static` + MCP list) as a content array with `cache_control: {"type": "ephemeral"}` on the single block. This is the KV cache prefix.
3. **Last tool tagged** — the final entry in the `tools` array receives `cache_control: {"type": "ephemeral"}`, caching the entire tool list as part of the prefix.
### Message order and cache stability
The full message array is structured so the stable prefix is as long as possible (see *Context Building* in [session.md](session.md)):
```text
[static system — cached] [scratchpad?] [summary?] [conversation] [dynamic tail] [tail reminder]
```
The dynamic tail (Honcho memories + date/time) is placed **after** the conversation, so it never shortens the cacheable prefix. Scratchpad is a separate message so a mid-turn write only invalidates that small block, not the large static prefix.
### Cache TTL
Anthropic's `ephemeral` cache has a sliding TTL of ~5 minutes (extended by each hit). A cache hit is reported in the response as `cache_read_input_tokens` in the usage block.
### DeepSeek automatic KV cache
DeepSeek's disk KV cache is **prefix-based and fully automatic** — no explicit markers or special headers are required. Because the static system message is always the first entry in the message array, it becomes the stable cache prefix on every turn.
`prompt_cache = false` for the `DeepSeek` provider: no Anthropic-style `cache_control` blocks are injected (DeepSeek does not understand them). The cache operates transparently. DeepSeek reports cache hits in the response under `usage.prompt_cache_hit_tokens` / `usage.prompt_cache_miss_tokens` (visible in the raw request log).
#### ⚠️ The dynamic tail and cache invalidation
DeepSeek's KV cache compares requests **token by token from position 0**. If any token differs at position N, everything from N onward is recomputed — there is no partial matching inside the sequence.
The dynamic tail (date/time + Honcho memories) is injected as a system message at the end of the message array, after the conversation history. Because it is placed *after* the conversation, it doesn't shorten the cacheable prefix for the system message and tools. However, the **exact timestamp** (`2026-05-28T10:56:34+02:00`) changes every second. This means the stored KV entry from the previous request ends with `[..conversation..][dyn_tail_T1]`, while the new request has `[..conversation..][new_user_msg][dyn_tail_T2]`. The break point occurs right after the last assistant message: everything beyond it must be recomputed.
In practice this means: **without timestamp rounding, only the static system message and tools are effectively cached.** The conversation history accumulates in the cache prefix, but the always-changing timestamp prevents the prefix from extending into the tail message of the stored entry.
**Observed impact (production data):**
| Configuration | `prompt_cache_hit_tokens` | `prompt_cache_miss_tokens` |
| --- | --- | --- |
| Exact timestamp (default before fix) | ~6,144 | ~21,583 |
| `round_minutes: 15` | ~38,272 | ~830 |
With rounding, the timestamp string stays byte-identical for up to 15 minutes, letting the full conversation + tools accumulate in the cache prefix. The remaining ~830 miss tokens represent only the current user message (unavoidably new on every request).
### `llm.datetime` — timestamp injection config
Controlled by `config.yml → llm.datetime`:
```yaml
llm:
datetime:
enabled: true # false = omit date/time from context entirely
round_minutes: 15 # round down to nearest N-minute boundary
# e.g. 10:56 → 10:50 with round_minutes: 10
# omit for exact timestamp (hurts KV cache on prefix-based providers)
```
`round_minutes` is the primary tuning knob for cache efficiency on DeepSeek and any other prefix-based KV cache provider. The trade-off is precision: the LLM sees a timestamp that may be up to `round_minutes` minutes in the past. For most conversational uses this is imperceptible; for time-critical tasks (cron triggers, calendar scheduling) prefer a smaller value or `null`.
The default in `default.config.yaml` is `round_minutes: 15` — a safe value that gives near-100% cache hit rates in typical conversations while keeping the timestamp accurate to within a quarter-hour.
### Future: Anthropic direct
`AnthropicClient` does not yet support `prompt_cache`. The implementation is different: the `system` parameter must be sent as a JSON array of content blocks rather than a plain string. Tracked as a future improvement.
---
---
## Reasoning Content / DeepSeek Thinking Mode
When DeepSeek is configured with `"thinking": {"type": "enabled"}` in `extra_params`, each response includes a `reasoning_content` field alongside the normal `content`. This is the model's chain-of-thought.
**DeepSeek requires that `reasoning_content` be echoed back in the assistant message on subsequent turns.** Omitting it causes a `400 invalid_request_error`.
### How it works
1. `OpenAiClient.chat_with_tools_raw` reads `message.reasoning_content` from the response and propagates it through `LlmTurn`.
2. `llm_loop` saves it to `chat_history.reasoning_content` alongside the assistant's text content.
3. `build_openai_messages` includes `reasoning_content` in the reconstructed assistant message whenever the field is non-null.
All other providers always return `reasoning_content: None`; the field is simply absent from their assistant messages in the history.
---
## LlmStrength Enum
Ordered (weakest → strongest): `VeryLow` < `Low` < `Average` < `High` < `VeryHigh`
Used by AUTO selection and `call_agent` to match agents to capable models.
---
## AUTO Selection Algorithm
When `client = "auto"` or no client is specified, `LlmManager::select()` runs four passes in order, returning the first match:
1. Not-Down + strength ≥ required + scope matches
2. Not-Down + strength ≥ required (scope relaxed)
3. Any Not-Down model
4. **Emergency fallback**: strongest model even if Down (logs a `WARN`)
Models are ordered by `priority ASC` in the DB; lower number = tried first.
---
## Health Tracking
| Threshold | Status |
| ---------------------------- | ------------------- |
| `consecutive_failures >= 3` | `Degraded` |
| `consecutive_failures >= 5` | `Down` |
| Next success | Reset to `Healthy` |
`mark_failure()` is called by `run_agent_turn` on LLM call errors. `mark_success()` is called on every successful response. Health state is preserved across `reload()` calls (e.g. after adding a new model).
---
## Automatic LLM Failover
When the primary model returns a retriable error (5xx, network error, 429), `run_agent_turn` automatically tries the next available model — up to **3 attempts per round**.
**Retry logic**:
- A fresh `tried_this_round` list is built at the start of every round.
- On error, `is_retriable_llm_error()` decides whether to try again. Client errors (400/401/403/404/422) are **not** retried — the request itself is invalid.
- `select_excluding(&tried)` picks the next model, applying the same scope/strength rules as AUTO selection but skipping already-tried ones.
- If a different model uses different `prompt_cache` settings, messages are rebuilt before the retry.
- `cur_name`/`cur_llm` persist for the rest of the turn once switched, so subsequent rounds use the new model without re-trying the failed one.
**Events emitted**:
| Event | When | Who reacts |
| --- | --- | --- |
| `model_fallback` | Each successful switch | Frontend shows an inline info note |
| `llm_failed` | All attempts exhausted | Frontend shows error + `_waiting = false`; Telegram sends a message |
Telegram ignores `model_fallback` (silent retry) but sends an error message for `llm_failed`, matching the same behaviour as `Error`.
---
## Valid Scope Values
`basic`, `writing`, `coding`, `reasoning`, `math`, `search`
Defined by convention; any string is accepted by the DB. Agents declare `scope` in `meta.json`; models declare matching scopes in the DB.
---
## Extra Params
Each model can store an optional `extra_params` JSON object. Its top-level keys are **merged into the request body** before every API call, overriding any default key with the same name.
`OpenAiClient` (covers OpenAI-compatible providers) merges extra params via `apply_extra`. `AnthropicClient` merges them too via `apply_extra` (constructed with `with_extra_body`), which additionally enforces the extended-thinking constraints (see *Reasoning* below).
**Example — DeepSeek thinking mode (native DeepSeek provider):**
```json
{ "thinking": {"type": "enabled"}, "reasoning_effort": "high" }
```
`reasoning_effort` accepts `"low"`, `"medium"`, or `"high"`. Only supported by DeepSeek reasoning models (e.g. `deepseek-reasoner`); sending these params to non-reasoning models returns a 400.
**Example — DeepSeek reasoning effort on OpenRouter:**
```json
{ "reasoning": { "effort": "high" } }
```
Set via the model edit modal in the LLM Models UI, or via `PUT /api/llm/models/{id}` with `extra_params` in the JSON body.
> For the model's **reasoning / thinking** knob, prefer the dedicated `reasoning` field (below) over hand-writing provider-specific keys in `extra_params`.
---
## Reasoning
Reasoning ("thinking") is a first-class, provider-agnostic per-model setting rather than a hand-written `extra_params` blob. Two `ApiProvider` methods own it:
```rust
fn reasoning_mode(&self, model_id: &str, capabilities: &[String]) -> Option<ReasoningMode>;
fn reasoning_request(&self, value: &serde_json::Value) -> Option<serde_json::Value>;
```
- **`reasoning_mode`** describes the *control* (drives the UI). `ReasoningMode` is either:
- `ValueSet { values, default }` — discrete choices (e.g. effort levels, or an on/off toggle), rendered as a dropdown.
- `Range { min, max, step, default, unit }` — a numeric budget, rendered as a number input.
- `None` — the model has no reasoning knob.
- **`reasoning_request`** translates the *selected value* (stored per model in `llm_models.reasoning`: a JSON string for a `ValueSet`, a JSON number for a `Range`, or `NULL` = off) into the provider-specific request fragment, merged into the request body at `build_llm` time via `providers::extra_with_reasoning`.
| Provider | `reasoning_mode` | Request fragment |
|---|---|---|
| `zai` | GLM-5.2+ → effort `ValueSet` (`disabled`/`minimal`…`max`); GLM-5.x/4.5/4.6/4.7 → on/off | `{"thinking":{"type":…}}` (+ `"reasoning_effort"` for GLM-5.2) |
| `deepseek` | reasoner / v4 → effort `ValueSet` (`disabled`/`low`…`max`) | `{"thinking":{"type":…}}` (+ `"reasoning_effort"`) |
| `openrouter` | per-model from the catalog `reasoning` object (`supported_efforts` / `supports_max_tokens`), else a fallback effort set | `{"reasoning":{"effort":…}}` or `{"reasoning":{"max_tokens":N}}` |
| `open_ai` | o-series / gpt-5 → effort `ValueSet` | `{"reasoning_effort":…}` |
| `anthropic` | thinking-capable models → `Range` (budget_tokens) | `{"thinking":{"type":"enabled","budget_tokens":N}}` |
The descriptor reaches the frontend two ways: attached to each catalog model (`RemoteLlmModelInfo.reasoning`) for the add-from-catalog form, on `LlmModelInfo.reasoning_mode` for the edit form, and via `GET /api/llm/providers/{id}/reasoning-mode?model_id=…` for the manual add form. The UI renders the matching control (dropdown / number input) with an "— off —" option (null value → no reasoning param sent).
**Anthropic** merges the fragment through `AnthropicClient::with_extra_body`; its `apply_extra` also enforces the extended-thinking constraints — `temperature` is dropped and `max_tokens` is bumped above `budget_tokens` when thinking is enabled.
---
## Model Metadata Fields
Each model record now stores additional metadata beyond the core LLM configuration:
| Field | Type | Source | Runtime use |
|---|---|---|---|
| `context_length` | `Option<i64>` | Provider catalog sync or manual input | Compaction threshold calculation, `max_tokens` limiting |
| `max_output_tokens` | `Option<i64>` | Provider catalog sync or manual input | Future: set `max_tokens` on LLM calls (currently `None`) |
| `knowledge_cutoff` | `Option<String>` | Provider catalog sync or manual input | Future: inject into system prompt |
| `capabilities` | `Vec<String>` | Provider catalog sync or manual input | Filtering by model feature (vision, function_calling, etc.) |
All fields are optional (`NULL` in the DB). When the provider catalog reports them,
they are automatically synced to existing DB records by `list_provider_models()`.
Manual values set via the API or UI take precedence when the provider does not
report a particular field (the sync uses `COALESCE` — only non-NULL catalog values
overwrite).
---
## LLM CRUD
All mutations go through `LlmManager` (not direct DB writes) because each operation calls `reload()` to rebuild the in-memory state:
- `add_provider()` / `update_provider()` / `delete_provider()`
- `add_model()` / `update_model()` / `delete_model()`
Setting `is_default = true` on a model automatically clears the flag on all others.
**Soft delete:** `delete_provider()` and `delete_model()` never issue `DELETE` statements. They set `removed_at = datetime('now')` on the row. Deleting a provider also cascades to all its models and clears the provider's `api_key`. Removed rows are excluded from `load_all_providers()` / `load_all_models()` and therefore from the in-memory state and AUTO selector. The `id` values remain valid as FK references in `chat_history.model_db_id`.
**Model identity is `name`.** `llm_models` has a single unique constraint, `name TEXT UNIQUE` — the alias, which is also the **resolution key** (`LlmManager` holds models in a `HashMap` keyed by name; `resolve(client_name)` / an agent's `client` look up by it). There is deliberately **no** `UNIQUE(provider_id, model_id)`: the same underlying model may be registered several times under one provider with different aliases and reasoning settings (e.g. `glm-4.6` vs `glm-4.6-thinking`). *(Migration v20 dropped the old `(provider_id, model_id)` constraint by rebuilding the table, preserving `id` values — they are FK targets in `llm_requests.model_db_id` / `chat_history.model_db_id`.)*
**Re-adding a removed model (revive):** rows are never hard-deleted, only soft-deleted (`removed_at`), and the `name` uniqueness ignores `removed_at`. So `insert_model()` upserts on `name`: `ON CONFLICT(name) DO UPDATE … removed_at = NULL`. Re-adding a model with the same alias **revives that row** (clearing `removed_at`, overwriting every field including `provider_id`/`model_id`/`reasoning`) instead of failing with a UNIQUE violation.
---
## ApiProvider — Service Types
Each provider declares which service kinds it supports via `ApiProvider::supported_types() -> &'static [ServiceType]`. Hardcoded per implementation — not stored in the DB.
`ServiceType` replaces the old `ModelType` enum (previously in `src/core/llm/providers/mod.rs`); it now lives in `src/core/provider/mod.rs` and is re-exported as `providers::ServiceType` for backwards compatibility.
| Provider (`type_id`) | `supported_types()` |
| -------------------- | ------------------------------------------------ |
| `openrouter` | `[Llm, Transcribe, ImageGenerate, Tts]` |
| `open_ai` | `[Llm, Transcribe, Tts]` |
| `anthropic` | `[Llm]` |
| `ollama` | `[Llm]` |
| `lm_studio` | `[Llm]` |
| `deepseek` | `[Llm]` |
| `elevenlabs` | `[Tts, Transcribe]` |
`supported_types` is included in the `GET /api/llm/providers` response so the frontend can filter provider dropdowns when adding TTS, transcription, LLM, or image generation models.
`GET /api/llm/providers/types` returns **all** registered provider types (no service-type filter). The frontend filters each picker independently using the `supported_types` array — e.g. the LLM model picker shows only providers where `supported_types.includes('llm')`.
---
## ApiProvider — Remote Model Catalog
`list_llm_models()` and `llm_model_info()` are methods on `ApiProvider`. They both receive the full `LlmProviderRecord` so they can read the `api_key` and `base_url` without constructing a separate credentials struct.
`RemoteLlmModelInfo` fields: `id`, `name`, `context_length`, `max_completion_tokens`,
`knowledge_cutoff`, `capabilities`, `vision: Option<bool>`, `price_input_per_million`, `price_output_per_million` (USD/M tokens).
`vision` is `Some(true/false)` when the provider reports it explicitly (e.g. OpenRouter `supported_parameters`), `None` when unknown.
| Provider (`type_id`) | `list_llm_models()` | `llm_model_info()` |
| --- | --- | --- |
| `openrouter` | `GET /api/v1/models` — sets `vision` from `supported_parameters` | — |
| `ollama` | `GET /api/tags` | `POST /api/show` |
| `anthropic` | `None` | `GET /v1/models/{id}` |
| `deepseek` | `GET /models` | `None` |
| `lm_studio` | `GET /v1/models` | `None` |
| `open_ai` | `None` | `None` |
| `elevenlabs` | `None` (LLM not supported) | `None` |
Provider instances are obtained via `ProviderRegistry::get(type_id)` — no on-demand factory needed.
### Model Catalog Cache
`LlmManager` caches `list_models()` results in memory, keyed by `provider_id`, with a **24-hour TTL**. The cache is discarded on process restart.
### Per-Model Metadata Cache
When `LlmManager::resolve()` is called, it lazily fetches `model_info()` for the resolved model
if the per-model cache is missing or older than **1 hour**. The `context_length` from the fresh
metadata is then propagated to the live `LlmEntry` in the model slot so subsequent turns use
the updated value.
Cache flow:
- Fast path: read lock on `model_meta_cache` → hit + fresh → return immediately.
- Miss / stale: fetch `model_info()` from the provider → update cache → update `LlmEntry.context_length`.
- Network failure: the old cached value (or DB value) is preserved — the error is silently ignored.
This ensures the compactor and any future `max_tokens` logic always have a reasonably current
`context_length` without blocking the first turn of the session.
```text
LlmManager::list_provider_models(provider_id)
→ cache hit (< 24h old) → return cached Vec<RemoteLlmModelInfo>
→ cache miss / expired → fetch via ApiProvider, store, return
```
API endpoint: `GET /api/llm/providers/{id}/models`
Used by the frontend "Add Model" wizard to populate the searchable model picker for OpenRouter, Ollama, and LM Studio providers.
---
## When to Update This File
- A new built-in provider is registered in `main.rs` (add row to the tables above)
- A new method is added to the `ApiProvider` trait
- The AUTO selection algorithm changes
- Health thresholds (`FAILURE_DEGRADED`, `FAILURE_DOWN`) change
- `ProviderRegistry` plugin API changes (register/unregister)
docs/logging-config.md
+151
View File
@@ -0,0 +1,151 @@
# Logging & Configuration
## Logging Setup
Log files are written to `logs/` using `tracing-appender` with daily rotation:
```
logs/skald.log.YYYY-MM-DD
```
A new file is created each day. The non-blocking writer is initialized in `main()` and the `_log_guard` is kept alive for the full process lifetime to ensure all buffered logs are flushed on shutdown.
The subscriber has **two layers** (`main.rs`):
| Layer | Writer | Filter | Format |
|---|---|---|---|
| File | `logs/skald.log` (daily) | `EnvFilter` (`RUST_LOG`, default `info`) | full structured (timestamp, level, target, fields) |
| Stdout | terminal | `boot` target only | minimal — message only, failures in red |
**Runtime output goes only to the file.** Stdout carries just the curated
**bootstrap** lines (see below); once the app is up nothing else is printed there.
---
## Bootstrap output (stdout)
During startup a small, ordered set of human-readable lines is printed to stdout
so you can see at a glance how the app is configured and how it is coming up:
```
skald v0.1.0 — starting
› Database ready (schema v16)
› MCP servers — connecting to 18 in background
✓ codebase-memory (14 tools)
› Plugins — 6 active, 1 failed, 2 available
✓ honcho, telegram, comfyui, elevenlabs, mobile-connector, whisper_local
✗ remote_connectivity — creating tailscale device
○ orpheus_tts_3b, kokoro_tts
✅ Ready — http://localhost:3000
✓ gcal (8 tools)
...
```
These are emitted via the helpers in `src/boot.rs` (`title`, `section`, `ok`,
`off`, `fail`, `ready`) on the `boot` tracing target. They are rendered by a
dedicated stdout layer that:
- shows **only** the `boot` target (it ignores `RUST_LOG`, so bootstrap output
always appears);
- strips timestamps/levels/targets and colours failures red (ANSI only on a TTY);
- still lets the same lines reach the **file** log as a high-level startup trace.
Note the glyph convention: `✓` started/connected, `✗` failed (with reason),
`○` available but disabled, `›` phase header, `✅` ready.
**MCP servers connect asynchronously** and do not block startup, so their `✓`/`✗`
lines stream in as each server responds — some may appear *after* the `✅ Ready`
line. The app is usable as soon as `Ready` prints (HTTP listening); MCP tools
become available as their servers connect.
To add a bootstrap line from anywhere in the binary crate, call
`crate::boot::section("…")` (or `ok`/`fail`/`off`). Keep them few and targeted —
this is a curated summary, not a log.
---
## Log Levels and RUST_LOG
Default level: **`info`**
Override with the `RUST_LOG` env var:
| Example | Effect |
|---|---|
| `RUST_LOG=info` | Default: info and above |
| `RUST_LOG=skald=debug,info` | Debug for this crate, info for dependencies |
| `RUST_LOG=trace` | Everything (very verbose) |
Level semantics:
| Level | Use for |
|---|---|
| `ERROR` | Failures requiring a code or config fix (config load failure, DB init error, LLM loop exhausted) |
| `WARN` | Non-critical anomalies to fix eventually (malformed API response, agent skipped) |
| `INFO` | Normal significant events (server started, session opened, job executed, response tokens) |
| `DEBUG` | Per-operation lifecycle (request sent to LLM, tool dispatched, context built) |
| `TRACE` | Fine-grained internals (round counters, full request bodies, message arrays) |
A dropped WebSocket connection, a cancelled request, or a completed session are **INFO** at most — they are expected runtime events, not errors.
---
## config.yml Structure
Loaded by `Config::load()` at startup. Copied from `default.config.yaml` if `config.yml` does not exist. **Never commit `config.yml`** — it may contain API keys.
| Section | Key | Type | Default | Notes |
|---|---|---|---|---|
| `server` | `host` | string | `127.0.0.1` | Bind address |
| `server` | `port` | u16 | `3000` | HTTP/WS port |
| `web` | `static_dir` | string | `./web` | Path to static frontend files |
| `db` | `path` | string | `./database.db` | SQLite file path |
| `llm` | `max_history_messages` | usize | `30` | Max messages kept per context window. Ignored when `compaction` is configured — the compactor manages the token budget instead. |
| `llm` | `max_tool_rounds` | usize? | `20` | Max tool-call rounds per message; falls back to `DEFAULT_MAX_TOOL_ROUNDS` |
| `llm.requests_log` | `enabled` | bool | `false` | Log every LLM call to the `llm_requests` table. **Disabling this also disables home-page LLM statistics.** |
| `llm.requests_log` | `request_payload_save` | bool | `true` | Persist request JSON (can be hundreds of KB per call) |
| `llm.requests_log` | `response_payload_save` | bool | `true` | Persist response JSON |
| `llm.requests_log` | `request_header_save` | bool | `true` | Persist request HTTP headers (api-key always redacted) |
| `llm.requests_log` | `response_header_save` | bool | `true` | Persist response HTTP headers |
| `llm.requests_log` | `cleanup_request_payload_after` | u32? | `null` | Set `request_json = ''` for rows older than N days |
| `llm.requests_log` | `cleanup_response_payload_after` | u32? | `null` | Set `response_json = NULL` for rows older than N days |
| `llm.requests_log` | `cleanup_headers_after` | u32? | `null` | Null out both header columns for rows older than N days |
| `llm.requests_log` | `cleanup_rows_after` | u32? | `null` | Physically delete rows older than N days (`null` = keep forever) |
The `llm.clients` block in `config.yml` is for reference only — actual runtime LLM providers and models are stored in the DB and managed via the UI or API.
---
## LLM Providers in config.yml
| Provider | Required fields | Optional fields |
|---|---|---|
| `lm_studio` | `model` | `base_url` (default: `http://localhost:1234/v1`) |
| `ollama` | `model` | `base_url` (default: `http://localhost:11434`) |
| `openai` | `model`, `api_key` | `base_url`, `strength`, `scope` |
| `anthropic` | `model`, `api_key` | `strength`, `scope` |
| `open_ai` (OpenRouter) | `model`, `api_key`, `base_url` | `strength`, `scope`, `extra_params` |
`extra_params` is merged into the request body top-level (used for provider-specific fields like `reasoning.effort`).
---
## config.yml vs DB
| What | Where | How to change |
|---|---|---|
| Server host/port | `config.yml` | Edit file, restart app |
| DB path | `config.yml` | Edit file, restart app |
| History/round limits | `config.yml` | Edit file, restart app |
| LLM providers | DB (`llm_providers`) | UI or REST API |
| LLM models | DB (`llm_models`) | UI or REST API |
| MCP servers | DB (`mcp_servers`) | `register_mcp` tool or UI |
| Cron jobs | DB (`scheduled_jobs`) | `execute_task` tool (mode=cron) or UI |
---
## When to Update This File
- `Config` struct gains or loses a field
- Log level semantics change (see also `memory/feedback_logging.md`)
- `config.yml` gains a new section or key
docs/mcp.md
+648
View File
@@ -0,0 +1,648 @@
# MCP (Model Context Protocol)
## Workspace Location
The MCP protocol layer lives in the standalone crate `crates/mcp-client`:
- `McpServer` — stdio subprocess client
- `McpHttpServer` — streamable HTTP client
- `McpServerClient` trait, `McpTool`, `McpServerConfig`, `McpTransport`
`McpManager` (`src/core/mcp/mod.rs`) remains in the main crate because it owns the `SqlitePool` and calls `crate::db::mcp_events` / `crate::db::mcp_servers`.
---
## What MCP Is Here
MCP allows external processes or HTTP services to expose tools to the LLM. The app connects to MCP servers at startup (or on demand via `register_mcp`), discovers their tools, and makes them available alongside built-in tools.
---
## McpManager Internals
```rust
McpManager {
pool: Arc<SqlitePool>
servers: RwLock<HashMap<String, Arc<dyn McpServerClient>>> // running servers
errors: RwLock<HashMap<String, String>> // startup failures
}
```
Initialization runs in a background `tokio::spawn` task. The manager is available immediately; servers connect asynchronously. A server failing to start is recorded in `errors` and does not block the app.
---
## Transports
| Transport | When to use | Required fields |
| --- | --- | --- |
| `stdio` | Local process (spawn subprocess) | `command`, optionally `args`, `env` |
| `http` | Remote HTTP server (streamable MCP) | `url`, optionally `api_key` |
| `sse` | Alias for `http` (backward compat) | same as `http` |
`${VAR}` interpolation is supported in `env` values and `api_key`.
### stdio process lifecycle
stdio subprocesses are spawned with `kill_on_drop(true)` and, on Unix, in their
own process group (`process_group(0)`). The new process group detaches them from
the terminal's foreground group, so a terminal Ctrl+C (SIGINT to the whole group)
does not reach them directly — otherwise Python-based servers would catch it and
dump a `KeyboardInterrupt` traceback. They are instead reaped via `kill_on_drop`:
when the app shuts down and the per-server reader task is dropped, the child gets
a silent SIGKILL.
The child's **stderr is captured** (`Stdio::piped()`, not inherited) and drained
into `tracing` at `debug` level under the `mcp_client` target, prefixed with the
server name. This keeps startup banners, deprecation warnings and INFO logs from
servers like FastMCP off the console at the default log level, while still making
them available for diagnostics via `RUST_LOG=mcp_client=debug`. Each stderr line is
**also** forwarded to that server's per-server log file — see [Per-server logs](#per-server-logs).
---
## Protocol version, header & pagination
**Protocol version** is a single shared constant — `PROTOCOL_VERSION` in
`crates/mcp-client/src/lib.rs` (currently **`2025-11-25`**, the revision Skald
targets). Both transports advertise it in their `initialize` request, so they can
never drift apart. **Capabilities are per-transport**, not shared: stdio declares
`{ "elicitation": {}, "experimental": { "tasks": {} } }` (form mode — see
Elicitation below); HTTP declares `{ "experimental": { "tasks": {} } }` because it
does not service the `ElicitationHandler` (stdio-only) and must not claim a
capability it can't honour. The `experimental.tasks` marker signals Skald is
task-*aware* (it recognises a deferred `CreateTaskResult`) without claiming the
full `tasks` capability — see *Cancellation & Tasks* below.
**Version negotiation is tolerant.** Skald reads `protocolVersion` from the
`initialize` response and, if the server negotiates a different (older) version,
logs a `warn!` and proceeds rather than disconnecting.
**`MCP-Protocol-Version` header (HTTP only).** Per the Streamable HTTP spec, every
*post-initialize* request must carry `MCP-Protocol-Version: <negotiated>`. The HTTP
transport captures the negotiated version into `protocol_version: Mutex<Option<String>>`
(mirroring how `session_id` is captured) and `request_headers()` injects it on every
`request`/`notify`. It is `None` only during the `initialize` call itself, so the
header is naturally omitted there (the spec scopes it to post-initialize requests).
**`tools/list` pagination.** Both transports follow the cursor: `tools/list` is
requested with `{ "cursor": <nextCursor> }` until the response omits `nextCursor`,
accumulating every page (previously only the first page was read, silently
truncating large servers). A `MAX_TOOL_PAGES` (50) cap guards against a server that
never clears the cursor. The per-tool field mapping lives in one place,
`McpTool::from_json`, shared by both transports.
---
## Structured tool results
MCP tools with an `outputSchema` return `structuredContent` (a JSON object) in
addition to (or instead of) text. Skald preserves the type end-to-end instead of
flattening everything to a string:
- `McpCallResult` (`crates/mcp-client/src/lib.rs`) — the transport-level result:
`Text(String)`, `Json(Value)`, or `Media { text, structured, items }` (see
*Media tool results* below). `extract_call_result` **prefers
`structuredContent`** when present (canonical per spec) and falls back to the
joined `text` items — which also fixes the silent empty-result case for servers
that return only `structuredContent` without the recommended text mirror.
> Tradeoff: when a server returns *both* a text mirror and `structuredContent`,
> the LLM sees the (compact) JSON, not the text. With a single `result` column +
> a type tag this is the correct one-representation choice (JSON is lossless and
> LLM-readable; the mirror is usually just `JSON.stringify` of the same object).
- `McpManager::call` maps `McpCallResult` → `ToolResult` (`crates/core-api/src/tool.rs`),
the host-side equivalent (`Text`/`Json`). `ToolResult::to_wire()` is the string
persisted in `chat_llm_tools.result` and replayed to the LLM (Json → compact JSON
string); `ToolResult::kind()` is the `"string"`/`"json"` tag.
- The tag is persisted in `chat_llm_tools.result_type` (schema **v19**, `DEFAULT
'string'`, `CHECK IN ('string','json')`) and sent to the frontend both live
(`ServerEvent::ToolDone.result_type`) and on history replay / approval-resolve
(`/api/sessions` items + `ResolveToolResponse`).
- Frontend: `copilot-render.js` renders a `result_type === 'json'` result as
pretty-printed JSON (`.copilot-tool-pre--json`); everything else stays plain text.
**In-repo example:** `scripts/weather_mcp_server.py`'s `get_air_quality` tool
emits `structuredContent` with a declared `outputSchema` — the payload carries a
human-readable `summary` string (emoji-formatted, the text mirror) **plus** the
raw numeric AQI and pollutant fields (`european_aqi`, `us_aqi`,
`pollutants_ug_m3`, …) for machine consumption. The other weather tools
(`get_current_weather`, `get_forecast`, `status`) return plain text. This is the
minimal in-repo reference for a server that emits structured results: it builds
the dict in the handler and the JSON-RPC layer wraps it as `structuredContent`
(`_structured_result`); error paths still return plain `Error:` text.
---
## Media tool results
MCP tool results can carry non-text content blocks — `image`, `audio`, embedded
`resource` (base64 `blob`), and `resource_link` (a remote URI). Previously these
were dropped: `extract_text` read only `content[].text`, so an MCP server that
generated an image returned an empty result unless it also mirrored the bytes in
`structuredContent`.
Now they are preserved end-to-end, **saved to disk and surfaced as a URL** (the
same model as the built-in `image_generate` tool — the bytes never enter the LLM
wire format):
- `crates/mcp-client/` stays a generic transport: `classify_content` walks
`content[]`, decodes the base64 of `image`/`audio`/`resource` blocks into bytes
and passes through `resource_link` URIs, producing `McpCallResult::Media { text,
structured, items: Vec<McpMedia> }`. The `Media` variant is emitted **only** when
at least one media block is present; pure text/JSON results are unchanged (no
regression).
- `McpManager::persist_media` (`src/core/mcp/mod.rs`) writes each inline item to
`data/mcp_media/<id>.<ext>` (`<ext>` from the MIME via `ext_for_mime`) and
composes a **markdown** `ToolResult::Text` referencing each item by URL
(`![image](/api/mcp-media/<id>.png) (image/png, 412 KB)`, `[audio](…)`,
`[file](…)`); `resource_link`s become `[<mime>](<uri>)`. Markdown (not JSON) so
the model can relay the URL into its message, where `renderMarkdown` displays it.
- Serving: `GET /api/mcp-media/{file}` (`src/frontend/api/mcp_media.rs`) reads from
`McpManager::media_dir()` with the `Content-Type` inferred by
`content_type_for_ext`; the filename is path-sanitized (flat `<id>.<ext>` only).
- **Not** sent to the model as a multimodal content block (the model does not
"see" the pixels) and **not** rendered inline in the tool card yet — both are
possible future enhancements.
`McpTool` also captures `title`, `output_schema`, `annotations` (2025-06-18+), and
`task_support` (`execution.taskSupport`, 2025-11-25). These are stored but **not
yet** validated/surfaced (output-schema validation, `readOnlyHint`/`destructiveHint`
UI hints, and per-tool task negotiation are future work).
---
## Cancellation & Tasks
Two 2025-11-25 base utilities the client now covers (`crates/mcp-client/`):
**`notifications/cancelled`.** When an in-flight `tools/call` is abandoned, the
client tells the server to stop instead of silently leaving it working. Both
transports arm a drop-guard **only for `tools/call`** (the spec forbids cancelling
`initialize`):
- `CancelOnDrop` (`server.rs`, stdio) / `HttpCancelOnDrop` (`http_server.rs`, HTTP)
hold the request `id`; if dropped while still *armed* they emit
`notifications/cancelled { requestId, reason }` (built by the shared
`cancelled_notification` helper in `lib.rs`, so the two transports can't drift).
- Fires in exactly two cases: a `/stop` drops the work future (`SimpleExecution`
→ `mcp.call` → `request()` future), and the 120 s `CALL_TIMEOUT_SECS` elapses
(reason `"timeout"`). Disarmed once the server replies or disconnects, where
cancelling is pointless.
- stdio also drops the now-orphaned `pending` entry (a small leak fix). HTTP is
**best-effort**: `requestId`↔POST correlation is weaker over Streamable HTTP, and
a non-timeout send error (server never received the request) disarms instead.
**Tasks (experimental, block-and-poll).** A server MAY defer an expensive
`tools/call` and return a `CreateTaskResult` (durable `taskId`, `status`,
`pollInterval`, `ttl`) instead of blocking. The client drives it to completion
synchronously:
- **Opt-in per request.** `call_tool` adds a `task` field to `tools/call` when the
tool advertises `execution.taskSupport` as `required`/`optional` (`McpTool.task_support`,
captured in `from_json`). Adding the field is the spec's opt-in for `tools/call`
(the client is the *requestor*), so no extra capability declaration is needed —
the `tasks` marker stays under `capabilities.experimental`.
- **Recognise.** `CreateTaskResult::parse` (top-level or nested under `task`) runs in
`extract_call_result` **before** the media/text logic, yielding `McpCallResult::Task`
so a handle isn't mistaken for an empty result.
- **Poll (`poll_task`, per transport).** Sleep `clamp_poll_interval` (server
`pollInterval`, clamped to [500 ms, 30 s]), then `tasks/get` until a terminal
status: `completed` → fetch the real result via `tasks/result` (parsed like any
normal result); `failed`/`cancelled` → error; `input_required` → error (mid-task
input is a follow-up). Bounded by `poll_deadline` (the task's `ttl`, else a 1 h
cap). **Each poll request is a normal short `request()`; only the overall wait is
unbounded — so a task-mode call no longer hits the 120 s `CALL_TIMEOUT_SECS` wall.**
- **Cooperative cancel.** A `TaskCancelOnDrop` / `HttpTaskCancelOnDrop` guard sends
`tasks/cancel { taskId }` (shared `tasks_cancel_request` helper) if the poll future
is dropped (a `/stop`, between polls) or the deadline is hit; disarmed on a terminal
status. Server-caps captured at `initialize` (`server_capabilities()`) remain for a
future poller to gate on.
**Trade-offs (v1, block-and-poll).** The session holds its `processing` lock for the
whole task (like any long tool call), and polling does **not** survive a Skald
self-restart. A *detached/durable* variant (DB-persisted tasks + a background poller
delivering via `inject_async_result`/`resume_turn`, surviving restart and freeing the
session) is the tracked follow-up. `McpManager::call`'s `McpCallResult::Task` arm is
now only a **defensive fallback** (polling normally resolves the task in `call_tool`).
---
## Tool Naming Convention
MCP tools are exposed to the LLM as **`mcp__<server_name>__<tool_name>`**.
Examples:
- Server `tavily`, tool `search` → `mcp__tavily__search`
- Server `fetch`, tool `get` → `mcp__fetch__get`
`parse_mcp_tool_name(name)` in `src/core/mcp/mod.rs` splits on `__` to extract server and tool names. This is how `run_agent_turn` routes MCP calls.
---
## Registering a Server
All MCP servers are stored in the **`mcp_servers` table** in SQLite. There is no static config file.
**Live registration** via `register_mcp` tool:
- LLM calls `register_mcp` with name, transport, connection details, and optionally `description` and `friendly_name`
- `McpManager::register()` does DB upsert + live `start_one()` connect
- Server is immediately available without a restart
**Tool parameters:**
| Parameter | Required | Type | Description |
| --- | --- | --- | --- |
| `name` | yes | string | Unique name for this MCP server (used to reference it in tool calls) |
| `transport` | yes | string | `stdio`, `http`, or `sse` |
| `command` | stdio only | string | Executable to spawn |
| `args` | stdio only | string[] | Command-line arguments |
| `env` | stdio only | object | Extra environment variables |
| `url` | http/sse only | string | Base URL of the remote server |
| `api_key` | http/sse only | string | API key (sent as `Authorization: Bearer <key>`) |
| `description` | no | string | Short description of what the server provides (shown in `list_items` type=mcp) |
| `friendly_name` | no | string | Human-readable display name for UI (e.g. "Google Calendar") |
**Startup timeout**: **`SERVER_START_TIMEOUT_SECS = 120`**. Servers that don't respond within 120 s are recorded as errors.
---
## Enabling / Disabling Servers
Use the built-in tool **`toggle_item`** (kind=mcp) to enable or disable an MCP server by name:
```text
toggle_item(kind="mcp", id="gcal", enabled=false) # disable
toggle_item(kind="mcp", id="gcal", enabled=true) # enable
```
**Important:** Toggling updates the `enabled` flag in the database, but **a restart is required** for the change to take effect on running servers. Disabled servers won't connect on next restart.
Use `list_items` (type=mcp) to see current server names and statuses.
## Deleting Servers
Use the built-in tool **`delete_mcp`** to permanently remove a server. It calls `McpManager::unregister`, which deletes the `mcp_servers` row and disconnects the running client in one step — no restart needed:
```text
delete_mcp(name="gcal")
```
This is **irreversible** (the configuration is discarded). To turn a server off while keeping its configuration, use `toggle_item(kind="mcp", enabled=false)` instead. Like `delete_cron_job`, `delete_mcp` is kept separate from the reversible `toggle_item` so it can carry its own approval rule.
---
## Example: Google Calendar MCP Server
A custom Python MCP server (`scripts/gcal_mcp_server.py`) provides full read/write access to Google Calendar:
| Tool | Description |
| --- | --- |
| `list_calendars` | Lists all calendars accessible to the authenticated user |
| `list_events` | Lists events with filters: `calendar_id`, `start_time`, `end_time`, `max_results`, `full_text`, `time_zone` |
| `get_event` | Returns a single event by `event_id` |
| `create_event` | Creates a new event (`summary`, `start`, `end`, optional description/location/attendees/recurrence) |
| `update_event` | Updates an existing event — only fields provided are changed |
| `delete_event` | Permanently deletes an event by `event_id` |
| `respond_to_event` | Sets RSVP status (`accepted`, `declined`, `tentative`, `needsAction`) |
**Credentials:** Stored in `./secrets/google_creds.json`. Run `python3 scripts/gcal_oauth_setup.py` to authenticate (requires `https://www.googleapis.com/auth/calendar` scope). Token refresh is handled automatically.
**Register:**
```text
register_mcp(name="gcal", transport="stdio", command="python3", args=["scripts/gcal_mcp_server.py"])
```
**Disable when not needed:**
```text
toggle_item(kind="mcp", id="gcal", enabled=false)
restart
```
---
## Push Notifications from MCP Servers
MCP servers can send **unsolicited events** to the app by writing JSON-RPC notification messages (no `id` field) to stdout. The app persists them to SQLite and processes them in batches via the TIC background agent.
### Protocol
A notification is a JSON-RPC 2.0 message without `id`:
```json
{"jsonrpc": "2.0", "method": "event/new_email", "params": {"subject": "...", "from": "..."}}
```
### How it flows
```text
MCP server writes notification to stdout
→ McpServer reader loop detects msg with no "id"
→ sends (server_name, msg) over notification_tx channel
→ McpManager::notification_consumer persists to mcp_events table
→ TicManager (every `tic.interval_secs`, default 900 s) fetches pending events, runs TIC agent
→ TIC calls notify(briefing) if user action is needed
```
**Exception — `notifications/message`.** The MCP *logging* utility method
`notifications/message` (`{ level, logger?, data }`) is **not** a business event: it
carries a diagnostic log record. The reader loop diverts it to the server's
[per-server log file](#per-server-logs) via `log_tx` instead of `notification_tx`, so
log records never reach `mcp_events`/TIC. Every other method (the custom `event/*`
notifications below) flows as shown above. Verified against production data: business
events use `event/*`; the only server observed emitting `notifications/message` was
`firecrawl`, and it was pure logging.
### Implementing notifications in an MCP server
**Node.js (WhatsApp)**:
```js
function notify(method, params) {
process.stdout.write(JSON.stringify({jsonrpc:'2.0', method, params}) + '\n');
}
client.on('message', async (msg) => {
if (msg.fromMe) return;
notify('event/whatsapp_message', { from: msg.from, body: msg.body });
});
```
**Python (Gmail, GCal)** — use a lock to avoid interleaving with MCP responses:
```python
import threading
_stdout_lock = threading.Lock()
def _emit_notification(method, params):
msg = json.dumps({"jsonrpc": "2.0", "method": method, "params": params})
with _stdout_lock:
sys.stdout.write(msg + "\n")
sys.stdout.flush()
```
Start a daemon polling thread in `main()` before entering the MCP serve loop. The MCP serve loop must also acquire `_stdout_lock` before writing responses.
### Implemented notification sources
| Source | Method | Trigger | Poll interval |
| --- | --- | --- | --- |
| `whatsapp` | `event/whatsapp_message` | Inbound WhatsApp message | Real-time (event) |
| `gmail` | `event/new_email` | New email in INBOX | 60 s (History API) |
| `gcal` | `event/new_calendar_event` | New calendar event created | 300 s (Events API) |
---
## Per-server logs
Each MCP server gets its own diagnostic log file at **`logs/mcp/<name>.log`** (the
server name is sanitized: non-`[A-Za-z0-9._-]` characters become `_`). This is a plain
append-only file — no SQLite — meant to be scanned later (e.g. by a diagnostics agent)
for `[error]`/`[warning]` lines.
**Sources captured** (all keyed by server name, all routed over the crate's `log_tx`
channel to `McpManager`'s `logs::log_consumer`):
| Tag | Source | Transports |
| --- | --- | --- |
| `[stderr]` | Raw child-process stderr line | stdio |
| `[<level>]` | `notifications/message` log record — tag is the MCP level (`debug`..`emergency`) | stdio |
| `[lifecycle]` | Start failure, startup timeout, connection, disconnect | all (incl. HTTP/SSE) |
**Line format** — ISO-8601 UTC timestamp, padded level tag, then text:
```text
2026-07-03T12:34:56.789Z [stderr] INFO: server ready
2026-07-03T12:34:56.789Z [warning] scraper: rate limit approaching
2026-07-03T12:34:56.789Z [lifecycle] failed to start: command not found
```
**Rotation:** when a file would exceed **5 MB** it is rotated to `<name>.log.1` (one
backup kept) so a chatty server can't grow the file without bound.
**Why stderr is primary.** The MCP *logging* utility (`notifications/message` +
`logging/setLevel`) is **deprecated** from the 2026-07-28 draft (SEP-2577); the official
migration path is `stderr` for stdio transports plus OpenTelemetry. So `stderr` is the
future-proof primary source and `notifications/message` is captured only for interop with
servers that still emit it. HTTP/SSE servers have no stderr and no async notification
listener, so their files contain lifecycle lines only.
**Code:** emission lives in `crates/mcp-client/src/log.rs` (`McpLogLine`, `McpLogTx`) and
the stdio reader loop in `crates/mcp-client/src/server.rs`; file writing lives in
`src/core/mcp/logs.rs` (`log_consumer`), spawned from `McpManager::new`. Lifecycle lines
are emitted by `McpManager::log_lifecycle`.
---
## Elicitation — server-initiated input (spec 2025-06-18)
An MCP server can ask the user for input **during** a tool call — a server→client
request, distinct from the unsolicited notifications above. Primary use case: the
SSH MCP asking for a sudo password on demand (see `data/mcp_ssh.md`). The value
never reaches the LLM, is never logged, and is never persisted.
Skald advertises the capability on the **stdio** transport's `initialize`
(`"capabilities": { "elicitation": {} }`) and surfaces requests in the Agent Inbox.
The `protocolVersion` is the shared `PROTOCOL_VERSION` const (see *Protocol version,
header & pagination*); `{ "elicitation": {} }` is form mode, which is what Skald
supports (URL-mode elicitation, new in 2025-11-25, is not yet handled).
### Elicitation protocol
A server→client request has **both** `method` and `id`:
```json
{"jsonrpc":"2.0","id":"e1","method":"elicitation/create","params":{
"message":"Enter sudo password",
"requestedSchema":{"type":"object","properties":{
"password":{"type":"string","format":"password"}}}}}
```
Skald replies on the same stdin: `{action: "accept"|"decline"|"cancel", content?: {…}}`.
### Elicitation flow
```text
MCP server writes elicitation/create (method + id) to stdout
→ McpServer reader loop routes it to handle_server_request (BEFORE the id/response
branch, since it has both method and id) and spawns a task (the user may take minutes)
→ ElicitationHandler bridge (src/core/elicitation) → ElicitationManager::register
→ ServerEvent::ElicitationRequested → Agent Inbox card ("Secrets" section)
→ user enters a value (masked if sensitive) and confirms / rejects
→ POST /api/inbox/elicitations/{id}/resolve → ElicitationManager::resolve
→ bridge maps the outcome → reader loop writes the JSON-RPC reply to the server's stdin
```
While an elicitation is in flight, the underlying `tools/call` does **not** time out
(`pending_elicitations` counter re-arms the call timeout). On a 5-min user-response
deadline, channel drop, or `decline`/`cancel`, the server receives a non-accept reply.
The schema is **v1-scoped**: a single field (masked when `format: password`,
`writeOnly: true`, or the name contains `password`/`passphrase`/`secret`/`token`),
or an empty `properties` ⇒ a yes/no confirmation. Elicitation is **stdio-only**.
A dependency-free demo server lives at `scripts/elicitation_demo_mcp.py`.
---
## SSH MCP Server
`scripts/ssh_mcp_server.py` is a stdio MCP server (depends on `paramiko>=3.4`) that
operates on remote hosts. Its filesystem tools intentionally produce the **same output
format** as Skald's native fs tools (`read_file`, `list_files`, `grep_files`, `edit_file`,
`replace_lines`) so the LLM treats local and remote uniformly — the only difference is a
leading `alias` argument. Full design notes: `data/mcp_ssh.md`.
**13 tools** (bare names; Skald prefixes `mcp__ssh__`):
- Aliases: `list_aliases`, `add_alias`, `remove_alias`.
- Filesystem (SFTP, login user): `read_file`, `list_files`, `grep_files`, `edit_file`, `replace_lines`.
- Exec/transfer: `exec` (with `sudo`/`sudo_user`), `upload`, `download` (both recursive on directories).
- Diagnostics: `sysinfo`, `systemd`.
**Aliases** live in `secrets/ssh_aliases.json` (auto-managed by the tools, written
atomically at `0600`, gitignored). The file holds **no secrets** (no password, ever).
Host keys are checked against `~/.ssh/known_hosts`; unknown hosts are rejected unless the
alias was added with `accept_new_host_key=true`. Connections are pooled per alias with
lazy TTL eviction (`SSH_MCP_POOL_TTL`, default 300s).
**Login auth** (per-alias `auth`, default `key`):
- `key` → SSH key / ssh-agent. If the chosen private key is encrypted, its passphrase is
requested **lazily** via elicitation (only when paramiko reports the key needs one).
`SSH_MCP_KEY_PASSPHRASE` still works as a non-interactive override.
- `password` → login password requested on demand via **elicitation** (a masked field in
the Agent Inbox); agent/key probing is skipped so paramiko goes straight to the password.
Elicited login secrets are cached only in the server's RAM (`SSH_MCP_LOGIN_PW_TTL`, default
300s), never sent to the LLM, never written to disk, and dropped on an authentication
failure so the next attempt re-prompts.
**sudo** (per-alias `sudo.method`, default `prompt`):
- `nopasswd` → `sudo -n`: non-interactive, **never prompts**. Pick this **only** when the alias user has a `NOPASSWD:` rule in the remote sudoers; on a normal host every sudo call fails fast with "a password is required" (no hung channel). When in doubt use `prompt`.
- `prompt` → `sudo -S`: the password is requested on demand via **elicitation** (see above),
a masked single field; fed to sudo's stdin, cached only in the server's RAM
(`SSH_MCP_SUDO_PW_TTL`, default 300s), never sent to the LLM, never written to disk.
- `none`: sudo disabled.
SFTP tools run as the login user (no root). For privileged writes the LLM is told to use
`exec(..., sudo=true)` with `tee`/`install`.
`upload` follows scp/rsync destination semantics for a single file: a `remote_path` ending
in `/` (or that is an existing remote directory) uploads the file **into** that directory
keeping its basename; otherwise `remote_path` is the exact destination file path. Missing
parent directories are created either way. (paramiko's `sftp.put` alone would reject a
directory-style path with a generic `Failure`.)
Register like any stdio server: `command: python3`, `args: ["scripts/ssh_mcp_server.py"]`.
---
## Lazy MCP Tool Loading
By default, injecting all MCP tool definitions into every LLM turn is expensive — 30+ tools can consume 10,000+ tokens per turn. Lazy loading solves this by only including tools for servers that have been explicitly activated.
### How It Works
1. At the start of each turn, `build_agent_config` reads `session_mcp_grants` for the current `session_id` and populates `active_mcp_grants` in memory.
2. **MCP tools are no longer part of `base_tool_defs`**. Instead, `AgentRunConfig::all_tool_defs()` re-queries `mcp.tools_for(active_mcp_grants)` on **every LLM round**. This means an `activate_tools` call in round N makes those tools available from round N+1 within the same turn — no cross-turn delay.
3. The system prompt contains a `<!-- MCP_LIST -->` tag (in `AGENT.md`) which is replaced at request time with a dynamic two-section block:
```text
## MCP servers
**Available** — call `activate_tools(["name"])` to load tools:
| Server | Description |
|------------|------------------------------------------|
| `tavily` | Web search and content extraction |
| `whatsapp` | Send and receive WhatsApp messages |
**Active** — tools callable as `mcp__<name>__<tool>`:
- `gmail`
```
### `<!-- MCP_LIST -->` Tag
Add this tag anywhere in an `AGENT.md` to inject the dynamic MCP availability block at that position. Agents that do not include the tag receive no MCP list injection.
Currently used in: `agents/main/AGENT.md`, `agents/tic/AGENT.md`.
Resolution pipeline:
- `agents::resolve_includes()` — replaces `<!-- MCP_LIST -->` with the `__MCP_LIST__` sentinel.
- `ChatSessionHandler::build_openai_messages()` — replaces `__MCP_LIST__` with the rendered block (via `render_mcp_list()`).
### `activate_tools` Tool
A synthetic interface tool (not in the global `ToolRegistry`):
```text
activate_tools(groups: ["server_name", ..., "config"])
```
- Takes an array of **tool-group** names. A group is either an MCP server name **or** the reserved keyword `config` (see [Config tool group](#config-tool-group) below).
- Updates the in-memory `active_mcp_grants` set immediately (the set holds server names and/or `"config"`).
- **Root agents** (`stack_id = None`): persists grants to `session_mcp_grants` — survives across turns and restarts.
- **Sub-agents** (`stack_id = Some(id)`): persists grants to `stack_mcp_grants` — survives restarts, but **deleted when the stack frame terminates** (no session leak).
- Returns a confirmation string listing which groups were activated and their scope (`session` or `stack <id>`).
### Config tool group
The same lazy mechanism gates the built-in **`Config`-category** tools (`set_secret`, `list_secrets`, `register_mcp`, `delete_mcp`, `configure_plugin`, `cron_jobs` delete, `toggle_item`). They are hidden from `base_tool_defs` and loaded on demand via `activate_tools(["config"])`:
- `build_agent_config` splits the registry with `ToolRegistry::openai_definitions_excluding_config()` (the always-on base) and `openai_definitions_config_only()` (carried as `AgentRunConfig.config_tool_defs`).
- `all_tool_defs()` appends `config_tool_defs` only when `active_mcp_grants` contains `"config"`. The `config_tool_defs` go through the same interactive-only and approval-visibility filters as `base_tool_defs`, so activating `config` never bypasses access control.
- The grant string `"config"` is persisted in `session_mcp_grants` / `stack_mcp_grants` exactly like an MCP server name, so it survives restarts (root) or is dropped on frame exit (sub-agent).
- Discoverability is a short static hint in `agents/main/AGENT.md` and `agents/project-coordinator/AGENT.md` (the orchestrators); the `<!-- MCP_LIST -->` block stays MCP-only.
- Not affected: `image_generate` is registered via the image-generator manager (not the `ToolRegistry`), so its `Config` category is inert and it stays always-on.
**Root agents**: injected in `build_agent_config` as an `InterfaceTool`.
**Sub-agents**: injected in `dispatch_sub_agent` — sub-agents always start with zero grants and activate what they need.
### Sub-Agent MCP Isolation
Sub-agents have a fully isolated MCP grant state:
| Aspect | Root agent | Sub-agent |
| --- | --- | --- |
| Initial grants | Loaded from `session_mcp_grants` DB | Empty (starts from zero) |
| `activate_tools` persists to | `session_mcp_grants` | `stack_mcp_grants` |
| Grants survive restart? | Yes | Yes (re-loaded by `dispatch_sub_agent`) |
| Grants cleaned up? | No (session lifetime) | Yes (on frame termination) |
| Session contamination? | N/A | None |
Sub-agents that don't include `<!-- MCP_LIST -->` in their `AGENT.md` receive no MCP list injection in the system prompt. The tool definitions are still included dynamically in `all_tool_defs()` based on grants, so they can call tools without the descriptive list — useful for agents with a narrow, pre-known tool set.
### `tic` Agent
`tic` uses lazy loading like any other root agent — it calls `activate_tools` for the servers it needs based on the pending events it receives. This avoids loading all MCP tool definitions on every tick when there may be nothing to process.
### Token Savings
| Situation | Approximate tokens |
| --- | --- |
| All MCP tools always loaded (old behaviour) | ~10,000–20,000 |
| Lazy mode, no grants yet | ~50–100 (compact list only) |
| Lazy mode, gmail + gcal granted | ~2,000–4,000 |
---
## When to Update This File
- A new transport type is added
- `PROTOCOL_VERSION` is bumped, the `MCP-Protocol-Version` header logic changes, or version-negotiation handling changes
- `tools/list` pagination (cursor loop, `MAX_TOOL_PAGES`) or `McpTool::from_json` changes
- The structured-result pipeline changes (`McpCallResult`/`ToolResult`, `result_type` column/event, `extract_call_result` preference, or the frontend JSON rendering)
- The tool naming convention changes
- `SERVER_START_TIMEOUT_SECS` changes
- `register_mcp` or `delete_mcp` tool parameters change (schema, required fields, description, friendly_name)
- `list_items` (type=mcp) return format changes (McpServerInfo fields)
- A new notification source is implemented
- The elicitation flow changes (capability/protocol version, schema parsing, the resolve route, or the in-flight timeout behaviour)
- Cancellation (`notifications/cancelled`, the `CancelOnDrop`/`HttpCancelOnDrop` guards) or Tasks (`CreateTaskResult` parsing, `McpCallResult::Task`, the block-and-poll `poll_task`, `wants_task` opt-in, `TaskCancelOnDrop`/`tasks/cancel`, `clamp_poll_interval`/`poll_deadline`, `server_capabilities`, the `experimental.tasks` marker) changes
- The SSH MCP server changes (tools, alias schema, sudo methods, or pooling/host-key behaviour in `scripts/ssh_mcp_server.py`)
- Lazy loading logic changes (`build_agent_config`, `dispatch_sub_agent`, `activate_tools`, grant tables)
- `ClientMessage` loses or gains fields relevant to MCP
docs/mcp/index.md
+20
View File
@@ -0,0 +1,20 @@
# MCP (Model Context Protocol)
External tool integration via Model Context Protocol servers.
## Files
- [mcp.md](../mcp.md) — McpManager, transports (stdio/HTTP), protocol version + `MCP-Protocol-Version` header, `tools/list` pagination, structured tool results, media tool results (image/audio/resource → `data/mcp_media/` + `/api/mcp-media/`), cancellation (`notifications/cancelled`) + Tasks (block-and-poll `tasks/get`/`result`/`cancel`), enable/disable, tool registration
- **Specification reference** (in `specs/` subdirectory) — one file per official MCP spec revision, as background for future Skald MCP work. See [specs/index.md](specs/index.md) for the comparative overview:
- [specs/2026-07-28-draft.md](specs/2026-07-28-draft.md) — Draft / RC: stateless redesign, per-request negotiation, extensions framework
- [specs/2025-11-25.md](specs/2025-11-25.md) — Latest Stable: OIDC Discovery, icons, URL-mode elicitation, experimental Tasks
- [specs/2025-06-18.md](specs/2025-06-18.md) — Stable: Streamable HTTP, Elicitation, structured tool output
- [specs/2024-11-05.md](specs/2024-11-05.md) — Legacy: first public release
- **Servers** (in `servers/` subdirectory):
- [servers/gmail.md](servers/gmail.md) — Gmail read+modify+send MCP server (custom Python)
- [servers/gcal.md](servers/gcal.md) — Google Calendar read+write MCP server (custom Python)
- [servers/gmaps.md](servers/gmaps.md) — Google Maps transit/directions MCP server (custom Python)
- [servers/serpapi_flights.md](servers/serpapi_flights.md) — SerpAPI Google Flights search MCP server (custom Python)
- [servers/whatsapp.md](servers/whatsapp.md) — WhatsApp read+send MCP server (custom Node.js)
See [../index.md#mcp-model-context-protocol](../index.md#mcp-model-context-protocol) for navigation.
docs/mcp/servers/gcal.md
+213
View File
@@ -0,0 +1,213 @@
# Google Calendar MCP Server (gcal)
## Overview
A Python MCP server providing **read + write** access to Google Calendar via the Google Calendar API v3.
**Server name:** `gcal`
**Transport:** `stdio` (spawns `python3 scripts/gcal_mcp_server.py`)
**Location:** `scripts/gcal_mcp_server.py`
The server also emits push notifications (`event/new_calendar_event`) to the agent when new events are created on the primary calendar (polled every 5 min).
---
## Tools
All tools are callable as `mcp__gcal__<tool>`; the table lists the bare `<tool>` names.
| Tool | Required params | Optional params | Description |
|------|-----------------|-----------------|-------------|
| `status` | *(none)* | *(none)* | Self-check: verifies credentials load, the token refreshes, and the Calendar API responds. Call first when another gcal tool fails. |
| `list_calendars` | *(none)* | *(none)* | List calendars accessible to the user (surfaces `calendar_id` values). |
| `list_events` | *(none)* | `calendar_id`, `time_min`, `time_max`, `max_results`, `full_text`, `time_zone` | Chronological event listing. `time_min` defaults to NOW. |
| `get_event` | `event_id` | `calendar_id` | Read a single event by ID, including attendees + RSVP status. |
| `create_event` | `summary`, `start`, `end` | `description`, `location`, `attendees`, `recurrence`, `time_zone`, `calendar_id`, `reminders` | Create an event; returns ID + HTML link. |
| `update_event` | `event_id` | `summary`, `start`, `end`, `description`, `location`, `attendees`, `time_zone`, `calendar_id`, `reminders` | Patch fields of an existing event; omitted fields are preserved. |
| `delete_event` | `event_id` | `calendar_id` | Permanently delete an event. Irreversible. |
| `respond_to_event` | `event_id`, `response` | `calendar_id` | Set RSVP (`accepted`, `declined`, `tentative`, `needsAction`). |
### `reminders` format
`reminders` (on `create_event` / `update_event`) accepts a JSON array whose items are **either** integers (minutes before the event, popup reminder) **or** objects `{"method": "popup"|"email", "minutes": <int>}`. Overrides calendar defaults. Examples: `[10, 30, 60]` or `[{"method": "email", "minutes": 60}, {"method": "popup", "minutes": 10}]`.
---
## Authentication
### Credentials File
OAuth 2.0 user credentials are stored in:
- **Default path:** `./secrets/google_creds.json` (relative to project root)
- **Override:** `GOOGLE_CREDS_PATH` env var
Required OAuth scope (granted by the setup script):
```
https://www.googleapis.com/auth/calendar
```
### Token Refresh
The access token expires after ~1 hour. The server **refreshes it automatically** in two places:
1. At startup, if the cached token is expired.
2. **Mid-session**, on the first 401 / `RefreshError` from any tool — `_call()` refreshes once, persists the new token to `secrets/google_creds.json`, and retries the call.
If the refresh token itself has been revoked or expired, `status` and every tool return an actionable `Error:` instructing to re-run `scripts/gcal_oauth_setup.py`.
### Git Safety
`./secrets/` is in `.gitignore` — credentials are never committed.
---
## Setup
### 1. Create OAuth credentials
1. Go to [Google Cloud Console → Credentials](https://console.cloud.google.com/apis/credentials).
2. Create an **OAuth client ID** of type *Desktop app*.
3. Put `client_id` and `client_secret` in `secrets/google_oauth_client.json`:
```json
{ "client_id": "....apps.googleusercontent.com", "client_secret": "GOCSPX-..." }
```
4. Enable the **Google Calendar API** under APIs & Services.
### 2. Run the OAuth flow
```bash
.venv/bin/python scripts/gcal_oauth_setup.py
```
Opens a browser, asks for the Calendar scope, and saves the token to `secrets/google_creds.json`.
### 3. Install dependencies
`google-auth`, `google-auth-oauthlib`, `google-api-python-client` are already listed in `requirements.txt`. Re-running `./run.sh` installs them automatically; or:
```bash
uv pip install -r requirements.txt
```
### 4. Register the server with the agent
```
register_mcp(
name="gcal",
transport="stdio",
command="python3",
args=["scripts/gcal_mcp_server.py"]
)
```
---
## Usage Examples
### Self-check: is Calendar working?
```
mcp__gcal__status()
```
Returns `Status: READY ✅ (ok)` if credentials load, the token refreshes, and the Calendar API responds; otherwise a `Status: … ❌/⚠️ (…)` report with a `What to do:` list. Call this first whenever another gcal tool fails.
### List events in the next 7 days
```
mcp__gcal__list_events(
calendar_id="primary",
time_min="2026-06-22T00:00:00+02:00",
time_max="2026-06-29T00:00:00+02:00",
max_results=50,
time_zone="Europe/Rome"
)
```
`time_min` / `time_max` are also accepted as `start_time` / `end_time` for backward compatibility. If `time_min` is omitted it defaults to NOW.
### Search events
```
mcp__gcal__list_events(
full_text="dentist",
time_min="2026-01-01T00:00:00+01:00",
time_max="2026-12-31T00:00:00+01:00"
)
```
### Get / create / RSVP
```
mcp__gcal__get_event(event_id="<id>")
mcp__gcal__create_event(
summary="Dentist",
start="2026-07-01T09:00:00",
end="2026-07-01T10:00:00",
reminders=[30, {"method": "email", "minutes": 120}]
)
mcp__gcal__respond_to_event(event_id="<id>", response="accepted")
```
---
## Enable / Disable
```
toggle_item(kind="mcp", id="gcal", enabled=false) # disable
toggle_item(kind="mcp", id="gcal", enabled=true) # re-enable
restart # required for changes to take effect
```
---
## Dependencies
| Package | Version | Purpose |
|---------|---------|---------|
| `google-api-python-client` | 2.197.0 | Google API Python client (Calendar v3) |
| `google-auth` | 2.55.0 | Auth / token refresh |
| `google-auth-oauthlib` | 1.4.0 | Local OAuth flow (setup script) |
---
## Error Handling
Every Google API exception is mapped to an actionable `Error:` string (flagged with `isError: true`):
| Condition | Response |
|-----------|----------|
| Credentials file missing | `"Error: Credentials file not found at …. Run scripts/gcal_oauth_setup.py…"` |
| Token refresh failed / revoked | `"Error: Calendar API token refresh failed …. Re-run scripts/gcal_oauth_setup.py…"` |
| HTTP 401 | `"Error: Calendar API rejected the access token (401)…. Re-run scripts/gcal_oauth_setup.py…"` |
| HTTP 403 | `"Error: Calendar API returned 403 Forbidden. The OAuth scopes … are insufficient, or the Calendar API is disabled in the Google Cloud Console…"` |
| HTTP 404 | `"Error: Calendar API returned 404 Not Found. Check the event/calendar ID…"` |
| HTTP 429 | `"Error: Calendar API rate limit exceeded (429). Wait a moment and retry."` |
| HTTP 400 | `"Error: Calendar API rejected the request as invalid (400). Check the parameters. Detail: …"` |
| HTTP 5xx | `"Error: Calendar API returned a server error (HTTP …). Retry in a moment."` |
| Missing required param | `"Error: Missing required parameter '<name>'"` |
| Unknown tool | `"Error: Unknown tool: <name>"` |
All errors are logged to stderr with the `[gcal_mcp]` prefix.
---
## Protocol
Implements JSON-RPC 2.0 over stdio (same as gmail / gmaps / whatsapp servers):
- **Requests:** read from stdin, one JSON object per line
- **Responses:** written to stdout
- **Notifications:** server-initiated (`event/new_calendar_event`), no `id`
- **Logs:** stderr only, prefixed `[gcal_mcp]`
Supported methods: `initialize`, `notifications/initialized`, `tools/list`, `tools/call`.
---
## When to Update This File
- A tool is added, removed, or renamed
- Auth mechanism changes
- Credential path or scope changes
- New error cases are mapped
- Protocol version changes
docs/mcp/servers/gmail.md
+283
View File
@@ -0,0 +1,283 @@
# Gmail MCP Server (gmail)
## Overview
A Python MCP server providing **read, modify, and send** access to Gmail via the Gmail API v1.
**Server name:** `gmail`
**Transport:** `stdio` (spawns `python3 scripts/gmail_mcp_server.py`)
**Location:** `scripts/gmail_mcp_server.py`
The server also emits push notifications (`event/new_email`) to the agent when new mail lands in the INBOX (polled every 60 s via the Gmail History API).
### Permissions (safe by design)
| Capability | Yes/No |
|------------|--------|
| Read messages & threads | ✅ |
| Search messages | ✅ |
| List labels | ✅ |
| Modify labels (mark read, star, archive) | ✅ |
| Send email | ✅ |
| Create labels | ✅ |
| Download attachments | ✅ |
| Trash message (reversible) | ❌ *removed from tools* |
| Untrash message | ❌ *removed from tools* |
| Permanently delete | ❌ *scope not granted* |
The server uses the `gmail.modify` + `gmail.labels` scopes, which allow all operations **except permanent deletion**.
---
## Tools
All tools are callable as `mcp__gmail__<tool>`; the table lists the bare `<tool>` names.
| Tool | Required params | Optional params | Description |
|------|-----------------|-----------------|-------------|
| `status` | *(none)* | *(none)* | Self-check: verifies credentials load, the token refreshes, and the Gmail API responds. Call first when another gmail tool fails. |
| `list_messages` | *(none)* | `query`, `max_results`, `label_ids`, `page_token` | List messages with optional Gmail search query and label filter. Returns subject, sender, date, message/thread IDs. Pass `page_token` (from a previous response) to fetch the next page. |
| `get_message` | `message_id` | `include_body` | Read a single message by ID (body included by default, truncated at 10000 chars). HTML-only emails are converted to readable text; attachment filenames are listed. |
| `get_thread` | `thread_id` | *(none)* | Read all messages in a thread, newest last. |
| `list_labels` | *(none)* | *(none)* | List labels with total + unread counts (resolves label IDs). |
| `modify_message` | `message_id` | `add_labels`, `remove_labels` | Add/remove labels (mark read, archive, star). Each label arg accepts a string or array. |
| `send_message` | `to`, `subject`, `body` | `cc`, `bcc`, `in_reply_to`, `thread_id`, `attachments` | Send an email; supports in-thread replies and file attachments. |
| `get_profile` | *(none)* | *(none)* | Account email, total message/thread count, history ID. |
| `create_label` | `name` | `label_list_visibility`, `message_list_visibility` | Create a label; returns the new label ID. |
| `download_attachments` | `message_id` | `folder` | Save all attachments from a message. Defaults to `data/gmail_attachments/`. |
> **Removed:** `search_messages` (a literal alias of `list_messages` — use `list_messages` with the `query` parameter instead, which supports full Gmail search syntax).
### Gmail Search Syntax
The `query` parameter on `list_messages` supports Gmail's native syntax:
| Example | Meaning |
|---------|---------|
| `from:john@example.com` | Messages from a sender |
| `to:maria@example.com` | Messages to a recipient |
| `subject:meeting` | Messages with "meeting" in subject |
| `is:unread` / `is:starred` | Unread / starred messages |
| `in:inbox` / `in:sent` | Messages in Inbox / Sent |
| `after:2024/01/01` / `before:2024/06/01` | Date range |
| `has:attachment` | Messages with attachments |
| `label:MY_LABEL` | Messages with a custom label |
Combine with spaces: `from:john is:unread after:2024/06/01`.
---
## Authentication
### Credentials File
OAuth 2.0 user credentials are stored in:
- **Default path:** `./secrets/gmail_creds.json` (relative to project root)
- **Override:** `GMAIL_CREDS_PATH` env var
Required OAuth scopes (granted by the setup script):
```
https://www.googleapis.com/auth/gmail.modify
https://www.googleapis.com/auth/gmail.labels
```
### Token Refresh
The access token expires after ~1 hour. The server **refreshes it automatically** in two places:
1. At startup, if the cached token is expired.
2. **Mid-session**, on the first 401 / `RefreshError` from any tool — `_call()` refreshes once, persists the new token to `secrets/gmail_creds.json`, and retries the call.
If the refresh token itself has been revoked or expired, `status` and every tool return an actionable `Error:` instructing to re-run `scripts/gmail_oauth_setup.py`.
### Git Safety
`./secrets/` is in `.gitignore` — credentials are never committed.
---
## Setup
### 1. Create OAuth credentials
1. Go to [Google Cloud Console → Credentials](https://console.cloud.google.com/apis/credentials).
2. Create an **OAuth client ID** of type *Desktop app*.
3. Put `client_id` and `client_secret` in `secrets/google_oauth_client.json`:
```json
{ "client_id": "....apps.googleusercontent.com", "client_secret": "GOCSPX-..." }
```
4. Enable the **Gmail API** under APIs & Services.
(The OAuth client file is shared with gcal; a single Desktop client works for both.)
### 2. Run the OAuth flow
```bash
.venv/bin/python scripts/gmail_oauth_setup.py
```
Opens a browser, asks for the Gmail scopes, and saves the token to `secrets/gmail_creds.json`.
### 3. Install dependencies
`google-auth`, `google-auth-oauthlib`, `google-api-python-client` are already listed in `requirements.txt`. Re-running `./run.sh` installs them automatically; or:
```bash
uv pip install -r requirements.txt
```
### 4. Register the server with the agent
```
register_mcp(
name="gmail",
transport="stdio",
command="python3",
args=["scripts/gmail_mcp_server.py"]
)
```
---
## Usage Examples
### Self-check: is Gmail working?
```
mcp__gmail__status()
```
Returns `Status: READY ✅ (ok)` with the account email if credentials load, the token refreshes, and the Gmail API responds; otherwise a `Status: … ❌/⚠️ (…)` report with a `What to do:` list. Call this first whenever another gmail tool fails.
### List unread messages
```
mcp__gmail__list_messages(query="is:unread", max_results=10)
```
When more results exist, the response ends with a `More results available. Use page_token='...'` line. Pass that token back to page forward:
```
mcp__gmail__list_messages(query="is:unread", max_results=10, page_token="09876...")
```
### Read a message
```
mcp__gmail__get_message(message_id="190abc123...", include_body=true)
```
The body is the `text/plain` part when present; for HTML-only emails it is converted to readable text and labelled `--- Body (converted from HTML) ---`. If the message has attachments, their filenames are listed on an `Attachments:` line — download them with `download_attachments` (which only needs the `message_id`).
### Mark as read / archive
```
mcp__gmail__modify_message(message_id="190abc123...", remove_labels=["UNREAD"])
mcp__gmail__modify_message(message_id="190abc123...", remove_labels="INBOX") # archive
```
### Send / reply in-thread
```
mcp__gmail__send_message(
to="friend@example.com",
subject="Hello!",
body="How are you?"
)
mcp__gmail__send_message(
to="sender@example.com",
subject="Re: Original subject",
body="This is my reply.",
in_reply_to="190abc123...",
thread_id="190thread456..."
)
```
`in_reply_to` adds RFC 2822 threading headers; `thread_id` attaches the message to the correct Gmail thread. For a proper reply visible in all clients, pass both.
### Send with attachments
```
mcp__gmail__send_message(
to="friend@example.com",
subject="Documents attached",
body="See the two files attached.",
attachments=[
"data/gmail_attachments/PREVENTIVO DI SPESA.pdf",
"uploads/gmail_attachments/00035.pdf"
]
)
```
`attachments` is an array of local file paths. Each path is either absolute or relative to the **project root**; the server reads every file from disk, guesses its MIME type, and adds it to a `multipart/mixed` message. **If any path does not exist the email is NOT sent** — the tool returns `Error: attachment not found: <path>` so the agent can correct it. Total attachment size is limited to ~25 MB (the Gmail `messages.send` request embeds the message as base64).
### Download attachments
```
mcp__gmail__download_attachments(message_id="190abc123...")
```
Saves into `data/gmail_attachments/` (served via `/data/gmail_attachments/...` in the frontend). Override with `folder`.
---
## Enable / Disable
```
toggle_item(kind="mcp", id="gmail", enabled=false) # disable
toggle_item(kind="mcp", id="gmail", enabled=true) # re-enable
restart # required for changes to take effect
```
---
## Dependencies
| Package | Version | Purpose |
|---------|---------|---------|
| `google-api-python-client` | 2.197.0 | Google API Python client (Gmail v1) |
| `google-auth` | 2.55.0 | Auth / token refresh |
| `google-auth-oauthlib` | 1.4.0 | Local OAuth flow (setup script) |
---
## Error Handling
Every Google API exception is mapped to an actionable `Error:` string (flagged with `isError: true`):
| Condition | Response |
|-----------|----------|
| Credentials file missing | `"Error: Credentials file not found at …. Run scripts/gmail_oauth_setup.py…"` |
| Credentials invalid & no refresh token | `"Error: Credentials invalid and cannot be refreshed. Re-run scripts/gmail_oauth_setup.py."` |
| Token refresh failed / revoked | `"Error: Gmail API token refresh failed …. Re-run scripts/gmail_oauth_setup.py…"` |
| HTTP 401 | `"Error: Gmail API rejected the access token (401)…. Re-run scripts/gmail_oauth_setup.py…"` |
| HTTP 403 | `"Error: Gmail API returned 403 Forbidden. The OAuth scopes … are insufficient, or the Gmail API is disabled in the Google Cloud Console…"` |
| HTTP 404 | `"Error: Gmail API returned 404 Not Found. Check the message/thread/attachment ID."` |
| HTTP 429 | `"Error: Gmail API rate limit exceeded (429). Wait a moment and retry."` |
| HTTP 400 | `"Error: Gmail API rejected the request as invalid (400). Check the parameters. Detail: …"` |
| HTTP 5xx | `"Error: Gmail API returned a server error (HTTP …). Retry in a moment."` |
| Missing required param | `"Error: Missing required parameter '<name>'"` |
| Unknown tool | `"Error: Unknown tool: <name>"` |
All errors are logged to stderr with the `[gmail_mcp]` prefix.
---
## Protocol
Implements JSON-RPC 2.0 over stdio (same as gcal / gmaps / whatsapp servers):
- **Requests:** read from stdin, one JSON object per line
- **Responses:** written to stdout
- **Notifications:** server-initiated (`event/new_email`), no `id`
- **Logs:** stderr only, prefixed `[gmail_mcp]`
Supported methods: `initialize`, `notifications/initialized`, `tools/list`, `tools/call`.
---
## When to Update This File
- A tool is added, removed, or renamed
- Auth mechanism or scopes change
- Credential path changes
- New error cases are mapped
- Protocol version changes
docs/mcp/servers/gmaps.md
+224
View File
@@ -0,0 +1,224 @@
# Google Maps MCP Server (gmaps)
## Overview
A Python MCP server providing **public-transit & mapping** capabilities via the Google Maps Platform APIs.
**Server name:** `gmaps`
**Transport:** `stdio` (spawns `python3 scripts/gmaps_mcp_server.py`)
**Location:** `scripts/gmaps_mcp_server.py`
---
## Tools
All tools are callable as `mcp__gmaps__<tool>`; the table lists the bare `<tool>` names.
| Tool | Required params | Optional params | Description |
|------|-----------------|-----------------|-------------|
| `status` | *(none)* | *(none)* | Self-check: verifies the API key is valid and the Geocoding API responds. Call first when another Maps tool fails. |
| `directions` | `origin`, `destination` | `mode`, `departure_time`, `transit_mode`, `transit_routing_preference`, `alternatives`, `language` | Step-by-step directions (transit, driving, walking, bicycling) |
| `geocode` | `address` | `language`, `region` | Address / place name → coordinates + place_id |
| `reverse_geocode` | `lat`, `lng` | `language` | Coordinates → formatted address |
| `search_places` | *(at least one of `query`/`location`)* | `radius`, `type`, `language` | Find nearby stations, stops, POIs |
| `distance_matrix` | `origins`, `destinations` | `mode`, `language` | Travel time & distance between multiple points |
---
## Authentication
### API Key
Unlike Gmail/Calendar (OAuth), Google Maps uses a **plain API key**.
**Priority order:**
1. Environment variable `GOOGLE_MAPS_API_KEY`
2. File `secrets/gmaps_api_key.txt` (first non-empty line)
The `secrets/` directory is in `.gitignore` — the key will not be committed.
### Required Google Cloud APIs
Enable all four in the [Google Cloud Console](https://console.cloud.google.com/apis/library):
| API | Used by |
|-----|---------|
| **Directions API** | `directions` |
| **Geocoding API** | `geocode`, `reverse_geocode` |
| **Places API** | `search_places` |
| **Distance Matrix API** | `distance_matrix` |
---
## Setup
### 1. Create API Key
1. Go to [Google Cloud Console → Credentials](https://console.cloud.google.com/apis/credentials)
2. Click **Create credentials → API key**
3. (Recommended) Restrict the key to the four APIs above
### 2. Save the key
```bash
echo "YOUR_API_KEY_HERE" > secrets/gmaps_api_key.txt
```
Or set the environment variable in your shell/`run.sh`:
```bash
export GOOGLE_MAPS_API_KEY=AIza...
```
### 3. Install the Python dependency
```bash
.venv/bin/pip install googlemaps
# or, if you re-run run.sh, it installs requirements.txt automatically
```
`googlemaps` is already listed in `requirements.txt`.
### 4. Register the server with the agent
Ask the agent:
```
register_mcp(
name="gmaps",
transport="stdio",
command="python3",
args=["scripts/gmaps_mcp_server.py"]
)
```
---
## Usage Examples
### Self-check: is Maps working?
```
mcp__gmaps__status()
```
Returns `OK: …` if the API key is present, valid, and the Geocoding API responds; otherwise an `Error:` string explaining what to fix. Call this first whenever another Maps tool fails.
### Transit directions home (now)
```
mcp__gmaps__directions(
origin="Piazza del Duomo, Milano",
destination="casa mia", ← or the real address saved in agent memory
mode="transit"
)
```
### Prefer train, fewer transfers
```
mcp__gmaps__directions(
origin="current location",
destination="Via Roma 1, Torino",
mode="transit",
transit_mode="train",
transit_routing_preference="fewer_transfers"
)
```
### Find the nearest metro station
```
mcp__gmaps__search_places(
query="metro",
location="Piazza Garibaldi, Napoli",
radius=500,
type="subway_station"
)
```
### How long does it take from A to B?
```
mcp__gmaps__distance_matrix(
origins="Stazione Centrale, Milano",
destinations="Aeroporto Malpensa",
mode="transit"
)
```
---
## Enable / Disable
```
toggle_item(kind="mcp", id="gmaps", enabled=false) # disable
toggle_item(kind="mcp", id="gmaps", enabled=true) # re-enable
restart # required for changes to take effect
```
---
## Dependencies
| Package | Version | Purpose |
|---------|---------|---------|
| `googlemaps` | latest | Google Maps Platform Python client |
---
## Parameter notes
### Coordinates format
Whenever a parameter accepts coordinates, pass them as a **`"latitude,longitude"` decimal string with no spaces**, e.g. `"45.4654,9.1866"`. Never pass an array or separate fields.
### `departure_time`
Must be the **literal string `"now"`** or an **ISO 8601 datetime string with timezone offset**, e.g. `"2025-06-15T08:30:00+02:00"`. Never pass a Unix timestamp integer — the tool now rejects non-string values with an explicit error.
### `transit_mode`
Restricts results to a specific vehicle: `"train"` = intercity/regional rail, `"subway"` = metro, `"tram"` = trams, `"bus"` = buses, `"rail"` = any rail. Omit to allow any vehicle.
---
## Error Handling
| Error | Response |
|-------|----------|
| API key not found | `"Error: Google Maps API key not found. Set GOOGLE_MAPS_API_KEY…"` |
| `googlemaps` not installed | `"Error: Missing dependency: No module named 'googlemaps'. Run: pip install googlemaps"` |
| `OVER_QUERY_LIMIT` | `"Error: <API> API quota exceeded (OVER_QUERY_LIMIT). Check usage and billing in the Google Cloud Console."` |
| `REQUEST_DENIED` | `"Error: <API> API request denied (REQUEST_DENIED). Verify that the API key … is valid and that the <API> API is enabled …"` |
| `INVALID_REQUEST` | `"Error: <API> API rejected the request as invalid (INVALID_REQUEST). Check that the addresses, coordinates, and parameters are well-formed."` |
| `MAX_ELEMENTS_EXCEEDED` | `"Error: <API> API returned MAX_ELEMENTS_EXCEEDED — too many origins×destinations at once. …"` |
| `NOT_FOUND` | `"Error: <API> API could not geocode one of the supplied places. …"` |
| Timeout | `"Error: <API> API request timed out. Retry in a moment."` |
| No route found | `"No routes found from '…' to '…'."` |
| Missing required param | `"Error: Missing required parameter '…'"` |
| Invalid `departure_time` | `"Error: 'departure_time' must be 'now' or an ISO 8601 string …"` |
All error responses are flagged with `isError: true`. The error label `<API>` reflects which Google Maps Platform API was being called (Directions, Geocoding, Places, Distance Matrix).
All errors are logged to stderr with `[gmaps_mcp]` prefix.
---
## Protocol
Implements JSON-RPC 2.0 over stdio (same as gcal and gmail servers):
- **Requests:** read from stdin, one JSON object per line
- **Responses:** written to stdout
- **Logs:** stderr only, prefixed `[gmaps_mcp]`
Supported methods: `initialize`, `notifications/initialized`, `tools/list`, `tools/call`.
---
## When to Update This File
- New tools added
- Auth mechanism changes (e.g. OAuth migration)
- New transport option added
- Error cases change
docs/mcp/servers/serpapi_flights.md
+177
View File
@@ -0,0 +1,177 @@
# SerpAPI Google Flights MCP Server (serpapi_flights)
## Overview
A Python MCP server providing **flight search** capabilities via the SerpAPI Google Flights engine.
**Server name:** `serpapi_flights`
**Transport:** `stdio` (spawns `python3 scripts/mcp/serpapi_flights/server.py`)
**Location:** `scripts/mcp/serpapi_flights/server.py`
---
## Tools
| Tool | Required params | Optional params | Description |
|------|-----------------|-----------------|-------------|
| `serpapi_search_flights` | `departure_id`, `arrival_id`, `outbound_date` | `return_date`, `adults`, `children`, `infants_in_seat`, `infants_on_lap`, `stops`, `currency`, `preferred_cabins`, `hl`, `max_results` | One-way or round-trip flight search on Google Flights |
The previous `serpapi_lookup_airport` tool (hardcoded dictionary of ~100 airports) was removed: the LLM already knows the common IATA codes, and SerpAPI itself accepts both airport codes (`JFK`, `FCO`) and city codes covering all airports of a city (`NYC`, `ROM`, `MIL`, `LON`).
---
## Authentication
### API key
SerpAPI uses a single API key (no OAuth).
**Priority order:**
1. Environment variable `SERPAPI_API_KEY`
2. File `secrets/serpapi_api_key.txt` (first non-empty line)
The `secrets/` directory is in `.gitignore` — the key will not be committed.
### Get a key
1. Sign up at [serpapi.com](https://serpapi.com).
2. Copy your API key from the dashboard.
### Save the key
```bash
echo "YOUR_SERPAPI_KEY_HERE" > secrets/serpapi_api_key.txt
```
Or set the environment variable:
```bash
export SERPAPI_API_KEY=your_key_here
```
---
## Setup
### 1. Install the Python dependency
`httpx` is listed in the project root `requirements.txt` and is installed automatically by `run.sh` into `.venv/`. To install manually:
```bash
uv pip install httpx
# or: .venv/bin/pip install httpx
```
The local `scripts/mcp/serpapi_flights/requirements.txt` is kept for standalone use and contains only `httpx>=0.27.0`.
### 2. Register the server with the agent
Ask the agent:
```
register_mcp(
name="serpapi_flights",
transport="stdio",
command="python3",
args=["scripts/mcp/serpapi_flights/server.py"]
)
```
---
## Usage Examples
### One-way, cheapest Milan → Rome next week
```
mcp__serpapi_flights__serpapi_search_flights(
departure_id="MIL",
arrival_id="ROM",
outbound_date="2026-07-01"
)
```
### Round-trip London → New York, 2 adults, business class, non-stop only
```
mcp__serpapi_flights__serpapi_search_flights(
departure_id="LON",
arrival_id="NYC",
outbound_date="2026-08-01",
return_date="2026-08-15",
adults=2,
preferred_cabins="business",
stops=0,
currency="USD"
)
```
---
## Parameter notes
### Airport vs city codes
`departure_id` and `arrival_id` must be **exactly 3 ASCII letters**, uppercased automatically. Both forms are accepted:
- **Airport code** — a specific airport: `JFK`, `FCO` (Rome Fiumicino), `LGW` (London Gatwick).
- **City code** — covers all airports of a city: `NYC`, `ROM`, `MIL`, `LON`. **Prefer city codes** when the user does not name a specific airport.
### `stops`
Integer enum: `0` = non-stop only, `1` = max one stop, `2` = max two stops. Omit to allow any.
### `type` (handled automatically)
The server sends SerpAPI `type=2` for one-way (no `return_date`) and `type=1` for round-trip. The caller never sets `type` directly.
### `currency`
ISO 4217 code (3 uppercase letters). Default `EUR`.
### `hl`
Language code for SerpAPI result text (e.g. `"en"`, `"it"`). Default `"en"`.
---
## Error Handling
| Error | Response |
|-------|----------|
| API key missing | `"Error: SerpAPI API key not found. Set SERPAPI_API_KEY env var or create secrets/serpapi_api_key.txt …"` |
| HTTP 401 | `"Error: Invalid SerpAPI API key. Check secrets/serpapi_api_key.txt or the SERPAPI_API_KEY env var."` |
| HTTP 429 | `"Error: SerpAPI rate limit exceeded. Wait a moment and retry."` |
| HTTP 400 | `"Error: Bad request — <body>. Verify airport codes (3-letter IATA) and dates."` |
| Timeout (30s) | `"Error: Request to SerpAPI timed out (30s). …"` |
| Missing/invalid param | `"Error: 'outbound_date' must be in YYYY-MM-DD format (got '…')."` |
| Unknown tool | `"Error: Unknown tool: <name>"` |
| SerpAPI error in body | `"Error: SerpAPI returned an error: <message>"` |
Every error is returned as a tool result with `isError: true` (detected by the `Error:` prefix).
All errors are logged to stderr with `[serpapi_flights_mcp]` prefix.
---
## Protocol
Implements JSON-RPC 2.0 over stdio (same shape as the `gmaps`, `gcal`, `gmail`, and `whatsapp` servers):
- **Requests:** read from stdin, one JSON object per line
- **Responses:** written to stdout
- **Logs:** stderr only, prefixed `[serpapi_flights_mcp]`
Supported methods: `initialize`, `notifications/initialized`, `tools/list`, `tools/call`.
The server is fully synchronous: stdin is read line-by-line, each `tools/call` performs a blocking `httpx.Client.get` to SerpAPI and returns the formatted result. No background threads, no FastMCP framework.
---
## When to Update This File
- New tools added
- Auth mechanism changes
- SerpAPI parameters added or removed
- Error cases change
docs/mcp/servers/whatsapp.md
+349
View File
@@ -0,0 +1,349 @@
# WhatsApp MCP Server (whatsapp)
## Overview
A Node.js MCP server that exposes WhatsApp as a set of tools for the LLM, using **whatsapp-web.js** + Puppeteer (headless Chromium).
**Server name:** `whatsapp`
**Transport:** `stdio` (spawns `node scripts/whatsapp_mcp/index.js`)
**Location:** `scripts/whatsapp_mcp/index.js`
### Capabilities
| Capability | Enabled |
|------------|---------|
| List chats and groups | ✅ |
| Read messages from a chat | ✅ |
| Search messages by keyword | ✅ |
| Search contacts by name | ✅ |
| Send messages | ✅ |
| Send media (image/video/audio/document, from file or URL) | ✅ |
| Download received media (photos, videos, documents) | ✅ |
| Logout / reset session without restart | ✅ |
| Address a contact by phone number (no lookup needed) | ✅ |
| Edit/delete messages | ❌ not implemented |
---
## ⚠️ Important notes on WhatsApp
- **whatsapp-web.js is unofficial**: it drives WhatsApp Web through Chromium. WhatsApp may ban the number in case of heavy or anomalous use.
- **Safe use**: reading your own groups and sending individual messages is in the tolerated grey area. Do not use it for spam or bulk automation.
- **Recommended number**: using a secondary number or a parallel WhatsApp Business account reduces the risk.
---
## Tools
All tools are callable as `mcp__whatsapp__<tool>`; the table lists the bare `<tool>` names.
| Tool | Parameters | Description |
|------|------------|-------------|
| `status` | *(none)* | Connection status as a plain-language report: the state, what it means, and step-by-step fix instructions when not operational. Cross-checks the live socket (`getState()`) to catch silently dropped sessions |
| `get_qr` | *(none)* | QR code to scan with the phone — path/URL to a PNG or HTML page, with ASCII fallback (only when status = QR_READY) |
| `logout` | *(none)* | Ends the session, **clears the cached credentials on disk** and re-initializes the client → new QR without restart. Use it when the session has expired/got stuck or to link a different phone |
| `list_chats` | `max_chats` (int, default 20, max 50) | List recent chats with name, ID and unread count |
| `get_messages` | `chat_id` **or** `number`, `limit` (int, default 20, max 100), `offset` (int, default 0) | Messages from a chat/group with pagination support. Media messages are tagged with their type and a `download id` |
| `send_message` | `chat_id` **or** `number`, `message` (required) | Send a text message |
| `send_media` | `chat_id` **or** `number`, `source` (required: file path or http(s) URL), `caption`, `as_document` (bool) | Send an image/video/audio/document |
| `download_media` | `message_id` (required) | Download the media attached to a message; saves it under `data/whatsapp_media/` and returns the path + `/data/` URL |
| `search_messages` | `query` (required), `max_results` (int, default 20, max 50) | Search by keyword across all chats |
| `search_contacts` | `query` (required), `max_results` (int, default 20, max 50) | Search saved contacts by name (partial, case-insensitive). Use it to find the ID of a contact not present in recent chats |
### chat_id format
- **Contact:** `39xxxxxxxxxx@c.us` (international prefix without `+`, followed by `@c.us`)
- **Group:** `xxxxxxxxxx-xxxxxxxxxx@g.us`
Correct chat_ids are obtained via `list_chats` (recent chats) or `search_contacts` (saved contacts not in recent chats).
**Shortcut for individual contacts:** `get_messages`, `send_message` and `send_media` also accept a plain `number` (phone number with country code, e.g. `393331234567` or `+39 333 123 4567`) instead of a `chat_id`. The server resolves it via `getNumberId` (which also verifies the number is on WhatsApp), so there is no need to look up the chat_id first. Groups must still be addressed by `chat_id`.
---
## Authentication
### First time (QR scan)
On the first launch there is no saved session. The client generates a QR code:
1. The LLM calls `status` → response `QR_READY`
2. The LLM calls `get_qr` → returns the QR
3. The user scans the QR with WhatsApp → **Settings → Linked Devices → Link a Device**
4. The state moves to `AUTHENTICATED`, then `READY`
The QR is also saved to a file (PNG at `data/whatsapp_qr.png`, HTML at `secrets/whatsapp_qr.html`, ASCII fallback at `secrets/whatsapp_qr.txt`).
### Subsequent sessions
The session is persisted in `secrets/whatsapp_session/` (managed by whatsapp-web.js's `LocalAuth`). On server restart the session is restored automatically, with no need to scan the QR again.
### Browser lifecycle & restart cleanup (self-healing)
The Puppeteer profile lives at `secrets/whatsapp_session/session` and Chromium takes an **exclusive lock** on it (`SingletonLock`): two browsers cannot open the same profile at once.
The host (skald) kills MCP servers with **SIGKILL** on shutdown/restart (`kill_on_drop` in the Rust MCP client — see `crates/mcp-client/src/server.rs`). SIGKILL is untrappable, so the node process dies instantly and the headless Chromium it spawned is **orphaned** (reparented to init), keeping the profile locked. The next launch would then fail with *"The browser is already running for &lt;profile&gt;. Use a different `userDataDir` or stop the running browser first."* — and every WhatsApp tool stays dead until the orphan is killed by hand.
To make this self-healing, on **every startup** (in `initClient()` → `cleanupStaleBrowser()`) the server:
1. Runs `pgrep -f <profile-dir>` and `SIGKILL`s every process still bound to the profile. Because skald runs a single WhatsApp MCP at a time and SIGKILLs the old node before spawning the new one, any such process is guaranteed to be a leftover orphan.
2. Removes the stale `SingletonLock` / `SingletonCookie` / `SingletonSocket` files so a fresh launch is unblocked.
For the shutdown paths that *are* trappable (clean stdin EOF, `SIGTERM`/`SIGINT` from a container/OS stop), the server closes the browser cleanly (`client.destroy()`, awaited with a 5 s cap) so it releases the lock on the way out. The startup cleanup is the backstop that covers the SIGKILL path. The saved login (under `session/Default/`) is untouched, so no QR re-scan is needed after a restart.
### Logout / expired session (without restart)
When the session expires or gets stuck (state `DISCONNECTED`), on restart `LocalAuth` would reload the invalid session from `secrets/whatsapp_session/`, immediately returning to the disconnected state. Previously the only fix was to delete that folder by hand and restart.
Now `logout` is enough:
1. Attempts a clean logout (`client.logout()`), tolerating failure if the browser page is already dead;
2. As a fallback, closes the browser (`destroy()`) to release the locks on the profile;
3. **Force-deletes `secrets/whatsapp_session/`** (the cached token);
4. Removes any stale QR files;
5. Re-initializes the client → generates a new QR within a few seconds.
```
mcp__whatsapp__logout()
# → wait a few seconds
mcp__whatsapp__get_qr()
# → scan the new QR
```
No server restart required.
### Token storage
| File/Directory | Contents |
|---|---|
| `secrets/whatsapp_session/` | Persistent WhatsApp session (LocalAuth) — deleted by `logout` |
| `secrets/whatsapp_qr.html` / `data/whatsapp_qr.png` | Temporary QR code (removed after authentication or a logout) |
| `secrets/whatsapp_qr.txt` | ASCII fallback of the QR (when `qrcode` is unavailable) |
| `data/whatsapp_media/` | Media downloaded via `download_media`, served at `/data/whatsapp_media/` |
Everything under `secrets/` is in `.gitignore` via the `secrets/` rule.
---
## Setup (one-time)
### 1. Install the Node.js dependencies
```bash
cd scripts/whatsapp_mcp
npm install
```
This installs `whatsapp-web.js`, `puppeteer` (includes Chromium, ~300MB), `qrcode` and `qrcode-terminal`.
### 2. Register the server (have the agent do it)
```
register_mcp(
name="whatsapp",
transport="stdio",
command="node",
args=["scripts/whatsapp_mcp/index.js"]
)
```
### 3. First authentication
```
mcp__whatsapp__status()
# → QR_READY
mcp__whatsapp__get_qr()
# → shows the QR, scan it with the phone
```
---
## Usage examples
### See recent chats
```
mcp__whatsapp__list_chats(max_chats=10)
```
### Read the latest messages from a group
```
mcp__whatsapp__get_messages(
chat_id="1234567890-9876543210@g.us",
limit=50
)
```
### Page through history (older messages)
`offset` skips the most recent messages, exposing the preceding window:
```
# Last 20 messages
get_messages(chat_id="...", limit=20, offset=0)
# Messages 21–40 (previous)
get_messages(chat_id="...", limit=20, offset=20)
# Messages 41–60 (even older)
get_messages(chat_id="...", limit=20, offset=40)
```
Limit: `limit + offset` cannot exceed 200 in a single call (a `fetchMessages` constraint).
### Find the contact of someone not in recent chats
```
mcp__whatsapp__search_contacts(query="Luca")
# → Luca Rossi [contact] | ID: 393331234567@c.us
```
### Search for what was said about a topic
```
mcp__whatsapp__search_messages(query="Monday meeting")
```
### Send a message
```
# By chat_id (groups, or chats already open)
mcp__whatsapp__send_message(
chat_id="393331234567@c.us",
message="Hi! Are you there?"
)
# Or directly by number — no list_chats/search_contacts needed first
mcp__whatsapp__send_message(
number="+39 333 123 4567",
message="Hi! Are you there?"
)
```
### Send media (image, video, document)
```
# From a local file (path relative to the project root, or absolute)
mcp__whatsapp__send_media(
number="393331234567",
source="data/report.pdf",
caption="Here is the report",
as_document=true
)
# From a URL
mcp__whatsapp__send_media(
chat_id="1234567890-9876543210@g.us",
source="https://example.com/photo.jpg",
caption="Look at this"
)
```
### Download received media
`get_messages` tags media messages with a `download id`:
```
mcp__whatsapp__get_messages(number="393331234567", limit=10)
# → [2026-06-22 10:01:00] Luca [image, download id="true_39...@c.us_3EB0..."]: invoice photo
mcp__whatsapp__download_media(message_id="true_39...@c.us_3EB0...")
# → saved to data/whatsapp_media/... (also served at /data/whatsapp_media/...)
```
---
## Connection states
`status` returns a self-describing report — it states the lifecycle state, explains it in plain language, and lists concrete next steps whenever it is not `READY`. The agent should not need this table; it is here for reference.
| State | Meaning | What to do |
|-------|---------|------------|
| `INITIALIZING` | Browser starting up, session loading | Wait a few seconds |
| `QR_READY` | QR scan needed | Call `get_qr` and scan |
| `AUTHENTICATED` | QR scanned, session being established | Wait (→ READY automatically) |
| `READY` | Operational | All tools available |
| `DISCONNECTED` | Connection/session lost | Call `logout` to reset and log in again (no restart) |
### Live socket cross-check
The lifecycle `state` above is driven by whatsapp-web.js **events**, so it can lag behind a session that drops silently. When `state` is `READY`, `status` also queries the live socket (`client.getState()`, a `WAState`) and reports a mismatch with tailored instructions:
| Live `WAState` while READY | Reported as | Fix suggested |
| --- | --- | --- |
| `CONNECTED` | `READY ✅ (ok)` | — |
| `UNPAIRED` / `UNPAIRED_IDLE` | `action needed` | Device unlinked from phone → `logout` + re-scan |
| `CONFLICT` | `action needed` | WhatsApp Web open elsewhere → close it, or `logout` + re-scan |
| `TIMEOUT` | `transient` | May auto-reconnect → wait and re-check; if stuck, `logout` |
| `DEPRECATED_VERSION` | `needs maintenance` | Update the `whatsapp-web.js` dependency (developer task) |
| `getState()` fails / other | `uncertain` | Browser may have crashed → wait and re-check; if stuck, `logout` |
---
## Enable / Disable
### Disable (when not needed)
```
toggle_item(kind="mcp", id="whatsapp", enabled=false)
restart
```
### Re-enable
```
toggle_item(kind="mcp", id="whatsapp", enabled=true)
restart
```
---
## Dependencies
| Package | Version | Purpose |
|---------|---------|---------|
| `whatsapp-web.js` | ^1.34.7 | WhatsApp Web client |
| `puppeteer` | ^25.1.0 | Headless Chromium (bundled) |
| `qrcode` | ^1.5.4 | Generates the QR as PNG / data-URL (HTML) |
| `qrcode-terminal` | ^0.12.0 | ASCII QR fallback |
**System requirements:**
- Node.js ≥ 18
- ~500MB of space for Puppeteer/Chromium
- A background Chromium process while the server is running
---
## Common errors
| Error | Cause | Fix |
|-------|-------|-----|
| `whatsapp-web.js not found` | `npm install` not run | `cd scripts/whatsapp_mcp && npm install` |
| `WhatsApp not ready (status: INITIALIZING)` | Server just started | Wait 15-30 seconds |
| `WhatsApp not ready (status: QR_READY)` | Session expired/missing | Call `get_qr` and scan |
| `WhatsApp not ready (status: DISCONNECTED)` | Connection/session lost | Call `logout`, wait, then `get_qr` and scan |
| Stays `DISCONNECTED` even after restart | Expired cached token in `secrets/whatsapp_session/` | Call `logout` (clears the cache and regenerates the QR) |
| `The browser is already running for …/session` | An orphaned Chromium from a hard-killed (SIGKILL) prior run still holds the profile lock | **Auto-healed on next startup** by `cleanupStaleBrowser()` (kills the orphan + clears the lock). See *Browser lifecycle & restart cleanup* |
| Chat ID not found | Wrong ID | Use `list_chats` to get the correct IDs |
---
## Protocol
Implements JSON-RPC 2.0 over stdio (same pattern as gmail and gcal):
- **Requests:** JSON on stdin (one per line)
- **Responses:** JSON on stdout
- **Logs:** stderr with the `[whatsapp_mcp]` prefix
Supported methods: `initialize`, `notifications/initialized`, `tools/list`, `tools/call`
---
## When to update this file
- New tools added to the server
- Changed session/QR paths under `secrets/`
- New connection states
- Changed dependency versions
docs/mcp/specs/2024-11-05.md
+94
View File
@@ -0,0 +1,94 @@
# MCP Specification — 2024-11-05 (Legacy)
> **Status:** Legacy (first public release)
> **Released:** 2024-11-05 · repo tags `2024-11-05` and `2024-11-05-final`
> **Spec site:** https://modelcontextprotocol.io/specification/2024-11-05
> **Authoritative schema:** [schema/2024-11-05/schema.ts](https://github.com/modelcontextprotocol/specification/blob/main/schema/2024-11-05/schema.ts)
This is the **first public release** of the Model Context Protocol specification (November 2024). It establishes the JSON-RPC 2.0 based, stateful, capability-negotiated protocol that all later revisions build upon: the Host/Client/Server architecture, the four server primitives (Resources, Prompts, Tools, Logging) and the two client features (Sampling, Roots), over stdio and the original HTTP+SSE transport. It is retained here as the historical baseline; later revisions (2025-03-26, 2025-06-18) added formal authorization, Streamable HTTP, structured tool output, and Elicitation — none of which exist in this version. The TypeScript schema (`schema.ts`) is the source of truth referenced by BCP 14 keywords (MUST / SHOULD / MAY).
## At a glance
- **Protocol style:** stateful connections
- **Capability negotiation:** connection-level (during `initialize`)
- **Transports:** stdio; HTTP+SSE (the original two-endpoint SSE transport)
- **Server features:** Resources, Prompts, Tools, Logging
- **Client features:** Sampling, Roots
- **Authorization:** **not part of the core spec** — clients and servers MAY negotiate custom auth; HTTP transports carry transport-level security guidance only
## Architecture
MCP follows a **client-host-server** architecture. A Host process creates and manages multiple Client instances; each Client has a 1:1 stateful session with exactly one Server. The Host enforces security policy, consent, and context aggregation; Servers are intentionally simple, composable, and isolated from one another — a server cannot read the whole conversation nor see into other servers. Servers receive only the contextual information they need, and the full conversation history stays with the Host.
Design principles: servers should be extremely easy to build, highly composable, mutually isolated, and incrementally extensible through capability negotiation. Servers may be local processes or remote services, and may request LLM access back through the client via Sampling.
## Base Protocol
### Messages
All messages **MUST** follow JSON-RPC 2.0. Three types are defined:
- **Requests** — **MUST** include a `string | number` `id` and a `method`. Unlike base JSON-RPC, the id **MUST NOT** be `null`, and **MUST NOT** have been reused by the requestor within the same session.
- **Responses** — **MUST** echo the request `id`; exactly one of `result` or `error` **MUST** be set (never both); error `code` **MUST** be an integer.
- **Notifications** — **MUST NOT** include an `id`.
### Lifecycle
A strict three-phase lifecycle: **Initialization → Operation → Shutdown**.
- Initialization **MUST** be the first interaction. The client sends an `initialize` request carrying `protocolVersion` (e.g. `"2024-11-05"`), its `capabilities`, and `clientInfo`. The server responds with its own negotiated `protocolVersion`, `capabilities`, and `serverInfo`. The client then **MUST** send an `notifications/initialized` notification before normal operations begin.
- Before the server's `initialize` response, the client **SHOULD NOT** send anything but `ping`; before the `initialized` notification, the server **SHOULD NOT** send anything but `ping` and logging.
- **Version negotiation:** the client sends a supported version (SHOULD be its latest); if the server supports it, it responds with the same version, otherwise with another supported version (SHOULD be its latest). If the client cannot accept the server's version, it **SHOULD** disconnect.
- **Capability negotiation:** client (`roots`, `sampling`, `experimental`) and server (`prompts`, `resources`, `tools`, `logging`, `experimental`) declare supported features. Sub-capabilities include `listChanged` (prompts/resources/tools) and `subscribe` (resources only). Both parties **SHOULD** respect negotiated capabilities for the whole session.
- **Shutdown** has no dedicated message: stdio clients close the server's stdin, then escalate to `SIGTERM` / `SIGKILL`; HTTP shutdown is signalled by closing the connection(s).
### Transports
Two standard transports are defined; clients **SHOULD** support stdio whenever possible.
- **stdio** — the client launches the server as a subprocess; messages are newline-delimited on stdin/stdout and **MUST NOT** contain embedded newlines; the server **MUST NOT** write non-MCP data to stdout; `stderr` **MAY** carry UTF-8 logs.
- **HTTP with SSE** — the server exposes two endpoints: an SSE endpoint (client connects, receives an `endpoint` event with a POST URI) and an HTTP POST endpoint to which the client sends all subsequent messages; server-to-client messages arrive as SSE `message` events. Security: servers **MUST** validate the `Origin` header, **SHOULD** bind to localhost only, and **SHOULD** require authentication.
- **Custom transports** MAY be implemented provided they preserve the JSON-RPC format and lifecycle.
### Authorization
**Authentication and authorization were not part of the core specification in this revision** (the `/basic/authorization` page does not exist for 2024-11-05). The base-protocol overview states auth "is not currently part of the core MCP specification" and that clients and servers **MAY** negotiate their own custom strategies. The only auth-related guidance is the transport-level security requirements above. (The OAuth 2.1 framework was introduced in a later revision, not this one.)
### Versioning
There is no dedicated versioning page in this revision; version behavior is specified within the lifecycle handshake (see above). The negotiated version is a literal string such as `"2024-11-05"`.
## Server Features
Servers advertise any implemented features via capabilities; each listed below **MUST** be declared before use.
### Resources
Application-driven contextual data, each identified by a [URI](https://datatracker.ietf.org/doc/html/rfc3986). Capability `resources` with optional `subscribe` and `listChanged`. Methods: `resources/list` (paginated), `resources/read` (by URI), `resources/templates/list` (RFC 6570 URI templates, paginated), plus `resources/subscribe` / `resources/unsubscribe` when `subscribe` is supported. List changes emit `notifications/resources/list_changed`; subscribed resource updates emit `notifications/resources/updated`.
### Prompts
User-controlled templated messages/workflows (typically surfaced as slash commands). Capability `prompts` with optional `listChanged`. Methods: `prompts/list` (paginated) and `prompts/get` (with `arguments`). A returned `PromptMessage` carries a `role` (`"user"` / `"assistant"`) and a content item (text or image). List changes emit `notifications/prompts/list_changed`. Arguments may be auto-completed via the completion utility.
### Tools
Model-controlled executable functions. Capability `tools` with optional `listChanged`. Methods: `tools/list` (paginated) and `tools/call`. A tool definition carries `name`, `description`, and an `inputSchema` (JSON Schema). A call response returns a `content` array of typed items (text and image content in this revision) plus a boolean `isError` flag. **There is no structured output** — results are unstructured content arrays. List changes emit `notifications/tools/list_changed`. Hosts **SHOULD** keep a human in the loop and confirm invocations.
### Utilities / Logging
Capability `logging` (declared as `{}`). Servers emit `notifications/message` with a syslog (RFC 5424) severity level (`debug`, `info`, `notice`, `warning`, `error`, `critical`, `alert`, `emergency`), an optional `logger` name, and arbitrary JSON-serializable `data`. Clients **MAY** set the minimum level via `logging/setLevel`. Additional cross-cutting utilities referenced by the spec include pagination (`server/utilities/pagination`), argument completion (`server/utilities/completion`), and `ping` (`basic/utilities/ping`).
## Client Features
### Sampling
Server-initiated LLM requests, enabling nested/agentic behaviors. Capability `sampling` (declared as `{}`). The server sends `sampling/createMessage` with `messages`, optional `modelPreferences` (sub-string `hints` plus normalized `costPriority` / `speedPriority` / `intelligencePriority` in 0–1), optional `systemPrompt`, and `maxTokens`. The client (with a human in the loop) approves/edits the prompt and returns `role`, `content`, `model`, and `stopReason`.
### Roots
Filesystem/URI boundaries the server may operate within. Capability `roots` with optional `listChanged`. The server calls `roots/list` and receives `Root` entries; each `uri` **MUST** be a `file://` URI in this revision (plus an optional display `name`). Clients supporting `listChanged` **MUST** emit `notifications/roots/list_changed` when the set changes. Clients **MUST** validate URIs against path traversal and **SHOULD** obtain user consent before exposing roots.
## Revision history
- **2024-11-05** — first public release of the specification (repo tag `2024-11-05`).
- **2024-11-05-final** — final revision of the 2024-11-05 spec line (repo tag `2024-11-05-final`).
- **2024-10-07** — pre-public revision tag that existed before the public release.
## Skald relevance
Skald's MCP client implementation lives in `crates/mcp-client/`: `McpServer` (stdio) and `McpHttpServer` (HTTP+SSE) both implement the `McpServerClient` trait defined in `crates/mcp-client/src/lib.rs`. The `McpManager` orchestrator in `src/core/mcp/mod.rs` registers and dispatches MCP tools to the agent tool-calling loop. This 2024-11-05 revision is the historical baseline the earliest MCP support targeted (stdio + HTTP+SSE, Resources/Prompts/Tools/Logging, Sampling/Roots); modern Skald also speaks newer revisions and therefore supports capabilities (e.g. Streamable HTTP, structured tool output) that did not exist in this legacy spec.
## References
- [Specification overview](https://modelcontextprotocol.io/specification/2024-11-05)
- [Architecture](https://modelcontextprotocol.io/specification/2024-11-05/architecture)
- [Base protocol](https://modelcontextprotocol.io/specification/2024-11-05/basic)
- [Lifecycle](https://modelcontextprotocol.io/specification/2024-11-05/basic/lifecycle)
- [Messages](https://modelcontextprotocol.io/specification/2024-11-05/basic/messages)
- [Transports](https://modelcontextprotocol.io/specification/2024-11-05/basic/transports)
- [Server features](https://modelcontextprotocol.io/specification/2024-11-05/server)
- [Tools](https://modelcontextprotocol.io/specification/2024-11-05/server/tools)
- [Resources](https://modelcontextprotocol.io/specification/2024-11-05/server/resources)
- [Prompts](https://modelcontextprotocol.io/specification/2024-11-05/server/prompts)
- [Logging](https://modelcontextprotocol.io/specification/2024-11-05/server/utilities/logging)
- [Client features / Roots](https://modelcontextprotocol.io/specification/2024-11-05/client)
- [Sampling](https://modelcontextprotocol.io/specification/2024-11-05/client/sampling)
- [Schema (schema.ts)](https://github.com/modelcontextprotocol/specification/blob/main/schema/2024-11-05/schema.ts)
- [Release 2024-11-05](https://github.com/modelcontextprotocol/modelcontextprotocol/releases/tag/2024-11-05)
docs/mcp/specs/2025-06-18.md
+88
View File
@@ -0,0 +1,88 @@
# MCP Specification — 2025-06-18 (Stable)
> **Status:** Stable
> **Released:** 2025-06-18
> **Spec site:** https://modelcontextprotocol.io/specification/2025-06-18
> **Authoritative schema:** [schema/2025-06-18/schema.ts](https://github.com/modelcontextprotocol/specification/blob/main/schema/2025-06-18/schema.ts)
Revision 2025-06-18 is the "modern stable" baseline of the Model Context Protocol. It supersedes the 2024-11-05 legacy revision by introducing the **Streamable HTTP** transport (single-endpoint, replacing the two-endpoint HTTP+SSE design), the **Elicitation** client feature (`elicitation/create`, server-initiated structured input from the user), and **structured tool output** (typed `structuredContent` validated against an optional `outputSchema`). It also tightens the authorization framework with OAuth 2.1 Resource Indicators (RFC 8707) and OAuth Protected Resource Metadata (RFC 9728), and removes JSON-RPC batching. The authoritative source of truth is the TypeScript schema; the spec text is normative where it restates requirements using BCP 14 keywords (MUST/SHOULD/MAY).
## At a glance
- **Protocol style:** stateful connections
- **Capability negotiation:** connection-level (`initialize`)
- **Transports:** stdio; **Streamable HTTP** (replaces HTTP+SSE)
- **Server features:** Resources, Prompts, Tools, Logging
- **Client features:** Sampling, Roots, **Elicitation**
- **Authorization:** OAuth 2.1 + Resource Indicators (RFC 8707)
## Architecture
Client-host-server topology built on JSON-RPC. A **host** creates and manages multiple **client** instances; each client holds a 1:1 **stateful session** with one **server** and enforces isolation between servers (a server cannot see the full conversation or other servers). Servers expose primitives (resources/prompts/tools) and MAY request client features (sampling/roots/elicitation). Design principles: servers must be easy to build, highly composable, mutually isolated, and progressively extensible via capability negotiation at `initialize`.
## Base Protocol
### Messages
All messages **MUST** be [JSON-RPC 2.0](https://www.jsonrpc.org/specification), UTF-8 encoded. Three kinds: **Requests** (with a string/integer `id`; the id **MUST NOT** be `null` and **MUST NOT** be reused within a session), **Responses** (`result` XOR `error`; both **MUST NOT** be set; error codes are integers), and **Notifications** (no `id`; receiver **MUST NOT** reply). The reserved `_meta` property carries extension metadata; key names with a `mcp`/`modelcontextprotocol` prefix are reserved. **JSON-RPC batching is disallowed** (each message is a standalone request/notification/response).
### Lifecycle
Three phases. (1) **Initialization** — client sends `initialize` with its supported `protocolVersion` (SHOULD be its latest), `capabilities`, and `clientInfo`; server responds with the negotiated `protocolVersion`, its `capabilities`, `serverInfo`, and optional `instructions`; client then sends `notifications/initialized`. The server SHOULD NOT send non-ping/logging requests before `initialized`. (2) **Operation** — both parties **MUST** respect the negotiated version and only use negotiated capabilities. (3) **Shutdown** — no dedicated message; the transport signals termination (stdio: close stdin → SIGTERM → SIGKILL; HTTP: `DELETE` the session or close streams). If versions mismatch the client **SHOULD** disconnect.
### Transports
Two standard transports. **stdio** — the client launches the server as a subprocess; newline-delimited JSON-RPC over stdin/stdout; messages **MUST NOT** contain embedded newlines; stderr is for logs only. **Streamable HTTP** — the server exposes a single **MCP endpoint** supporting `POST` and optionally `GET` (e.g. `https://example.com/mcp`), replacing the 2024-11-05 HTTP+SSE two-endpoint design:
- Every client JSON-RPC message is a fresh `POST`; the request **MUST** send `Accept: application/json, text/event-stream`. For a *request* the server responds either with `application/json` or by opening an SSE `text/event-stream`; for a *notification/response* the server returns `202 Accepted`.
- Server→client traffic (notifications/requests) flows over an SSE stream that the client opens via `GET` (server MAY decline with `405`). The server **MUST NOT** broadcast the same message across multiple concurrent streams.
- **Sessions:** the server MAY assign an `Mcp-Session-Id` (cryptographically secure, visible ASCII 0x21–0x7E) on the initialize response; the client **MUST** echo it on all subsequent requests; a `404` forces re-initialization; a `DELETE` terminates the session.
- **Resumability:** servers MAY attach SSE event `id`s; the client resumes via the `Last-Event-ID` header on `GET` (per-stream cursor; no cross-stream replay).
- Clients using HTTP **MUST** send `MCP-Protocol-Version: <version>` on all post-initialize requests. Servers **MUST** validate the `Origin` header (DNS-rebinding defence) and **SHOULD** bind localhost when local.
### Authorization
Optional, for HTTP transports only (stdio retrieves credentials from the environment). Built on OAuth 2.1 ([draft-ietf-oauth-v2-1-13](https://datatracker.ietf.org/doc/html/draft-ietf-oauth-v2-1-13)) with PKCE. **Resource Server classification:** the MCP server is an OAuth 2.1 *resource server*; the MCP client is the OAuth 2.1 *client*; tokens are issued by a separate authorization server. Discovery: MCP servers **MUST** implement OAuth 2.0 Protected Resource Metadata ([RFC 9728](https://www.rfc-editor.org/rfc/rfc9728.html)) and signal the metadata URL via `WWW-Authenticate` on `401`; clients **MUST** use Authorization Server Metadata ([RFC 8414](https://datatracker.ietf.org/doc/html/rfc8414)) and **SHOULD** use Dynamic Client Registration ([RFC 7591](https://datatracker.ietf.org/doc/html/rfc7591)). **Resource Indicators (RFC 8707):** the client **MUST** send the `resource` parameter (identifying the target MCP server) in both the authorization and token requests. Token refresh is supported; scopes are server-defined.
### Versioning
Protocol versions are date strings (e.g. `2025-06-18`). Version negotiation happens only in `initialize` (see Lifecycle): the client sends the version it supports; the server returns it if supported, otherwise its latest other version; the client disconnects on an unsupported response. A `CHANGELOG.md` records revisions; SDKs adopt versions at their own pace and rely on negotiation for backwards/forwards compatibility.
## Server Features
Servers advertise implemented primitives in their capabilities. Control model: **Prompts** are user-controlled, **Resources** application-controlled, **Tools** model-controlled.
### Resources
Application-driven context exposed via URIs. Capability `resources` with optional `subscribe` and `listChanged`. Methods: `resources/list`, `resources/read`, `resources/templates/list` (RFC 6570 URI templates), plus `resources/subscribe`/`unsubscribe` and `notifications/resources/*`. **Resource links** — tools and resources can reference related resources; a `resource` content item (with `uri`, optional `mimeType`, and inline `text`/`blob`) embeds a resource, and tool results can emit resource links so clients can pull further context. All content items carry optional `annotations` (`audience`, `priority`, `lastModified`).
### Prompts
User-controlled templated messages. Capability `prompts` (with optional `listChanged`). Methods: `prompts/list`, `prompts/get` (takes `name` + `arguments`). A `PromptMessage` has `role` (`user`/`assistant`) and typed `content` (text/image/audio/embedded-resource). Arguments can be autocompleted via the completion utility.
### Tools
Model-controlled functions. Capability `tools` (with optional `listChanged`). Methods: `tools/list`, `tools/call`. A tool definition carries `name`, optional `title`/`description`, `inputSchema` (JSON Schema), and optional `outputSchema` (JSON Schema) and `annotations` (treated as untrusted). **Structured output:** a `tools/call` result returns a `content[]` array (text/image/audio/resource items) **plus an optional `structuredContent`** JSON object. When `outputSchema` is declared, the server **MUST** return structured results conforming to it and clients **SHOULD** validate; for backwards compatibility a tool returning `structuredContent` **SHOULD** also emit the serialized JSON in a `TextContent` block. Execution failures use `isError: true`; protocol errors use standard JSON-RPC codes.
### Utilities / Logging
Cross-cutting utilities: `ping` (liveness), `notifications/progress` (progress tracking), `notifications/cancelled` (cancellation via request id), `notifications/initialized`, and `completion/complete` (argument autocompletion). **Logging:** servers with the `logging` capability emit `notifications/message` with a level (`debug`..`emergency`) and structured data; clients set level via `logging/setLevel`.
## Client Features
### Sampling
Server-initiated LLM generation. Client capability `sampling`. Method `sampling/createMessage` — the server sends `messages`, optional `systemPrompt`, `maxTokens`, and `modelPreferences` (`hints` + normalized `intelligencePriority`/`speedPriority`/`costPriority` in 0–1); the client performs human-in-the-loop review, selects the model, and returns `role`/`content`/`model`/`stopReason`. Human approval is expected; the server never supplies API keys.
### Roots
Server-initiated inquiry into client filesystem boundaries. Client capability `roots` (with optional `listChanged`). Method `roots/list` returns `file://` URIs the server may operate within; `notifications/roots/list_changed` signals updates. Clients **MUST** validate URIs (path traversal) and obtain user consent.
### Elicitation
Server-initiated structured input from the end user. Client capability `elicitation`. Method `elicitation/create` — the server sends a human-readable `message` plus a `requestedSchema`, a *restricted* JSON Schema (flat object of primitive properties only: string/number/boolean/enums, no nesting). The client presents a form and responds with `action: "accept"` (+ `content` object), `"decline"`, or `"cancel"`. **Privacy/safety:** servers **MUST NOT** use elicitation to request sensitive information (passwords, API keys, credentials); clients **SHOULD** make the requesting server clear and provide explicit decline/cancel. Because responses may be sensitive, they must not be echoed into logs or persisted beyond need.
## Changes vs 2024-11-05
- **Streamable HTTP** replaces the HTTP+SSE two-endpoint transport (single `POST`/`GET` MCP endpoint, `Mcp-Session-Id` header, SSE resumability via `Last-Event-ID`, `MCP-Protocol-Version` header).
- **Elicitation** client feature added (`elicitation/create` with a restricted JSON Schema form; accept/decline/cancel).
- **Structured tool output** — tools may return `structuredContent` (typed JSON) plus an optional `outputSchema`; structured results SHOULD also serialize to a `TextContent` block.
- **Resource links** — tools/resources can reference related resources (embedded `resource` content items, annotations).
- **JSON-RPC batching removed** (was permitted in 2024-11-05; each message is now standalone).
- **Authorization strengthened:** OAuth 2.1 Resource Indicators (RFC 8707) with the `resource` parameter; OAuth 2.0 Protected Resource Metadata (RFC 9728) + Resource Server classification; Authorization Server Metadata (RFC 8414); Dynamic Client Registration (RFC 7591) recommended.
- **Protocol remains stateful** with connection-level capability negotiation; client features are now Sampling, Roots, and Elicitation.
## Skald relevance
Skald's MCP stack implements this revision. `crates/mcp-client/` provides the transport/runtime primitives — `McpServerClient` trait (`src/lib.rs:67`), stdio server, and `McpHttpServer` (`http_server.rs`) — while `src/core/mcp/mod.rs` (`McpManager`) registers and drives connected servers, and `src/core/elicitation/mod.rs` bridges the new `elicitation/create` server→client request through `ElicitationManager` (surfaced in the Inbox) with secrets never logged or persisted. 2025-06-18 is therefore the baseline that introduced the Elicitation feature Skald implements end-to-end.
## References
- [Specification overview](https://modelcontextprotocol.io/specification/2025-06-18)
- [Architecture](https://modelcontextprotocol.io/specification/2025-06-18/architecture)
- [Base protocol](https://modelcontextprotocol.io/specification/2025-06-18/basic)
- [Server features](https://modelcontextprotocol.io/specification/2025-06-18/server)
- [Client features](https://modelcontextprotocol.io/specification/2025-06-18/client)
- [Schema (schema.ts)](https://github.com/modelcontextprotocol/specification/blob/main/schema/2025-06-18/schema.ts)
- [Release 2025-06-18](https://github.com/modelcontextprotocol/modelcontextprotocol/releases/tag/2025-06-18)
docs/mcp/specs/2025-11-25.md
+135
View File
@@ -0,0 +1,135 @@
# MCP Specification — 2025-11-25 (Latest Stable)
> **Status:** Latest Stable (current `Latest` release)
> **Released:** 2025-11-25 · repo tags `2025-11-25` (stable), `2025-11-25-RC` (pre-release)
> **Spec site:** https://modelcontextprotocol.io/specification/2025-11-25
> **Authoritative schema:** [schema/2025-11-25/schema.ts](https://github.com/modelcontextprotocol/specification/blob/main/schema/2025-11-25/schema.ts)
This is the current stable revision of the Model Context Protocol. It keeps the stateful, JSON-RPC 2.0 core of 2025-06-18 and refines authorization (adding OpenID Connect Discovery and OAuth Client ID metadata documents), introduces optional **icons** metadata across server primitives, **URL-mode elicitation**, **incremental scope consent** (step-up authorization), **tool-calling in sampling**, and ships an **experimental Tasks** mechanism for long-running, pollable operations.
## At a glance
- **Protocol style:** stateful connections
- **Capability negotiation:** connection-level (`initialize`)
- **Transports:** stdio; Streamable HTTP
- **Server features:** Resources, Prompts, Tools, Logging
- **Client features:** Sampling, Roots, Elicitation
- **Authorization:** OAuth 2.1 + Resource Indicators (RFC 8707) + **OIDC Discovery** + Protected Resource Metadata (RFC 9728) + Client ID metadata documents + incremental scope consent
- **Experimental:** Tasks
## Architecture
Client-host-server topology built on JSON-RPC, focused on context exchange and sampling coordination.
- **Host:** creates/manages multiple clients, enforces consent and security policy, aggregates context. Each client has a 1:1 relationship with one server and maintains isolation between servers.
- **Client:** establishes one stateful session per server, negotiates capabilities, routes messages, owns subscriptions/notifications.
- **Server:** exposes resources/tools/prompts, may request sampling/roots/elicitation through client interfaces; can be a local process or remote service.
Design principles: servers should be easy to build, highly composable, must **not** read the whole conversation or "see into" other servers, and features can be added progressively. **Capability negotiation** is the contract: server features (resources, prompts, tools, logging) and client features (sampling, roots, elicitation, tasks) must be advertised in `capabilities` to be usable.
## Base Protocol
### Messages
All messages **MUST** be [JSON-RPC 2.0](https://www.jsonrpc.org/specification), UTF-8 encoded. Three kinds:
- **Requests** — `id` (string|integer) **REQUIRED**; **MUST NOT** be `null`; the requestor **MUST NOT** reuse an `id` within a session.
- **Responses** — `result` (any object) on success; `error` (`code` integer + `message`) on failure. The `id` **MUST** match the originating request (except when the request `id` was unreadable due to malformation).
- **Notifications** — one-way; **MUST NOT** carry an `id`; the receiver **MUST NOT** respond.
The `_meta` field is reserved for protocol-level and extension metadata; certain keys are reserved by MCP and implementers **MUST NOT** assume meaning for reserved keys. **JSON Schema dialect:** schemas without an explicit `$schema` default to [JSON Schema 2020-12](https://json-schema.org/draft/2020-12/schema); implementations **MUST** support at least 2020-12 and **SHOULD** document any additional dialects. The `icons` property is a standardized optional visual-identifier array (`src`, `mimeType`, `sizes`) usable on implementations/resources/tools/prompts; icon payloads **MAY** contain executable content (e.g. scripted SVG) so consumers **MUST** fetch them without credentials and treat them as untrusted.
### Lifecycle
Three phases: **Initialization → Operation → Shutdown**.
1. Client sends `initialize` with its `protocolVersion` (this **SHOULD** be the latest the client supports), client `capabilities`, and `clientInfo`.
2. Server responds with the negotiated `protocolVersion`, its `capabilities`, `serverInfo` (`name`/`title`/`version`/`description`/`icons`/`websiteUrl`), and optional `instructions`.
3. Client sends an `notifications/initialized` notification; only then does normal operation begin. Before `initialize` is answered the client **SHOULD NOT** send non-`ping` requests; before `initialized` the server **SHOULD NOT** send non-`ping`/non-logging requests.
**Version negotiation:** if the server supports the requested version it echoes it; otherwise it returns the latest version *it* supports. If the client cannot accept that version it **SHOULD** disconnect. Over HTTP the client **MUST** send `MCP-Protocol-Version: <version>` on every subsequent request.
### Transports
- **stdio** — client launches the server as a subprocess; newline-delimited JSON-RPC on stdin/stdout; messages **MUST NOT** contain embedded newlines; stdout is reserved exclusively for valid MCP messages; `stderr` is for optional logging. Clients **SHOULD** support stdio whenever possible.
- **Streamable HTTP** — the server exposes a **single MCP endpoint** supporting POST and GET. The client POSTs each JSON-RPC message with `Accept: application/json, text/event-stream`; the server answers synchronously (`application/json`) or opens an SSE stream. **Session management:** the server **MAY** assign an `Mcp-Session-Id` on the `InitializeResult` response; once issued, the client **MUST** send that header on subsequent requests and the server **MAY** reject mismatches with 404. **Resumability:** servers **MAY** attach SSE `id`s; clients resume after disconnect via HTTP GET with `Last-Event-ID`. Servers **MUST** validate the `Origin` header (403 on mismatch) and **SHOULD** bind to localhost locally to prevent DNS-rebinding. Replaces the deprecated HTTP+SSE transport from 2024-11-05.
### Authorization
Authorization is **OPTIONAL**. HTTP-based implementations **SHOULD** conform; stdio **SHOULD NOT** (use environment credentials instead). The model: the MCP server is an OAuth 2.1 **resource server**; the MCP client is the OAuth 2.1 **client**; a separate **authorization server** issues tokens.
- Built on **OAuth 2.1** (draft-ietf-oauth-v2-1-13) with PKCE and **Resource Indicators (RFC 8707)**.
- **Protected Resource Metadata (RFC 9728)** — MCP servers **MUST** implement it; clients **MUST** use it for authorization-server discovery (via `WWW-Authenticate: resource_metadata=…` on 401, or a `.well-known/oauth-protected-resource` URI). Servers **SHOULD** advertise required `scope` in the 401 challenge.
- **Authorization-server metadata** — the authorization server **MUST** expose either **OAuth 2.0 Authorization Server Metadata (RFC 8414)** *or* **OpenID Connect Discovery 1.0** (`.well-known/openid-configuration`); clients **MUST** support **both**, trying RFC 8414 first.
- **OAuth Client ID Metadata Documents** (draft-ietf-oauth-client-id-metadata-document-00) — clients/servers **SHOULD** support this public-client identification mechanism; RFC 7591 Dynamic Client Registration **MAY** be supported.
- **Incremental scope consent** — the step-up / scope-challenge flow: clients start with the minimal `scopes_supported` (or the 401 `scope`) and request additional scopes only when the server returns an insufficient-scope error (`WWW-Authenticate` with a narrowed challenge), satisfying least-privilege progressively.
### Versioning
There is no dedicated versioning page in this revision; protocol-version agreement is defined within **Lifecycle** (the `initialize` handshake and the `MCP-Protocol-Version` header). The version string for this revision is `2025-11-25`. The spec site's `/basic/versioning` URL does **not** resolve for 2025-11-25.
## Server Features
### Resources
Application-driven context exposed by URI. Capability `resources` with optional `subscribe` and `listChanged`. Operations: `resources/list` (paginated), `resources/read`, `resources/templates/list` (RFC 6570 URI templates, auto-completable). Resources carry `uri`, `name`, optional `title`/`description`/`mimeType`/`icons`/`annotations` (`audience`, `priority`, `lastModified`). Changes are signalled via `notifications/resources/*`; **resource links** let any `text` content embed typed links to other resources through their URI. Clients **MUST** obtain user consent before exposing resource data.
### Prompts
User-controlled templates (e.g. slash commands). Capability `prompts` with optional `listChanged`. Operations: `prompts/list` (paginated), `prompts/get` (with arguments). A prompt has `name`, optional `title`/`description`/`arguments`/`icons`; messages are `role` + typed `content` (text/image/audio/embedded-resource), each supporting `annotations`.
### Tools
Model-controlled functions. Capability `tools` with optional `listChanged`. Operations: `tools/list` (paginated) and `tools/call`. A tool defines `name`, optional `title`/`description`/`icons`, `inputSchema` (defaults to JSON Schema 2020-12), optional `outputSchema`, and `annotations` (e.g. `readOnlyHint`, `destructiveHint`, `idempotentHint`, `openWorldHint`). **Structured output:** when `outputSchema` is present, the server **MUST** return conforming results as a JSON object in `structuredContent`, and **SHOULD** also mirror it in a `TextContent` block for backwards compatibility; clients **SHOULD** validate against the schema. Results return `content[]`, an `isError` flag, and the optional `structuredContent`. A human **SHOULD** remain in the loop to authorize invocations.
### Utilities / Logging
Cross-cutting utilities: **pagination** (`cursor`/`nextCursor` on list ops), **completion** (`completion/complete` for argument auto-complete), **ping** (`ping`), **cancellation** (`notifications/cancelled`), **progress** (`notifications/progress`), and **logging** — capability `logging`; clients configure level via `logging/setLevel`, servers emit `notifications/message` with severity, logger name, and structured data.
## Client Features
### Sampling
Server-initiated LLM completions via `sampling/createMessage`. Clients **MUST** declare `sampling`; **human-in-the-loop** review of the prompt and result is **RECOMMENDED**. Requests carry `messages`, optional `systemPrompt`, `modelPreferences` (`hints`, `intelligencePriority`/`speedPriority`), `maxTokens`, and (when the client advertises `sampling.tools`) a `tools` array plus optional `toolChoice`. **Tool-calling support:** the client's LLM may emit `tool_use`; the server executes the tool and re-issues `sampling/createMessage` with the tool results, looping until `stopReason: "endTurn"`. The legacy `includeContext: "thisServer"|"allServers"` values (gated behind `sampling.context`) are soft-deprecated.
### Roots
Server-initiated discovery of filesystem boundaries. Capability `roots` (optional `listChanged`). Server sends `roots/list`; client returns `uri` (currently **MUST** be a `file://` URI) plus optional `name`. Clients emit `notifications/roots/list_changed` when the set changes. Clients **MUST** validate root URIs against path traversal and **SHOULD** prompt for consent before exposing them.
### Elicitation
Server-initiated requests for user information via `elicitation/create`. Capability `elicitation` with sub-modes `form` and `url` (an empty object is equivalent to `form`-only; at least one mode **MUST** be supported). Two modes:
- **Form mode** — in-band structured input. `requestedSchema` is restricted to a flat object of primitive properties (string/number/integer/boolean/enum, including `oneOf`/multi-select arrays); formats `email`, `uri`, `date`, `date-time`. Response `action` is `accept` (with `content`), `decline`, or `cancel`.
- **URL mode** (new in 2025-11-25) — out-of-band interaction for sensitive data (secrets, OAuth, payments) that must **not** transit the client. Parameters: `mode: "url"`, `url`, `elicitationId`, `message`. The client only collects consent and opens the URL; the response is `action: "accept"` with **no** content. Servers **MAY** fire `notifications/elicitation/complete` and **MAY** return error `-32042` (`URLElicitationRequiredError`) carrying required elicitations. Servers **MUST** use URL mode (never form mode) for passwords/API keys/tokens/credentials.
Privacy: clients **MUST** show which server is asking, provide clear decline/cancel controls, let users review form responses before sending, and never log/retain secrets.
## Experimental: Tasks
Introduced in 2025-11-25 as **experimental**. Tasks are durable state machines that wrap a request for pollable, deferred-result execution (expensive computation, batch jobs, external job APIs). Either party may be a **requestor** or **receiver**. A task is created by adding a `task` field (with optional `ttl`) to an augmentable request; the receiver returns a `CreateTaskResult` with a `taskId`, `status` (`working`→`completed`/`failed`/`cancelled`, plus `input_required`), `pollInterval`, and timestamps. The real result is fetched later via `tasks/result`; status via `tasks/get`, optional `notifications/tasks/status`, listing via `tasks/list`, termination via `tasks/cancel`. Capability `tasks` is structured per request category (e.g. `tasks.requests.tools.call` server-side; `tasks.requests.sampling.createMessage` / `tasks.requests.elicitation.create` client-side), with tool-level negotiation through `execution.taskSupport` (`required`/`optional`/`forbidden`). The design may be formalized or modified in future revisions.
## Changes vs 2025-06-18
- **OpenID Connect Discovery 1.0** added as a first-class authorization-server metadata mechanism (clients must support it alongside RFC 8414).
- **Icons metadata** — optional `icons[]` on implementations, resources, tools, and prompts.
- **Incremental scope consent** — step-up authorization via scope challenges (request additional scopes only on insufficient-scope errors).
- **URL-mode elicitation** (`mode: "url"`, `URLElicitationRequiredError` `-32042`, `notifications/elicitation/complete`) for sensitive out-of-band interactions.
- **Sampling tool-calling** — `sampling.tools` capability; servers may pass `tools`/`toolChoice` and run a multi-turn tool loop.
- **OAuth Client ID metadata documents** (draft-ietf-oauth-client-id-metadata-document-00) for public-client identification.
- **Experimental Tasks** — durable, pollable, deferred-result request augmentation.
- Unchanged: stateful JSON-RPC 2.0 core (batching already removed in 2025-06-18), connection-level capability negotiation, stdio + Streamable HTTP transports, structured tool output (`outputSchema`/`structuredContent`, from 2025-06-18), elicitation form mode, resource links.
## Skald relevance
This is the revision Skald's MCP client targets. The client lives in `crates/mcp-client/` (stdio + Streamable HTTP, with an `elicitation` integration test); runtime connection management is `src/core/mcp/mod.rs` (`McpManager`); server-initiated `elicitation/create` is handled by `src/core/elicitation/mod.rs` (`ElicitationManager` + bridge), surfaced in the unified **Inbox**, with secrets never logged or persisted. **Non-text tool-result content** (`image`/`audio`/`resource`/`resource_link`) is now preserved rather than dropped: it is persisted under `data/mcp_media/` and surfaced as a `/api/mcp-media/` URL in a markdown result (see [`../../mcp.md`](../../mcp.md) → *Media tool results*).
**Cancellation** (`notifications/cancelled`) is implemented: an abandoned `tools/call` (a `/stop` that drops the work future, or the 120 s timeout) now tells the server to stop via a per-transport drop-guard, instead of leaving it working on abandoned output. **Tasks** are polled **synchronously (block-and-poll)**: `call_tool` opts a task-capable tool in with the `task` field, then `poll_task` polls `tasks/get` until terminal and fetches the real result via `tasks/result` — so a task-mode call no longer hits the 120 s wall — sending `tasks/cancel` cooperatively if the call is dropped. Both are documented in [`../../mcp.md`](../../mcp.md) → *Cancellation & Tasks*. The block-and-poll variant holds the session for the task's duration and doesn't survive a self-restart; the **detached/durable** poller (DB-persisted tasks + background delivery via `inject_async_result`/`resume_turn`) is the tracked follow-up. Still unimplemented from this revision: **sampling**, **roots**, **URL-mode elicitation**, mid-task `input_required`, progress/completion/logging utilities, and OAuth authorization (HTTP transport).
## References
- [Specification overview](https://modelcontextprotocol.io/specification/2025-11-25)
- [Architecture](https://modelcontextprotocol.io/specification/2025-11-25/architecture)
- [Base protocol](https://modelcontextprotocol.io/specification/2025-11-25/basic)
- [Server features](https://modelcontextprotocol.io/specification/2025-11-25/server)
- [Client features](https://modelcontextprotocol.io/specification/2025-11-25/client)
- [Schema (schema.ts)](https://github.com/modelcontextprotocol/specification/blob/main/schema/2025-11-25/schema.ts)
- [Release 2025-11-25](https://github.com/modelcontextprotocol/modelcontextprotocol/releases/tag/2025-11-25)
docs/mcp/specs/2026-07-28-draft.md
+114
View File
@@ -0,0 +1,114 @@
# MCP Specification — 2026-07-28 Draft (RC)
> **Status:** Draft / Release Candidate (pre-release). Subject to change — not final.
> **Tag:** `2026-07-28-RC` · rendered at `/specification/draft`
> **Spec site:** https://modelcontextprotocol.io/specification/draft
> **Authoritative schema:** [schema/draft/schema.ts](https://github.com/modelcontextprotocol/specification/blob/main/schema/draft/schema.ts)
The `2026-07-28` draft is a **foundational redesign** of the Model Context Protocol, not an incremental revision. It moves from the prior stateful-connection model to a **stateless base protocol with self-contained requests**, replaces connection-level capability negotiation (the `initialize` handshake) with **per-request capability negotiation**, and introduces a formal **extensions framework** for opt-in modular functionality. Several older client/server features are deprecated under a new feature-lifecycle policy. This is the direction future Skald MCP work should anticipate; the per-request shift and the Tasks extension are the most impactful changes to plan for.
## At a glance
- **Protocol style:** stateless, self-contained requests (no session state inferred across requests)
- **Capability negotiation:** per-request, via `_meta.io.modelcontextprotocol/clientCapabilities`
- **Transports:** **stdio** (newline-delimited JSON-RPC over a client-launched subprocess) and **Streamable HTTP** (single MCP endpoint; `POST` per message, request-scoped SSE for replies). Custom transports **MAY** be defined. (HTTP+SSE was deprecated earlier, in `2025-03-26`.)
- **Server features:** Resources, Prompts, Tools
- **Client features (core):** **Elicitation only**. (Sampling and Roots are deprecated — see below.)
- **Extensions (opt-in):** Tasks (`io.modelcontextprotocol/tasks`), MCP Apps (`io.modelcontextprotocol/ui`), Skills over MCP (in Working Group / experimental), OAuth client-credentials, enterprise-managed authorization, …
- **Authorization:** OAuth 2.1 + PKCE, HTTP transports only; stdio uses environment credentials
## The redesign (why this matters)
Three structural changes define this draft:
1. **Stateless base protocol.** All information needed to process a request is contained in the request itself — protocol version, client identity, and capabilities travel in the `_meta` field on every request. Servers **MUST NOT** rely on prior requests to establish context, and **SHOULD** handle requests associated with multiple tasks/threads/conversations.
2. **Per-request capability negotiation.** The connection-level `initialize` handshake is gone for modern clients; capabilities are declared on each request, and the server's capabilities are discoverable via `server/discover`. This replaces session-scoped negotiation entirely.
3. **Extensions framework.** Optional, always opt-in, modular capabilities (Tasks, MCP Apps, …) negotiated through the `extensions` field of capabilities. Core client features are trimmed to Elicitation; Sampling, Roots, and Logging move to **deprecated** status under a formal lifecycle policy (not silent removal).
## Architecture
The client-host-server topology is retained. A **host** creates and manages multiple **client** instances; each client has a 1:1 relationship with exactly one **server** and enforces isolation (a server cannot see the whole conversation or other servers). The change is that clients now "attach protocol version and capabilities to every request" rather than establishing a session. Servers expose resources/tools/prompts and **MAY** request client input (sampling/elicitation/roots) by returning an `InputRequiredResult` within a reply. Design principles are unchanged: servers should be easy to build, highly composable, mutually isolated, and progressively extensible.
## Base Protocol
### Messages
All messages **MUST** be [JSON-RPC 2.0](https://www.jsonrpc.org/specification), UTF-8 encoded. Three kinds: **Requests** (with a string/integer `id`; the id **MUST NOT** be `null` and **MUST NOT** match any in-flight request id), **Responses**, and **Notifications** (no `id`; receiver **MUST NOT** reply). A notable addition is **polymorphic results**: every result **MUST** include a `resultType` field — `"complete"` (final content), `"input_required"` (an `InputRequiredResult`, driving a multi-round-trip request), or extension-defined values; an absent `resultType` **MUST** be treated as `"complete"` for backward compatibility. JSON-RPC's reserved server-error range is now partitioned: `-32000`–`-32019` is legacy (new codes **MUST NOT** be allocated there); `-32020`–`-32099` is reserved for the MCP specification. Defined codes: `-32020` `HeaderMismatch`, `-32021` `MissingRequiredClientCapability`, `-32022` `UnsupportedProtocolVersion`. Three message patterns are supported: **Request/Response**, **Multi Round-Trip Requests (MRTR)**, and **Subscribe/Notify** (`subscriptions/listen`).
### Lifecycle / Negotiation
There is **no negotiation handshake** in the modern protocol. Every request declares its protocol version in `_meta` (also mirrored in the `MCP-Protocol-Version` header on HTTP). If the server does not support the requested version it **MUST** respond `UnsupportedProtocolVersionError` (`-32022`) listing its `supported` versions; the client **SHOULD** pick a mutually supported version and retry. Servers **MUST** implement `server/discover`; clients **MAY** call it first to learn supported versions and capabilities up front, but are free to invoke any RPC inline and handle the error. Capability negotiation is therefore **per-request**: clients advertise capabilities in `_meta.io.modelcontextprotocol/clientCapabilities`; servers expose theirs through `server/discover`. Extensions are advertised the same way, via a `capabilities.extensions` map keyed by extension identifier.
### Transports
A transport is a binding (framing + metadata + cancellation signalling); message semantics are identical on every transport. Two standard bindings:
- **stdio** — newline-delimited JSON-RPC over a client-launched subprocess; `notifications/cancelled` abandons an in-flight request. Clients **SHOULD NOT** tie the subprocess lifetime to a single task/thread/conversation.
- **Streamable HTTP** — every client message is a `POST` to a single MCP endpoint; a reply arrives as a JSON object or a request-scoped SSE stream. Selected `_meta` fields are mirrored into HTTP headers so intermediaries can route without parsing the body (body remains source of truth). Cancellation closes the response stream.
Custom transports **MAY** be defined but **MUST** preserve JSON-RPC format, the message patterns, and the per-request metadata model; those over a reliable byte stream **SHOULD** reuse stdio framing.
### Authorization
Optional. For HTTP transports it is built on **OAuth 2.1** with PKCE; stdio **SHOULD NOT** use it and instead retrieves credentials from the environment. The MCP server is an OAuth 2.1 *resource server* and **MUST** implement OAuth 2.0 Protected Resource Metadata ([RFC 9728](https://www.rfc-editor.org/rfc/rfc9728.html)), signalling the metadata URL via `WWW-Authenticate` on `401`. Authorization servers **MUST** provide discovery via OAuth 2.0 Authorization Server Metadata ([RFC 8414](https://datatracker.ietf.org/doc/html/rfc8414)) or OIDC Discovery; clients **MUST** support both. Client registration priority: **Client ID Metadata Documents** (new, preferred) → pre-registered → **Dynamic Client Registration** ([RFC 7591](https://datatracker.ietf.org/doc/html/rfc7591), now **deprecated**). Resource Indicators ([RFC 8707](https://www.rfc-editor.org/rfc/rfc8707.html)) and the `iss` parameter ([RFC 9207](https://datatracker.ietf.org/doc/html/rfc9207)) apply. Servers **SHOULD** advertise required `scope` in the `WWW-Authenticate` challenge; clients follow a least-privilege scope-selection strategy with step-up authorization.
### Versioning
Protocol versions are date strings (e.g. `2026-07-28`). Terminology: **Modern** = per-request metadata (this revision and later); **Legacy** = `initialize`-handshake versions (`2025-11-25` and earlier); **Dual-era** = implementations supporting both. A dual-era server selects behaviour from how the client opens: a request carrying modern `_meta` is served statelessly; an `initialize` request selects legacy semantics. Clients detect a server's era via transport-specific probing (stdio: `server/discover` and fall back on any non-modern error; HTTP: attempt a modern request and inspect a `400` body). The era is a property of the server and **SHOULD** be cached for the process/origin lifetime. A full compatibility matrix (Modern/Legacy/Dual-era × client/server) is specified.
## Server Features
Servers advertise implemented primitives in their capabilities. Control model unchanged: **Prompts** user-controlled, **Resources** application-controlled, **Tools** model-controlled.
### Resources
Context/data exposed by the server, addressed by URI. Discovery and reading are RPC-driven; clients open a `subscriptions/listen` stream to receive `notifications/*` (tagged with a `subscriptionId`) when resources change. The old `-32002` "resource not found" code is replaced by `-32602`; clients **SHOULD** still accept `-32002` from legacy servers.
### Prompts
Templated messages/workflows invoked by user choice (e.g. slash commands, menu options). Listed and fetched on demand; the model does not pick them autonomously.
### Tools
Functions exposed to the LLM (model-controlled). Tool invocation requires the server to declare tool capabilities. Tools **MAY** return structured output, and — via extensions such as MCP Apps — can reference interactive UI resources.
### Utilities
Cross-cutting concerns now consist of **Configuration**, **Progress tracking**, **Cancellation**, and **Error reporting**. **Logging is no longer a core utility: it is deprecated** (SEP-2577). The migration path is to log to `stderr` for stdio transports and to use [OpenTelemetry](https://opentelemetry.io/) for observability.
## Client Features
### Elicitation
The **only core client feature**. Servers **MAY** request user input during request processing by returning an `InputRequiredResult` containing an `elicitation/create` request. Two modes:
- **Form mode** — in-band structured data with optional JSON-Schema validation. Schemas are restricted to flat objects of primitive properties (string/number/integer/boolean/enum). Servers **MUST NOT** use form mode to request secrets (passwords, API keys, tokens, payment credentials).
- **URL mode** — out-of-band interaction via URL navigation; data (other than the URL) is **not** exposed to the client. Servers **MUST** use URL mode for sensitive information.
Clients supporting elicitation **MUST** declare `_meta.io.modelcontextprotocol/clientCapabilities.elicitation` (with `form` and/or `url` sub-capabilities; empty object ≡ `form` only) on each request. Clients **MUST** make clear which server is asking, and provide decline/cancel options.
## Extensions (new)
### Overview
Extensions are optional additions beyond the core protocol — modular, specialized, or experimental. They are identified by `{vendor-prefix}/{extension-name}` (official ones use `io.modelcontextprotocol/`), advertised in the `capabilities.extensions` map (per-request), and **always disabled by default** — requiring explicit opt-in from both sides. If one side supports an extension the other lacks, it **MUST** fall back to core behaviour or reject with an appropriate error. Extensions evolve independently and **SHOULD** use capability flags or in-extension versioning rather than new identifiers for breaking changes. Official extensions live in `ext-*` repositories; experimental ones (incubating in Working/Interest Groups) use the `experimental-ext-` prefix.
### Tasks
Extension `io.modelcontextprotocol/tasks` (repo `ext-tasks`). Lets a server return a durable handle instead of blocking on long-running operations (CI pipelines, batch jobs, human approvals). Flow: the client declares the extension per-request; when a request will be long-running the server returns a `CreateTaskResult` (`resultType: "task"`) carrying a `taskId`, initial status, TTL, and suggested `pollIntervalMs`; the client polls `tasks/get`; if status becomes `input_required`, the response carries `inputRequests` which the client fulfils via `tasks/update`; terminal states are `completed` / `failed` / `cancelled`. The client **MAY** send `tasks/cancel` (cooperative). Servers **MAY** push `notifications/tasks` via `subscriptions/listen`; polling remains the default. Task IDs are durable and survive client disconnect/restart.
### Skills over MCP
An emerging extension (Skills Over MCP Working Group; current direction is SEP-2640, a Resources-based Extensions-Track proposal; reference work in `experimental-ext-skills`). It defines how "agent skills" — rich, structured instructions for agent workflows — are discovered, distributed, and consumed through MCP. **Not yet a finalized official extension** at the time of this draft.
### MCP Apps
Extension `io.modelcontextprotocol/ui` (repo `ext-apps`). Lets a server return interactive HTML (charts, forms, video players, dashboards) rendered inline in the conversation. A tool declares a UI reference via `_meta.ui.resourceUri` pointing at a `ui://` resource; the host fetches and renders the HTML inside a **sandboxed iframe**, isolated from the parent page. Communication uses a JSON-RPC dialect over `postMessage` with a `ui/` method prefix (e.g. `ui/initialize`); `_meta.ui.csp` and `_meta.ui.permissions` control loading and capabilities. Already supported by several clients (Claude, Claude Desktop, VS Code Copilot, Microsoft 365 Copilot, Goose, Postman, and others).
## Changes vs 2025-11-25
- **Stateless base protocol**: requests are self-contained; servers **MUST NOT** rely on connection/session state.
- **Per-request capability negotiation** replaces the `initialize` handshake; `server/discover` provides up-front discovery.
- **Polymorphic results** (`resultType`: `complete` / `input_required` / extension values) and a formal MRTR pattern via `InputRequiredResult`.
- **New spec-reserved error range** (`-32020`–`-32099`) and codes (`HeaderMismatch`, `MissingRequiredClientCapability`, `UnsupportedProtocolVersion`).
- **Elicitation is the only core client feature**; **Sampling and Roots are deprecated** (SEP-2577), retained ≥12 months, eligible for removal in the first revision released on/after **2027-07-28**. Migration: Roots → tool parameters/resource URIs/server configuration; Sampling → integrate directly with LLM provider APIs.
- **Logging deprecated** as a utility (SEP-2577) → `stderr` / OpenTelemetry.
- **Dynamic Client Registration deprecated** (→ Client ID Metadata Documents).
- **Extensions framework** introduced; **Tasks** stabilized as an official extension (`io.modelcontextprotocol/tasks`), **MCP Apps** added (`io.modelcontextprotocol/ui`), **Skills over MCP** incubating.
## Skald relevance
Skald's MCP surface lives in `crates/mcp-client/` (the `McpServer` stdio wrapper, `McpHttpServer`, and the `McpServerClient` trait), `src/core/mcp/` (`McpManager`), and `src/core/elicitation/` (`ElicitationManager` + bridge, surfaced in the Inbox). Two draft changes matter most for future planning: (1) the **stateless / per-request** shift means capability and version metadata must be attached to every request rather than assumed from a session, and `server/discover` becomes the discovery primitive; (2) the **Tasks extension** maps closely onto Skald's async/background work and would let MCP servers return durable, pollable handles for long-running operations. Skald's existing elicitation bridge already corresponds to the one remaining core client feature, so that path forward is well aligned. Until the draft finalizes, treat Sampling/Roots/Logging as deprecated-but-present for interop only.
## References
- [Specification (draft)](https://modelcontextprotocol.io/specification/draft)
- [Architecture](https://modelcontextprotocol.io/specification/draft/architecture)
- [Base protocol](https://modelcontextprotocol.io/specification/draft/basic)
- [Versioning and compatibility](https://modelcontextprotocol.io/specification/draft/basic/versioning)
- [Transports](https://modelcontextprotocol.io/specification/draft/basic/transports)
- [Authorization](https://modelcontextprotocol.io/specification/draft/basic/authorization)
- [Server features](https://modelcontextprotocol.io/specification/draft/server)
- [Client features — Elicitation](https://modelcontextprotocol.io/specification/draft/client/elicitation)
- [Deprecated features registry](https://modelcontextprotocol.io/specification/draft/deprecated)
- [Extensions overview](https://modelcontextprotocol.io/extensions/overview)
- [Tasks extension](https://modelcontextprotocol.io/extensions/tasks/overview)
- [MCP Apps extension](https://modelcontextprotocol.io/extensions/apps/overview)
- [Skills over MCP Working Group](https://modelcontextprotocol.io/community/working-groups/skills-over-mcp)
- [Schema (schema.ts)](https://github.com/modelcontextprotocol/specification/blob/main/schema/draft/schema.ts)
- [Release 2026-07-28-RC](https://github.com/modelcontextprotocol/modelcontextprotocol/releases/tag/2026-07-28-RC)
docs/mcp/specs/index.md
+74
View File
@@ -0,0 +1,74 @@
# MCP Specification Reference
Quick-reference mirrors of each **official Model Context Protocol specification revision**,
maintained as background for future Skald MCP work. The authoritative source of truth for
every revision is the TypeScript schema (`schema/<version>/schema.ts`) in
[`modelcontextprotocol/specification`](https://github.com/modelcontextprotocol/specification);
these notes summarize each revision's structure, requirements, and deltas for fast lookup
when implementing against `crates/mcp-client/` and `src/core/mcp/`.
Each file follows the same template — *At a glance → Architecture → Base Protocol
(Messages / Lifecycle / Transports / Authorization / Versioning) → Server Features →
Client Features → Changes vs previous → Skald relevance → References* — so revisions can be
diffed section by section.
## Revisions
| Revision | Status | Released | Protocol style | Headline | File |
| --- | --- | --- | --- | --- | --- |
| **2026-07-28** | **Draft / RC** | ~2026-05 (pre-release) | Stateless, per-request caps | Foundational redesign: stateless base, per-request negotiation, extensions framework; Sampling/Roots/Logging deprecated | [2026-07-28-draft.md](2026-07-28-draft.md) |
| **2025-11-25** | **Latest Stable** | 2025-11-25 | Stateful | OIDC Discovery, icons, URL-mode elicitation, sampling tool-calling, experimental Tasks | [2025-11-25.md](2025-11-25.md) |
| **2025-06-18** | Stable | 2025-06-18 | Stateful | Streamable HTTP, Elicitation, structured tool output, OAuth Resource Indicators (RFC 8707) | [2025-06-18.md](2025-06-18.md) |
| **2024-11-05** | Legacy | 2024-11-05 | Stateful | First public release: stdio + HTTP+SSE, Resources/Prompts/Tools/Logging, Sampling/Roots | [2024-11-05.md](2024-11-05.md) |
Additional repo tags not given their own file (point-revisions of the spec lines above):
- `2024-11-05-final` — final revision of the 2024-11-05 line (covered in [2024-11-05.md](2024-11-05.md)).
- `2024-10-07` — pre-public preliminary tag, superseded by the public 2024-11-05 release.
- `2025-03-26` — intermediate revision (deprecated the HTTP+SSE transport); superseded by 2025-06-18.
- `2025-11-25-RC` — release candidate of the 2025-11-25 line.
## Feature availability across revisions
| Capability | 2024-11-05 | 2025-06-18 | 2025-11-25 | 2026-07-28 (draft) |
| --- | :---: | :---: | :---: | :---: |
| **stdio transport** | ✔ | ✔ | ✔ | ✔ |
| **HTTP+SSE transport** | ✔ | ✘ (replaced) | ✘ | ✘ (deprecated since 2025-03-26) |
| **Streamable HTTP transport** | ✘ | ✔ | ✔ | ✔ |
| **Stateful connections** | ✔ | ✔ | ✔ | ✘ (stateless, self-contained requests) |
| **Per-request capability negotiation** | ✘ | ✘ | ✘ | ✔ |
| **Resources** | ✔ | ✔ | ✔ | ✔ |
| **Prompts** | ✔ | ✔ | ✔ | ✔ |
| **Tools** (structured output `outputSchema`/`structuredContent`) | ✘ | ✔ | ✔ | ✔ |
| **Tools** (unstructured `content[]` only) | ✔ | ✔ | ✔ | ✔ |
| **Logging** (server feature / utility) | ✔ | ✔ | ✔ | ✘ deprecated (→ stderr / OpenTelemetry) |
| **Icons metadata** | ✘ | ✘ | ✔ | ✔ |
| **Sampling** | ✔ | ✔ | ✔ | ✘ deprecated (SEP-2577) |
| **Roots** | ✔ | ✔ | ✔ | ✘ deprecated (SEP-2577) |
| **Elicitation** (form mode) | ✘ | ✔ | ✔ | ✔ (only core client feature) |
| **Elicitation** (URL mode) | ✘ | ✘ | ✔ | ✔ |
| **OAuth 2.1 authorization framework** | ✘ (not in core spec) | ✔ (RFC 8707, RFC 9728) | ✔ + OIDC Discovery + Client ID metadata | ✔ (DCR deprecated → Client ID metadata) |
| **JSON-RPC batching** | ✔ | ✘ (removed) | ✘ | ✘ |
| **Tasks** | ✘ | ✘ | experimental | extension `io.modelcontextprotocol/tasks` |
| **Extensions framework** | ✘ | ✘ | ✘ | ✔ |
## Feature-lifecycle policy (introduced in the 2026-07-28 draft)
The draft formalizes how features are deprecated and removed ([SEP-2577](https://modelcontextprotocol.io/specification/draft/deprecated)):
- A deprecated feature is retained for **at least 12 months** and remains usable for interop.
- It becomes **eligible for removal** in the first revision released on/after **2027-07-28**.
- Currently deprecated: **Sampling**, **Roots**, **Logging**, **Dynamic Client Registration**.
When implementing against the draft, treat these as present-but-don't-build-new-dependencies.
## Source of truth
- **Schema (authoritative):** [`schema/<version>/schema.ts`](https://github.com/modelcontextprotocol/specification/tree/main/schema) — the TypeScript schema is normative; the prose restates it with BCP 14 keywords (MUST / SHOULD / MAY).
- **Spec site:** [modelcontextprotocol.io/specification](https://modelcontextprotocol.io/specification) — per-version browsable spec.
- **Releases:** [github.com/modelcontextprotocol/modelcontextprotocol/releases](https://github.com/modelcontextprotocol/modelcontextprotocol/releases) — tags, RCs, changelogs.
## Related Skald docs
- [../index.md](../index.md) — MCP subsystem overview (servers, transports, integration)
- [../../mcp.md](../../mcp.md) — `McpManager`, transports, naming convention, enable/disable
docs/memory.md
+118
View File
@@ -0,0 +1,118 @@
# Memory System
`src/core/memory/mod.rs`
---
## Purpose
Provides a pluggable long-term memory layer for the LLM. Before every user turn,
the active memory backend is queried for relevant context and the result is
injected into the system prompt. Memory backends can also expose optional LLM
tools (e.g. `memory_query`).
The file-based MD memory (`data/memory/`) continues to exist in parallel and is
managed autonomously by the LLM via the `read_file` / `write_file` tools — it is
not part of this system.
---
## Memory Trait
```rust
#[async_trait]
pub trait Memory: Send + Sync {
fn id(&self) -> &str;
fn is_available(&self) -> bool;
async fn query_context(session_id: i64, user_message: &str) -> Option<String>;
fn tools(&self) -> Vec<Arc<dyn Tool>> { vec![] } // default: no tools
}
```
| Method | Description |
| --- | --- |
| `id()` | Unique backend identifier (e.g. `"honcho"`) |
| `is_available()` | `true` when the backend is up and ready. The manager skips unavailable backends silently. |
| `query_context()` | Returns a formatted string to inject into the system prompt, or `None` if nothing relevant is available (cold start, backend down, etc.) |
| `tools()` | Optional LLM-callable tools exposed by this backend. Called per turn. |
---
## MemoryManager
`Skald::memory_manager: Arc<MemoryManager>`
Holds **at most one** active backend.
### Singleton rule
| Situation | Result |
| --- | --- |
| No backend registered | New backend accepted, logged at `INFO` |
| Same id re-registers (plugin restart) | Old entry replaced, logged at `INFO` |
| Different id tries to register | Rejected with `error!`; existing backend kept |
### Methods
| Method | Description |
| --- | --- |
| `register(Arc<dyn Memory>)` | Register (or replace) a backend |
| `query_context(session_id, msg)` | Delegates to the backend if available; returns `None` otherwise |
| `tools()` | Returns backend tools if available; empty `Vec` otherwise |
| `tool_defs()` | OpenAI-format JSON definitions of the backend's tools |
---
## Integration in the LLM loop
### Context injection (read path)
In `ChatSessionHandler::handle_message`, **before** `build_agent_config`:
```
memory_manager.query_context(session_id, user_message)
→ Some(ctx) → prepended to extra_system_context
→ None → extra_system_context unchanged
```
Called for **all sessions** — interactive, cron, and tic alike. Automated agents
benefit from knowing user preferences and context just as much as interactive
ones (e.g. a cron agent that knows "Daniele prefers Italian" produces better
output). Only the **write path** filters by `is_interactive`/`is_ephemeral`.
### Tool dispatch (per turn)
`build_agent_config` calls `memory_manager.tools()` and stores the result in
`AgentRunConfig::memory_tools`. These tools are:
1. Added to the LLM's tool list via `all_tool_defs()` (after base + MCP tools,
before interface tools).
2. Dispatched in `run_agent_turn` before the global `ToolRegistry` fallthrough.
3. Inherited by sub-agents via `AgentRunConfig::for_sub_agent`.
---
## Adding a Memory Backend
1. Implement `Memory` on a struct (usually inside a plugin module).
2. In the plugin's `Plugin` trait impl, override `fn memory() -> Option<Arc<dyn Memory>>` to return `Some(...)`.
3. `PluginManager` calls `state.memory_manager.register(plugin.memory())` automatically after each successful `start()` / `reload()`.
No changes to `main.rs` or the session handler are needed.
---
## Current Backends
| Backend | Source | Plugin |
| --- | --- | --- |
| `honcho` | `src/core/plugin/honcho/mod.rs` | [honcho.md](honcho.md) |
---
## When to Update This File
- `Memory` trait methods change
- `MemoryManager` singleton rules change
- A new backend is added or removed
- Integration points in the session handler change
docs/notifications.md
+121
View File
@@ -0,0 +1,121 @@
# Notifications
## Overview
TIC is a background agent that runs every 15 minutes, reads incoming events (email, WhatsApp, Google Calendar) and decides what is worth surfacing to the user. Its filtering behaviour is controlled by the file `data/notifications.md`.
When TIC runs, that file is **automatically injected into its system prompt** (via `inject_memory` in `agents/tic/meta.json`). TIC reads the rules before evaluating each event batch.
---
## How notifications reach the main agent
TIC (and other background agents like cron workers) calls the `notify` interface tool with a **structured notification** — `{source, event_type, summary, event_time, refs}` (the `Notification` struct in `src/core/notification.rs`) — queued via `ChatHub::notify()`. The `summary` is a neutral, third-person statement of fact; the **main agent** owns the user-facing wording. The notification delivery chain:
1. `ChatHub::notification_consumer` batches notifications (200 ms window), then:
2. Appends a **synthetic Assistant message** to `chat_history` (`is_synthetic=true`) with a reasoning trace in `reasoning_content`:
> "The system signaled pending notifications. Let me read them and surface anything relevant to the user."
3. Inserts a **pre-completed tool call** in `chat_llm_tools`: `read_notification()` with `status='done'`, `result_type='json'`, and `result` containing the JSON array of notification objects.
4. Calls `ChatHub::resume()` → `resume_turn` detects the synthetic tool call on the last assistant message and runs the LLM loop.
5. The conversational agent (depth=0) sees the tool result as if it had just called `read_notification`, then writes the user-facing message — naming the source and adding context, rather than echoing the raw `summary`.
The `read_notification` tool is a real `Tool` registered in `ToolRegistry` (category `Introspection`, `root_agent_only=true`). When the LLM calls it normally, it returns an empty array `[]` — notifications are only ever injected synthetically by `ChatHub`. The tool is visible only to depth=0 agents (filtered out of sub-agent configs via `root_only_tool_names`).
This replaces the previous mechanism (synthetic User message with `[SYSTEM - NOTIFICATION]` prefix and `extra_system_dynamic` framing instructions). The new approach:
- Uses the **natural tool-call pattern** (tool → result → response)
- Applies to the **assistant itself** (synthetic reasoning + tool call)
- Avoids injecting fake user turns and behavioural framing in system messages
---
## How the main agent updates preferences
When the user says something like:
- *"Notify me when I get an email from Mario"*
- *"Stop alerting me about group WhatsApp messages"*
- *"Always tell me if a meeting starts in less than an hour"*
The main agent must **update `data/notifications.md`** directly using `edit_file` or `write_file`.
No restart is needed — TIC reads the file fresh on every tick.
---
## File format
The file is plain Markdown. TIC reads it as free prose, so any clear natural-language instruction works. The recommended structure is:
```markdown
# Notification preferences
## Always notify
- <rule>
- <rule>
## Never notify
- <rule>
- <rule>
## Custom rules
- <rule>
- <rule>
```
### Section semantics
| Section | Meaning |
|---|---|
| `## Always notify` | Events matching these rules should always be surfaced, even if they seem low-priority by default |
| `## Never notify` | Events matching these rules should be silently dropped, even if they seem important by default |
| `## Custom rules` | Everything else: conditional rules, time-based rules, contact-specific overrides |
---
## Examples
```markdown
## Always notify
- Emails from Kandice Phillips or anyone at Dawson Cornwell
- Any calendar event that was added today and starts within 24 hours
- WhatsApp messages from my sister (saved as "Valentina")
## Never notify
- Newsletters, marketing, automated digests
- LinkedIn notifications
- Calendar reminders for recurring weekly meetings
## Custom rules
- Only notify about WhatsApp group messages if I am explicitly mentioned
- For emails about the Serena legal matter, always notify regardless of sender
```
---
## How TIC uses this file
TIC reads the rules at the **Step 1 — Read memory** phase (the file is pre-injected in context). It then applies them during **Step 3 — Decide**:
- A rule in `Always notify` raises the threshold for suppression — TIC will surface the event unless something is clearly spam.
- A rule in `Never notify` acts as a hard filter — TIC drops the event without calling `notify`.
- `Custom rules` are evaluated as natural-language conditions. TIC is instructed to interpret them strictly and not over-generalise.
If the file does not exist or is empty, TIC falls back to its built-in heuristics (see `agents/tic/AGENT.md`).
---
## File location
```
data/notifications.md
```
This file lives in `data/` (not `data/memory/`) because it is system configuration, not personal memory. It is user-editable and version-control friendly.
---
## When to Update This File
- The notification delivery mechanism changes (e.g. `read_notification` tool, synthetic injection flow)
- TIC's behaviour or decision process is modified
- A new background agent that calls `notify` is introduced
- The `data/notifications.md` format or location changes
docs/plugins.md
+269
View File
@@ -0,0 +1,269 @@
# Plugin System
Plugins are long-running subsystems compiled into the binary and managed by `PluginManager`. They receive a `PluginContext` (a bundle of `Arc<dyn Trait>` deps) and run independently from the Axum web server.
---
## Plugin Trait
```rust
#[async_trait]
pub trait Plugin: Send + Sync {
fn id(&self) -> &str;
fn name(&self) -> &str;
fn description(&self) -> &str;
fn is_running(&self) -> bool;
async fn start(&self, ctx: PluginContext) -> Result<()>;
async fn reload(&self, enabled: bool, config: Value, ctx: PluginContext) -> Result<()>;
async fn stop(&self) -> Result<()>;
/// Optional: contribute one Axum router nested under `/api/plugin/<id>/`.
fn http_router(&self) -> Option<axum::Router> { None }
fn as_any(&self) -> &dyn Any;
}
```
`start()` and `reload()` must **spawn internal tasks and return immediately**.
`stop()` must cancel all internal tasks and **await their completion**.
`PluginContext` (`crates/core-api/src/plugin.rs`) carries typed `Arc<dyn Trait>` deps sourced from `Skald` — plugins use only what they need. Beyond the registry/provider deps, it includes:
- `db: Arc<sqlx::SqlitePool>` — Skald's shared SQLite pool (create/use plugin-owned tables here).
- `inbox: Arc<dyn InboxApi>` — unified approvals + clarifications façade (`list_pending`, `approve`, `reject`, `answer`); idempotent by `request_id`.
- `chat_hub.events(...)` — subscribe to the global `GlobalEvent` bus (including the four Inbox events; see [approval/index.md](approval/index.md)).
### Plugin HTTP routes (`http_router`)
A plugin can mount HTTP routes by overriding `http_router()` (default `None`). After `start_enabled()`, `WebFrontend::start` collects routers via `PluginManager::collect_plugin_routers()` and nests each enabled plugin's router under `/api/plugin/<id>/`, behind Skald's normal auth. The router must close over the plugin's own state (no axum `State` is passed). Only routes — no nav entries / JS assets / Lit components. The mesh-facing router (`router_factory`) does not include plugin routes.
---
## PluginManager — `src/core/plugin/mod.rs`
| Method | Purpose |
|---|---|
| `register(plugin)` | Add a plugin before startup (build phase) |
| `set_skald(Arc<Skald>)` | Wire core after `Skald` is built |
| `set_router_factory(factory)` | Provide Axum router factory (called by `WebFrontend`) |
| `set_web_port(port)` | Provide HTTP port for plugins that need it (called by `WebFrontend`) |
| `start_enabled()` | Start all DB-enabled plugins |
| `collect_plugin_routers()` | Gather `(id, Router)` of enabled plugins to nest under `/api/plugin/<id>/` (called by `WebFrontend` after `start_enabled`) |
| `stop_all()` | Graceful shutdown on SIGINT |
| `toggle(id, enabled)` | Enable/disable and start/stop at runtime |
| `list()` | `Vec<PluginInfo>` with enabled + running flags |
### Startup sequence
```
main.rs:
plugins = vec![Arc::new(HonchoPlugin::new()), Arc::new(TelegramPlugin::new(...)), …]
Skald::new(core_cfg, plugins):
PluginManager::new(pool) → register_arc(plugin) for each plugin → Arc::new(plugin_manager)
→ build tool_registry (list_items / toggle_item reference Arc<PluginManager>)
→ Arc::new(Skald { … })
→ plugin_manager.set_skald(Arc::clone(&skald))
WebFrontend::start():
→ plugin_manager.set_router_factory(factory)
→ plugin_manager.set_web_port(port)
→ plugin_manager.start_enabled().await
→ plugin_manager.collect_plugin_routers().await // nest under /api/plugin/<id>/
```
### Enabled/disabled persistence
Plugin state and configuration are stored exclusively in the `plugins` SQLite table (`id TEXT PK, enabled INTEGER, config TEXT`). `config.yml` has no plugin section — plugins are configured at runtime via the REST API (`PUT /api/plugins/{id}`) or by asking the agent to use `toggle_item` (kind=plugin).
---
## Adding a New Plugin
Plugins live in independent workspace crates (see [crates/workspace.md](crates/workspace.md)):
1. Create `crates/plugin-<name>/` with a `Cargo.toml` depending on `core-api` and any needed external crates.
2. Implement `core_api::plugin::Plugin` in `crates/plugin-<name>/src/lib.rs`.
3. Add the crate to the workspace `members` list and as a dependency in the main `Cargo.toml`.
4. In `src/main.rs`, add `Arc::new(plugin_name::MyPlugin::new(...))` to the `plugins` vec before `Skald::new()`.
5. Rebuild — no restart needed for toggle.
---
## LLM Tools
| Tool | Description |
|---|---|
| `list_items` (type=plugins) | All plugins with enabled + running status |
| `toggle_item` (kind=plugin) | Enable/disable a plugin by id (immediate + persisted) |
| `configure_plugin` | Update a plugin's config JSON and restart it immediately |
### Plugin-provided global LLM tools
The `Plugin` trait has **no** `tools()` method: it cannot register global LLM tools by itself. The pattern (used by `mobile-connector`) is:
1. The plugin exposes a domain control trait (e.g. `RelayAgent`) and implements it on the plugin struct.
2. The plugin crate exports a factory `fn xxx_tools(agent: Arc<dyn Trait>) -> Vec<Arc<dyn Tool>>`.
3. In `Tools::build` (`src/core/skald/bundles.rs`), while the registry is being assembled, the plugin is fetched via `PluginManager::get_plugin_typed::<P>()`, cast to the trait, and its tools are registered with `ToolRegistry::register_arc(...)`.
The tools call into the plugin lazily, so they can be registered before the plugin's runloop starts (they return an error while the plugin is stopped). This keeps the `core-api` `Plugin` trait minimal while still allowing tool-based control surfaces (plugin.md §11).
---
## Available Plugins
| Plugin id | Source | Doc |
|---|---|---|
| `honcho` | `crates/plugin-honcho/src/lib.rs` | [honcho.md](honcho.md) |
| `remote_connectivity` | `crates/plugin-tailscale-remote/src/lib.rs` | [remote.md](remote.md) |
| `telegram` | `crates/plugin-telegram-bot/src/lib.rs` | [plugins/telegram.md](plugins/telegram.md) |
| `whisper_local` | `crates/plugin-transcribe-whisper-local/src/lib.rs` | [whisper-local.md](whisper-local.md) |
| `comfyui` | `crates/plugin-comfyui/src/lib.rs` | [providers/image.md](providers/image.md) |
| `mobile-connector` | `crates/plugin-mobile-connector/src/lib.rs` | [mobile-connector.md](mobile-connector.md) |
---
## Transcribe Providers and TranscribeManager
Speech-to-Text is decoupled from the plugin system via `TranscribeManager` (`src/core/transcribe/mod.rs`).
Plugins that provide transcription (e.g. `whisper_local`) register an `Arc<dyn Transcribe>` in `skald.transcribe_manager` at `start()` and deregister at `stop()`. Non-plugin providers (e.g. a future OpenRouter client) can register directly at startup without needing a full plugin lifecycle.
```rust
// trait — src/core/transcribe/mod.rs
pub trait Transcribe: Send + Sync {
fn id(&self) -> &str;
async fn transcribe(&self, audio: Vec<u8>, format: &str) -> Result<String>;
}
```
| Method | Purpose |
|---|---|
| `register(Arc<dyn Transcribe>)` | Add/replace a provider by id |
| `unregister(id)` | Remove a provider |
| `get()` | Returns the first available provider |
Selection strategy is currently **first registered**. Callers (e.g. Telegram) ask `skald.transcribe_manager.get().await` — they never reference a concrete type.
See [whisper-local.md](whisper-local.md) for the only current provider.
---
## Image Generators and ImageGeneratorManager
Image generation is decoupled from the plugin system via `ImageGeneratorManager` (`src/core/image_generate/`) and two traits in `core-api::image_generate` — same split as `TranscribeProvider` / `TranscribeRegistry`.
Two kinds of providers coexist:
| Kind | Source | Example |
| --- | --- | --- |
| **DB-backed** | `image_generate_models` table, built from `llm_providers` credentials | OpenRouter `x-ai/grok-2-vision` |
| **Plugin-registered** | Ephemeral — registered at runtime by plugins | ComfyUI workflows |
Plugins register via `ctx.image_generate_registry` (type `Arc<dyn ImageGenerateRegistry>`) in `PluginContext`. No dependency on the main crate is needed — only `core-api`.
```rust
// crates/core-api/src/image_generate.rs
pub trait ImageGenerate: Send + Sync { fn id(&self) -> &str; fn name(&self) -> &str; async fn generate(&self, prompt: &str) -> Result<Vec<u8>>; }
pub trait ImageGenerateRegistry: Send + Sync { async fn register(&self, provider: Arc<dyn ImageGenerate>); async fn unregister(&self, id: &str); }
```
| Method | Purpose |
|---|---|
| `ctx.image_generate_registry.register(...)` | Add a plugin provider (ephemeral) |
| `ctx.image_generate_registry.unregister(id)` | Remove a plugin provider |
| `add_model / update_model / delete_model` | DB-backed CRUD (called by REST API) |
| `list()` | Returns all active providers (plugin + DB) — used by LLM tool |
| `get(id)` | Returns a specific provider by id |
| `generate(provider_id, prompt)` | Generates and saves image, returns `(PathBuf, url)` |
The LLM interacts with providers via two tools: `image_generate_providers_list` and `image_generate`. See [providers/image.md](providers/image.md) for the full flow.
---
## TTS and TtsManager
Text-to-speech follows the same split pattern as transcribe and image_generate. `TtsManager` (`src/core/tts/`) manages both DB-backed and plugin-registered providers. Traits live in `core-api::tts`.
| Kind | Source | Example |
| --- | --- | --- |
| **DB-backed** | `tts_models` table, built from `llm_providers` credentials | OpenAI `tts-1-hd` |
| **Plugin-registered** | Ephemeral — registered at runtime by plugins | `OrpheusTtsPlugin` |
Plugins register via `ctx.tts_registry` (type `Arc<dyn TtsRegistry>`) in `PluginContext`.
```rust
// crates/core-api/src/tts.rs
pub trait TextToSpeech: Send + Sync {
fn id(&self) -> &str;
fn name(&self) -> &str;
fn description(&self) -> Option<&str>;
fn instructions(&self) -> Option<&str>; // default voice style
async fn synthesize(&self, text: &str, instructions: Option<&str>) -> Result<Vec<u8>>;
}
pub trait TtsRegistry: Send + Sync {
async fn register(&self, provider: Arc<dyn TextToSpeech>);
async fn unregister(&self, id: &str);
}
```
| Method | Purpose |
|---|---|
| `ctx.tts_registry.register(...)` | Add a plugin TTS provider (ephemeral) |
| `ctx.tts_registry.unregister(id)` | Remove a plugin provider |
See [providers/tts.md](providers/tts.md) for the full manager API and DB schema.
---
## ApiProvider and ApiProviderRegistry
Plugins that supply an `ApiProvider` (e.g. a cloud TTS + Transcribe integration without subprocess) register via `ctx.api_provider_registry`:
```rust
// crates/core-api/src/provider.rs
#[async_trait]
pub trait ApiProvider: Send + Sync {
fn type_id(&self) -> &'static str;
fn display_name(&self) -> &'static str;
fn supported_types(&self)-> &'static [ServiceType];
async fn list_tts_models(&self, record: &LlmProviderRecord) -> Result<Option<Vec<RemoteTtsModelInfo>>>;
async fn list_transcribe_models(&self, ..) -> ..;
async fn list_llm_models(&self, ..) -> ..;
fn build_tts(&self, ..) -> Option<Result<Arc<dyn TextToSpeech>>>;
fn build_transcriber(&self, ..) -> Option<Result<Arc<dyn Transcribe>>>;
fn build_llm(&self, ..) -> Option<Result<BuiltLlmClient>>;
fn build_image_generator(&self, ..) -> Option<Result<Arc<dyn ImageGenerate>>>;
fn ui_meta(&self) -> ProviderUiMeta;
}
pub trait ApiProviderRegistry: Send + Sync {
fn register_plugin(&self, provider: Arc<dyn ApiProvider>);
fn unregister_plugin(&self, type_id: &str);
}
```
`ProviderRegistry` in `src/core/provider/mod.rs` implements `ApiProviderRegistry`. It is exposed as `ctx.api_provider_registry` in `PluginContext`.
Plugin-registered providers shadow builtin ones: `ProviderRegistry::get(type_id)` checks the plugin list first.
**When to use `ApiProvider` vs `TtsRegistry`/`TranscribeRegistry`:**
- Use `TtsRegistry` / `TranscribeRegistry` for ephemeral local engines (subprocess, on-device model) that don't rely on the `llm_providers` credential DB.
- Use `ApiProvider` for cloud services that the user configures via the LLM-providers UI (API key stored in `llm_providers`, models managed via `tts_models` / `transcribe_models` CRUD).
All types used by `ApiProvider` (`LlmProviderRecord`, `LlmModelRecord`, `TtsModelRecord`, `TranscribeModelRecord`, `ImageGenerateModelRecord`, `RemoteLlmModelInfo`, `RemoteTtsModelInfo`, `RemoteTranscribeModelInfo`, `LlmStrength`) live in `core-api::provider` or their respective `core-api` modules. Plugin crates depend only on `core-api`.
---
## Plugin catalogue
| Plugin ID | Crate | Description |
|---|---|---|
| `honcho` | `crates/plugin-honcho` | Honcho long-term memory backend |
| `telegram_bot` | `crates/plugin-telegram-bot` | Private Telegram bot interface |
| `whisper_local` | `crates/plugin-transcribe-whisper-local` | Local STT via whisper.cpp |
| `tailscale_remote` | `crates/plugin-tailscale-remote` | Remote access via Tailscale mesh |
| `comfyui` | `crates/plugin-comfyui` | ComfyUI image generation workflows |
| `orpheus_tts_3b` | `crates/plugin-tts-orpheus-3b` | Local TTS via Orpheus 3B (Python subprocess) |
| `kokoro_tts` | `crates/plugin-tts-kokoro` | Local TTS via Kokoro ONNX (lightweight, multilingual) |
| `elevenlabs` | `crates/plugin-elevenlabs` | ElevenLabs cloud TTS and transcription (ApiProvider) |
| `mobile-connector` | `crates/plugin-mobile-connector` | Bridges the Inbox to mobile apps over the relay (E2E encrypted) — see [mobile-connector.md](mobile-connector.md) |
docs/plugins/honcho.md
+209
View File
@@ -0,0 +1,209 @@
# Honcho Memory Plugin
Plugin: `crates/plugin-honcho/src/lib.rs`
HTTP client: `crates/honcho-client/` (separate workspace crate)
---
## Purpose
Streams completed chat turns to a [Honcho](https://honcho.dev) server so that it can extract long-term conclusions about the user (write path), and reads that context back into every LLM turn via the [`Memory`](memory.md) trait (read path).
---
## Self-hosted Docker package
A ready-to-run Docker Compose setup is in the [`honcho/`](../honcho/) folder at the project root.
It starts four services: the Honcho API, the deriver background worker, PostgreSQL + pgvector, and Redis.
**Quick start:**
```sh
cd honcho
cp .env.example .env
# Edit .env — set at least LLM_OPENAI_API_KEY=sk-...
docker compose up -d
# API available at http://localhost:8000
```
Full instructions, LLM provider options (OpenAI, OpenRouter, Ollama), and troubleshooting are in [`honcho/README.md`](../honcho/README.md).
---
## Setup
1. Start the Honcho server (see above).
2. Enable the plugin via the agent or REST API:
```json
PUT /api/plugins/honcho
{
"enabled": true,
"config": {
"base_url": "http://localhost:8000",
"api_key": "",
"workspace_id": "personal-agent"
}
}
```
Or ask the main agent: _"enable the honcho plugin"_.
---
## Configuration
Stored in the `plugins` SQLite table (`config` JSON blob). Managed at runtime — no entry in `config.yml`.
| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `base_url` | string | `http://localhost:8000` | Honcho server URL |
| `api_key` | string | _(empty)_ | API key; leave empty for local/unauthenticated instances |
| `workspace_id` | string | `personal-agent` | Honcho workspace identifier for this agent instance |
---
## Honcho Object Model
```
workspace (workspace_id from config — one per agent instance)
├── peer "user" observe_others=true
├── peer "assistant" observe_me=true
└── session one per local chat_sessions.id
├── message peer_id="user"
├── message peer_id="assistant"
└── …
```
**Workspace and peers** are created (idempotently) each time the plugin starts. If they already exist, the API returns an error which is logged at `WARN`/`DEBUG` and ignored.
**Sessions** are created lazily on the first event for a new `chat_sessions.id`, then cached in memory for the life of the listener task. The Honcho session UUID is stored in the session cache but not persisted to SQLite — restarting the plugin creates new Honcho sessions for subsequent events.
---
## Event Filtering
An event is forwarded only when **all** of the following conditions hold:
| Condition | Reason |
| --- | --- |
| `is_interactive = true` | A real user is in the conversation |
| `is_ephemeral = false` | Not a short-lived automated session (cron, tic) |
| `is_synthetic = false` | Message content was typed by the user, not injected by the system |
| `role` is `User` or `Assistant` | Sub-agent messages (`Agent` role) are skipped |
| `content` is non-empty | Guard against empty strings |
---
## Lifecycle
1. **`start()`** — subscribes to `skald.event_bus`, calls `ensure_workspace_ready` (best-effort), then spawns the listener task.
2. **Listener task** — `tokio::select!` loop on the bus receiver and a `CancellationToken`. On `RecvError::Lagged`, logs a warning and continues (some turns are missed but the task stays alive).
3. **`stop()`** — cancels the token and awaits the task.
4. **`reload()`** — follows the standard plugin pattern: start/stop/restart-on-change.
---
## Error Handling
All Honcho API errors are **fire-and-forget**: logged as `warn!` and never propagated to the session handler or the user. A Honcho outage has zero impact on chat functionality.
`HonchoError::Request`'s `Display` walks the full `source()` chain, so transport
failures surface the real cause in logs (e.g. `Request failed: error sending
request for url (...): Connection reset by peer`) instead of just reqwest's
opaque top-line. This makes host↔container issues (e.g. a stale Docker Desktop
port-forward after a container recreation) diagnosable from the `warn!` alone.
---
## Read Path
`HonchoMemory` implements the [`Memory`](../memory.md) trait. Before each LLM turn,
`query_context` is called automatically by `ChatSessionHandler::handle_message` — for
**all** session types: interactive, cron, and tic.
### Flow
1. Checks `is_available()` — returns `None` immediately if the plugin is stopped.
2. Looks up the Honcho session UUID for the local `session_id` in the shared `session_map`.
3. **If a mapping exists** (interactive session with at least one turn written):
- Calls `client.session_context(workspace_id, honcho_session_id, tokens=2000, search_query=user_msg)`.
- Returns the formatted result on success.
- On error: logs `warn!` and falls through to the peer-context fallback **without** `search_query`
(avoids a second embedding of the same user message — `session_context` already embedded it before failing).
4. **Fallback — `peer_context("user")`** (no mapping, or session_context error):
- Cold start / cron / tic (no `session_map` entry): calls with `search_query=user_msg` for relevance.
- After a `session_context` failure: calls **without** `search_query` to avoid double-embedding.
- Returns global user knowledge derived from all sessions Honcho has observed.
- On error: logs `warn!` and returns `None`.
The formatted context is prepended to `extra_system_context` and injected into the system prompt. Errors are never propagated — they degrade gracefully to `None`.
### Context format
`format_context()` extracts, in priority order:
1. `conclusions[].content` → "Known facts about the user: …"
2. `summary` → "Conversation summary: …"
3. Fallback: pretty-printed raw JSON
The result is wrapped in `--- Honcho memory context --- / --- end ---` markers.
---
## LLM-callable Tools
`HonchoMemory::tools()` returns **five** tools whenever the plugin is active
(`is_available()` true). They give the LLM direct, on-demand access to every
layer of Honcho's API, complementing the automatic pre-turn `query_context`
injection. All operate on the `user` peer and are inherited by sub-agents via
`AgentRunConfig::memory_tools`.
The [official Honcho documentation](https://honcho.dev/docs/v3/documentation/features/chat) recommends exposing these as tools so the agent decides on its own when to read or write memory, rather than only relying on automatic injection.
| Tool | Endpoint | Cost | What it does |
| --- | --- | --- | --- |
| `memory_query` | `POST .../peers/user/chat` | High (LLM synthesis) | Natural-language question → synthesized answer (dialectic reasoning, `reasoning_level=low`) |
| `honcho_search` | `GET .../peers/user/context?search_query=…` | Low | Semantic search over derived facts; returns raw ranked excerpts (with ids when present) |
| `honcho_context` | `GET .../peers/user/context` | Low | Full context snapshot (conclusions + summary), no synthesis; optional focus `query` |
| `honcho_profile` | `GET`/`PUT .../peers/user/card` | Low | Read the peer card, or overwrite it with a list of fact strings (`card`) |
| `honcho_conclude` | `POST .../conclusions` / `DELETE .../conclusions/{id}` | Low | Write a new fact (`conclusion`) or delete one by id (`delete_id`); exactly one required |
**Peer model — all tools operate on the `user` peer as both observer and observed.**
This plugin configures the `user` peer with `observe_me = true`, so the user's
self-knowledge lives in the `observer = user / observed = user` slot. Therefore
`honcho_conclude` writes with `observer_id = observed_id = user`, and `honcho_search`
uses `peer_context` (not the `conclusions/query` endpoint, which requires explicit
observer/observed filters) — the same proven path as the automatic read-path
injection. This differs from setups where the assistant observes the user
(`observer = assistant`); keeping observer = user is what lets the read-path see
facts written by `honcho_conclude`.
**When to use vs. the automatic injection:**
| Mechanism | When it fires | Best for |
| --- | --- | --- |
| `query_context` (auto) | Before every LLM turn | Background context, cold-start facts |
| `memory_query` (tool) | LLM calls it explicitly | On-demand deep reasoning mid-conversation |
| `honcho_search` / `honcho_context` (tools) | LLM calls them explicitly | Cheap raw recall without LLM synthesis |
| `honcho_profile` / `honcho_conclude` (tools) | LLM calls them explicitly | Actively curating long-term memory |
**Implementation note:** `Tool::execute` is synchronous but the Honcho calls are
async. All five tools share the `run_blocking` helper, which uses
`tokio::task::block_in_place` + `Handle::current().block_on(...)` to drive the
future from within the Tokio multi-thread scheduler without spawning a new thread.
---
## Future Work
- **Session persistence** — store the Honcho session UUID in a new `chat_sessions.honcho_session_id` column so the mapping survives a plugin restart.
---
## When to Update This File
- Config fields change
- Honcho object model or peer setup changes
- Filtering rules change
- `query_context` flow changes (session vs peer fallback logic)
- Docker Compose setup in `honcho/` changes significantly
- Public API of `crates/honcho-client/` changes
docs/plugins/index.md
+15
View File
@@ -0,0 +1,15 @@
# Plugin System
Extensible plugin architecture: lifecycle, trait contract, built-in plugins.
## Files
- [../plugins.md](../plugins.md) — Plugin trait, PluginManager, HTTP router integration
- **Built-in Plugins:**
- [honcho.md](honcho.md) — Honcho long-term memory plugin: setup, config, filtering, lifecycle
- [mobile-connector.md](mobile-connector.md) — Mobile app relay bridge, E2E encryption, Inbox sync (v2 protocol)
- [telegram.md](telegram.md) — Telegram bot: setup, pairing, whitelist, HITL approval
- [whisper-local.md](whisper-local.md) — Local STT via whisper.cpp (Metal-accelerated)
- [remote.md](remote.md) — Tailscale mesh remote connectivity
See [../index.md#plugin-system](../index.md#plugin-system) for navigation.
docs/plugins/mobile-connector.md
+288
View File
@@ -0,0 +1,288 @@
# Mobile Connector Plugin (`mobile-connector`)
Bridges Skald's **Inbox** (approvals + clarifications + MCP elicitations) to mobile apps over the
**relay**, implementing the **agent** role of the v2 relay protocol. The plugin is
the namespace owner and the sole authority over authorized devices. Skald is
never exposed on the internet: only this plugin connects out, and only to the
relay.
- Crate: `crates/plugin-mobile-connector` — the **application** layer (thin).
- Networking: `crates/skald-relay-client` — the **standalone, payload-agnostic**
relay client (WS v2 transport, E2E crypto, anti-replay counters, pairing,
device authorization, SQLite persistence). It depends only on
`skald-relay-common` (never on `core-api`), so it is reusable and unit/
integration-tested in isolation. See [Crate split](#crate-split-skald-relay-client).
- Shared crypto + protobuf: `crates/skald-relay-common` (byte-for-byte interop
with the reference vectors in [relay/test-vectors.md](../relay/test-vectors.md))
- **Protocol documentation** (canonical, in English):
- [relay/index.md](../relay/index.md) — architecture, actors, threat model
- [relay/relay-protocol.md](../relay/relay-protocol.md) — protobuf schema, auth, pairing, live channel, presence
- [relay/framing.md](../relay/framing.md) — E2E plaintext framing (version + compression)
- [relay/payloads.md](../relay/payloads.md) — JSON payload schemas (inbox_request, inbox_update, …)
- [relay/crypto.md](../relay/crypto.md) — crypto contract, key derivation, AEAD, anti-replay
---
## Crate split: skald-relay-client
The plugin is the **application** layer; all networking lives in the standalone
`skald-relay-client` crate. The boundary is **payload-agnostic**: the client
exchanges opaque decrypted `Vec<u8>` payloads keyed by device pubkey and emits
inbound traffic as `RelayEvent`s; it never interprets the JSON. The plugin
consumes `client.events()`, applies the JSON schemas (`payloads.rs`) and the
`InboxApi`, and calls `client.send(dest, bytes, live)`.
Consequences of the split:
- **Authorization policy stays in the plugin.** On `RelayEvent::ClientPaired`,
the client has already derived the `aes_key`, persisted the device as Pending,
and consumed the pairing token. The plugin's event loop decides: if
`require_device_confirmation` it notifies; otherwise it calls
`client.authorize(ed)` and then `broadcast_inbox()`.
- **`client.authorize()` is payload-agnostic** — it does NOT push an Inbox
snapshot. The plugin sends the snapshot after authorizing (both the auto path
and the `RelayAgent::authorize_client` tool path).
- **v2 framing (`compress/decompress_payload`) is transport** — handled inside
the client, so the plugin only ever sees clean JSON.
- **Identity seed** is injected via `SeedSource::Path("data/relay/seed")` (same
relative path as before) so existing identities/devices survive the upgrade.
### Module map — `skald-relay-client` (networking)
| Module | Role |
|---|---|
| `config.rs` | `RelayClientConfig` + `SeedSource` (`Bytes` / `Path`) |
| `events.rs` | `RelayEvent` (`Connected`/`Disconnected`/`Message`/`ClientPaired`/`ClientRevoked`), broadcast |
| `identity.rs` | Seed load/generate (`0600`, injected path) + derived Ed25519/X25519 keys + `namespace_id` |
| `db.rs` | `relay_clients` table — devices + anti-replay counters (atomic counter helpers, `delete_all`) |
| `pairing.rs` | In-memory single-window pairing sessions (`code → session`) + `QrCodeData` |
| `state.rs` | Networking-only runtime: per-client `aes_key` cache, seal/open, counters, emits events |
| `ws.rs` | Permanent reconnecting agent WebSocket (v2 binary transport). Challenge → `Auth` → role dispatch → forward loop |
| `client.rs` | `RelayClient` — the public façade (`new`/`start`/`shutdown`/`send`/pairing/authorize/revoke/`clear_all`/`events`) |
### Module map — `plugin-mobile-connector` (application)
| Module | Role |
|---|---|
| `payloads.rs` | E2E JSON payload schemas (`inbox_update`, `notification`, client responses incl. `inbox_request`). Zlib-compressible per v2 framing.md |
| `app.rs` | `RelayApp`: Inbox dispatch (`broadcast_inbox`/`apply_client_payload`), authorization policy, the `events()` consumer loop |
| `notifier.rs` | `DelayedNotifier`: debounces phone pushes (`notify_delay_secs`) so resolving on the computer suppresses the push. See [Delayed push](#delayed-push) |
| `proxy.rs` | Accepts relay **pipes** of `stream_type = "http-local-proxy"` and reverse-proxies each to `127.0.0.1:<web_port>`. See [HTTP reverse proxy](#http-reverse-proxy-http-local-proxy) |
| `router.rs` | The QR-code HTTP endpoint (`/pairingqrcode`), resolves the current `RelayApp` → `client.lookup_pairing` |
| `agent.rs` | `RelayAgent` control trait (pairing, list, authorize, revoke) |
| `tools.rs` | The three LLM tools, registered in the main crate's `ToolRegistry` |
| `lib.rs` | `MobileConnectorPlugin` (`Plugin` + `RelayAgent`), lifecycle, bus subscriber, `RelayClient`/`RelayApp` wiring |
---
## Configuration
Stored in the `plugins` table (JSON, edited via the plugin UI / `configure_plugin`):
```yaml
relay_url: "wss://relay.skaldagent.net/v1/ws" # empty ⇒ plugin idle (no WS)
pairing_ttl: 300 # seconds, max 600
require_device_confirmation: true # manual confirm new devices (recommended)
notify_delay_secs: 20 # debounce before pushing to the phone (0 = immediate)
```
`enabled` (the standard plugin flag) starts/stops the runloop.
`notify_delay_secs` debounces the **phone push** for a new approval/clarification
(see [Delayed push](#delayed-push)). The mobile push is only useful when you're
away from the computer; if you answer at the computer within the window, no phone
notification is sent. Set `0` to push immediately.
---
## Persistence (plugin.md §9)
| Data | Location | Why |
|---|---|---|
| `seed` (32 B) | filesystem `data/relay/seed`, `0600` | the only persistent secret; keys + `namespace_id` are derived at runtime |
| Pairing session | **in-memory** only | transient (≤ TTL); lost on restart ⇒ just re-pair |
| Devices + `send/recv_counter` | DB `relay_clients` | **must** survive restarts |
### Why counters live in the DB
Skald self-restarts by design. If counters reset to 0 on restart:
- `send_counter → 0` reuses an AES-GCM nonce under the same key (breaks
confidentiality + integrity for that device).
- `recv_counter → 0` re-opens the replay window.
So `send_counter` is incremented **and persisted before** sealing/sending
(`db::next_send_counter`, a transaction), and `recv_counter` is persisted only
**after** a valid `open`.
### `aes_key` cache
The per-client AES-256-GCM key is `HKDF(X25519(seed_x_priv, client_x_pub))`. It
is derived once and cached in memory (`HashMap<ed25519_pub, aes_key>` in
`RelayState`), never persisted; on a cache miss it is re-derived from the
client's stored `x25519_pub`. The cache entry is dropped on revoke.
---
## Pairing flow
1. The agent calls `mobile_start_pairing(ttl?)` (gated behind approval).
2. The plugin generates a 32-byte `pairing_token` (CSPRNG), sends
`pairing_start{token, ttl}` to the relay, and registers an in-memory session
keyed by a separate random `code` (latest-wins: any prior active session is
marked *Superseded*). It returns the URL
`/api/plugin/mobile-connector/pairingqrcode?code=<code>`.
3. The copilot renders the URL as an image. The endpoint serves a PNG of the QR
while the session is **Active**, else a placeholder (`QR scaduto` /
`QR già usato`). The QR payload is the normative `QrCodeData` JSON (never on
disk, never in the URL).
4. The client scans, connects as `role:"pairing"`, the relay consumes the token
and forwards `client_paired` to the agent.
5. On `client_paired`: derive + cache `aes_key`, persist the client as
**Pending** (counters 0), mark the session **Consumed**, then apply the
policy:
- `require_device_confirmation = false` ⇒ auto-authorize.
- `require_device_confirmation = true` ⇒ leave Pending; the human authorizes
via the control surface (a `notification` is pushed to existing devices).
`authorize` always reflects the full local set (replacement semantics): adding a
device sends the complete list including it; revoking sends it without.
---
## Message flows
- **Inbox → clients:** the bus subscriber reacts to the six Inbox events
(`approval_requested`, `approval_resolved`, `clarification_requested`,
`clarification_resolved`, `elicitation_requested`, `elicitation_resolved`) and
routes them through the **debouncer** (see
[Delayed push](#delayed-push)) before building an `InboxSnapshot` via
`inbox.list_pending()` and sending a sealed `inbox_update` to every Authorized
client. Each approval
carries a humanised `summary` (from `Tool::describe(Short)`, computed in
`Inbox::list_pending`) for the card/notification plus the raw `arguments`
(untruncated) for the detail dialog — so the user sees the full `execute_cmd`
command, not a truncated label. Each clarification carries its
`suggested_answers`. Each elicitation carries **only** its prompt metadata
(`server_name`, `message`, `field_name`, `sensitive`, `is_confirmation`) — never
a value; the value is supplied by the device in `elicitation_response.content`.
- **Clients → Inbox:** inbound `message` is checked (`from` ∈ Authorized,
nonce direction + counter > `recv_counter`), opened, and dispatched by `kind`:
`approval_response` → `inbox.approve/reject`, `clarification_response` →
`inbox.answer`, `elicitation_response` → `inbox.resolve_elicitation` (its
`content` may be a secret — never logged/persisted in clear), `hello` → persist
`device_info`, `inbox_request` → send a **targeted** `inbox_update` back to
`from` only (see below), `logout` → revoke.
After any response the Inbox is re-snapshotted. `request_id` is mapped
`string ↔ i64` (non-parsing ids are dropped). Inbox ops are idempotent by
`request_id`.
- **Reconnect snapshot (`inbox_request`):** the relay does **not** notify the
agent when a client reconnects, so the client sends `inbox_request` on the
**live channel** (`Message.live=true`) after every `auth_ok` (e.g. when the app
is opened from a push). The agent replies with an `inbox_update` sealed to the
requester only — not a broadcast — so other devices are not needlessly
re-aligned. A pull of stale state is useless, so the live channel is correct:
if the agent is offline, the client gets `PeerOffline` immediately instead of
waiting. Side-effect-free and idempotent (by `request_id`). See
`data/ios-app/v2/relay-protocol.md` §3.1.
---
## HTTP reverse proxy (`http-local-proxy`)
So a remote device can reach Skald's web UI **without** a VPN/Tailscale or an open
port, the plugin reuses the relay **pipe** (relayed E2E byte-stream, see
[relay/pipe.md](../relay/pipe.md)) as a reverse proxy to the local HTTP server.
`proxy.rs` subscribes to `RelayClient::incoming_pipes()` and, for each invite with
`stream_type == "http-local-proxy"`, accepts the pipe and splices it byte-for-byte
to a **fresh** `TcpStream` to `127.0.0.1:<web_port>` (`PluginContext::web_port`).
Per pipe it `split`s the connection into independent send/receive halves and runs
each direction in its own task (full-duplex, so neither blocks the other;
`PipeSender::send` is backpressured by the pipe's ~10 MiB send buffer, and
`recv`/`read` are cancel-safe — see [relay/pipe.md §6.1](../relay/pipe.md#61-full-duplex--client-side-backpressure)).
When either direction ends it cancels a shared token so the other unwinds. Invites
of other `stream_type`s are **ignored** (not rejected) since `incoming_pipes` is a
broadcast shared with possible future consumers.
The native app side (later) opens one pipe per outbound connection and points a
WebView at it; because the tunnel is a transparent TCP splice, HTTP/1.1 keep-alive,
parallel connections, and the chat WebSocket upgrade all work unchanged.
**Security.**
- The destination is **pinned** to `127.0.0.1:<web_port>` — the client cannot pick
host/port, so this is not an open localhost proxy (no SSRF to other local services).
- Access is gated by the relay's pipe auth (`pipe.md §3.1`): only the namespace
agent or an **authorized** client can establish a pipe.
- It exposes the full local web UI remotely — that is the intent; pair/authorize
devices accordingly.
**Relay tuning** (env, `pipe.md §3.3`): a browser opens several connections, so
`RELAY_PIPE_MAX_PER_NS` (default 8) may need raising; an idle chat-WS pipe can be
reaped at `RELAY_PIPE_IDLE_TIMEOUT_SECS` (120 s) — the frontend auto-reconnects.
Teardown: `proxy_one` takes a child of the plugin cancel token, so plugin stop
closes active tunnels; `stop_inner` also `shutdown()`s the relay client.
---
## Delayed push
A phone push is only valuable when the user is *away* from the computer. When
they're at the chat, every approval/clarification would otherwise fire an
instant — and pointless — notification, since they answer on the computer within
seconds. `DelayedNotifier` (`notifier.rs`) debounces this between the bus events
and `broadcast_inbox()`. (Elicitations are not chat-inline, so they are exempt —
see below.)
- **`*_requested`** arms a timer (`notify_delay_secs`, default 20s) keyed by
`(kind, request_id)` — approvals, clarifications, and elicitations use
independent id counters, so the kind is part of the key. If the timer elapses unresolved, the
key is marked *notified* and the Inbox is pushed (`broadcast_inbox`, `live=false`
→ store-and-forward / offline push).
- **Elicitations are the exception**: they live *only* in the Inbox (never
inline in the chat, unlike approvals/clarifications), so there is no
computer-side answer to debounce against. They skip the timer and are pushed
**immediately**, regardless of `notify_delay_secs`.
- **`*_resolved`** before the timer fires ⇒ the timer is cancelled and **nothing
is sent**. If the push already went out, the resolution is broadcast so the
phone clears the item. (Untracked ids fall back to a broadcast for snapshot
freshness.)
This only affects the **phone**: the desktop/web approval UI runs over the
per-session WebSocket (`ApprovalRequired`/`AgentQuestion`) and is never delayed.
Phone-driven responses (`apply_client_payload`) still `broadcast_inbox()`
immediately; the subsequent `*_resolved` bus event is handled idempotently. Armed
timers are cancelled on plugin stop (`cancel_all`). Set `notify_delay_secs: 0`
for the previous instant-push behaviour.
---
## LLM tools (plugin.md §11)
| Tool | Effect | Approval |
|---|---|---|
| `mobile_start_pairing(ttl?)` | Open the pairing window, return the QR URL | **Gated** (a default `require` rule is seeded, like `execute_cmd`/`restart`) |
| `mobile_list_devices()` | List devices (state, platform, device_info, last_seen) | read-only |
| `mobile_revoke_device(pubkey)` | Revoke a device by hex ed25519 pubkey | `Config` category |
These tools are not contributed through the `Plugin` trait (which has no
`tools()` method). They are registered in `Tools::build` (`src/core/skald/bundles.rs`):
the plugin is fetched via `get_plugin_typed::<MobileConnectorPlugin>()`, cast to
`Arc<dyn RelayAgent>`, and bound into the tools via
`plugin_mobile_connector::mobile_tools(agent)` → `ToolRegistry::register_arc`.
`mobile_start_pairing`'s approval gate is the default rule seeded in
`ApprovalManager::seed_defaults` (`src/core/approval/mod.rs`): opening a window
emits a secret (the QR) into chat, so it must be a deliberate human action, not
LLM-triggerable via prompt injection.
---
## HTTP endpoint
`GET /api/plugin/mobile-connector/pairingqrcode?code=<random>` — runtime PNG of
the QR (or placeholder), behind Skald's normal auth. Mounted by `WebFrontend`
via `Plugin::http_router()` (the router closes over the live `RelayState`). The
`code` is a non-enumerable capability; a URL leaked into `chat_history`
self-revokes once the window closes.
docs/plugins/remote.md
+163
View File
@@ -0,0 +1,163 @@
# Remote Connectivity
Exposes the Skald web app on a private mesh network so remote clients (iOS app, NAS, etc.) can connect without port forwarding or internet exposure.
---
## Architecture
```
core-api plugin-tailscale-remote crate
───────────────────────────── ──────────────────────────────────────────────
trait RemoteAccess ←── RemotePlugin (src/lib.rs)
(core_api::remote) TailscaleSystemProvider (src/tailscale_sys.rs) ← default
TailscaleEmbeddedProvider (src/tailscale.rs)
[feature: remote-tailscale]
```
- **`RemoteAccess` trait** (`core_api::remote`): vendor-agnostic interface. The core (`Skald`, WS handler, etc.) only knows this trait.
- **`TailscaleSystemProvider`** (`crates/plugin-tailscale-remote/src/tailscale_sys.rs`): **recommended provider**. Reads the mesh IP via `tailscale ip -4` (requires `tailscaled` running on the host). Binds a standard `tokio::net::TcpListener` — no experimental dependencies.
- **`TailscaleEmbeddedProvider`** (`crates/plugin-tailscale-remote/src/tailscale.rs`): embedded alternative using `tailscale-rs`. Feature-gated: `remote-tailscale`, **off by default** (enable via the root crate's `embedded-tailscale` feature). No system daemon required, but currently pre-1.0 with known DERP/reconnect issues, and it drags the `aws-lc-rs` C crypto build back into the tree (see [Feature Flag](#feature-flag)). Use only when a daemon cannot be installed (e.g. unrooted NAS).
- **`RemotePlugin`** (`crates/plugin-tailscale-remote/src/lib.rs`): wires the provider into the plugin lifecycle. Spawns a second Axum server on the mesh interface using the same router as the local server.
### `RemoteAccess` trait
```rust
pub trait RemoteAccess: Send + Sync {
fn provider_name(&self) -> &str;
async fn device_ip(&self) -> Result<Ipv4Addr>;
fn is_connected(&self) -> bool;
async fn shutdown(&self);
}
```
Stored in `Skald::remote: Arc<RwLock<Option<Arc<dyn RemoteAccess>>>>`. `None` when the plugin is disabled.
### Dual-bind strategy
The plugin (not the `WebServer`) is responsible for the mesh-facing server:
1. `RemotePlugin::start(state)` calls `extract_deps(state)` once, storing three named fields:
- `port: u16` — TCP port to bind on the mesh interface
- `remote_slot: Arc<RwLock<Option<Arc<dyn RemoteAccess>>>>` — slot in `Skald` to register the active provider
- `router_factory: Arc<dyn Fn() -> Router + Send + Sync>` — closure that rebuilds the Axum router
2. The internal helpers (`start_tailscale_sys`, `start_tailscale`) use only those three fields — no `Arc<Skald>` reference after extraction.
3. `start_tailscale_sys` binds `tokio::net::TcpListener::bind((mesh_ip, port))`.
4. `start_tailscale` calls `provider.axum_listener(port)` → `tailscale::axum::Listener`.
5. Both call `router_factory()` to get a fresh router and spawn `axum::serve(listener, router)` guarded by a `CancellationToken`.
`extract_deps` uses `std::sync::OnceLock` — idempotent across `reload()` calls. The values are stable for the lifetime of the process (port and static dir come from config, the remote slot is the same `Arc`).
The local server on `127.0.0.1:PORT` is unaffected.
---
## Configuration
Config is stored in the **`plugins` SQLite table** (not in `config.yml`).
| Key | Type | Default | Description |
|---|---|---|---|
| `provider` | string | `"tailscale_sys"` | `tailscale_sys` (system daemon, recommended) or `tailscale` (embedded, no daemon). |
| `auth_key` | string | — | Tailscale auth key (`tskey-auth-...`). Only required for the `tailscale` embedded provider on first join. |
| `hostname` | string | `"personal-agent"` | Hostname on the tailnet. Only used by the embedded `tailscale` provider. |
| `key_file` | string | `"data/tailscale_keys.json"` | Path for persisting node identity between restarts. Only used by the embedded `tailscale` provider. |
Config is persisted automatically and survives restarts.
---
## Agent Workflow
### Using the system Tailscale daemon (recommended)
Requires Tailscale already installed and logged in on the host machine.
```
1. toggle_item(kind="plugin", id="remote_connectivity", enabled=true)
2. restart
→ On next boot: plugin reads IP from tailscale ip -4, mesh server starts, app reachable at <ts-ip>:3000
```
No auth key needed — the system daemon is already authenticated.
### Using the embedded provider (no daemon)
Requires a binary **built with `--features embedded-tailscale`** (off by default;
see [Feature Flag](#feature-flag)). Without it, selecting `provider="tailscale"`
fails at start with `unknown provider 'tailscale'`.
```
1. configure_plugin "remote_connectivity" {"provider":"tailscale","auth_key":"tskey-auth-...","hostname":"personal-agent"}
2. toggle_item(kind="plugin", id="remote_connectivity", enabled=true)
3. restart
→ On next boot: embedded tailscale connects, mesh server starts, app reachable at <ts-ip>:3000
```
After first setup, the plugin auto-starts on every boot (persisted `enabled=true` in DB).
To check status: `list_items` (type=plugins) → look for `remote_connectivity`, check `running` and `runtime_status.ip`.
---
## Data Streams (iOS → Server)
Remote clients can push typed data over the existing WebSocket connection:
```json
{"type": "data", "stream": "location", "payload": {"lat": 45.1, "lng": 9.2, "accuracy": 10.0}}
```
Handled in `src/frontend/api/ws.rs` → `handle_data_msg()`. Dispatched to:
| Stream | Handler | Notes |
|---|---|---|
| `location` | `state.location_manager.update("remote", ...)` | Stores in-memory; existing `latest()` / `all()` queries work |
| other | `warn!` log | Reserved for future streams (health, etc.) |
---
## Limitations
### `tailscale_sys` (system daemon)
- **Requires `tailscaled` on the host**: if the daemon is not running or the device is not logged in, `start()` fails immediately with a clear error.
- **IP can change**: if the device rejoins the tailnet with a different IP, the plugin must be restarted to pick up the new address. The server binds to a specific IP, not `0.0.0.0`.
### `tailscale` (embedded, tailscale-rs v0.3)
- **DERP-relay only**: direct WireGuard holepunching is not yet implemented (issue #151). Latency is slightly higher than native WireGuard.
- **Known reconnect bugs**: after a node restart, existing connections can hang for ~15 s (issue #11). DERP connectivity can be lost after a control plane reconnect (issue #26).
- **Unaudited cryptography**: do not use for highly sensitive data until an official audit is complete.
- **Breaking changes**: the library is pre-1.0. The `TS_RS_EXPERIMENT=this_is_unstable_software` env var is required at runtime (set by `run.sh`).
- **Auth key expiry**: auth keys expire. Regenerate and call `configure_plugin` with the new value if the plugin fails to connect.
---
## Feature Flag
`tailscale-rs` is compiled only when the `remote-tailscale` feature is active.
It is **off by default**: the crate pulls the pure-Rust `tailscale` library, which
internally forces the `aws-lc-rs` crypto backend (a cmake/NASM C build) back into
the whole workspace, defeating the ring-only static-binary story
(see [build-and-distribution.md](../build-and-distribution.md)). The recommended
`tailscale_sys` provider needs none of this and is always compiled.
```toml
# crates/plugin-tailscale-remote/Cargo.toml
[features]
default = [] # embedded provider OFF by default
remote-tailscale = ["dep:tailscale"]
# root Cargo.toml — opt in from the top-level build:
[features]
embedded-tailscale = ["plugin-tailscale-remote/remote-tailscale"]
```
To build **with** the embedded provider (accepts the aws-lc-rs C build):
```sh
cargo build --features embedded-tailscale
```
The default build already omits it — no `--no-default-features` needed.
docs/plugins/telegram.md
+234
View File
@@ -0,0 +1,234 @@
# Telegram Plugin
A private Telegram bot that forwards messages to the LLM and supports Human-in-the-Loop approvals via inline keyboard buttons.
---
## Setup
1. Create a bot with [@BotFather](https://t.me/BotFather) and copy the token.
2. Add to `config.yml`:
```yaml
plugins:
telegram:
token: "123456789:AABBCC..."
```
3. Restart the app. The bot starts automatically if the token is present.
---
## Pairing — how to authorize a user
Access control is managed entirely through the file `secrets/telegram_whitelist.json`.
### File format
```json
{
"whitelist": [123456789],
"pending_pairings": [
{
"code": "A3KX7P",
"chat_id": 987654321,
"issued_at": "2026-05-19T10:30:00+02:00"
}
]
}
```
- `whitelist` — array of authorized `chat_id` values (integers). Users in this array can send messages to the agent.
- `pending_pairings` — users who have contacted the bot but are not yet authorized. Each entry has a `code` shown to the user in Telegram chat, their `chat_id`, and the `issued_at` timestamp. Entries older than **24 hours** are pruned automatically the next time an unauthorized user contacts the bot, so abandoned codes do not pile up.
### Pairing flow
1. An unknown user sends any message to the bot.
2. The bot replies with a 6-character pairing code and writes an entry to `pending_pairings` in `secrets/telegram_whitelist.json`.
3. The user communicates the code to you (e.g., through a separate channel).
4. You ask the agent: *"Telegram pairing code A3KX7P — authorize it"*.
5. The agent reads `secrets/telegram_whitelist.json`, finds the entry with `code: "A3KX7P"`, moves the `chat_id` from `pending_pairings` to `whitelist`, and writes the file back.
6. Within 10 seconds the plugin's watchdog detects the file change, logs the event, and **sends a welcome message** to the newly authorized user on Telegram.
### To authorize manually (without asking the agent)
Use `edit_file` or `write_file` to move the `chat_id` from `pending_pairings` to `whitelist` in `secrets/telegram_whitelist.json`.
### To revoke access
Remove the `chat_id` from the `whitelist` array in `secrets/telegram_whitelist.json`. The change takes effect on the user's next message (whitelist is re-read on every message).
---
## Watchdog
The plugin polls `secrets/telegram_whitelist.json` every **10 seconds** for modification-time changes.
When it detects a change:
- Reloads the whitelist.
- Identifies any `chat_id` values newly added to `whitelist`.
- Sends each newly authorized user a welcome message on Telegram.
- Logs the event at INFO level.
This means there is no restart needed after editing the file — authorization takes effect automatically.
---
## Commands
| Command | Effect |
|---|---|---|
| `/new` or `/clear` | Create a new chat session (clears LLM context) |
| `/stop` | Interrupt the agent mid-turn (clears pending approvals and clarifications) |
| `/models` | List available LLM models ordered by priority (numbered `0..N`, index 0 is `auto`) |
| `/model <N\|name\|auto>` | Pin the model for this chat by index, name (substring allowed), or reset to `auto`. State is held in `ChatHub.selected_clients["telegram"]` and broadcast to all clients of the source via `ClientSelected`. Cleared on server restart |
| `/context` | Show last turn's token usage (`↑X tok · ↓Y tok`) |
| `/cost` | Show total spend for this session in USD (sync sub-agents included; async tasks excluded) |
| `/compact` | Force context compaction (bypasses the token threshold) |
| `/resettools` | Remove all activated tool groups (MCP servers + `config`) from the session |
| `/sethome` | Set Telegram as the home source for background notifications |
| `/help` | Show available commands |
| any other `/command` | Unknown — replies with a "Unknown command" notice + the help list, never forwarded to the LLM |
| any text | Forwarded to the LLM agent |
---
## Human-in-the-Loop Approvals
When the LLM triggers a tool that requires user approval (`execute_cmd`, `restart`, write-file tools outside `memory/`):
1. The bot sends a message with the operation details and a content preview.
2. Four inline keyboard buttons appear in two rows:
```text
[✅ Approve] [❌ Reject]
[⏱ 15 min] [🔄 Session]
```
3. Tapping a button resolves the pending approval and execution continues or is cancelled.
4. **⏱ 15 min** — approves and suppresses approval prompts for tools of the same category/MCP server for 15 minutes.
5. **🔄 Session** — approves and suppresses all approval prompts for the rest of the session.
6. The approval message is **deleted** once resolved, whether via Telegram or the web UI.
Bypass buttons call `ApprovalApi::approve_with_bypass` (scope auto-detected from the tool's category or MCP server). See [../approval/index.md](../approval/index.md) for bypass semantics.
---
## Output Formatting
Telegram's HTML parse mode supports only a limited tag set: `<b>` `<i>` `<u>` `<s>` `<code>` `<pre>` `<a>` `<blockquote>`. Structural elements (`<table>`, `<ul>`, `<li>`, `<div>`) are **not supported**.
The plugin injects a compact formatting context into every LLM session (`TELEGRAM_FORMAT_CONTEXT` in `mod.rs`) and a shorter tail reminder (`TELEGRAM_FORMAT_REMINDER`) instructing it to:
- Use Telegram HTML tags only.
- Never use Markdown (`**`, `*`, `` ` ``, `#`, `_`, `|`).
- Replace structured data (tables) with bullet lists (`•`).
| Element | Correct | Wrong |
| --------------- | --------------------- | ------------------ |
| Bold | `<b>text</b>` | `**text**` |
| Italic | `<i>text</i>` | `*text*` |
| Code | `<code>text</code>` | `` `text` `` |
| Code block | `<pre>text</pre>` | ` ```text``` ` |
| Structured data | bullet list `•` | `\| col \| col \|` |
Long responses are automatically split into chunks of ≤ 4000 characters via `send_long()` in `helpers.rs`.
### Markdown sanitizer (post-processing safety net)
Because LLMs occasionally emit Markdown despite instructions, `send_long` applies `sanitize_for_telegram()` on every HTML-mode send **before** chunking. This provides a reliable fallback independent of model compliance:
1. **Markdown tables → bullet lists** — detects `| col | col |` blocks, emits the header row as `<b>header — header</b>` and each data row as `• val — val`.
2. **`**bold**` → `<b>bold</b>`** — converts residual Markdown bold.
3. **`## Header` → `<b>Header</b>`** — converts residual Markdown headers.
### Fallback behavior
If the Telegram API rejects a chunk (e.g., due to malformed HTML), `send_long` retries **without** `ParseMode::Html`. Before retrying it strips all HTML tags (`<…>`) from the chunk using a regex so the user sees plain text rather than raw `<b>…</b>` markup.
---
## Voice (Speech Integration)
If the Speech plugin is configured and running, the Telegram plugin gains two additional capabilities:
### Incoming voice messages (STT)
When the user sends a voice note, the plugin:
1. Downloads the OGG audio from Telegram.
2. Passes it to `SpeechPlugin::transcribe()`.
3. Forwards the resulting text to the LLM as a normal message.
### Outgoing voice replies (TTS)
The LLM has access to a `send_voice_message(text)` tool. When it calls it, the plugin:
1. Passes the text to the active `TextToSpeech` synthesiser (`synthesize()`).
2. **Transcodes the audio to Ogg/Opus** (`to_ogg_opus` in `tools.rs`) — the only format Telegram renders as a playable voice message. The synthesiser's `output_format()` decides the input: `opus`/`ogg` pass through untouched; raw `pcm` (e.g. Gemini TTS) is decoded as 24 kHz/mono/s16le; every other container (mp3, wav, …) is auto-detected. Conversion runs `ffmpeg` over stdin/stdout pipes (no temp files).
3. Sends the resulting Ogg/Opus bytes back to the user as a Telegram voice message.
The LLM is instructed to use voice only for short, conversational replies with no code or complex formatting. The TTS engine's formatting guide (SSML-like tags) is also injected into the system context so the LLM can control pacing and emphasis.
> **Requires `ffmpeg`** on `PATH` for any non-Opus synthesiser. If it is missing, `send_voice_message` returns a clear error (`ffmpeg not available …`) and the LLM falls back to a text reply.
### Requirements
Both `plugins.speech.stt_model` and `plugins.speech.tts_model` must be set in `config.yml`. The Speech plugin must be enabled and running before the Telegram plugin starts.
---
## File & Media Attachments
The Telegram plugin downloads incoming attachments and forwards them to the conversation. The LLM sees them in timeline order — it knows which file was most recently sent without any special indexing.
| Type | Saved to disk | How it reaches the LLM |
| --- | --- | --- |
| Document (PDF, ZIP, …) | `data/uploads/telegram/<chat_id>/<filename>` | Structured `metadata.attachments` (shared with web/mobile) |
| Photo | `data/uploads/telegram/<chat_id>/<file_id>.jpg` | Structured `metadata.attachments` |
| Location | — | `[TELEGRAM SYSTEM INFO]` text (latitude, longitude, accuracy, Google Maps URL) |
**Document and Photo are aligned with the web/mobile attachment model**: `download_and_save`
returns a `core_api::message_meta::Attachment` (project-root-relative path so `/data/…` serves
it), `handle_attachment` puts it in `SendMessageOptions.metadata`, and the message builder
generates the shared `[SYSTEM INFO]` block on the fly. Viewing the `telegram` source from the
copilot therefore shows these as **chips**, not raw text. See
[frontend.md](../frontend.md#attachments) and [database.md](../database.md) (`chat_history.metadata`).
**Location** has no file, so it keeps the legacy `[TELEGRAM SYSTEM INFO]` text path
(`system_info_message`). Captions typed alongside a Document/Photo become the user turn's text.
### Live locations
When the user shares a live location, two things happen:
1. **Initial message** (`message` event) — the LLM is notified via a `[TELEGRAM SYSTEM INFO]` message and the position is written to `skald.location_manager` under the key `"telegram"`.
2. **Subsequent updates** (`edited_message` events) — the position in `location_manager` is updated silently, with no LLM notification. This keeps the store current for any background scripts or tools that read `user_location("telegram")`.
`LocationManager` is in-memory only. On restart, the store starts empty and is repopulated as soon as Telegram delivers the next live location tick (typically within seconds if sharing is still active).
The `uploads/` directory is gitignored.
### Extending attachment types
To add a new type (e.g. sticker, contact):
1. Add a variant to `TelegramAttachment` in `crates/plugin-telegram-bot/src/attachments.rs`.
2. Implement `download_and_save`: return `Ok(Some(Attachment))` for a file-backed type (it flows into `metadata.attachments`) or `Ok(None)` for a file-less one (then add a `system_info_message` arm and handle it in the `None` branch of `handle_attachment`).
3. Detect the message type in `classify_message` in `handlers.rs` and return `IncomingEvent::Attachment(...)`.
---
## Interface Tools
The Telegram plugin can inject custom LLM-callable tools into any session via the `interface_tools` parameter of `SendMessageOptions`. These tools are only visible to the root agent — sub-agents do not inherit them.
To add a Telegram-specific tool, construct an `InterfaceTool` with an OpenAI tool definition and an async handler closure that captures `Arc<Bot>` and `ChatId`, then pass it in the `interface_tools` vec inside `SendMessageOptions`.
`InterfaceTool` and `ToolFuture` are defined in `crates/core-api/src/interface_tool.rs` (re-exported via `crate::chat_hub`). `AgentRunConfig` remains in `src/core/session/handler/interface_tools.rs` (main crate only).
---
## Secrets directory
`secrets/telegram_whitelist.json` is gitignored. The directory is created automatically on first pairing request. Never commit this file.
docs/plugins/whisper-local.md
+123
View File
@@ -0,0 +1,123 @@
# WhisperLocal Plugin
Local Speech-to-Text via [whisper.cpp](https://github.com/ggerganov/whisper.cpp), Metal-accelerated on Apple Silicon.
Implemented in pure Rust using the `whisper-rs` crate — no Python involved.
---
## Setup
### 1. Download a GGML model
```sh
mkdir -p models
curl -L -o models/ggml-large-v3.bin \
https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3.bin
```
Other available sizes (smaller = faster, less accurate):
| Model | Size | Notes |
|---|---|---|
| `ggml-tiny.bin` | ~75 MB | Very fast, lower accuracy |
| `ggml-base.bin` | ~142 MB | Good balance for testing |
| `ggml-small.bin` | ~466 MB | Good accuracy |
| `ggml-medium.bin` | ~1.5 GB | High accuracy |
| `ggml-large-v3.bin` | ~3.1 GB | Best accuracy, recommended |
| `ggml-large-v3-turbo.bin` | ~1.6 GB | large-v3 speed-optimised |
All models: `https://huggingface.co/ggerganov/whisper.cpp`
### 2. Configure `config.yml`
```yaml
plugins:
whisper_local:
model: "models/ggml-large-v3.bin"
language: "it" # BCP-47 code, or "auto" for detection
load_at_startup: false # false (default) = lazy load on first use
idle_timeout_secs: 1200 # unload after 20 min idle; 0 = never unload
```
| Option | Default | Effect |
|---|---|---|
| `model` | — (required) | Path to the GGML `.bin` file |
| `language` | `auto` | BCP-47 code or `auto`. Applied live — runtime changes take effect on the next transcription without a reload |
| `load_at_startup` | `false` | **When** the model first loads: `false` = lazily on the first transcription, `true` = eagerly in `start()` (warm, no first-call latency) |
| `idle_timeout_secs` | `1200` | **When** the model unloads: after this many seconds of inactivity. `0` = never unload (stays resident once loaded) |
The two timing options are orthogonal and cover the whole spectrum:
| `load_at_startup` | `idle_timeout_secs` | Behaviour |
|---|---|---|
| `false` | `1200` | **Default** — load on first use, free ~2 GB after 20 min idle |
| `true` | `0` | Always resident — eager load, never unload (legacy behaviour) |
| `true` | `1200` | Warm at startup, but freed if unused |
| `false` | `0` | Load on first use, then stay resident |
### 3. Build
The first `cargo build` compiles whisper.cpp (a few minutes). Subsequent builds are cached.
---
## How it works
```
Telegram voice message (OGG/Opus)
│
▼ ffmpeg → 16 kHz mono WAV
▼ hound → Vec<f32> PCM samples
▼ whisper.cpp (Metal GPU) → text
│
▼ forwarded to LLM as a normal text message
```
Audio conversion uses the system `ffmpeg` binary (must be installed: `brew install ffmpeg`).
Inference runs on Apple Silicon GPU via Metal. Falls back to CPU if Metal is unavailable.
---
## Memory management (lazy load + idle unload)
The GGML weights are ~2 GB, so the plugin keeps them in memory only while they are
actually useful. The model lives in a shared, droppable cell (`LazyModel`):
- **`start()`** validates the model path and registers a lightweight transcriber, but
does **not** load the weights unless `load_at_startup: true`. The registered handle
holds no strong reference to the weights, so they can be freed at any time.
- **First transcription** triggers `ensure_loaded()`, which loads the weights once
(concurrent first-callers wait on a single load) and records a last-used timestamp.
- A background **eviction task** ticks every 60 s and unloads the model once it has
been idle for `idle_timeout_secs`. Set `idle_timeout_secs: 0` to disable eviction.
- **Unloading** is refcount-safe: an in-flight transcription holds its own handle to
the weights, so memory is reclaimed only after it finishes. The actual free runs on
a blocking thread (whisper.cpp GPU cleanup).
Trade-off: after an unload, the next transcription pays the reload cost (a few
seconds). The OS page cache usually keeps the `.bin` warm, so the reload is mostly
memory copy + Metal allocation rather than disk I/O. Use `load_at_startup: true` /
`idle_timeout_secs: 0` if you prefer zero first-call latency over reclaiming the RAM.
---
## Integration with TranscribeManager
`WhisperLocalPlugin` does **not** expose itself as `Arc<dyn Transcribe>` directly. At `start()` it registers a lightweight `WhisperLocalTranscriber` handle into `skald.transcribe_manager`; at `stop()` it deregisters it. Callers never reference the plugin type — they ask the manager:
```rust
if let Some(t) = skald.transcribe_manager.get().await {
let text = t.transcribe(audio, "ogg").await?;
}
```
See [../plugins.md](../plugins.md) for the `TranscribeManager` API and the `Transcribe` trait.
---
## When to Update This File
- The audio conversion pipeline changes
- Default recommended models change
- Registration/deregistration logic in `start()`/`stop()` changes
- The lazy-load / idle-eviction lifecycle or its config options change
docs/projects.md
+133
View File
@@ -0,0 +1,133 @@
# Projects
Filesystem-linked **projects**, each a unit of work tied to a directory on disk. A project gives
agents a standing context (path, description, permissions) so they know what they're working on
without the user re-explaining it every time.
Two ways to work on a project:
1. **Tickets** — fire-and-forget background tasks (one agent run per ticket).
2. **Interactive chat** — a persistent conversation with the project's coordinator agent, which
delegates to specialist sub-agents.
For the database schema (`projects`, `project_tickets`) see [database.md](database.md). This file
documents the subsystem behavior.
---
## Modules
| Path | Role |
| ---- | ---- |
| `src/core/projects/mod.rs` | `ProjectManager` — CRUD; free fn `build_runtime_run_context` |
| `src/core/projects/tickets.rs` | `ProjectTicketManager` — ticket CRUD + lifecycle |
| `src/core/db/projects.rs`, `src/core/db/project_tickets.rs` | DAOs |
| `src/frontend/api/projects.rs` | REST + chat-session endpoints |
| `web/components/projects/` | `<projects-page>`, `<project-list-section>`, `<project-board-section>` |
---
## RunContext: `build_runtime_run_context`
`projects::build_runtime_run_context(project, base) -> RunContext` is the single place that turns a
project into a runtime [`RunContext`](approval.md). It layers project-runtime fields over an
optional `base` RC (which carries only static config set at creation, e.g. `security_group`):
- `working_directory` ← `project.path` (always overwritten).
- `allow_fs_writes` ← extended with the project tree and `{skald_cwd}/data` (pre-authorizes writes
there, so tool calls in those trees skip the approval gate).
- `system_prompt` ← project-context fragments prepended: a project header (name + description) and
a hint pointing at the personal-data dir.
It is shared by **both** execution paths below, so a ticket job and an interactive chat see an
identical project context. Edit it in one place.
---
## Tickets (background)
A ticket is an individual work item with a `title`, `description`, `agent_id`, and optional
`run_context` (static config only). Lifecycle in `ProjectTicketManager`:
- `create` / `delete` / `reset` — CRUD; every mutation calls `db::projects::touch` so the list
orders by recency.
- `start(ticket_id)` — resolves the base RC (ticket override → project default), calls
`build_runtime_run_context`, then spawns a background job via `TaskManager.spawn_async_job` with
`origin_ref = "PROJECT_TASK:{id}"`. The ticket row records the `job_id` and moves to running.
- Completion is event-driven: `start_listener` subscribes to the system bus and, on
`SystemEvent::JobCompleted` whose `origin_ref` matches `PROJECT_TASK:`, calls `on_job_completed`
to persist the result/error and final status.
The board UI (`web/components/projects/project-board.js`) renders tickets in a single scrollable
list divided into three sections:
- **Running** — active tickets (status `pending` or `in_progress`), in start order.
- **Todo** — pending tickets, sorted by `created_at` descending (newest first).
- **Completed** — done/failed tickets, sorted by `completed_at` descending (most recent first).
The LLM result of a done ticket is rendered as markdown. Failed tickets show raw error text.
The view polls every 5 s while any ticket is running. Each ticket links to its session.
---
## Interactive project chat
A persistent conversation about the project, driven by the **`project-coordinator`** agent (see
[agents.md](agents.md)). A project can be of **any kind** — software, but also travel, study,
writing, events, personal goals — and the coordinator adapts to its nature (read from the injected
project description). The user talks to one bot that already knows the project; it does everyday
planning and writing itself and delegates specialized work — research via `researcher`, or code via
`tech-lead`/`software-architect`/`software-engineer` — to sub-agents through `execute_task`.
**Project memory (`SKALD.md`).** The coordinator's `meta.json` declares `"inject_memory": ["$WD/SKALD.md"]`.
The `$WD` placeholder expands to the session's working directory (the project path), so a `SKALD.md`
placed in the project root is auto-loaded into the system prompt as a `<memory_file>` block — the
per-project analogue of how `main` loads `data/memory/*`. If the file doesn't exist yet, a
`(file not created yet)` placeholder is injected, which the coordinator can fill in via `write_file`.
See `inject_memory` in [agents.md](agents.md).
**Source.** The chat is bound to source id `project-{id}` in the `sources` table (hyphen, not `:`,
so it stays URL-safe in `/api/{source}/messages`). The session is **interactive and
non-ephemeral**, so it persists and is resumed on reopen — unlike the disposable per-client
sessions [`ChatHub`](architecture.md) normally manages.
**Provisioning.** `api::projects::provisioning_for_source(skald, source)` maps a source to its
`(agent_id, RunContext)`:
- `project-{id}` → (`project-coordinator`, `build_runtime_run_context(project, project.run_context)`)
- anything else → (`main`, `None`)
This single resolver is reused by both endpoints so open and reset never diverge:
| Endpoint | Effect |
| --- | --- |
| `POST /api/projects/{id}/session` | `ChatHub::provision_session(source, agent, rc, reset=false)` — open or resume; returns `{ source, session_id }` |
| `POST /api/sessions?source=project-{id}` | same with `reset=true` — recreates the session with the **coordinator** (not `main`) |
`provision_session` is the only entry point for the source→session mapping ChatHub owns; the RC is
persisted at session creation (via `ChatSessionManager::create_session`) so it's present before the
handler is built. Because the session is interactive, `execute_task` is auto-injected, giving the
coordinator sub-agent delegation for free.
**UI (desktop).** The desktop copilot shows browser-style tabs: `General` (the `web` source, always
present) plus one tab per open project chat. The board's **Open Chat** button `POST`s the session
endpoint, then dispatches a `project-chat-open` window event (`{source, label}`); the copilot
adds/focuses the tab and calls `ChatSession._switchSource(source)` to swap the live WebSocket.
Closing a project tab is UI-only — the session persists and can be reopened from the board.
**UI (mobile).** The mobile web app (`<mobile-app>`) has a **Projects** bottom-nav entry rendering
`<projects-page>` (list from `GET /api/projects`). Tapping a project `POST`s the same session
endpoint and emits a `project-open` event; the shell navigates to `#chat/project-{id}`, which its
hash router turns into the `<chat-page>` `source` prop `project-{id}` (→ `_switchSource`) with the
project name in the header. A back button returns to `#chat` (the main `mobile` session), and the
hash survives refresh. It reuses the **same** `project-{id}` session as the desktop, so a project
chat is continuous across desktop, mobile browser, and — since the native iOS shell renders this web
app over the relay — remote. See [frontend.md](frontend.md).
---
## When to update this file
- Changing the project/ticket lifecycle or `build_runtime_run_context`.
- Changing how project chats are provisioned, sourced, or surfaced in the UI.
- Schema changes go in [database.md](database.md); the coordinator agent in [agents.md](agents.md).
docs/providers/image.md
+248
View File
@@ -0,0 +1,248 @@
# Image Generation
Framework for generating images from text prompts. Supports DB-backed providers (configured via UI) and plugin-registered providers (ephemeral, registered at runtime).
---
## Architecture
```text
crates/core-api/src/image_generate.rs
— ImageGenerate trait (provider interface)
— ImageGenerateRegistry trait (plugin write-side: register/unregister)
src/image_generate/
mod.rs — record types, re-exports ImageGenerate from core-api
manager.rs — ImageGeneratorManager (DB-backed + plugin slots, impls ImageGenerateRegistry)
db.rs — CRUD for image_generate_models table
openrouter_image.rs — OpenRouterImageGenerator (chat completions + modalities)
src/tools/
image_generate.rs — LLM tools: image_generate_providers_list, image_generate
src/api/
image_generate_models.rs — REST CRUD for image_generate_models
images.rs — GET /api/images/:id (serve generated files)
```
Two kinds of providers coexist:
| Kind | Source | Example |
| ---- | ------ | ------- |
| **DB-backed** | Rows in `image_generate_models`, built from `llm_providers` credentials | OpenRouter `x-ai/grok-2-vision` |
| **Plugin-registered** | Ephemeral — registered at runtime by plugins | future: `StableDiffusionPlugin` |
Plugin-registered providers take precedence over DB-backed ones in `get()`.
---
## Traits (crates/core-api)
```rust
// core_api::image_generate
#[async_trait]
pub trait ImageGenerate: Send + Sync {
fn id(&self) -> &str;
fn name(&self) -> &str;
async fn generate(&self, prompt: &str) -> Result<Vec<u8>>; // raw PNG bytes
}
/// Write-side used by plugins to register/unregister ephemeral providers.
/// Implemented by ImageGeneratorManager in the main crate.
#[async_trait]
pub trait ImageGenerateRegistry: Send + Sync {
async fn register(&self, provider: Arc<dyn ImageGenerate>);
async fn unregister(&self, id: &str);
}
```
`ImageGenerateRegistry` is also available on `PluginContext` as `ctx.image_generate_registry`, so plugin crates that depend only on `core-api` can register providers without importing anything from the main crate.
---
## Manager API
```rust
// Async constructor — loads DB models on startup
ImageGeneratorManager::new(pool: Arc<SqlitePool>, data_root: impl Into<PathBuf>)
-> Result<Arc<Self>>
// Plugin registration (ephemeral — called by a plugin's start()/stop())
image_generator_manager.register(Arc::new(provider)).await;
image_generator_manager.unregister("my_provider_id").await;
// DB-backed CRUD (called by REST API handlers)
image_generator_manager.add_model(record).await // → Result<i64>
image_generator_manager.update_model(id, record).await
image_generator_manager.delete_model(id).await // soft delete
image_generator_manager.get_model(id).await // → Option<ImageGenerateModelRecord>
// Listings
image_generator_manager.list_models_info().await // DB-backed only → Vec<ImageGenerateModelInfo>
image_generator_manager.list_all_info().await // plugin + DB → Vec<ImageGenerateModelInfo>
image_generator_manager.list().await // lightweight → Vec<ImageGenerateInfo> (for LLM tool)
// Resolution
image_generator_manager.get(id).await // → Option<Arc<dyn ImageGenerate>>
// Generation (called by image_generate tool via block_in_place)
image_generator_manager.generate(provider_id, prompt).await // → Result<(PathBuf, String)> (path, url)
// Image storage path
image_generator_manager.images_dir() // → PathBuf (data/images/)
```
---
## LLM Tools
Two tools are injected per-turn when at least one provider is active (absent otherwise):
### `image_generate_providers_list`
Lists all currently active image generation providers.
```text
Parameters: (none)
Returns: JSON array of {id: string, name: string}
```
### `image_generate`
Generates an image synchronously. Blocks the tool round until the image is ready.
```text
Parameters:
provider_id string (required) — ID from image_generate_providers_list
prompt string (required) — text prompt
Returns: {"path": "/abs/path/data/images/<id>.png", "url": "/api/images/<id>"}
```
**Typical agent flow:**
```text
1. image_generate_providers_list() → [{id: "grok-imagine", name: "grok-imagine"}]
2. image_generate("grok-imagine", "a red sunset") → {"path": "...", "url": "/api/images/abc123"}
```
---
## Image Storage
Generated images are written to `data/images/<random_id>.png` (relative to the working directory). The directory is created automatically on first use.
---
## REST API
### Image serving
```text
GET /api/images/:id
```
Serves the generated PNG. Returns `404` if the file does not exist, `400` for invalid IDs. No authentication (local server).
### Model management
```text
GET /api/image-generate/models — list all active providers (plugin + DB)
POST /api/image-generate/models — add a DB-backed model
GET /api/image-generate/models/{id} — get a model record
PUT /api/image-generate/models/{id} — update a model record
DELETE /api/image-generate/models/{id} — soft-delete a model
```
**POST / PUT body:**
```json
{
"provider_id": 1,
"model_id": "x-ai/grok-2-vision",
"name": "grok-imagine",
"priority": 100
}
```
`name` becomes the `provider_id` used in the `image_generate` LLM tool. If omitted, `model_id` is used.
---
## OpenRouter provider
`OpenRouterImageGenerator` calls the OpenRouter chat completions endpoint with `modalities: ["image"]` (image-only — do **not** use `["image", "text"]`, which is for multimodal models and causes a 404 on image-only models like `grok-imagine-image-quality`). The response image is returned as a base64 data URL at `choices[0].message.images[0].image_url.url`.
To register an OpenRouter image model:
1. Add/use an existing `llm_providers` row with `type = "open_router"` and a valid API key.
2. `POST /api/image-generate/models` with that `provider_id` and the desired `model_id`.
---
## Plugin Registration
Plugin crates depend only on `core-api` — no reference to the main crate needed.
```rust
// In crates/plugin-foo/Cargo.toml:
// core-api = { path = "../core-api" }
use core_api::image_generate::ImageGenerate;
struct MyImageGenerator { /* ... */ }
#[async_trait]
impl ImageGenerate for MyImageGenerator {
fn id(&self) -> &str { "my_generator" }
fn name(&self) -> &str { "My Generator" }
async fn generate(&self, prompt: &str) -> Result<Vec<u8>> {
// call external API or local model, return PNG bytes
}
}
// In Plugin::reload() when enabled:
ctx.image_generate_registry.register(Arc::new(MyImageGenerator { ... })).await;
// In Plugin::stop() or reload() when disabled:
ctx.image_generate_registry.unregister("my_generator").await;
```
---
## ComfyUI plugin (`crates/plugin-comfyui`)
Each JSON file in `data/comfyui/workflows/` becomes a separate `ImageGenerate`
provider. The plugin monitors ComfyUI health every 5s and unregisters all
providers if the server is unreachable.
Workflow files must be exported from ComfyUI as "API Format". The plugin reads
an optional `_personal_agent` key for metadata:
```json
"_personal_agent": {
"name": "Realistic Portrait",
"description": "Ritratti realistici, formato verticale. Default 768×1024.",
"prompt_node": "6",
"negative_prompt_node": "7",
"prompt_field": "clip_l",
"prompt_field_extra": ["clip_g", "t5xxl"],
"extra_params": { "width_node": "8", "height_node": "8", "steps_node": "3" }
}
```
- `prompt_field` (optional): input field to write the prompt into. Default `"text"` (for `CLIPTextEncode`). Use `"clip_l"` for `CLIPTextEncodeSD3`.
- `prompt_field_extra` (optional): additional input fields to copy the same prompt into. For SD3.5: `["clip_g", "t5xxl"]`.
- `negative_prompt_field` / `negative_prompt_field_extra`: same for the negative prompt node.
Provider id: `comfyui-{filename}` (e.g. `realistic-portrait.json` → `comfyui-realistic-portrait`).
See [../comfyui-workflow-format.md](../comfyui-workflow-format.md) for the complete
guide on reading and modifying workflow files.
---
## When to Update This File
- A new concrete `ImageGenerate` implementation is added (e.g. a new provider backend)
- Image storage path or REST endpoint changes
- LLM tool signatures change
docs/providers/index.md
+11
View File
@@ -0,0 +1,11 @@
# Model Providers
Trait contracts and implementations for LLM, TTS, transcription, and image generation backends.
## Files
- [tts.md](tts.md) — Text-to-Speech: trait, manager, provider catalogue, tts_models DB table
- [transcribe.md](transcribe.md) — Speech-to-Text: OpenAI-compatible audio API, transcribe_models DB table
- [image.md](image.md) — Image generation: trait, manager, async task system, LLM tools, REST endpoint
See [../index.md#model-providers](../index.md#model-providers) for navigation. See also [../llm-clients.md](../llm-clients.md) for LLM client trait and selection.
docs/providers/transcribe.md
+148
View File
@@ -0,0 +1,148 @@
# Transcription Providers
Cloud Speech-to-Text via any OpenAI-compatible audio transcription endpoint.
---
## Architecture
```text
crates/core-api/src/transcribe.rs
— Transcribe trait (provider interface)
— TranscribeProvider trait (resolve active provider)
— TranscribeRegistry trait (plugin write-side: register/unregister)
— TranscribeModelRecord (DB record type — moved here from main crate)
— RemoteTranscribeModelInfo (remote catalog type — moved here from main crate)
src/transcribe/
mod.rs — TranscribeModelInfo (API response type), re-exports from core-api
db.rs — SQL layer for transcribe_models table
manager.rs — TranscribeManager (DB-aware, owns the table)
openai_audio.rs — OpenAiAudioTranscriber: impl Transcribe via HTTP multipart
elevenlabs_audio.rs — ElevenLabsTranscriber: impl Transcribe via ElevenLabs Scribe API
```
`TranscribeManager` holds two kinds of providers:
| Kind | Source | Example |
| ---- | ------ | ------- |
| **DB-backed** | `transcribe_models` table, built from `llm_providers` credentials | `OpenAiAudioTranscriber` |
| **Plugin-registered** | Ephemeral — registered at runtime by plugins | `WhisperLocalTranscriber` |
`get()` returns the first plugin provider (if any is running), then falls back to the first DB-backed provider ordered by `priority ASC`. Callers never reference a concrete type:
```rust
if let Some(t) = skald.transcribe_manager.get().await {
let text = t.transcribe(audio, "ogg").await?;
}
```
### Manager API
```rust
// DB-backed CRUD — only TranscribeManager touches transcribe_models
transcribe_manager.add_model(record).await?
transcribe_manager.update_model(id, record).await?
transcribe_manager.delete_model(id).await? // soft-delete
transcribe_manager.get_model(id).await // → Option<TranscribeModelRecord>
transcribe_manager.list_models_info().await // → Vec<TranscribeModelInfo> (DB-backed only)
transcribe_manager.list_all_info().await // → Vec<TranscribeModelInfo> (plugins first, then DB — used by API)
// Remote model catalog (calls ApiProvider::list_transcribe_models)
transcribe_manager.list_provider_models(provider_id).await // → Result<Vec<RemoteTranscribeModelInfo>>
// Plugin registration (ephemeral — called by WhisperLocalPlugin)
transcribe_manager.register(Arc::new(transcriber)).await
transcribe_manager.unregister("whisper_local").await
```
`RemoteTranscribeModelInfo` fields: `id`, `name`, `description`, `languages` (BCP-47 codes).
### REST API
| Method | Path | Description |
| ------ | ---- | ----------- |
| `GET` | `/api/transcribe/models` | List all models — plugin-registered first (`from_plugin: true`), then DB-backed |
| `POST` | `/api/transcribe/models` | Add a new transcription model |
| `GET` | `/api/transcribe/models/{id}` | Get a DB-backed model record |
| `PUT` | `/api/transcribe/models/{id}` | Update a DB-backed model |
| `DELETE` | `/api/transcribe/models/{id}` | Soft-delete a DB-backed model |
| `GET` | `/api/transcribe/providers/{id}/models` | List remote transcription models from a configured provider (`RemoteTranscribeModelInfo[]`) |
---
## OpenAiAudioTranscriber
Implemented in `src/core/transcribe/openai_audio.rs`.
Calls `POST {base_url}/audio/transcriptions` with a `multipart/form-data` body:
| Field | Value |
| ---------- | ------------------------------------------------ |
| `file` | Raw audio bytes with extension-derived MIME type |
| `model` | Provider model ID (e.g. `openai/whisper-1`) |
| `language` | BCP-47 code (optional — omitted for auto-detect) |
Accepted formats: `ogg`, `mp3`, `mp4`, `m4a`, `wav`, `webm`, `flac`.
No local conversion needed — the provider handles decoding server-side.
### Supported providers
| Provider | `base_url` | Notes |
| -------- | ---------- | ----- |
| OpenRouter | `https://openrouter.ai/api/v1` | Model: `openai/whisper-1`, etc. |
| OpenAI | `https://api.openai.com/v1` | Model: `whisper-1` |
---
## ElevenLabsTranscriber
Implemented in `src/core/transcribe/elevenlabs_audio.rs`.
Calls `POST https://api.elevenlabs.io/v1/speech-to-text` with auth header `xi-api-key` (not Bearer) and a `multipart/form-data` body:
| Field | Value |
| ----- | ----- |
| `file` | Raw audio bytes |
| `model_id` | ElevenLabs Scribe model (e.g. `scribe_v1`) — stored as `model_id` in the DB record |
Returns `{ "text": "..." }`. Provider type: `elevenlabs`.
`ElevenLabsProvider::list_transcribe_models()` calls `GET https://api.elevenlabs.io/v1/models` and filters entries whose `model_id` starts with `scribe` (or `can_do_voice_conversion: true` as a fallback). Returns `RemoteTranscribeModelInfo` with id, name, description, and supported languages.
Which providers support transcription is declared statically via `ApiProvider::supported_types()` — see [llm-clients.md](llm-clients.md#apiprovider--service-types).
---
## DB: transcribe_models table
```sql
CREATE TABLE transcribe_models (
id INTEGER PRIMARY KEY AUTOINCREMENT,
provider_id INTEGER NOT NULL REFERENCES llm_providers(id),
model_id TEXT NOT NULL,
name TEXT NOT NULL UNIQUE,
language TEXT, -- BCP-47 or NULL for auto-detect
priority INTEGER NOT NULL DEFAULT 100,
removed_at TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE(provider_id, model_id)
)
```
`provider_id` references `llm_providers` — the same provider table used by LLM models.
Only providers that declare `ServiceType::Transcribe` in `supported_types()` should have rows here.
---
## DB insert — soft-delete revival
`transcribe_models` has two `UNIQUE` constraints: `name` and `(provider_id, model_id)`. `db::insert()` attempts to revive a soft-deleted row before falling back to a plain `INSERT` — same pattern as `tts/db.rs`. See [tts-providers.md](tts-providers.md#db-insert--soft-delete-revival) for the full description.
---
## When to Update This File
- A new concrete `Transcribe` implementation is added
- `transcribe_models` schema changes
- A provider gains or loses transcription support
docs/providers/tts.md
+332
View File
@@ -0,0 +1,332 @@
# Text-to-Speech Providers
Cloud TTS via OpenAI-compatible or ElevenLabs endpoints, plus plugin-registered local engines.
---
## Architecture
```text
crates/core-api/src/tts.rs
— TextToSpeech trait (provider interface)
— TtsProvider trait (resolve active provider)
— TtsRegistry trait (plugin write-side: register/unregister)
— TtsModelRecord (DB record type — moved here from main crate)
— RemoteTtsModelInfo (remote catalog type — moved here from main crate)
src/core/tts/
mod.rs — TtsModelInfo (API response type), re-exports from core-api
db.rs — SQL layer for tts_models table
manager.rs — TtsManager (DB-aware, owns the table, impls TtsProvider + TtsRegistry)
openai_tts.rs — OpenAiTtsSynthesiser: impl TextToSpeech via OpenAI-compatible HTTP JSON
crates/plugin-elevenlabs/src/lib.rs
ElevenLabsTtsSynthesiser: impl TextToSpeech via ElevenLabs v1 API
ElevenLabsTranscriber: impl Transcribe via ElevenLabs Scribe API
ElevenLabsProvider: impl ApiProvider (model listing, build_tts, build_transcriber)
ElevenLabsPlugin: impl Plugin — registers ElevenLabsProvider on start
```
Two kinds of providers coexist:
| Kind | Source | Example |
| ---- | ------ | ------- |
| **DB-backed** | `tts_models` table, built from `llm_providers` credentials | `OpenAiTtsSynthesiser`, `ElevenLabsTtsSynthesiser` |
| **Plugin-registered** | Ephemeral — registered at runtime by plugins | `OrpheusTtsPlugin`, `KokoroTtsPlugin` |
`get()` returns the first plugin provider (if any is running), then the first DB-backed provider ordered by `priority ASC`.
---
## Traits (crates/core-api)
```rust
// core_api::tts
#[async_trait]
pub trait TextToSpeech: Send + Sync {
fn id(&self) -> &str;
fn name(&self) -> &str;
fn description(&self) -> Option<&str>; // default None
fn instructions(&self) -> Option<&str>; // default voice style stored in DB
async fn synthesize(&self, text: &str, instructions: Option<&str>) -> Result<Vec<u8>>;
}
/// Read-side used by callers to get the active provider.
#[async_trait]
pub trait TtsProvider: Send + Sync {
async fn get(&self) -> Option<Arc<dyn TextToSpeech>>;
}
/// Write-side used by plugins to register/unregister ephemeral providers.
#[async_trait]
pub trait TtsRegistry: Send + Sync {
async fn register(&self, provider: Arc<dyn TextToSpeech>);
async fn unregister(&self, id: &str);
}
```
### `instructions` semantics
| Level | Where set | Precedence |
|-------|-----------|------------|
| **DB-level** | `tts_models.instructions` column | Default for this model config |
| **Call-time** | `synthesize(text, Some(override))` | Overrides DB-level for this call |
This lets the LLM (or a plugin) say "respond in a cheerful tone" on a per-turn basis without changing the model's default configuration.
---
## Manager API
```rust
// Async constructor — loads DB models on startup
TtsManager::new(pool: Arc<SqlitePool>, registry: Arc<ProviderRegistry>) -> Result<Arc<Self>>
// Resolution
tts_manager.get().await // → Option<Arc<dyn TextToSpeech>> (plugins first, then DB)
// Plugin registration (ephemeral)
tts_manager.register(Arc::new(synthesiser)).await
tts_manager.unregister("kokoro_local").await
// DB-backed CRUD (called by REST API handlers)
tts_manager.add_model(record).await // → Result<i64>
tts_manager.update_model(id, record).await
tts_manager.delete_model(id).await // soft delete
tts_manager.get_model(id).await // → Option<TtsModelRecord>
// Listings
tts_manager.list_models_info().await // DB-backed only → Vec<TtsModelInfo>
tts_manager.list_all_info().await // plugin + DB → Vec<TtsModelInfo>
// Remote model catalog (calls ApiProvider::list_tts_models)
tts_manager.list_provider_models(provider_id).await // → Result<Vec<RemoteTtsModelInfo>>
```
`RemoteTtsModelInfo` fields: `id`, `name`, `description`, `languages` (BCP-47 codes), `cost_factor: Option<f64>` (relative cost multiplier, e.g. `1.0` = standard), `instructions: Option<String>` (usage guidance for LLM and UI pre-fill).
---
## OpenAiTtsSynthesiser
Implemented in `src/core/tts/openai_tts.rs`.
Calls `POST {base_url}/audio/speech` with a JSON body:
| Field | Value |
|-------|-------|
| `model` | Provider model ID (e.g. `tts-1`, `tts-1-hd`, `gpt-4o-mini-tts`) |
| `input` | Text to synthesise |
| `voice` | From `tts_models.voice_id`; `NULL` ⇒ `"alloy"` |
| `response_format` | From `tts_models.response_format`; `NULL` ⇒ `"mp3"` |
| `instructions` | Optional natural-language style/tone/speed override |
Returns raw audio bytes in the requested `response_format` (default `mp3`).
### `voice`
The `voice` field is taken from the per-model `tts_models.voice_id` column, falling back to `alloy` when unset. Voice names are **provider-specific**: OpenAI uses `alloy`/`echo`/`fable`/`onyx`/`nova`/`shimmer`; Gemini uses `Kore`/`Puck`/`Zephyr`/`Charon`/… An unknown voice name can make the provider error (OpenRouter→Gemini surfaces this as a generic `500`), so set `voice_id` to a value valid for the chosen model.
### `response_format`
The audio container/codec is taken from the per-model `tts_models.response_format` column (`mp3`, `opus`, `aac`, `flac`, `wav`, `pcm`). Leaving it empty falls back to `mp3`. Some models reject `mp3` and require a specific value — e.g. `google/gemini-*-tts-*` on OpenRouter returns `400 … only supports response_format="pcm"`. Set the column (UI dropdown in the model form) to the value the model demands.
> **Note:** `pcm` is raw, headerless audio. Consumers that need a playable container handle the transcode themselves — the Telegram `send_voice_message` tool converts whatever `output_format` reports (mp3/wav/**pcm**/…) to Ogg/Opus via ffmpeg before sending. See [plugins/telegram.md](../plugins/telegram.md).
### `output_format()`
`TextToSpeech::output_format()` reports the container/codec of the bytes returned by `synthesize` (`mp3`, `opus`, `wav`, `pcm`, …; default `"mp3"`). `OpenAiTtsSynthesiser` returns its configured `response_format`. Consumers that need a specific container use this to decide whether and how to transcode — e.g. raw `pcm` is headerless and must be described to the decoder, so the hint is essential there.
### Supported providers
| Provider | `base_url` | Notes |
| -------- | ---------- | ----- |
| OpenAI | `https://api.openai.com/v1` | Models: `tts-1`, `tts-1-hd`, `gpt-4o-mini-tts` |
| OpenRouter | `https://openrouter.ai/api/v1` | OpenAI-compatible endpoint |
---
## ElevenLabsTtsSynthesiser
Implemented in `crates/plugin-elevenlabs/src/lib.rs` (via `plugin-elevenlabs`).
Calls `POST https://api.elevenlabs.io/v1/text-to-speech/{voice_id}` with auth header `xi-api-key` (not Bearer).
| Field in DB record | Meaning |
| ------------------ | ------- |
| `model_id` | ElevenLabs **generation model** (e.g. `eleven_multilingual_v2`, `eleven_turbo_v2_5`) |
| `voice_id` | ElevenLabs **voice ID** (e.g. `21m00Tcm4TlvDq8ikWAM`). Required for ElevenLabs. |
| `instructions` | Injected into LLM system prompt; not sent to ElevenLabs API |
**Legacy fallback:** if `voice_id` is `NULL` (records created before the field split), `model_id` is treated as the voice ID and the generation model defaults to `eleven_multilingual_v2`. This keeps existing records working after the migration.
Returns raw MP3 bytes. Provider type: `elevenlabs` — requires an `xi-api-key` stored in `llm_providers.api_key`. No `base_url` needed.
### Remote model catalog
`ElevenLabsProvider::list_tts_models()` calls `GET https://api.elevenlabs.io/v1/models`, filters entries where `can_do_text_to_speech: true`, and returns `RemoteTtsModelInfo` with:
- `cost_factor` from the `token_cost_factor` field
- `instructions` from `elevenlabs_tts_instructions(model_id)` — per-model usage guidance (supported tags, non-verbal sound syntax, etc.)
---
## REST API
| Method | Path | Description |
| ------ | ---- | ----------- |
| `GET` | `/api/tts/models` | All models — plugin-registered first (`from_plugin: true`), then DB-backed |
| `POST` | `/api/tts/models` | Add a new TTS model |
| `GET` | `/api/tts/models/{id}` | Get a DB-backed model record |
| `PUT` | `/api/tts/models/{id}` | Update a DB-backed model |
| `DELETE` | `/api/tts/models/{id}` | Soft-delete a DB-backed model |
| `GET` | `/api/tts/providers/{id}/models` | List remote TTS models from a configured provider (`RemoteTtsModelInfo[]`) |
The provider models endpoint calls `TtsManager::list_provider_models()` → `ApiProvider::list_tts_models()`. Returns an error if the provider does not support model listing.
Handled by `src/frontend/api/tts_models.rs`.
---
## DB: tts_models table
```sql
CREATE TABLE tts_models (
id INTEGER PRIMARY KEY AUTOINCREMENT,
provider_id INTEGER NOT NULL REFERENCES llm_providers(id),
model_id TEXT NOT NULL, -- generation model (e.g. eleven_multilingual_v2, tts-1-hd)
voice_id TEXT, -- speaker voice (required for ElevenLabs; NULL for OpenAI)
name TEXT NOT NULL UNIQUE,
description TEXT, -- human-readable, shown in UI
instructions TEXT, -- default voice style / tone / speed
response_format TEXT, -- audio format (mp3/opus/aac/flac/wav/pcm); NULL ⇒ mp3
priority INTEGER NOT NULL DEFAULT 100,
removed_at TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE(provider_id, model_id)
)
```
`voice_id` was added in **schema version 2** via `ALTER TABLE tts_models ADD COLUMN voice_id TEXT`. `response_format` was added in **schema version 18** via `ALTER TABLE tts_models ADD COLUMN response_format TEXT`. See [database.md](database.md#migration-pattern).
---
## Plugin Registration
`TtsRegistry` is exposed on `PluginContext` as `ctx.tts_registry`. Plugin crates depend only on `core-api`.
```rust
use core_api::tts::TextToSpeech;
struct MyTtsSynth { /* ... */ }
#[async_trait]
impl TextToSpeech for MyTtsSynth {
fn id(&self) -> &str { "kokoro_local" }
fn name(&self) -> &str { "Kokoro Local" }
async fn synthesize(&self, text: &str, instructions: Option<&str>) -> Result<Vec<u8>> {
// call local engine, return MP3 bytes
}
}
// In Plugin::start() when enabled:
ctx.tts_registry.register(Arc::new(MyTtsSynth { ... })).await;
// In Plugin::stop():
ctx.tts_registry.unregister("kokoro_local").await;
```
---
## Kokoro TTS (`plugin-tts-kokoro`)
Lightweight local TTS using the Kokoro ONNX model (~310 MB model + ~27 MB voices). No PyTorch or GPU required — runs fully on CPU via ONNX Runtime.
**Crate:** `crates/plugin-tts-kokoro/`
**Plugin ID:** `kokoro_tts`
### How it works
The Python server (`kokoro_server.py`) is embedded in the crate via `include_str!`. On start the plugin writes it to a temp path and spawns it as a FastAPI subprocess. The server downloads `kokoro-v1.0.onnx` and `voices-v1.0.bin` from GitHub Releases on first run, then exposes `POST /synthesize` returning WAV bytes. The plugin registers itself with `TtsManager` and deregisters on stop.
### Setup
```text
toggle_item(kind="plugin", id="kokoro_tts", enabled=true)
```
Optional config:
```json
{ "voice": "if_sara", "lang": "it", "speed": 1.0 }
```
### Config
| Field | Values | Default |
| ----- | ------ | ------- |
| `voice` | Any Kokoro voice ID (e.g. `if_sara`, `im_nicola`, `af_heart`) | `if_sara` |
| `lang` | BCP-47 language code | `it` |
| `speed` | Speech rate multiplier | `1.0` |
Python deps (in `requirements.txt`): `kokoro-onnx`, `soundfile`.
---
## Orpheus TTS 3B (`plugin-tts-orpheus-3b`)
Local, on-device TTS using the Orpheus 3B model. Runs a Python subprocess for inference.
**Crate:** `crates/plugin-tts-orpheus-3b/`
**Plugin ID:** `orpheus_tts_3b`
**Note:** the FP16 model is large (~6 GB) and uses significant RAM during inference. Prefer `int8` quantization on memory-constrained machines, or use `plugin-tts-kokoro` as a lighter alternative.
**How it works:** the Python inference server (`orpheus_server.py`) is embedded in the plugin binary via `include_str!`. On start, the plugin writes it to `models/orpheus-3b/orpheus_server.py` and spawns it. The server prints `PORT:<n>` to stdout when ready; the plugin reads that port and registers itself as a `TextToSpeech` provider. On stop, the subprocess is killed.
**Setup:**
```text
set_secret("HUGGINGFACE_TOKEN", "hf_...")
configure_plugin("orpheus_tts_3b", {"quantization": "int8", "voice": "tara"})
toggle_item(kind="plugin", id="orpheus_tts_3b", enabled=true)
```
**Config:**
| Field | Values | Default |
| ----- | ------ | ------- |
| `quantization` | none / int8 / int4 | int8 |
| `voice` | tara / dan / leah / zac / zoe / mia / julia / leo | tara |
---
## DB insert — soft-delete revival
`tts_models` has two `UNIQUE` constraints: `name` and `(provider_id, model_id)`. Soft-deleted rows (where `removed_at IS NOT NULL`) still hold those unique values, which would cause a plain `INSERT` to fail when re-adding a previously deleted model.
`db::insert()` handles this by first attempting to revive the soft-deleted row: it runs an `UPDATE … RETURNING id` that matches on `removed_at IS NOT NULL AND (provider_id=? AND model_id=? OR name=?)`. If a row is found it is restored with the new values and its `removed_at` is set to `NULL`; only if no match is found does a plain `INSERT` run. The same pattern is applied in `transcribe/db.rs` and `image_generate/db.rs`.
---
## Telegram `send_voice_message` tool
When the Telegram plugin is active and at least one TTS provider is available, the LLM-callable tool `send_voice_message` is injected into every Telegram session. It is absent when no TTS provider is configured.
| Aspect | Detail |
| --- | --- |
| Tool name | `send_voice_message` |
| Parameter | `text: String` — the text to synthesise |
| Provider selection | Highest-priority active provider (`TtsProvider::get()`) |
| Transport | `bot.send_voice()` — Telegram voice message |
| Instructions | The provider's `instructions()` string is embedded in the tool description so the LLM knows how to format text for that engine |
The tool resolves the synthesiser at call time (not at registration time), so a TTS provider that becomes available mid-conversation is picked up automatically on the next call.
---
## When to Update This File
- A new concrete `TextToSpeech` implementation is added
- `tts_models` schema changes
- A provider gains or loses TTS support
docs/relay/crypto.md
+429
View File
@@ -0,0 +1,429 @@
# Crypto Contract
> This file is the **single source of truth** for cryptography. Relay, plugin, and app MUST
> implement exactly what follows. Any divergence breaks interoperability. The words MUST / MUST NOT /
> SHOULD carry the RFC 2119 meaning. Verify your implementation against
> [test-vectors.md](test-vectors.md) **before** integrating.
Field encoding: see [index.md §5](index.md). In short: keys/signatures/ids/nonces in **lowercase
hex**, ciphertext in **standard base64 with padding**.
---
## 1. Domain Constants (NORMATIVE)
All strings are ASCII/UTF-8, without NUL terminator unless noted as `\x00`.
| Name | Value (bytes) | Use |
|------|---------------|-----|
| `KDF_SALT` | `"skald-kdf-v1"` | HKDF seed → keypair (§3) |
| `KDF_INFO_X25519` | `"x25519"` | HKDF info, X25519 branch (§3) |
| `KDF_INFO_ED25519` | `"ed25519"` | HKDF info, ed25519 branch (§3) |
| `SESSION_SALT` | `"skald-session-v1"` | HKDF shared_secret → aes_key (§5) |
| `SESSION_INFO` | `"aes-256-gcm"` | HKDF info, AEAD key (§5) |
| `NS_DOMAIN` | `"skald-namespace-v1"` | `namespace_id` derivation (§7) |
| `AUTH_DOMAIN` | `"skald-relay-auth-v1"` | Challenge-response signature (§8) |
| `NONCE_DIR_AGENT_TO_CLIENT` | `0x00 0x00 0x00 0x01` | Nonce prefix, agent→client direction (§6) |
| `NONCE_DIR_CLIENT_TO_AGENT` | `0x00 0x00 0x00 0x02` | Nonce prefix, client→agent direction (§6) |
| `PIPE_AUTH_DOMAIN` | `"skald-pipe-auth-v1"` | Pipe data-plane challenge signature ([pipe.md §3.1](pipe.md)) |
| `PIPE_KDF_SALT` | `"skald-pipe-v1"` | HKDF salt: ephemeral ECDH → per-pipe AES key ([pipe.md §4](pipe.md)) |
| `PIPE_KDF_INFO` | `"pipe-aes-256-gcm"` | HKDF info, per-pipe AES key ([pipe.md §4](pipe.md)) |
| `NONCE_DIR_PIPE_INITIATOR` | `0x00 0x00 0x00 0x03` | Nonce prefix, pipe initiator→responder ([pipe.md §4](pipe.md)) |
| `NONCE_DIR_PIPE_RESPONDER` | `0x00 0x00 0x00 0x04` | Nonce prefix, pipe responder→initiator ([pipe.md §4](pipe.md)) |
Algorithms: **X25519** (RFC 7748), **Ed25519** (RFC 8032), **HKDF-SHA256** (RFC 5869),
**AES-256-GCM** (NIST SP 800-38D), **SHA-256** (FIPS 180-4).
> The **pipe** (relayed byte-stream, [pipe.md](pipe.md)) reuses this entire suite — X25519 ECDH,
> HKDF, AES-256-GCM with the `DIR ‖ counter` nonce (§6) — keyed by a **per-pipe ephemeral** DH
> (Perfect Forward Secrecy), with `aad = connection_id`. No new primitives.
---
## 2. Persistent Material: the Seed
Every actor with a cryptographic identity (agent and each client) holds **one single persistent
secret**: a **32-byte seed** generated from CSPRNG.
- Agent: `data/relay/seed`, 32-byte binary file, permissions `0600`. Generated on first start.
- iOS client: 32 bytes in Keychain, attribute `kSecAttrAccessibleWhenUnlockedThisDeviceOnly`.
- Android client: 32 bytes in Keystore / EncryptedSharedPreferences (hardware-backed if available).
Two keypairs are derived from this seed (§3). The seed MUST NOT leave the device and MUST NOT
ever be transmitted. Private keys are regenerated from the seed on each startup; they are not
persisted separately.
> **Why two keypairs?** Ed25519 is for **signing** (authentication toward the relay).
> X25519 is for **ECDH** (E2E key agreement). They are related curves with distinct roles and APIs
> on all platforms (CryptoKit separates them: `Curve25519.Signing` vs `Curve25519.KeyAgreement`).
> **Never** convert an ed25519 key into X25519 by reinterpreting the bytes: this is cryptographically
> wrong. Both are derived independently from the seed.
---
## 3. Key Derivation from Seed (NORMATIVE)
Identical across all platforms. `HKDF` = HKDF-SHA256, 32-byte output.
```
x25519_priv = HKDF(ikm = seed, salt = KDF_SALT, info = KDF_INFO_X25519, len = 32)
ed25519_priv = HKDF(ikm = seed, salt = KDF_SALT, info = KDF_INFO_ED25519, len = 32)
```
- `x25519_priv` (32 bytes) is the X25519 private **scalar**. Libraries apply RFC 7748 *clamping*
internally; do not clamp manually. `x25519_pub = X25519(x25519_priv, basepoint)`.
- `ed25519_priv` (32 bytes) is the **Ed25519 seed** (the 32-byte "private key" of RFC 8032).
`ed25519_pub` (32 bytes) is derived from it per RFC 8032.
> Terminology note: in Ed25519, the 64-byte "private key" is `seed(32) ‖ pub(32)`. Here the secret
> material is the **32-byte seed** (`ed25519_priv` above). Do not confuse the 32 bytes of *our* seed
> (§2) with the 32 bytes of the *Ed25519 seed* (HKDF output): they are different things.
### Rust (agent / relay-side verification)
```rust
use hkdf::Hkdf;
use sha2::Sha256;
use ed25519_dalek::SigningKey; // ed25519-dalek = "2"
use x25519_dalek::{StaticSecret, PublicKey}; // x25519-dalek = "2"
fn derive_keys(seed: &[u8; 32]) -> (SigningKey, StaticSecret) {
let hk = Hkdf::<Sha256>::new(Some(b"skald-kdf-v1"), seed);
let mut x = [0u8; 32];
hk.expand(b"x25519", &mut x).unwrap();
let x25519_priv = StaticSecret::from(x); // internal clamping
let mut e = [0u8; 32];
hk.expand(b"ed25519", &mut e).unwrap();
let ed25519_priv = SigningKey::from_bytes(&e);
(ed25519_priv, x25519_priv)
}
// pub keys:
// ed25519_pub = signing_key.verifying_key().to_bytes() // 32B
// x25519_pub = PublicKey::from(&x25519_priv).to_bytes() // 32B
```
### Swift (iOS, CryptoKit)
```swift
import CryptoKit
func deriveKeys(seed: Data) -> (signing: Curve25519.Signing.PrivateKey,
agreement: Curve25519.KeyAgreement.PrivateKey) {
let ikm = SymmetricKey(data: seed)
let salt = Data("skald-kdf-v1".utf8)
let xRaw = HKDF<SHA256>.deriveKey(inputKeyMaterial: ikm, salt: salt,
info: Data("x25519".utf8), outputByteCount: 32)
let eRaw = HKDF<SHA256>.deriveKey(inputKeyMaterial: ikm, salt: salt,
info: Data("ed25519".utf8), outputByteCount: 32)
let agreement = try! Curve25519.KeyAgreement.PrivateKey(
rawRepresentation: xRaw.withUnsafeBytes { Data($0) })
let signing = try! Curve25519.Signing.PrivateKey(
rawRepresentation: eRaw.withUnsafeBytes { Data($0) })
return (signing, agreement)
}
```
### Kotlin (Android — reference)
Use **BouncyCastle / Tink**: `HKDFBytesGenerator(SHA256Digest)` with the same salt/info, then
`X25519PrivateKeyParameters` and `Ed25519PrivateKeyParameters` from the 32 derived bytes.
---
## 4. ECDH — Key Agreement (X25519, ONLY path)
The agent and each client exchange their **X25519 public key** (the agent via QR; the client via
the pairing frame — see [relay-protocol.md](relay-protocol.md)). The shared secret:
```
shared_secret = X25519(my_x25519_priv, peer_x25519_pub) // 32 bytes
```
It is symmetric: `X25519(a_priv, b_pub) == X25519(b_priv, a_pub)`. **MUST** always and only use
X25519 keys. Ed25519 keys NEVER enter ECDH.
```rust
let shared = my_x25519_priv.diffie_hellman(&PublicKey::from(peer_x25519_pub_bytes));
let shared_secret: [u8; 32] = *shared.as_bytes();
```
```swift
let peerPub = try Curve25519.KeyAgreement.PublicKey(rawRepresentation: peerX25519PubBytes)
let shared = try myAgreementPriv.sharedSecretFromKeyAgreement(with: peerPub)
// `shared` is a SharedSecret; do NOT use it raw: pass through HKDF (§5).
```
> **Point validation.** Standard libraries (x25519-dalek, CryptoKit) handle low-order points;
> an implementation that does not MUST reject an all-zero shared secret.
---
## 5. AEAD Key Derivation (HKDF)
The raw shared secret is never used directly as a key. It is derived:
```
aes_key = HKDF(ikm = shared_secret, salt = SESSION_SALT, info = SESSION_INFO, len = 32)
```
```rust
let hk = Hkdf::<Sha256>::new(Some(b"skald-session-v1"), &shared_secret);
let mut aes_key = [0u8; 32];
hk.expand(b"aes-256-gcm", &mut aes_key).unwrap();
```
```swift
let aesKey = shared.hkdfDerivedSymmetricKey(using: SHA256.self,
salt: Data("skald-session-v1".utf8),
sharedInfo: Data("aes-256-gcm".utf8),
outputByteCount: 32)
```
`aes_key` is **per-peer** (one per agent↔client pair) and static for the life of the pairing
(no PFS in the current protocol).
---
## 6. AEAD — AES-256-GCM with Counter Nonce and AAD (NORMATIVE)
**All** E2E messages are encrypted this way. There is no separate MAC: **GCM is already
authenticated**. (No separate HMAC — it would be redundant and violate key-separation.)
### 6.1 Nonce — Monotonic Counter, NOT Random
The GCM nonce is **12 bytes** and is built deterministically to prevent reuse and provide
**anti-replay**:
```
nonce (12B) = DIR (4B) ‖ counter (8B, big-endian)
```
- `DIR` = `NONCE_DIR_AGENT_TO_CLIENT` if the encryptor is the agent, `NONCE_DIR_CLIENT_TO_AGENT`
if it is the client. Ensures the two directions never collide even though they share `aes_key`.
- `counter` is a **strictly increasing** 64-bit integer, **persisted per-peer and per-direction**.
Starts at `1`. Increments by 1 per sent message. MUST be persisted **before** sending (so a
crash cannot cause reuse).
The **receiver** maintains `last_seen_counter` for (peer, direction) and MUST reject any message
with `counter <= last_seen_counter` (replay or reorder). Under FIFO store-and-forward delivery,
counters arrive in order; a forward gap is allowed (messages lost), a value `<=` is not.
> Consequence: counters are the primary **anti-replay state**. They survive reconnections and
> restarts because they are persisted. If the send counter is irreversibly reset (e.g. seed
> restored without state), a **re-pairing** is required (new `aes_key`, counters reset together).
### 6.2 AAD — Routing Binding
The AAD (Additional Authenticated Data) binds the ciphertext to routing metadata, so a malicious
relay relabelling `from`/`to` causes decryption to **fail**:
```
AAD (96B) = namespace_id_raw (32B) ‖ from_pubkey (32B) ‖ to_pubkey (32B)
```
- `namespace_id_raw` = the 32 raw bytes of the hash from §7 (NOT the hex string).
- `from_pubkey`, `to_pubkey` = **ed25519** public keys (32 raw bytes) of sender and recipient
(same values used for routing in the envelope).
- The receiver reconstructs the AAD from the `from`/`to` fields of the received envelope and its
own `namespace_id`. If they do not match those used in encryption → invalid GCM tag → discard.
### 6.3 Encrypted Block Format
```
sealed = ciphertext ‖ tag(16B) // GCM "combined" output, WITHOUT nonce
```
The `nonce` travels **in plaintext** in a separate envelope field (it is public by definition;
its integrity is guaranteed because GCM uses it as an authenticated IV). On the wire (inside the
E2E JSON payload, before the framing of §… is applied):
```json
{ "nonce": "<hex 24>", "ciphertext": "<base64 of (ciphertext‖tag)>" }
```
In the protobuf transport (`Message` frame) the fields are raw bytes — no hex, no base64.
### 6.4 Rust
```rust
use aes_gcm::{Aes256Gcm, KeyInit, Nonce};
use aes_gcm::aead::{Aead, Payload};
fn seal(aes_key: &[u8;32], dir: [u8;4], counter: u64,
aad: &[u8;96], plaintext: &[u8]) -> (Vec<u8> /*nonce*/, Vec<u8> /*sealed*/) {
let mut nonce = [0u8; 12];
nonce[..4].copy_from_slice(&dir);
nonce[4..].copy_from_slice(&counter.to_be_bytes());
let cipher = Aes256Gcm::new(aes_key.into());
let sealed = cipher.encrypt(Nonce::from_slice(&nonce),
Payload { msg: plaintext, aad }).expect("encrypt");
(nonce.to_vec(), sealed)
}
fn open(aes_key: &[u8;32], nonce: &[u8;12], aad: &[u8;96], sealed: &[u8]) -> Option<Vec<u8>> {
let cipher = Aes256Gcm::new(aes_key.into());
cipher.decrypt(Nonce::from_slice(nonce), Payload { msg: sealed, aad }).ok()
}
```
### 6.5 Swift
```swift
func seal(aesKey: SymmetricKey, dir: [UInt8], counter: UInt64,
aad: Data, plaintext: Data) throws -> (nonce: Data, sealed: Data) {
var n = Data(dir) // 4B
var be = counter.bigEndian
n.append(Data(bytes: &be, count: 8)) // +8B = 12B
let nonce = try AES.GCM.Nonce(data: n)
let box = try AES.GCM.seal(plaintext, using: aesKey,
nonce: nonce, authenticating: aad)
// box.ciphertext ‖ box.tag == "sealed"
return (n, box.ciphertext + box.tag)
}
func open(aesKey: SymmetricKey, nonce: Data, aad: Data, sealed: Data) throws -> Data {
let ct = sealed.prefix(sealed.count - 16)
let tag = sealed.suffix(16)
let box = try AES.GCM.SealedBox(nonce: AES.GCM.Nonce(data: nonce),
ciphertext: ct, tag: tag)
return try AES.GCM.open(box, using: aesKey, authenticating: aad)
}
```
### 6.6 Static Key Operational Limit
With a static `aes_key` and a 64-bit counter there is no practical risk of nonce exhaustion or
reuse (the counter is unique by construction). The NIST limit for AES-GCM with a single key is
~2³² messages before considering rotation: unreachable for this workload (approvals/clarifications).
Key rotation via **re-pairing** is nevertheless recommended if compromise is suspected.
---
## 7. `namespace_id` Derivation (NORMATIVE)
The namespace id is **immutably bound** to the agent's identity key — preventing takeover without
requiring relay-side state to guarantee it:
```
namespace_id_raw = SHA256( NS_DOMAIN ‖ 0x00 ‖ agent_ed25519_pub(32B) ) // 32 bytes
namespace_id = hex(namespace_id_raw) // 64 chars
```
- The relay, upon receiving the agent's auth, MUST verify that `namespace_id` derives from the
presented `agent_ed25519_pub` and that the challenge signature is valid under that key.
- The client, from the QR, MUST verify `namespace_id == hex(SHA256(NS_DOMAIN ‖ 0x00 ‖ agent_ed25519_pub))`
using the `agent_ed25519_pub` from the QR. This way it does not trust the relay for the id.
- `namespace_id_raw` is also the value used in the AAD (§6.2).
---
## 8. Challenge-Response (Key Ownership Proof)
On WS open the **relay speaks first** and sends a challenge. The connecting peer (any role) signs
and responds. Transport details in [relay-protocol.md](relay-protocol.md); here the primitive.
```
challenge_nonce = 32 random bytes (CSPRNG on the relay side), sent as raw bytes in protobuf
msg_to_sign = AUTH_DOMAIN ‖ 0x00 ‖ challenge_nonce_raw(32B)
signature = Ed25519_sign(ed25519_priv, msg_to_sign) // 64 bytes
```
The relay verifies `Ed25519_verify(pub, signature, msg_to_sign)`. The **domain separation**
(`AUTH_DOMAIN`) prevents an auth signature from being reusable in other contexts.
```rust
let mut msg = Vec::with_capacity(20 + 32);
msg.extend_from_slice(b"skald-relay-auth-v1");
msg.push(0x00);
msg.extend_from_slice(&challenge_nonce_raw); // 32B
let sig = ed25519_priv.sign(&msg); // ed25519-dalek: Signer
```
```swift
var msg = Data("skald-relay-auth-v1".utf8); msg.append(0x00); msg.append(challengeNonceRaw)
let sig = try signingPriv.signature(for: msg) // 64B
```
> Ed25519 internally hashes the message: do **not** pre-hash with SHA-256. Sign
> `AUTH_DOMAIN ‖ 0x00 ‖ nonce` directly.
---
## 9. Pairing Token (Capability Bearer, NOT a Signature)
The `pairing_token` is a **single-use bearer secret**, not a signature:
```
pairing_token = 32 random bytes (CSPRNG on the agent side), as raw bytes in protobuf
```
- The agent generates it on each `pairing_start`, puts it in the QR, and sends it to the relay
(`PairingStart` frame). 256-bit entropy: not guessable.
- The relay treats it as an opaque blob: **byte-for-byte** comparison, **expiry**, **single-use**
(consumed on first successful pairing), valid only while the namespace is in pairing mode.
- The client presents it in the pairing frame. It cannot verify it cryptographically (bearer token):
security comes from **out-of-band QR** + **short TTL** + **single-use** + **explicit agent confirmation**
of the new device.
> No Ed25519 signature on the token: nobody would verify it (security theater). A 256-bit random
> secret is simpler and equally strong as a capability.
---
## 10. Key Storage
### Agent (filesystem + DB)
```
data/relay/
└── seed # 32 bytes, 0600. The only persistent secret.
```
DB table `relay_clients` (see [../plugins/mobile-connector.md](../plugins/mobile-connector.md)):
stores per-client `x25519_pub`, `send_counter`, `recv_counter`. `shared_secret` and `aes_key`
are **never persisted**: re-derived from `seed` + `x25519_pub` on each startup (smaller attack
surface; negligible cost).
### Client (Keychain / Keystore)
- `seed` (32B) with `kSecAttrAccessibleWhenUnlockedThisDeviceOnly`, shared with the
**Notification Service Extension** via **Keychain Access Group** (the NSE must be able to
derive `aes_key`).
- `namespace_id`, `relay_url`, `agent_ed25519_pub`, `agent_x25519_pub`, `send_counter`,
`recv_counter`: in the same shared storage.
- App uninstall → keys lost → re-pairing required.
---
## 11. Algorithm Summary
| Operation | Algorithm | Input → Output |
|-----------|-----------|----------------|
| Seed | CSPRNG | → 32B |
| Key derivation | HKDF-SHA256 (`KDF_SALT`, info) | seed 32B → x25519_priv 32B, ed25519_priv 32B |
| ECDH | X25519 | my_x25519_priv + peer_x25519_pub → shared 32B |
| AEAD key derivation | HKDF-SHA256 (`SESSION_SALT`, `SESSION_INFO`) | shared 32B → aes_key 32B |
| Encryption | AES-256-GCM | aes_key + nonce(DIR‖counter) + AAD(96B) → ciphertext‖tag |
| `namespace_id` | SHA-256 (`NS_DOMAIN`) | agent_ed25519_pub → 32B (hex) |
| Auth | Ed25519 sign/verify (`AUTH_DOMAIN`) | ed25519_priv + challenge → sig 64B |
| Pairing token | CSPRNG | → 32B single-use bearer |
## 12. Security Considerations
- **PFS**: not in the current protocol. Static `aes_key` → traffic capture + later seed theft =
plaintext for historical messages. Roadmap: ephemeral ECDH per session.
- **Replay**: prevented by monotonic counter (§6.1) + `request_id` idempotency
([payloads.md](payloads.md)) + `ts` freshness.
- **Malicious relay**: cannot read content (E2E) and cannot relabel `from`/`to` (AAD, §6.2);
it can only **drop/hold/reorder** → mitigated by fail-safe + TTL pending on the agent side.
- **Timing**: Ed25519 and AES-GCM are constant-time in the reference implementations
(ed25519-dalek, aes-gcm with AES-NI feature, CryptoKit). Tag/token comparisons MUST be
constant-time (`subtle` / `constantTimeAreEqual`).
- **Input validation**: reject malformed hex/base64, wrong lengths, and every failed decryption
**without** distinguishing the cause in error messages.
docs/relay/describe-and-push.md
+114
View File
@@ -0,0 +1,114 @@
# Tool Description & Push Delivery (`describe` + `blocks` + APNs)
_Normative for approval rendering on the client side._
> How the app receives and displays **what** it is approving, and how the push notification stays
> lightweight. Primarily concerns **approvals** (tool arguments are hard to read as raw JSON);
> clarifications already carry a human-written `question` from the LLM.
>
> This file defines the **wire contract** (what the client receives). How the blocks are
> produced on the agent side (built-in vs MCP, templates) is in the agent's tool description
> infrastructure.
---
## 1. Two Representations, One Item
Every `approvals[]` entry in an `inbox_update` ([payloads.md §3.1](payloads.md)) carries **two**
views of the same tool call:
| Field | What it is | Used for |
|-------|-----------|----------|
| `summary` | Short human string, generated by `describe(Short)` on the agent (e.g. *"Send an email to mario@acme.com"*) | Card row, **push notification**, conversation log |
| `blocks` | **Structured** parameter description, generated by `describe_view(args)` | Detail screen (forms/tables/diffs) |
| `arguments` | Raw args (tool's JSON) | Final fallback / debug |
`summary` is the source of truth for "narrow" surfaces (notification, badge); `blocks` for the
detail screen. `arguments` remain as a safety net.
---
## 2. `blocks` Schema (wire)
A tool call is described as a **list of typed blocks**. The vocabulary is **small and stable**:
tools are unlimited, block types are not. Each client maps types to its native widgets *once*,
and it works for any present and future tool.
```json
{
"v": 1,
"summary": "Send an email to to@mail.com",
"blocks": [
{ "type": "key_value", "key": "Email-To", "value": "to@mail.com", "value_type": "email" },
{ "type": "field", "label": "Subject", "value": "Q3 Estimate", "value_type": "string" },
{ "type": "block", "label": "Body", "value": "…body…", "value_type": "text" }
]
}
```
Example for `write_file`:
```json
{
"v": 1,
"summary": "Write /path/x.rs",
"blocks": [
{ "type": "key_value", "key": "File", "value": "/path/x.rs", "value_type": "path" },
{ "type": "block", "label": "Diff", "value": "= same\n- old line\n+ new line", "value_type": "diff" }
]
}
```
**`type`** = layout hint (three values suffice):
| `type` | Rendering |
|--------|-----------|
| `key_value` | Compact `key: value` row |
| `field` | Labelled field, value on one line |
| `block` | Extended content with label (multi-line / dedicated viewer) |
**`value_type`** = value semantics → widget + formatting:
`string` · `text` · `markdown` · `code` · `diff` · `command` · `email` · `url` · `path` · `json` · `number` · `boolean` · `datetime` · `secret`
Rendering notes:
- `diff` → diff viewer (green/red). `command` / `code` → monospace. `email` / `url` → tappable.
- `secret` → **masked by default** (e.g. API key in args).
- Block fields: `key` (for `key_value`) or `label` (for `field`/`block`), `value`, `value_type`.
---
## 3. Delivery Model: Lightweight Push, Detail via WS
**Principle: the push never carries `blocks`/`arguments`. Only the `summary`.**
| Channel | What travels |
|---------|-------------|
| **WS** (live or `inbox_request` on tap) | **Complete** `inbox_update`: `summary` + `blocks` + `arguments` |
| **APNs/FCM push** | A minimal `notification` (kind §3.2 in payloads.md) with `body` = `summary` (`describe`), **no blocks/args** |
Flow: the app receives the notification with the `summary` line → user taps → app opens WS and
sends `inbox_request` ([payloads.md §4.6](payloads.md)) → receives the complete `inbox_update`
with `blocks` → shows the detail screen.
### Why (zero-trust + size)
- The relay is **zero-trust**: it cannot read the encrypted content, so it **cannot** extract
`summary` from the `inbox_update` blob. It is the **agent** that emits, for the push, a
lightweight `notification` payload (E2E, decrypted by the NSE) containing only the `summary`.
- This way the push **always** fits under the content-in-push threshold (3500B b64,
[server.md §5](server.md)) → notification always readable, without the content-vs-wake juggling
needed for rich payloads.
- The `aps.alert` in plaintext ("Skald / Action required") remains the generic fallback visible
to Apple; the real text (`summary`) is in the encrypted blob that the NSE replaces.
---
## 4. Forward-Compat & Degradation
- `v` at the top of `blocks`. An unknown `type`/`value_type` is not an error: the client
degrades to `key_value` / raw string (never crash).
- If `blocks` is absent (older agent, or tool with no description), the client shows `summary`
and optionally raw `arguments`. Fully backward-compatible.
- Non-form surfaces (e.g. Telegram) **flatten** blocks to text (`key: value`, diff in monospace):
every block is linearisable by construction.
docs/relay/framing.md
+104
View File
@@ -0,0 +1,104 @@
# E2E Plaintext Framing
> Defines the structure of the **bytes that are encrypted** in the `ciphertext` field of the
> `Message` frame ([relay-protocol.md §6](relay-protocol.md)). **The cryptography does not
> change** ([crypto.md](crypto.md)): a blob of bytes is always encrypted with AES-256-GCM. What
> changes is *what* those bytes are: a **versioned frame** wrapping the JSON payload.
>
> The relay remains **blind**: it sees only ciphertext, nothing about versions or compression.
---
## 1. Structure
The **plaintext** (what is encrypted) is:
```
plaintext = version (1 byte) ‖ comp (1 byte) ‖ payload
```
| Field | Byte | Values | Meaning |
|-------|------|--------|---------|
| `version` | 1 | `0x01` \| `0x02` | Framing version. `0x01` = JSON app payload; `0x02` = **pipe signaling** (MsgPack, see below). Unknown value → receiver discards with log. |
| `comp` | 1 | `0x00` \| `0x01` | Compression algorithm applied to `payload` (§2). |
| `payload` | N | — | The content: **JSON UTF-8** ([payloads.md](payloads.md)) for `0x01`, **MsgPack `PipeSignal`** ([pipe.md §2](pipe.md)) for `0x02`; optionally compressed. |
> **`version 0x02` (pipe signaling).** Reserved for the pipe control plane ([pipe.md](pipe.md)):
> `0x02 ‖ 0x00 ‖ <MsgPack PipeSignal>` (uncompressed). It rides this same E2E channel; a receiver
> peeks the first byte to route `0x02` to its pipe layer and `0x01` to the JSON app path. The
> existing `decompress_payload` still only accepts `0x01` — the pipe layer handles `0x02` itself.
`version` and `comp` are **in plaintext inside the plaintext** (readable only after decryption):
they cannot go in the AAD or outside the ciphertext, or the relay would see them. They are
integrity-protected by the GCM tag along with the rest.
> **Two versioning planes, do not confuse.** `version` (this byte, `0x01`) versions the
> **framing** (the binary envelope). The JSON field `v` inside the `payload`
> ([payloads.md §1](payloads.md)) versions the **payload schema**. They are independent: framing
> can evolve while a `kind`'s schema stays fixed, and vice versa. In these documents "version" =
> framing byte; "`v`" = payload schema. (The name `v` is unchanged from the original design for
> consistency with existing payloads.)
## 2. Compression
| `comp` | Algorithm | Notes |
|--------|-----------|-------|
| `0x00` | none | `payload` = JSON UTF-8 as-is. |
| `0x01` | **zlib / DEFLATE** (RFC 1950/1951) | Default for large payloads. Safe interop: Rust `flate2` ↔ iOS `Compression` framework (`COMPRESSION_ZLIB`). |
| `0x02…` | _reserved_ | E.g. `lz4` in the future. Addable without breakage: a receiver that does not know a `comp` value discards with log. |
Rules:
1. **Compress-then-encrypt, always in this order.** The ciphertext is not compressible; compressing
after would give no gain.
2. Compression is **optional on the sender side**, **mandatory on the receiver side**: anyone
receiving MUST handle both `0x00` and `0x01`.
3. **Threshold**: compress only if `len(payload)` exceeds ~1 KiB. Below that, the zlib header
overhead wipes out any gain → use `0x00`.
4. Compression operates on `payload` **only**, not on the two header bytes.
## 3. Decoding (receiver side)
For each decrypted `Message` envelope:
1. AES-GCM → obtain the `plaintext` blob (AAD/anti-replay identical to the crypto contract,
[crypto.md §6](crypto.md)).
2. Read `version = plaintext[0]`. If `!= 0x01` → discard with log.
3. Read `comp = plaintext[1]`. If unknown → discard with log.
4. `body = plaintext[2:]`; if `comp == 0x01` → decompress (zlib).
5. Parse `body` as JSON; validate `v`/`kind` ([payloads.md §6](payloads.md)); apply action
idempotently.
## 4. No Version Disambiguation
There is no v1/v2 transport coexistence in production (clean break, no distributed v1 clients).
Therefore **no disambiguation trick is needed**: every payload is a versioned frame (`version = 0x01`).
A receiver reading a `version` different from `0x01` discards with log (§3 step 2).
## 5. Sizes & Limits
The `ciphertext` travels as **raw bytes** in the `Message` protobuf
([relay-protocol.md §10](relay-protocol.md)): **no base64**, so the frame limit applies almost
entirely to the ciphertext. Full chain:
```
payload →(zlib?)→ body →(GCM: +16B tag)→ raw ciphertext →(protobuf: +~tens of bytes)→ frame
```
**Normative constants** (frame limit differs per channel):
```
# Standard frame 64 KiB (control + Message live=false store-and-forward)
MAX_CIPHERTEXT_BYTES = 65000 # raw ciphertext (GCM tag included)
# Live frame 512 KiB (Message live=true, authenticated connection)
MAX_LIVE_CIPHERTEXT_BYTES = 524000 # raw ciphertext (GCM tag included)
```
Values leave a few hundred bytes of margin for the protobuf envelope (`peer` 32B, `nonce` 12B,
field tags, `live`) under the respective `MAX_*_FRAME_BYTES`. Anyone composing a large payload
**MUST** close the packet before exceeding `MAX_LIVE_CIPHERTEXT_BYTES`, estimating the size
**after** compression and tag.
Compression helps fit more data per frame: health-type data (JSON numeric and repetitive)
typically compresses 5–10×.
docs/relay/index.md
+173
View File
@@ -0,0 +1,173 @@
# Skald Remote Control — Architecture & Index
> **Purpose.** Specify, unambiguously, how to build the system that lets a mobile app (iOS/Android)
> remotely control a person's **Skald instance** — even when Skald runs at home behind NAT.
> Documents are written as **implementation contracts**: a coding agent must be able to implement
> its component (relay, plugin, app) by reading only these files and achieve byte-for-byte
> interoperability with all other components.
## 1. The Problem
Skald is self-hosted: anyone who installs it locally ends up **behind NAT**, unreachable from the
internet. We want a mobile app that:
1. receives **push notifications** when Skald needs human input (approvals, clarifications);
2. **responds** (approve / reject / clarify) even with Skald behind NAT.
Push notification systems (APNs/FCM) do not allow an arbitrary sender to push to someone else's
app: a component holding the push credentials is required. Hence the **relay**.
The entire architecture exists **only** to solve: (a) bidirectional communication through NAT,
(b) push notifications. Nothing more. The relay is designed to be **content-blind**.
> **What this is NOT.** Not a chat, not a streaming system, not a sub-agent protocol.
> The mobile client is a **remote control surface** (a human-in-the-loop remote) for the
> **single Skald instance** that owns the namespace. The approvals and clarifications the client
> sees are those exposed by that Skald instance through its Inbox; how Skald generates them
> internally (tools, scheduled jobs, etc.) is an internal detail outside this spec.
## 2. Actors
| Actor | Abbr | Role |
|-------|------|------|
| **Skald Agent** | `agent` | The Skald instance. **Namespace owner.** Holds the identity key. Opens a permanent WS connection to the relay. Encrypts/decrypts E2E. |
| **Relay Client** | `agent` impl | `crates/skald-relay-client/`: the **standalone, payload-agnostic** library that implements the `agent` role — keys, WS v2 transport, E2E crypto, anti-replay counters, pairing, device authorization, SQLite persistence. Exchanges opaque decrypted bytes via `RelayEvent`; depends only on `skald-relay-common` (never on Skald/`core-api`). |
| **Mobile Connector Plugin** | — | The thin **application** crate inside Skald (`crates/plugin-mobile-connector/`) on top of the relay client: it owns the JSON payload schemas, the Inbox↔relay routing, the authorization policy, and the QR endpoint. The bridge to mobile apps; today via relay, in the future also via direct transports (TCP/port-forward). See [server.md](server.md) and [../plugins/mobile-connector.md](../plugins/mobile-connector.md). |
| **Relay Server** | `relay` | The only centralised component. APNs/FCM bridge, store-and-forward, namespace routing. **Zero-trust on content.** See [server.md](server.md). |
| **Shared Crate** | — | `crates/skald-relay-common/`: protocol frame types (protobuf) + cryptographic primitives, shared **byte-for-byte** between relay, relay client, and server (no duplication). |
| **Client** | `client` | Mobile app (iOS/Android). Pairs via QR, encrypts/decrypts E2E, shows Inbox, responds. Implementation documented in the iOS app repository. |
A **namespace** is the isolated zone of one person: their agent + their authorised clients.
Different namespaces are unaware of each other. Multiple devices can share a namespace
(iPhone + iPad).
## 3. Architecture
```
Home / NAT Cloud Pocket
┌───────────────────────┐ ┌────────────────────────┐ ┌──────────────────────┐
│ Skald Agent │ │ Relay Server │ │ Client (iOS/Android) │
│ (namespace owner) │ │ (zero-trust) │ │ │
│ ┌──────────────────┐ │ WSS │ • APNs/FCM bridge │ WSS │ ┌─────────────────┐ │
│ │ Mobile Connector │◀─┼───────▶│ • store-and-forward │◀───▶│ │ CryptoEngine │ │
│ │ ed25519 + X25519 │ │ (perm.)│ • namespace routing │ │ │ ed25519 + X25519 │ │
│ └──────────────────┘ │ │ • does NOT decrypt │ │ └─────────────────┘ │
└───────────────────────┘ └───────────┬────────────┘ └──────────────────────┘
│ push (wake / encrypted blob)
▼
APNs (Apple) / FCM (Google)
```
- All actors connect to the **same** WebSocket endpoint on the relay.
- Agent↔client communication is **end-to-end encrypted**: the relay sees only opaque blobs.
- The relay routes by public key within the namespace and, if the recipient is offline,
queues and sends a push.
## 4. Threat Model (read before implementing)
### 4.1 Guarantees
| Guarantee | Mechanism |
|-----------|-----------|
| **Content confidentiality** end-to-end | AES-256-GCM with key derived from ECDH X25519. The relay has no key. |
| **Content integrity + authenticity** | GCM tag + binding of `from`/`to`/`namespace_id` in AAD. A relay that flips one byte breaks decryption. |
| **Peer authentication at pairing** | The agent's X25519 public key arrives **out-of-band** via QR (TOFU). The E2E channel is authenticated toward whoever controls that key. |
| **Anti-replay** | Per-direction **monotonic counter** nonce + `request_id` idempotency + `ts` freshness. See [crypto.md](crypto.md). |
| **Key ownership proof** (to the relay) | Challenge-response with Ed25519 signature, with domain separation. |
| **No namespace takeover** | `namespace_id = SHA256(domain ‖ agent_ed25519_pub)`: the id is immutably bound to the key. |
| **Device authorisation controlled by the owner** | Only the agent decides the authorised list. Pairing produces a **pending** device until the agent confirms. Pairing token is **single-use**. |
### 4.2 What the Relay CAN See and Do (declared limits)
> "Zero-trust" here means **content-confidential**, **not** metadata-private. This must be stated
> explicitly in the privacy policy.
| The relay sees | Notes |
|----------------|-------|
| Public keys of agent and clients | Public identifiers, not linked to real identities. |
| `device_token` (APNs/FCM), `platform` | Required for push delivery. |
| IP addresses (TCP/TLS layer) | Unavoidable. |
| Relationship graph (who talks to whom), timing, message sizes | Routing metadata. The relay learns **when** you are active. |
| The relay does NOT see | Why |
|------------------------|-----|
| Content / message type | E2E encrypted; the AAD is authenticated but the routing fields are only pubkeys. |
| Detailed `device_info` (model, OS, app version) | Sent **E2E** to the agent after pairing (`hello`), not to the relay. |
| The relay CAN do (and we defend against it) | Defence |
|---------------------------------------------|---------|
| **Drop / hold / reorder** messages and pushes | A lost approval = no action (fail-safe). Pending items have **TTL on the agent side**: a held-then-released "approve" is **no longer acted upon** after expiry. |
| **Replay** an encrypted blob | Monotonic counter per direction + `request_id` idempotency: a replay is discarded. |
| **Relabel** `from`/`to` | `from`/`to`/`namespace_id` are in the GCM AAD: decryption fails. |
### 4.3 Out of Scope (assumptions)
- **Compromised host** (agent or device): if the attacker has the seed, they have everything. Unavoidable.
Mitigation: minimal-permission storage / Keychain `ThisDeviceOnly`.
- **Apple/Google push channel compromise**: content stays E2E-protected; at worst availability is lost.
- **Perfect Forward Secrecy**: **not** in the current protocol (static shared secret after pairing). Roadmap.
Accepted consequence: traffic capture + later seed theft = plaintext for historical messages.
## 5. Encoding Conventions (NORMATIVE — apply to all files)
To eliminate ambiguity between implementations, the encoding of **every** binary field is fixed here.
| Data type | Wire encoding (JSON) | Example |
|-----------|----------------------|---------|
| Public keys (ed25519, X25519), 32 bytes | **lowercase hex**, 64 chars | `"3b6a…"` |
| Ed25519 signatures, 64 bytes | **lowercase hex**, 128 chars | `"9f1c…"` |
| `namespace_id` (SHA-256, 32 bytes) | **lowercase hex**, 64 chars | `"a17e…"` |
| `pairing_token` (32 bytes random) | **lowercase hex**, 64 chars | `"5d20…"` |
| Challenge `nonce` (32 bytes random) | **lowercase hex**, 64 chars | `"c4f0…"` |
| AEAD `nonce` (12 bytes) | **lowercase hex**, 24 chars | `"000000016a…"` |
| **Ciphertext** AEAD (variable, ciphertext‖tag) | **standard base64 with padding** (RFC 4648 §4) | `"q1B2…=="` |
Rules:
1. **Hex for fixed-length material** (keys, signatures, ids, nonces): easy to compare and debug.
Hex MUST always be lowercase; an implementation receiving uppercase MUST accept it but MUST emit lowercase.
2. **Standard base64 (not url-safe), with padding** for variable-length blobs (only ciphertext qualifies).
3. These rules apply to **JSON payloads** (the E2E content). The relay transport layer uses protobuf
binary frames where all binary fields travel as **raw bytes** — no hex, no base64.
4. Application timestamps: **unix epoch in milliseconds** (integer). Relay routing timestamps:
ISO-8601 UTC string (advisory only).
5. Unknown fields in JSON are ignored (forward-compat). Integers without decimal point.
## 6. Document Map
| File | Content | Primary audience |
|------|---------|-----------------|
| [index.md](index.md) | This file: vision, actors, threat model, encoding | Everyone |
| [crypto.md](crypto.md) | **Crypto contract**: seed, key derivation, ECDH, HKDF, AEAD, AAD, anti-replay, signatures | All implementors |
| [relay-protocol.md](relay-protocol.md) | **WebSocket protocol**: protobuf transport, auth, pairing, message envelope, live channel, presence, errors, limits | Relay, plugin, app |
| [framing.md](framing.md) | **E2E plaintext framing** `[version][comp][payload]` + optional zlib compression | Plugin, app |
| [pipe.md](pipe.md) | **Relayed byte-stream** (TURN-style): control-plane signaling + `/v1/pipe` data plane, per-pipe ephemeral DH (PFS), splice + limits | Relay, relay client, app |
| [payloads.md](payloads.md) | **E2E payload schemas** (the encrypted content the relay never sees) | Plugin, app |
| [describe-and-push.md](describe-and-push.md) | **Approval rendering**: `summary` + structured `blocks`, push delivery model | Plugin, app |
| [server.md](server.md) | **Relay server** implementation (Rust): zero-trust, store-and-forward, push bridge, deploy | Relay coding agent |
| [test-vectors.md](test-vectors.md) | **Crypto test vectors** + reference generator for byte-for-byte interop | All implementors |
> **Recommended reading order for a coding agent:** index → crypto → relay-protocol → framing →
> payloads → (your component's file) → test-vectors.
## 7. Versioning
- Protocol version in the URL: `/v1/ws`. Payload schema version in the `v` field (integer) of each
E2E JSON.
- Crypto domain constants (salt/info/prefix) contain `v1`. A future protocol would use different
constants: no cross-version confusion possible.
- **All** normative constants live in [crypto.md §1](crypto.md). No other file redefines them.
- The WebSocket transport uses **protobuf binary frames** (`RelayFrame`, package `skald.relay.v2`)
with raw bytes for all binary fields. The proto schema lives in `crates/skald-relay-common`.
- E2E plaintext framing is versioned by the `version` byte (`0x01` = JSON app payload, `0x02` = pipe
signaling), independently of the JSON payload schema version (`v` field). See [framing.md](framing.md).
- The pipe data plane adds **one** endpoint, `/v1/pipe` (relayed byte-stream). See [pipe.md](pipe.md).
## 8. Links
- Skald backend: `crates/` (workspace root)
- Shared crate: `crates/skald-relay-common/`
- Mobile connector plugin: `crates/plugin-mobile-connector/`
- Relay server: `crates/skald-relay-server/`
- iOS app: `/Users/dguiducci/projects/skald-ios/` (target `SkaldInbox` + Notification Service Extension)
- iOS skill: `skills/ios-development/SKILL.md`
docs/relay/payloads.md
+389
View File
@@ -0,0 +1,389 @@
# E2E Payloads — Encrypted Content Schemas
> This file defines the **plaintext** that is encrypted (AES-256-GCM, [crypto.md §6](crypto.md))
> and transported in the `ciphertext` field of the `Message` frame
> ([relay-protocol.md §6](relay-protocol.md)). **The relay never sees any of this.** Only the
> agent and the client see it.
>
> The plaintext is **JSON UTF-8**, wrapped in the framing envelope ([framing.md](framing.md))
> before encryption. No canonical form is required: it is encrypted as a byte blob and re-parsed
> by the recipient; it is never hashed separately.
---
## 1. Common Envelope
Every E2E payload has these base fields, plus kind-specific ones:
```json
{
"v": 1,
"kind": "<string>",
"id": "<uuid-v4>",
"ts": 1750000000000
}
```
| Field | Type | Required | Meaning |
|-------|------|----------|---------|
| `v` | int | yes | Payload schema version. `1` here. Different value → receiver discards with log. |
| `kind` | string | yes | Discriminant (table §2). |
| `id` | string (uuid-v4) | yes | Unique message id. Used for dedup at payload level and for acks. |
| `ts` | int (unix ms) | yes | Sender-side creation timestamp. Freshness check (§6). |
Common rules:
- **Forward-compat**: unknown fields are ignored. An unknown `kind` is discarded (with log),
not a fatal error.
- **Idempotency**: the receiver MUST handle every payload idempotently relative to its action
identifier (`request_id` for responses; `id` for generic dedup).
- **Anti-replay**: guaranteed by the nonce counter ([crypto.md §6.1](crypto.md)); `id`/`ts` are
additional application-level defences.
---
## 2. Kind Catalogue
| `kind` | Direction | Purpose |
|--------|-----------|---------|
| `inbox_update` | agent → client | Full Inbox snapshot (pending approvals + clarifications + elicitations). |
| `notification` | agent → client | Generic notification (title/body), for informational pushes. |
| `hello` | client → agent | First message after pairing: detailed `device_info`. |
| `inbox_request` | client → agent | Explicit Inbox snapshot request; agent responds with a **targeted** `inbox_update`. |
| `approval_response` | client → agent | Outcome of an approval request. |
| `clarification_response` | client → agent | Answer to a clarification. |
| `elicitation_response` | client → agent | Reply to an MCP elicitation (carries the requested value E2E). |
| `logout` | client → agent | Device removes itself from the namespace. |
| `ack` | bidirectional | Delivery confirmation (optional, for reliability). |
---
## 3. Agent → Client
### 3.1 `inbox_update` — Inbox Snapshot
**Full snapshot**, not a delta: contains **all** currently pending items. Idempotent by
construction (replaces local state). So a lost push does not cause state loss: the next snapshot
realigns.
```json
{
"v": 1,
"kind": "inbox_update",
"id": "0c5b…",
"ts": 1750000000000,
"badge": 2,
"approvals": [
{
"request_id": "appr_8f2a…",
"tool_name": "send_email",
"agent_label": "Skald",
"summary": "Send an email to mario@acme.com",
"detail": "Subject: Q3 Estimate\nBody: …",
"arguments": { "to": "mario@acme.com", "subject": "Q3 Estimate" },
"created_at": 1749999990000
}
],
"clarifications": [
{
"request_id": "clar_3b1c…",
"question": "Proceed with the €240 payment?",
"context": "Invoice #1234, supplier X",
"suggested_answers": ["Yes, proceed", "No, cancel"],
"agent_label": "Skald",
"created_at": 1749999991000
}
],
"elicitations": [
{
"request_id": "elic_5d7e…",
"server_name": "ssh",
"message": "Enter the SSH password for deploy@host",
"field_name": "password",
"sensitive": true,
"is_confirmation": false,
"created_at": 1749999992000
}
]
}
```
| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `badge` | int | yes | Total pending item count (= len(approvals)+len(clarifications)+len(elicitations)). Used by the client for badge. |
| `approvals[]` | array | yes | May be empty. |
| `approvals[].request_id` | string | yes | **Action identifier.** Stable while the item is pending. Used for response idempotency. |
| `approvals[].tool_name` | string | yes | Name of the tool requesting approval (e.g. `send_email`, `execute_cmd`). |
| `approvals[].agent_label` | string | yes | Human-readable origin label (typically `"Skald"`). |
| `approvals[].summary` | string | yes | Short line for notification/card (≤ ~120 chars). |
| `approvals[].detail` | string | no | Extended text for the detail screen. |
| `approvals[].arguments` | object | no | **Raw tool arguments** (JSON passed by the LLM). Source of truth for the detail screen: the client shows these so the user knows *what* they are approving (critical for `execute_cmd` → show `arguments.command`). May be absent for tools without arguments. E2E encrypted along with the rest of the payload. |
| `approvals[].created_at` | int (unix ms) | yes | When the request was created on the Skald side. |
| `clarifications[]` | array | yes | May be empty. |
| `clarifications[].request_id` | string | yes | Action identifier. |
| `clarifications[].question` | string | yes | Question to display. |
| `clarifications[].context` | string | no | Optional context. |
| `clarifications[].suggested_answers` | array of strings | no | Pre-defined answers suggested by the LLM. May be empty/absent. The client shows them as quick-tap options; free-form input is always possible too. The choice is sent as `clarification_response.answer` (§4.3). |
| `clarifications[].agent_label` | string | yes | Origin label. |
| `clarifications[].created_at` | int (unix ms) | yes | — |
| `elicitations[]` | array | yes | May be empty. MCP server-initiated input requests (e.g. an SSH/sudo password). |
| `elicitations[].request_id` | string | yes | Action identifier. Echoed back in `elicitation_response` (§4.4). |
| `elicitations[].server_name` | string | yes | MCP server that asked for input (e.g. `"ssh"`). |
| `elicitations[].message` | string | yes | Prompt to display to the user. |
| `elicitations[].field_name` | string \| null | no | Key the requested value must be stored under in `elicitation_response.content`. `null` for a bare confirmation. |
| `elicitations[].sensitive` | bool | yes | When `true`, the value is a secret: the client SHOULD mask input and MUST NOT cache/persist it. |
| `elicitations[].is_confirmation` | bool | yes | When `true`, this is a yes/no confirmation (no value field); `accept`/`decline` suffice and `content` is omitted. |
| `elicitations[].created_at` | int (unix ms) | yes | — |
> **Elicitation values never appear here.** This snapshot carries only the *prompt* metadata. The
> value the user supplies travels **only** in the client→agent `elicitation_response.content`
> (§4.4) and is handed straight to the MCP server; the agent never logs or persists it in clear.
>
> **Push privacy.** When this snapshot is sent to an offline client, the relay delivers it
> (encrypted) in the push *content-in-push* if it fits the APNs/FCM limit. Keep `summary`/`detail`
> short. If it exceeds the limit, the relay sends a *wake* and the client downloads the snapshot
> over WS ([server.md §5](server.md)).
### 3.2 `notification` — Generic Notification
```json
{ "v":1, "kind":"notification", "id":"…", "ts":…, "title":"Skald", "body":"Nightly job completed" }
```
| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `title` | string | yes | Notification title. |
| `body` | string | yes | Notification body. |
No response required. Does not affect the badge unless accompanied by an `inbox_update`.
### 3.3 `ack` (optional)
```json
{ "v":1, "kind":"ack", "id":"…", "ts":…, "ref_id":"<id of confirmed payload>" }
```
Confirms that a payload with `id == ref_id` was received/processed. Optional (store-and-forward
+ idempotent snapshots suffice for v1).
---
## 4. Client → Agent
### 4.1 `hello` — Post-Pairing Application Handshake
First E2E message the client sends after it is authorised and connected as `client`. Transfers
detailed `device_info` **outside the relay's view**.
```json
{
"v": 1,
"kind": "hello",
"id": "…",
"ts": …,
"device_info": {
"platform": "ios",
"model": "iPhone 16 Pro",
"os_version": "18.5",
"app_version": "1.0.0",
"device_name": "Daniele's iPhone"
}
}
```
| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `device_info.platform` | string | yes | `"ios"` \| `"android"`. |
| `device_info.model` | string | no | Hardware model. |
| `device_info.os_version` | string | no | OS version. |
| `device_info.app_version` | string | no | App version. |
| `device_info.device_name` | string | no | Human-readable name for the agent's device list UI. |
The agent persists this data and shows it in the device list.
### 4.2 `approval_response` — Approval Outcome
```json
{
"v": 1,
"kind": "approval_response",
"id": "…",
"ts": …,
"request_id": "appr_8f2a…",
"decision": "approved",
"reason": null,
"bypass_secs": 900
}
```
| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `request_id` | string | yes | MUST match an `approvals[].request_id` received. |
| `decision` | enum string | yes | **Only** `"approved"` \| `"rejected"`. Other values → agent discards. |
| `reason` | string \| null | no | Reason (typically for `rejected`). |
| `bypass_secs` | int | no | With `decision="approved"` only. Approve **and** register a bypass for similar tools: `900` = 15 minutes, `0` = for the entire session. **Absent** = single approval (current behaviour). The scope (tool category / MCP server / all) is auto-detected by the agent: the client only sends the seconds. |
Agent behaviour (see [../plugins/mobile-connector.md](../plugins/mobile-connector.md)):
1. Resolves the request via Skald's Inbox/ApprovalManager (`resolve(request_id, decision, reason)`).
If `decision="approved"` and `bypass_secs` is present, uses `approve_with_bypass` instead
of simple approve (registers the session bypass with auto-detected scope).
2. **Idempotency**: if `request_id` is already resolved (or no longer pending), the operation
is a **no-op** (log and ignore). Neutralises replays and double deliveries.
3. Sends a new `inbox_update` (the snapshot will no longer contain that item) to realign clients.
### 4.3 `clarification_response` — Clarification Answer
```json
{
"v": 1, "kind": "clarification_response", "id": "…", "ts": …,
"request_id": "clar_3b1c…",
"answer": "Yes, proceed."
}
```
| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `request_id` | string | yes | MUST match a `clarifications[].request_id`. |
| `answer` | string | yes | Free-form answer text. |
Same `request_id` idempotency as §4.2.
### 4.4 `elicitation_response` — MCP Elicitation Reply
Reply to an `elicitations[]` entry (§3.1): an MCP server asked for an input the LLM must not see
(e.g. an SSH/sudo password). The requested value travels **only** in this payload's `content`,
sealed E2E — the relay never sees it and the agent hands it straight to the MCP server without
logging or persisting it.
```json
{
"v": 1, "kind": "elicitation_response", "id": "…", "ts": …,
"request_id": "elic_5d7e…",
"action": "accept",
"content": { "password": "hunter2" }
}
```
| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `request_id` | string | yes | MUST match an `elicitations[].request_id`. |
| `action` | enum string | yes | **Only** `"accept"` \| `"decline"` \| `"cancel"`. Other values → agent discards. `decline` rejects the prompt; `cancel` aborts the whole request. |
| `content` | object \| null | conditional | Present **only** with `action="accept"` for a value prompt: a single key equal to the elicitation's `field_name`, whose value is the user's input (possibly a secret). Absent/null for `decline`/`cancel` and for confirmations (`is_confirmation=true`). A non-object `content` is dropped. |
Agent behaviour:
1. Resolves the request via Skald's Inbox (`resolve_elicitation(request_id, action, content)`), which
forwards the outcome to the `ElicitationManager` and unblocks the waiting MCP call.
2. **Idempotency**: a `request_id` already resolved (or no longer pending) is a **no-op**.
3. Sends a new `inbox_update` to realign clients.
> **Secret hygiene.** `content` may carry a secret. It is never written to logs, the DB, or any
> trace on the agent side; it lives only long enough to satisfy the MCP `elicitation/create` call.
### 4.5 `logout` — Device Self-Removal
```json
{ "v":1, "kind":"logout", "id":"…", "ts":… }
```
The agent, on receipt:
1. removes `client_ed25519_pub` from the local authorised list;
2. sends an updated `Authorize` (without that client) to the relay → the relay closes the
device's WS, purges its queue, and forgets its `device_token`
([relay-protocol.md §5](relay-protocol.md));
3. forgets the client's keys/counters.
> Revocation can also be initiated **by the agent** (lost/stolen device): the user removes it via
> the Skald UI and the agent sends `Authorize` without that device. `logout` E2E is only the
> "device-initiated" case.
### 4.6 `inbox_request` — Explicit Inbox Snapshot Request
The client sends this payload to ask the agent for the current Inbox state.
**MUST be sent after `AuthOk` on every WS (re)connection** (including app open from a push),
because the agent does **not** receive a reconnect signal from the relay: without `inbox_request`
the client's Inbox would stay empty until a new bus event triggers a broadcast.
```json
{ "v":1, "kind":"inbox_request", "id":"…", "ts":… }
```
No specific fields beyond the common envelope (§1).
Agent behaviour:
1. Builds the current Inbox snapshot (`list_pending()`).
2. Sends an `inbox_update` (§3.1) **targeted to the requester only** (not a broadcast): the
message is sealed with the requesting client's `aes_key`, leaving other devices unaffected.
3. Idempotent and side-effect-free on the Inbox: safe to send on every connection. If there are
no pending items, the snapshot has `badge:0` and empty arrays.
> This follows the *targeted request → targeted response* pattern. The payload travels on the
> **live channel** (`Message.live=true`, [relay-protocol.md §6.4](relay-protocol.md)): a stale
> Inbox snapshot is useless, so route-or-fail is correct — if the agent is offline, the client
> learns immediately via `PeerOffline`.
### 4.7 `ack` (optional)
Same as §3.3, opposite direction.
---
## 5. Inbox State Machine (client side)
```
inbox_update (snapshot)
┌──────────────────────────────────────┐
▼ │
[ local list ] ──user approves/rejects──▶ send approval_response
▲ │ (optimistic: remove card)
│ ▼
└──────────── next inbox_update ◀─── agent resolves and re-snapshots
```
- The client updates the UI **optimistically** (removes the card on response send), but the
**source of truth** is the next `inbox_update`. If the response is lost, the item reappears
on the next snapshot.
- Local `badge` = `badge` of the last snapshot, minus items already responded to locally
(reconciled on next snapshot).
## 6. Freshness & Validation (receiver side)
For every decrypted E2E payload, the receiver MUST:
1. verify the nonce **counter** (`> last_seen`, [crypto.md §6.1](crypto.md)) → otherwise discard;
2. verify `v == 1` → otherwise discard with log;
3. (SHOULD) discard if `|now - ts|` > 7 days (aligned with the queue TTL): extra defence against
very late replays;
4. validate required fields and types; a malformed payload is discarded without crash;
5. apply the action **idempotently** by `request_id` (responses) or `id` (generic dedup).
## 7. Complete Round-Trip Examples
**Approval (foreground):**
```
agent → inbox_update { approvals:[{request_id:"appr_1", tool_name:"send_email", …}], badge:1 }
client → approval_response { request_id:"appr_1", decision:"approved" }
agent → inbox_update { approvals:[], badge:0 } // realign
```
**Clarification (background, via content-in-push):**
```
agent → inbox_update { clarifications:[{request_id:"clar_9", question:"Proceed?"}], badge:1 }
(relay: client offline → push with encrypted blob)
client → (NSE decrypts, shows notification) → user opens app → clarification_response { request_id:"clar_9", answer:"Yes" }
agent → inbox_update { clarifications:[], badge:0 }
```
**MCP elicitation (SSH password, secret E2E):**
```
(MCP `ssh` server calls elicitation/create → agent blocks the tool call)
agent → inbox_update { elicitations:[{request_id:"elic_5", server_name:"ssh", field_name:"password", sensitive:true}], badge:1 }
client → (masked input) → elicitation_response { request_id:"elic_5", action:"accept", content:{ "password":"hunter2" } }
agent → (hands content to the MCP server, unblocks the call; value never logged) → inbox_update { elicitations:[], badge:0 }
```
**App opened from notification (reconnect):**
```
client → (connects as role:"client", auth_ok)
client → inbox_request { } // live channel (Message.live=true)
agent → inbox_update { approvals:[…], clarifications:[…], badge:N } // targeted to requester only
```
docs/relay/pipe.md
+215
View File
@@ -0,0 +1,215 @@
# Pipe — Relayed Byte-Stream over Skald Relay
> **Implementation reference.** A generic, content-blind, end-to-end-encrypted **byte-stream**
> channel between two members of a namespace, **relayed** (TURN-style) through the Skald relay. It
> sits ON TOP of the existing transport: signaling rides the existing E2E `Message` channel (no new
> `RelayFrame`); the data plane is **one new relay endpoint** (`/v1/pipe`). The relay splices opaque
> ciphertext and never reads it.
>
> **Status (v1, implemented).** Scope = **client↔agent** (the shared E2E key already exists, so the
> ephemeral handshake is authenticated by the channel that carries it). Suite = `x25519-sealed`.
> Compression = `none` (negotiation present for forward-compat). client↔client is deferred (needs a
> signed roster/manifest + a self-authenticating suite — see §7).
>
> Read after: `index.md` → `relay-protocol.md` → `crypto.md` → `framing.md`.
## 1. Why
The message channel (`relay-protocol.md`) is for **discrete** E2E payloads (approvals, clarifications,
health sync): ≤60 msg/min, ≤512 KiB/frame, store-and-forward. It serves **stream-shaped, high-volume**
flows poorly — log tailing, file transfer, audio, remote shell, real-time sensors. The pipe is the
**reusable streaming primitive** for those. It is **TURN's relayed mode**: a control plane brokers a
rendezvous; a separate connection carries a raw encrypted byte stream the relay blindly splices, so
TCP/WS gives reliability/ordering/flow-control for free (no reinvented windowing).
```
Control plane (existing E2E Message channel) Data plane (new WSS /v1/pipe)
A ──pipe_invite (live)──▶ R ──▶ B A ──▶ R ◀── B (each dials out; NAT-friendly)
A ◀──pipe_accept (live)── R ◀── B R verifies auth, matches by connection_id,
(ephemeral X25519 exchanged → per-pipe key, PFS) then splices opaque ciphertext frames
B offline ⇒ A gets PeerOffline ⇒ abort A ⇄ B: AES-256-GCM stream, relay sees ciphertext
```
## 2. Control plane — signaling
Pipe signaling rides the **existing** `Message{live=true}` E2E frame. It is **not** a new
`RelayFrame`; the relay stays content-blind. To distinguish it from JSON app payloads on the same
channel, the decrypted plaintext uses a reserved framing header (`crypto.md §1`):
```
FRAMING_VERSION_PIPE (0x02) ‖ COMP_NONE (0x00) ‖ <MsgPack PipeSignal>
```
The receiver peeks the first byte (`crypto::is_pipe_signal`): `0x02` ⇒ route to the pipe layer;
`0x01` ⇒ the existing JSON app path, unchanged. `live=true` is required — a stale "please connect" is
useless; if the peer is offline the initiator gets `PeerOffline` (`relay-protocol.md §6.4`) and aborts.
**Wire format = MsgPack** (`rmp-serde`, named maps). `PipeSignal` is externally tagged
(`{ "Invite": {…} }`) so a blob is self-describing. Byte fields are length-validated on decode.
| Message | Fields |
|---------|--------|
| `Invite` | `connection_id` (32B), `suite`, `handshake` (opaque; initiator ephemeral X25519 pub for `x25519-sealed`), `stream_type` (app-defined), `compress` (advertised list), `headers` (arbitrary `String→String`) |
| `Accept` | `connection_id`, `suite`, `handshake` (responder ephemeral pub), `compress` (selected codec) |
| `Reject` | `connection_id`, `reason` |
- **`connection_id`**: 32 random bytes, single-use, short-lived. The rendezvous key, known only to A
and B (sent E2E). **Not** a security boundary on its own — the data-plane signature (§3) is.
- **`suite`** is a discriminator and **`handshake` is opaque**: adding a Noise suite (§7) is a new
variant with the **same wire shape**. Signaling is **symmetric** (initiator/responder by role,
never agent-vs-client) so client↔client is not blocked.
- **`headers`**: app metadata for the stream (filename/size for a transfer, filters for a log tail).
By `pipe_accept` both sides have the peer's ephemeral pubkey and derive the per-pipe key (§4).
## 3. Data plane — `WSS /v1/pipe`
A **second WebSocket**, separate from the control WS, binary frames carrying **raw bytes** (no
protobuf). Chosen over HTTP `CONNECT` / raw TCP for reachability: 443/TLS, traverses CDN / L7 LB /
mobile carriers, camouflaged as a normal WS. The socket **is** the tunnel (one connection per pipe);
the control WS stays separate and alive.
### 3.1 Auth handshake (relay-mediated, MsgPack)
Mirrors the main WS "relay speaks first":
```
A → WSS /v1/pipe
R → PipeChallenge { nonce: 32B } (relay speaks first)
A → PipeAuth {
connection_id, pubkey (ed25519, 32B),
dest = SHA256(peer_ed25519_pub) (32B), (declares intended counterparty)
namespace_id (raw 32B),
signature = sign_ed25519(priv, PIPE_AUTH_DOMAIN ‖ 0x00 ‖ nonce ‖ connection_id) (64B)
}
R verifies, in order:
1. signature valid under pubkey (verify_strict) → else close
2. pubkey is the agent of namespace_id, OR an authorized client → else close
3. (on the second side) cross-refs match (§3.2) → else close both
```
The reply is a **signature**, not an echo — it proves control of `pubkey`, exactly like the main WS
auth. `connection_id` is **not** trusted as identity.
### 3.2 Matching & splice (relay state machine)
```
challenge → pipe_auth → pending → matched → streaming → teardown
```
- **pending**: first authenticated side for `connection_id` is parked (TTL); the namespace pipe count
is incremented.
- **matched**: second side authenticates → relay verifies the cross-refs
`SHA256(A.pubkey)==B.dest AND SHA256(B.pubkey)==A.dest AND same namespace`, then hands the second
side's socket halves to the first.
- **streaming**: the first side owns a bidirectional forward loop of binary-frame payloads. The relay
reads nothing else; WS-level pings are answered on the originating leg; data is rate-limited per
direction.
- **teardown**: either side closing/erroring tears down both (no orphans). *(v1 closes both; FIN
half-close propagation is a future refinement.)*
### 3.3 Relay limits (NORMATIVE; env-overridable)
The relay becomes a **stateful connection proxy** (TURN resource model): fd+buffers per pipe, idle
reaping, pending TTL, per-namespace concurrency cap, backpressure (no unbounded buffering).
| Limit | Env var | Default | Why |
|-------|---------|---------|-----|
| Pending half-open TTL | `RELAY_PIPE_PENDING_TTL_SECS` | 30 s | A dialed, B never showed → reap. |
| Idle pipe timeout | `RELAY_PIPE_IDLE_TIMEOUT_SECS` | 120 s | Reclaim dead pipes. |
| Max concurrent pipes / namespace | `RELAY_PIPE_MAX_PER_NS` | 8 | Bound proxy resource use. |
| Max data-plane frame | `RELAY_PIPE_MAX_FRAME_BYTES` | 1 MiB | Bulk transfer; separate from the message-channel quota. |
| Bandwidth cap (per connection, per direction) | `RELAY_PIPE_MAX_BPS` | 0 (unlimited) | Token bucket; stops the pipe being a free unmetered tunnel. |
## 4. Secure channel — reused AES-256-GCM, ephemeral DH (PFS)
The A↔B stream reuses the **existing** crypto primitives (`crypto.md`), not Noise/TLS — the same
AES-256-GCM / X25519 / HKDF stack already interop-tested against the iOS client:
- **Per-pipe key**: each side samples a fresh ephemeral X25519, exchanges the pubkey in the signaling,
and computes `pipe_key = HKDF(ECDH(eph), salt=PIPE_KDF_SALT, info=PIPE_KDF_INFO)`. Ephemeral DH ⇒
**Perfect Forward Secrecy** per pipe (closes the gap in `index.md §4.3` for this channel).
- **Authentication**: the ephemeral pubkeys travel **inside the E2E-sealed signaling**, so for
client↔agent they are authenticated by the existing channel — no signatures needed in the
handshake (that is the `x25519-sealed` suite). client↔client (no pre-shared key) needs a
self-authenticating suite (§7).
- **Frame crypto**: each chunk is `AES-256-GCM(pipe_key, nonce, aad)`. The 12-byte nonce is
`DIR (4B) ‖ counter (8B)` with a per-direction counter (`DIR_PIPE_INITIATOR` / `DIR_PIPE_RESPONDER`),
**not transmitted** (reconstructed by the receiver — strict in-order WS/TCP delivery). `aad =
connection_id` (binds frames to the rendezvous). Counters start at 1.
The relay never holds `pipe_key`; mismatched keys fail the GCM tag (confidentiality holds even if the
relay mis-splices).
## 5. Compression
Negotiated in the handshake (`compress` advertise→select), per direction, `none | zlib`. **v1 ships
`none` only** — the negotiation field exists for forward-compat. Stateful streaming `zlib` is deferred
(it is a shared-dictionary deflate context, and on a *generic* bus it reintroduces the CRIME/BREACH
class for `stream_type`s mixing attacker-controlled plaintext with secrets).
## 6. Library API (`skald-relay-client`)
```
RelayClient::open_pipe(peer, stream_type, headers) -> PipeConnection // initiator
RelayClient::incoming_pipes() -> broadcast::Receiver<IncomingPipe> // responder feed
RelayClient::accept_pipe(&IncomingPipe) -> PipeConnection // responder
RelayClient::reject_pipe(&IncomingPipe, reason)
PipeConnection::{ send(&[u8]), recv() -> Option<Vec<u8>>, close() } // sealed/opened transparently
PipeConnection::split() -> (PipeSender, PipeReceiver) // independent halves
PipeSender::send(&[u8]) // seals + queues; blocks only when the send buffer is full
PipeReceiver::recv() -> Option<Vec<u8>>
```
Inbound pipe invites surface on a **separate channel** (`incoming_pipes`), **not** as a `RelayEvent`
variant — so adding the pipe is purely additive and the `plugin-mobile-connector` consumer compiles
unchanged. The relay client owns the pipe control plane end-to-end (it intercepts only the `pipe_*`
signaling kinds; every other payload stays pass-through).
### 6.1 Full-duplex & client-side backpressure
A `PipeConnection` is **full-duplex**: on `connect` the data-plane socket is split into a sink + stream
and a background **writer task** takes the sink. `send` seals the chunk and hands it to the writer
(returning before the flush); `recv` reads the stream directly. So a slow flush on one direction never
stalls the other — and `split() -> (PipeSender, PipeReceiver)` lets each direction run in its own task
with no shared `&mut`. The per-direction counter nonce stays ordered because `send` is single-writer
(`&mut self`) and the writer drains FIFO.
**Backpressure** is an in-memory byte budget (`SEND_BUFFER_BYTES`, ~10 MiB) held as a semaphore: `send`
reserves the sealed frame's bytes and the writer releases them *after* the frame is flushed. While the
socket drains normally `send` returns immediately; if the peer/relay stops draining, the budget empties
and `send` **blocks** until space frees up (bounding memory). This sits under the relay's own per-pipe
limits (§3.3). WS-level pings stay responsive: the read half forwards `Pong`s to the writer on a
separate control channel that the budget never throttles. Dropping both halves (or `close()`) tears the
pipe down — the writer closes the socket once its channels are gone.
### 6.2 Agent-side consumers (by `stream_type`)
- **`http-local-proxy`** — `plugin-mobile-connector` (`src/proxy.rs`) accepts these pipes and
reverse-proxies each, byte-for-byte, to the local web server at `127.0.0.1:<web_port>`, letting a
native WebView render the Skald web UI over the relay (no NAT hole / Tailscale). It `split`s each
pipe and runs the two directions in **separate tasks** (remote→local writes to the web server;
local→remote reads it and `send`s back, backpressured by the send buffer), so a stalled direction
can't block the other. Destination is pinned (not client-chosen); access is already gated by §3.1
(agent or authorized client). See
[../plugins/mobile-connector.md](../plugins/mobile-connector.md#http-reverse-proxy-http-local-proxy).
## 7. client↔client (deferred)
The data plane is **already** client↔client-capable: the relay authenticates by namespace membership +
cross-dest, not by agent-vs-client. Two things are missing above it, both additive:
1. **Key/identity distribution** — clients don't know each other's keys. Plan: the agent signs a
**manifest** (versioned roster of authorized members' pubkeys) the relay caches and serves; clients
verify the agent's ed25519 signature.
2. **Self-authenticating handshake** — without a pre-shared client↔client key the ephemeral exchange
can't be sealed in an existing channel. Plan: a new `suite` (e.g. `noise-nn+ed25519`) — same
signaling wire shape, new `handshake` interpretation. The relay does **not** change.
## 8. Verification (implemented)
- **relay-common** (`crypto.rs`, `pipe.rs` tests): MsgPack round-trips; `pipe_auth` sign/verify binds
nonce + connection_id (and rejects an `AUTH_DOMAIN` signature — domain separation); pipe-key
symmetry over ephemeral DH; pipe-signal framing peek.
- **relay-server** (`tests/pipe.rs`): two raw WS peers → `pending→matched→streaming→teardown`;
rejects bad signature, cross-dest mismatch, non-member; teardown closes the peer.
- **relay-client** (`src/pipe.rs` net tests): two `PipeConnection`s stream bytes (incl. a 200 KiB
blob) both ways through the **real** relay; a wrong key fails AEAD open (relay never had plaintext);
`split` both ends and stream 1 MiB **each way simultaneously** (full-duplex — neither direction
blocks the other). Signaling routing (`src/state.rs`): invite → `incoming_pipes`, accept/reject → the
waiter.
- **Non-regression**: `plugin-mobile-connector` builds unchanged; existing `protocol.rs` /
`integration.rs` suites still pass.
## 9. Out of scope (deferred)
- Stateful `zlib` compression (negotiation reserved).
- HTTP/2 extended `CONNECT` multiplexing (many pipes over one socket).
- client↔client (§7) and the specific app `stream_type` consumers.
docs/relay/relay-protocol.md
+521
View File
@@ -0,0 +1,521 @@
# Relay Protocol — WebSocket
> Transport protocol between **any actor** (agent, client, pairing) and the **relay**, over
> **a single WebSocket**. No REST. This file defines the **protobuf frame schema**, the
> **authentication handshake**, the **E2E message envelope**, the **live channel**, **presence**,
> and the **pairing flow**. The **encrypted content** inside the envelope is in
> [payloads.md](payloads.md); the **cryptography** is in [crypto.md](crypto.md).
>
> MUST/SHOULD carry the RFC 2119 meaning.
**Transport**: every WebSocket frame is a **binary frame** (opcode `0x2`) carrying exactly one
`RelayFrame` protobuf. All binary fields (keys, signatures, nonces, namespace_id) travel as
**raw bytes** — no hex, no base64. The encoding rules in [index.md §5](index.md) apply only
inside the E2E JSON payloads, not to the transport layer.
---
## 1. Concepts
- **Namespace**: created implicitly when an `agent` authenticates for the first time. Identified by
`namespace_id = hex(SHA256(NS_DOMAIN ‖ 0x00 ‖ agent_ed25519_pub))` ([crypto.md §7](crypto.md)).
Expires after **7 days** without any connection.
- **Owner**: the `agent` holding the namespace private key. **Sole authority** over the authorised
client list.
- **Client**: a mobile device **authorised by the agent**. Before pairing it does not exist; after
pairing it is `pending` until the agent authorises it.
## 2. Endpoint
```
wss://<relay-host>/v1/ws
```
Single endpoint for all actors. The **role** is established in the `Auth` frame. `namespace_id`
is NOT in the query string: it travels inside `Auth`. Transport: **WSS mandatory** (TLS); the
relay MUST reject plain WS.
## 3. RelayFrame Schema (NORMATIVE)
Lives in `crates/skald-relay-common`, generated for Rust (`prost`) and iOS (`SwiftProtobuf`).
Package name: `skald.relay.v2`.
```proto
syntax = "proto3";
package skald.relay.v2;
// One WebSocket binary frame = one RelayFrame.
message RelayFrame {
oneof frame {
Challenge challenge = 1;
Auth auth = 2;
AuthOk auth_ok = 3;
AuthError auth_error = 4;
Authorize authorize = 5;
AuthorizeOk authorize_ok = 6;
PairingStart pairing_start = 7;
PairingReady pairing_ready = 8;
PairingStop pairing_stop = 9;
PairingStopOk pairing_stop_ok = 10;
ClientPaired client_paired = 11;
Message message = 12;
PeerOffline peer_offline = 13;
PresenceRequest presence_request = 14;
PresenceList presence_list = 15;
PresenceEvent presence_event = 16;
Error error = 17;
}
reserved 18, 19; // ex Ping/Pong: keepalive via native WS frames (§8), not protobuf
}
// --- Data plane (E2E). The relay routes, does NOT read ciphertext/nonce. ---
message Message {
bytes ciphertext = 1; // E2E payload: JSON+framing (framing.md). Opaque to relay.
bytes nonce = 2; // 12B, AEAD nonce
bytes peer = 3; // 32B: 'to' on send (sender→relay), 'from' on delivery (relay→dest)
bool live = 4; // true = live channel (§6): route-or-fail, no queue/push
}
message PeerOffline { bytes peer = 1; } // 32B: recipient not connected (live channel only)
// --- Presence (§7). Control frames, not E2E. ---
message PresenceRequest {}
message PresenceList { repeated bytes online = 1; } // 32B each
message PresenceEvent { bytes pubkey = 1; Status status = 2; } // 32B
enum Status { STATUS_UNSPECIFIED = 0; STATUS_ONLINE = 1; STATUS_OFFLINE = 2; }
// --- Handshake / auth / pairing: role is implicit in the sub-message set.
// No enum Role (its default 0 would mean "AGENT" — security footgun). ---
message Challenge { bytes nonce = 1; } // 32B
message Auth {
oneof role {
AuthAgent agent = 1;
AuthClient client = 2;
AuthPairing pairing = 3;
}
bytes signature = 4; // 64B, over AUTH_DOMAIN‖0x00‖nonce
}
message AuthAgent { bytes agent_ed25519_pub = 1; } // 32B; namespace_id = hash(pubkey)
message AuthClient {
bytes namespace_id = 1; // 32B
bytes client_ed25519_pub = 2; // 32B
string device_token = 3; // push token (opaque)
Platform platform = 4;
}
message AuthPairing {
bytes namespace_id = 1; // 32B
bytes client_ed25519_pub = 2; // 32B
bytes client_x25519_pub = 3; // 32B
bytes pairing_token = 4; // 32B
string device_token = 5; // push token (opaque)
Platform platform = 6;
}
enum Platform { PLATFORM_UNSPECIFIED = 0; PLATFORM_IOS = 1; PLATFORM_ANDROID = 2; }
message AuthOk { bytes namespace_id = 1; }
message AuthError { string code = 1; string message = 2; }
message Authorize { repeated bytes clients = 1; } // 32B each (replaces full list)
message AuthorizeOk { uint32 authorized = 1; }
message PairingStart { bytes pairing_token = 1; uint32 ttl = 2; }
message PairingReady { uint32 ttl = 1; }
message PairingStop {}
message PairingStopOk {}
message ClientPaired {
bytes client_ed25519_pub = 1;
bytes client_x25519_pub = 2;
Platform platform = 3;
}
message Error { string code = 1; string message = 2; }
// Keepalive: native WebSocket ping/pong frames (§8), not protobuf messages.
```
> **Validation (proto3 has no `required`).** The role split prevents cross-role confusion but
> does not enforce non-empty fields: the relay MUST still validate the **presence and length** of
> `bytes` fields (32B pubkeys, 64B signatures, …) and reject with `bad_request`. The
> `*_UNSPECIFIED = 0` enum values make "absent enum field" distinguishable and rejectable.
## 4. Authentication Handshake
**The relay speaks first.** As soon as the WS is open, it sends a `Challenge`. Until `AuthOk`
arrives, the only frame accepted from the peer is `Auth`.
```
PEER (agent | pairing | client) RELAY
│ ── WSS connect ───────────────────────────── ▶│
│ ◀──── Challenge { nonce: 32B } ───────────────│ relay speaks first
│ ── Auth { role:..., signature: 64B } ────────▶│
│ ◀──── AuthOk { namespace_id } ────────────────│
│ OR AuthError { code, message } ────────────│
```
- `Challenge.nonce`: 32 random bytes. Unique per connection. Expires after **30 s**: no `Auth`
in time → `challenge_timeout` and close.
- `Auth.signature`: Ed25519 signature of `AUTH_DOMAIN ‖ 0x00 ‖ nonce_raw` ([crypto.md §8](crypto.md)).
- The relay MUST verify the signature under the role-appropriate public key **before** any other
logic.
### 4.1 `role: agent` — the Skald instance
The namespace may not exist yet: it is created here.
```proto
Auth {
agent: AuthAgent { agent_ed25519_pub: <32B> },
signature: <64B>
}
```
Relay checks (in order):
1. `agent_ed25519_pub` is exactly 32 bytes.
2. `signature` valid over `AUTH_DOMAIN ‖ 0x00 ‖ nonce_raw` under `agent_ed25519_pub`.
3. Compute `namespace_id = SHA256(NS_DOMAIN ‖ 0x00 ‖ agent_ed25519_pub)`.
4. If namespace doesn't exist → create it (bind `namespace_id ↔ agent_ed25519_pub`, immutable).
If it exists → pubkey MUST match (by construction it does, since the id is a hash of the key;
a mismatch is a bug → `not_found`).
5. If an `agent` WS is already open for this namespace → close the old one (one agent connection
per namespace at a time).
Response: `AuthOk { namespace_id: <32B raw> }`.
Right after, the agent SHOULD send an `Authorize` frame (§5) with the current authorised client
list (possibly empty).
### 4.2 `role: pairing` — not-yet-authorised client
For initial connection before authorisation. Accepted only if the namespace is in **pairing mode** (§9).
```proto
Auth {
pairing: AuthPairing {
namespace_id: <32B>,
client_ed25519_pub: <32B>,
client_x25519_pub: <32B>,
pairing_token: <32B>,
device_token: "<push token>",
platform: PLATFORM_IOS | PLATFORM_ANDROID
},
signature: <64B>
}
```
Relay checks:
1. `signature` valid under `client_ed25519_pub`.
2. `namespace_id` exists and is in pairing mode.
3. `pairing_token` matches **byte-for-byte** the one from `PairingStart`, **not expired**,
**not yet consumed** (single-use).
4. Mark the token **consumed**. Register the client as **`pending`** (NOT yet authorised):
store `client_ed25519_pub`, `client_x25519_pub` (opaque), `device_token`, `platform`.
5. Forward a `ClientPaired` frame to the agent (§9.4).
Response: `AuthOk { namespace_id: <32B raw> }`.
After `AuthOk` the pairing client **closes** the WS. It becomes operational by reconnecting with
`role: client` **once the agent has authorised it** (the app may retry with backoff until it
receives `AuthOk` instead of `unauthorized`).
> `device_token` and `platform` are the **only** device data the relay knows: required for push.
> Model, OS, app version do NOT pass through the relay: the app sends them **E2E** to the agent
> via a `hello` message ([payloads.md](payloads.md)).
### 4.3 `role: client` — authorised device
```proto
Auth {
client: AuthClient {
namespace_id: <32B>,
client_ed25519_pub: <32B>,
device_token: "<push token>",
platform: PLATFORM_IOS | PLATFORM_ANDROID
},
signature: <64B>
}
```
Relay checks:
1. `signature` valid under `client_ed25519_pub`.
2. `namespace_id` exists.
3. `client_ed25519_pub` is in the **authorised** list (NOT `pending`). Otherwise `unauthorized`.
4. Update `device_token` (it can change: APNs/FCM rotate it).
5. If a `client` WS is already open for the same pubkey → close the old one.
6. Deliver any queued messages (store-and-forward, §6.3).
Response: `AuthOk { namespace_id: <32B raw> }`.
## 5. Client Authorisation (agent only)
The agent is the **sole authority**. The authorised list is declared with:
```proto
Authorize { clients: [ <32B>, <32B>, … ] }
```
- **Replacement semantics**: this list **replaces** the previous one (not an append). To add a
device, send the full list including it; to revoke one, send the list without it.
- Relay effects, atomic:
- keys present now but absent before → become `authorised` (exit `pending`);
- keys absent now but present before → **revoked**: the relay MUST (a) close that client's active
WS if any, (b) **purge its store-and-forward queue**, (c) forget its `device_token`.
- Response: `AuthorizeOk { authorized: N }` (N = number of active authorised clients).
## 6. E2E Messages
After `AuthOk`, agent and client exchange **opaque** messages routed by pubkey.
### 6.1 Sending (sender → relay)
```proto
Message {
ciphertext: <bytes>, // E2E blob: framed JSON, encrypted AES-256-GCM
nonce: <12B>,
peer: <32B>, // destination ed25519 pubkey
live: false // or true for live channel (§6.4)
}
```
- `peer` = ed25519 pubkey of the recipient (the agent, or a client).
- The relay knows the sender: it is the pubkey authenticated on **this** WS. It does NOT trust any
`from` field supplied by the sender.
- The relay MUST verify that `peer` belongs to the **same namespace** (namespace agent, or an
authorised client). Otherwise `not_found`.
- The relay NEVER reads or alters `nonce` and `ciphertext`.
### 6.2 Receiving (relay → recipient)
The relay rewrites `Message.peer` from `to` (the destination sent by the sender) to `from`
(the authenticated pubkey of the sender, which the relay guarantees), and adds a routing
`timestamp` (advisory, ISO-8601 UTC) via delivery metadata if needed.
```proto
Message {
ciphertext: <bytes>,
nonce: <12B>,
peer: <32B>, // 'from': authenticated sender pubkey
live: <bool>
}
```
The recipient:
1. reconstructs the AAD = `namespace_id_raw ‖ peer_pub(from) ‖ my_pub` ([crypto.md §6.2](crypto.md));
2. selects the `aes_key` for peer `from`;
3. decrypts; verifies the **counter** in the nonce (`> last_seen`, [crypto.md §6.1](crypto.md));
4. strips the framing header ([framing.md](framing.md)), parses payload ([payloads.md](payloads.md)).
Idempotent by `request_id`.
### 6.3 Store-and-Forward (`live=false`)
If the recipient is not connected when the message arrives:
1. The relay queues the message (`peer` as destination, sender pubkey, `nonce`, `ciphertext`,
`created_at`).
2. If the recipient is a **client** with a `device_token`, the relay sends a **push**
([server.md §5](server.md)).
3. On the recipient's (re)connection, the relay drains the queue **in FIFO order** over the WS,
then deletes delivered messages.
4. Queue TTL: **7 days**. Beyond that → silently dropped.
Queue limits per recipient: see §10.
### 6.4 Live Channel (`live=true`)
`live=true` selects a different delivery class:
| `live` | Relay semantics |
|--------|-----------------|
| `false` | **Store-and-forward**: if recipient is offline, queue (max 200, TTL 7d) and push. For approvals/clarifications. |
| `true` | **Route-or-fail**: forward **only** if the recipient is connected now. If offline → do NOT queue, do NOT push, reply to the sender with `PeerOffline { peer: <32B> }`. For state pulls and high-volume flows. Relay is **stateless** for this channel. |
On delivery the relay rewrites `Message.peer` from `to` (destination) to `from` (authenticated
sender), as in §6.2. `nonce`/`ciphertext` are never read.
### 6.5 Pull vs Notification: which traffic uses live (NORMATIVE)
The value of `live` is not a free choice: it depends on the **semantic nature** of the payload.
> **State pull → `live=true`. Event-driven notification that must wake the human → `live=false`
> (store-and-forward + push).**
A **pull** ("give me the current state") served stale is useless or harmful: route-or-fail is
correct — if the peer is absent, the sender knows immediately (`PeerOffline`) and shows an offline
state instead of hanging or receiving a stale snapshot hours later. A **notification** that must
reach an offline phone, however, *must* be able to wait in queue and be pushed.
| Payload | Direction | `live` | Why |
|---------|-----------|--------|-----|
| `inbox_request` (app open / reconnect) | client → agent | **`true`** | State pull: stale = useless. Agent offline → app learns immediately. |
| `inbox_update` in **response** to an `inbox_request` | agent → client | **`true`** | Client just asked: it is online by construction. |
| `inbox_update` for a **new event** (approval/clarification) | agent → client | **`false`** | Must reach an offline phone → queue + push. |
The sender, upon receiving `PeerOffline`, **stops** sending to that peer and retries on the next
`PresenceEvent { STATUS_ONLINE }` (§7) or on reconnect.
> **Why `PeerOffline` is needed even with presence.** Presence declares `ONLINE` with up to
> ~120 s delay on disconnect (idle-timeout). `PeerOffline` covers that blind window.
> Presence = *when to start*; `PeerOffline` = *correctness backstop*.
## 7. Presence
The relay exposes who is connected in the namespace. Scope is **strictly per namespace**: never
propagated outside. It only reveals pubkeys already known to the relay ([index.md §4.2](index.md)).
- `PresenceRequest {}` → relay replies `PresenceList { online: [<32B>, …] }` (snapshot, includes
the requester).
- On `AuthOk` of a connection, **and** on its close (WS close or 120 s idle-timeout), the relay
sends `PresenceEvent { pubkey: <32B>, status }` to **all other** connected namespace members.
Normative rules:
1. **Namespace scope**: no cross-namespace `PresenceEvent`.
2. **`OFFLINE` is best-effort and delayed** (up to ~120 s): not a guarantee of unreachability →
the live channel has its own backstop `PeerOffline` (§6.4).
3. Idempotency: two consecutive `ONLINE` events for the same pubkey = no-op on the receiver.
## 8. Keepalive
- The relay sends **native WS ping frames** every **30 s**; the peer responds with a **pong frame**.
These are native WebSocket opcodes, not protobuf messages.
- No traffic for **120 s** → the relay closes the connection.
- The agent reconnects with exponential backoff **1s, 2s, 4s, 8s, …, max 60s** (+ jitter).
- The client manages the WS according to its foreground/background lifecycle
(see the iOS app repository documentation).
## 9. Pairing
Explicit process: the agent opens a window; the relay accepts `role: pairing` only during the
window; the token is **single-use**.
```
AGENT (perm. WS) RELAY CLIENT (new WS)
│ ─ PairingStart ──────────▶│ │
│ {token, ttl} │ │
│ ◀─ PairingReady ──────────│ │
│ show QR ──────────────────────────────────────────── ▶│
│ │ ◀─ ws connect ─────────────│
│ │ ── Challenge ─────────────▶│
│ │ ◀─ Auth role:pairing ───────│
│ │ token, client pubkeys, │
│ │ device_token, platform │
│ │ verify: window? token ok? │
│ │ TTL? single-use? → consume│
│ │ ── AuthOk ────────────────▶│
│ ◀─ ClientPaired ──────────│ (client → close WS) │
│ client pubkeys, plat. │ │
│ ─ Authorize [.. new] ────▶│ (agent decides: authorise)│
│ ◀─ AuthorizeOk ───────────│ │
│ ─ PairingStop ───────────▶│ close window │
│ ◀─ PairingStopOk ─────────│ │
```
### 9.1 `PairingStart` (agent → relay)
```proto
PairingStart { pairing_token: <32B>, ttl: 300 }
```
- `pairing_token`: 32 random bytes (single-use bearer, [crypto.md §9](crypto.md)).
- `ttl`: seconds (default 300, max 600). The relay computes `expiry = now + ttl` and stores
`{token, namespace_id, expiry, consumed: false}`.
- Response: `PairingReady { ttl: 300 }`.
If the agent calls `PairingStart` again while a window is open, the **new** token replaces the
previous one (the old token is immediately invalidated).
### 9.2 `PairingStop` (agent → relay)
```proto
PairingStop {}
```
Closes the window; the current token is invalidated. Response: `PairingStopOk {}`.
### 9.3 Implicit Stop
On `ttl` expiry without `PairingStop`, the relay closes the window automatically. A consumed
token stays consumed; an unused token becomes unusable.
### 9.4 `ClientPaired` (relay → agent)
```proto
ClientPaired {
client_ed25519_pub: <32B>,
client_x25519_pub: <32B>,
platform: PLATFORM_IOS | PLATFORM_ANDROID
}
```
The agent:
1. computes `shared_secret = X25519(agent_x25519_priv, client_x25519_pub)` and the `aes_key`
([crypto.md §4-5](crypto.md));
2. persists the client (pubkeys, counters at 0);
3. applies the **authorisation policy** (auto or user confirmation) and sends updated `Authorize`;
4. waits for the client's `hello` E2E message for detailed `device_info`.
## 10. Limits & Quotas (NORMATIVE, relay side)
| Limit | Value | Error |
|-------|-------|-------|
| Max frame size (pre-auth and store-and-forward) | 64 KiB | `payload_too_large` |
| Max frame size (live channel, post-auth) | 512 KiB | `payload_too_large` |
| Challenge timeout | 30 s | `challenge_timeout` |
| Idle timeout | 120 s | (silent close) |
| Store-and-forward queue TTL | 7 days | (silent drop) |
| Max queued messages per client | 200 | `queue_full` (rejects new until drained) |
| Max new connections per IP | 30 / minute | `rate_limited` |
| Max messages per connection | 60 / minute | `rate_limited` |
| Inactive namespace TTL | 7 days | (garbage collection) |
| Pairing `ttl` | default 300, max 600 s | (clamped) |
The 512 KiB limit for the live channel applies **only** to `RelayFrame { message { live: true } }`
and **only after `auth_ok`**. Any pre-auth frame over 64 KiB → `payload_too_large` (denies
unauthenticated flood amplification).
Values are reasonable defaults; the relay exposes them via config. Per-IP quotas contain
unauthenticated flood on the public endpoint.
## 11. Namespace Lifecycle
```
agent auth → namespace created (if new)
├── agent disconnected → namespace "idle"
├── client connected → namespace active
├── agent reconnected → resumed
└── 7 days without any connection → deleted (queues, tokens, authorised list, device_tokens)
```
Deletion is never explicit. If the agent reconnects after GC, the namespace is recreated from
scratch (same `namespace_id`, because it derives from the same key) but **without** any clients:
devices must re-pair.
## 12. Errors
Uniform format:
```proto
Error { code: "<code>", message: "<description>" }
```
`AuthError` uses the same shape, emitted during the handshake instead of `AuthOk`.
| Code | Meaning |
|------|---------|
| `challenge_timeout` | No `Auth` within 30 s. |
| `invalid_signature` | Challenge signature not valid. |
| `unauthorized` | Client not in authorised list. |
| `not_found` | Namespace or recipient not found / outside namespace. |
| `pairing_closed` | Namespace not in pairing mode, or token expired/consumed/wrong. |
| `rate_limited` | Per-IP or per-connection quota exceeded. |
| `payload_too_large` | Frame exceeds size limit. |
| `queue_full` | Recipient queue full. |
| `bad_request` | Malformed protobuf, missing field, wrong byte length. |
On all `auth_error` cases and after fatal errors, the relay closes the WS.
## 13. Summary: Everything on One WS
| Direction | Frames |
|-----------|--------|
| relay → anyone | `Challenge`, `AuthOk` / `AuthError`, `Message` (with `from`), `Error`, native WS ping |
| agent → relay | `Auth`(agent), `Authorize`, `PairingStart`, `PairingStop`, `Message`, native WS pong |
| relay → agent | `ClientPaired`, `AuthorizeOk`, `PairingReady`, `PairingStopOk`, `Message`, `PresenceEvent` |
| client → relay | `Auth`(pairing/client), `Message`, `PresenceRequest`, native WS pong |
| relay → client | `Message`, `PeerOffline`, `PresenceList`, `PresenceEvent` |
No REST endpoint exists. `namespace_id` is never in a query string.
docs/relay/server.md
+342
View File
@@ -0,0 +1,342 @@
# Relay Server — Implementation
> Guide for the coding agent building `crates/skald-relay-server`. The **protocol** is in
> [relay-protocol.md](relay-protocol.md); the **cryptography** (which the relay barely touches)
> is in [crypto.md](crypto.md). Here: internal architecture, persistence, push bridge, deploy,
> quotas.
---
## 1. Role (and Non-Role)
The relay is the **only centralised component**. It does **only** four things:
1. **Authenticates** connections (Ed25519 challenge-response) and routes by `namespace_id`.
2. **Forwards** opaque messages between agent and clients of the same namespace.
3. **Store-and-forward**: queues for offline recipients.
4. **Push bridge**: for offline clients it talks to APNs (Apple) and FCM (Google).
It does **nothing else**: no business logic, no decryption, no content reading, no user accounts.
Deliberately dumb. Its only "truth" is: `pubkey → namespace`, `pubkey → device_token`, and a
FIFO queue of blobs.
### Zero-Trust: What It Means Here (precise)
The relay is **content-confidential**, **not** metadata-private (see [index.md §4](index.md)).
It sees pubkeys, device_tokens, IPs, the relationship graph, and timing; it does **not** see
content or detailed `device_info` (which travel E2E). Everything the relay persists is either
non-sensitive or E2E-encrypted.
---
## 2. Stack & Structure
Language: **Rust**. Static musl binary ~5–7 MB, ~30 MB RAM, cold start < 100 ms.
| Crate | Use |
|-------|-----|
| `axum` | HTTP server + WebSocket upgrade, healthcheck |
| `tokio` / `tokio-tungstenite` | async runtime + per-connection WS |
| `prost` | protobuf encode/decode (`RelayFrame` from `skald-relay-common`) |
| `sqlx` (sqlite) | persistence (namespaces, clients, queue) |
| `ed25519-dalek` = "2" | challenge-response signature verification |
| `sha2`, `hex` | hashing and encoding |
| `a2` | APNs HTTP/2 + JWT |
| `reqwest` | FCM HTTP v1 (Android) |
| `tracing` | structured logs (metrics only, **never** content) |
| `clap` | CLI flags |
| `governor` | per-IP / per-connection rate limiting |
```
crates/skald-relay-server/
├── Cargo.toml
└── src/
├── main.rs # config, init, axum server, graceful shutdown
├── ws.rs # WS handler: challenge → auth(role) → forward loop
├── auth.rs # signature verification, namespace_id derivation, role gating
├── routing.rs # live connection registry (namespace → agent/clients)
├── store.rs # sqlx: namespaces, clients, queue, pairing
├── push.rs # APNs + FCM bridge, content-in-push vs wake
├── limits.rs # quotas, rate-limit, timeouts
└── types.rs # serde types for JSON (used only for push payloads / logging)
```
> **Shared crate.** Frame types and auth crypto (signature verification + `namespace_id`
> derivation) are **common** with the plugin and live in `crates/skald-relay-common`.
> The relay depends on that crate. X25519/HKDF/AES-GCM remain in the shared crate for E2E
> (plugin + app) and for the `gen-vectors` binary, not used by the relay itself.
---
## 3. Data Model (SQLite)
Minimal schema. **No sensitive data in plaintext**: `ciphertext` is E2E; pubkeys are public
identifiers.
```sql
CREATE TABLE namespaces (
namespace_id TEXT PRIMARY KEY, -- hex(SHA256(domain‖pub)), immutable
agent_ed25519_pub BLOB NOT NULL UNIQUE, -- 32B, binds id to key
created_at INTEGER NOT NULL, -- unix ms
last_active INTEGER NOT NULL, -- for 7-day GC
-- pairing window (at most one active per namespace):
pairing_token BLOB, -- 32B random, NULL if closed
pairing_expiry INTEGER, -- unix ms
pairing_consumed INTEGER NOT NULL DEFAULT 0 -- 0/1 single-use
);
CREATE TABLE clients (
namespace_id TEXT NOT NULL REFERENCES namespaces(namespace_id) ON DELETE CASCADE,
client_ed25519_pub BLOB NOT NULL, -- 32B, routing + auth
client_x25519_pub BLOB NOT NULL, -- 32B, opaque (forwarded to agent)
device_token TEXT, -- push token (APNs/FCM)
platform TEXT NOT NULL, -- 'ios' | 'android'
state TEXT NOT NULL, -- 'pending' | 'authorized'
last_seen INTEGER, -- unix ms
PRIMARY KEY (namespace_id, client_ed25519_pub)
);
CREATE TABLE queue (
id INTEGER PRIMARY KEY AUTOINCREMENT,
namespace_id TEXT NOT NULL REFERENCES namespaces(namespace_id) ON DELETE CASCADE,
to_pub BLOB NOT NULL, -- 32B recipient
from_pub BLOB NOT NULL, -- 32B sender (guaranteed by relay)
nonce BLOB NOT NULL, -- 12B
ciphertext BLOB NOT NULL, -- opaque (ciphertext‖tag)
created_at INTEGER NOT NULL -- unix ms (for 7-day TTL)
);
CREATE INDEX idx_queue_dest ON queue(namespace_id, to_pub, id);
```
Notes:
- `client_x25519_pub` is persisted for **robustness** (re-forwarding `ClientPaired` if the agent
missed it), even though the relay does not use it for crypto.
- The relay does NOT store `shared_secret`/`aes_key` (it neither has them nor can compute them).
- `state='pending'` until the agent sends `Authorize` including the pubkey (→ `authorized`).
A `role:"client"` connection is only accepted from `authorized`.
### ⚠️ Constraint: SQLite on EFS ⇒ Single Instance
In v1 the relay runs as a **single Fargate task** with SQLite on an EFS volume. SQLite on NFS/EFS
**does not support** concurrent writes from multiple processes (unreliable locking → corruption).
Therefore:
- **Do NOT** scale horizontally with this configuration (no HA, no multi-task).
- Store-and-forward assumes a **single writer**.
- **Scale path** (post-v1, if HA is needed): replace `store.rs` with Postgres (RDS) for the
queue and a distributed connection registry (e.g. Redis pub/sub) for cross-instance routing.
The `store.rs` API is designed to make this substitution localised.
---
## 4. Concurrency & Routing
Model: **one Tokio task per WS**.
- `routing.rs` holds in memory `DashMap<namespace_id, NamespaceConns>` where
`NamespaceConns { agent: Option<Sender>, clients: HashMap<pubkey, Sender> }` and `Sender` is
a `tokio::sync::mpsc::Sender<RelayFrame>` toward that WS's task.
- **Forwarding**: on receiving a `Message` from an authenticated WS, check `peer` (destination):
- if the recipient has a live connection in the same namespace → send on its `Sender`;
- otherwise → `store::enqueue(...)` and, for a client, `push::notify(...)`. Unless
`Message.live=true`, in which case → send `PeerOffline { peer }` back to the sender.
- **Single agent**: one `agent` connection per namespace; a new one displaces the old (close old).
- **Single client per pubkey**: same for devices.
- **Keepalive**: ping task every 30 s; close on 120 s silence
([relay-protocol.md §8](relay-protocol.md)).
### Store-and-Forward (delivery)
```rust
async fn deliver_pending(tx: &Sender<RelayFrame>, store: &Store,
ns: &str, to_pub: &[u8;32]) -> anyhow::Result<()> {
for m in store.fetch_pending(ns, to_pub).await? { // ORDER BY id ASC (FIFO)
tx.send(build_message_frame(m.from_pub, m.nonce, m.ciphertext, false)).await?;
store.delete_pending(m.id).await?; // delete after delivery
}
Ok(())
}
```
Queue full (> 200 for recipient, [relay-protocol.md §10](relay-protocol.md)) → reject new
messages with `queue_full` until drained. TTL: a periodic task deletes messages older than 7 days
and namespaces inactive for 7 days.
---
## 5. Push (APNs / FCM Bridge)
When a message is destined for an **offline client** with a `device_token`, the relay sends a
push. Two modes, decided by the **size of the encrypted blob**:
### 5.1 Content-in-Push (preferred, enables "approve from notification")
If `len(raw ciphertext)` fits within the payload limit (**APNs ~4 KiB**, **FCM ~4 KiB**), the
relay includes the **already E2E-encrypted blob** in the push. The device decrypts it in the
Notification Service Extension and shows a rich notification with Approve/Reject actions,
**without** opening the app.
**APNs payload**:
```json
{
"aps": {
"alert": { "title": "Skald", "body": "Action required" },
"badge": 1,
"sound": "default",
"mutable-content": 1,
"category": "skald_inbox"
},
"d": {
"ns": "<namespace_id hex>",
"from": "<agent_ed25519_pub hex>",
"n": "<nonce hex 24>",
"c": "<ciphertext base64>"
}
}
```
- `mutable-content: 1` activates the Notification Service Extension (decrypts `d.c`).
- `aps.alert` is a **generic fallback** shown if the NSE fails: **never** sensitive content.
- The relay does NOT know what is in `d.c`: it copies it as-is from the queue.
> Note: inside `d`, the values use hex/base64 encoding for JSON compatibility (nonce hex 24 chars,
> ciphertext base64). This is the one context where the encoding conventions from
> [index.md §5](index.md) apply outside the E2E JSON payload.
### 5.2 Wake-Only (fallback when blob exceeds limit)
```json
{
"aps": { "alert": { "title":"Skald", "body":"Action required" },
"badge":1, "sound":"default", "content-available":1 },
"d": { "ns": "<namespace_id hex>", "wake": true }
}
```
The device wakes, opens a **temporary WS**, downloads queued messages, and shows the Inbox.
No content in the push.
> **Choice rule (normative):** content-in-push if `len(raw_ciphertext_bytes) <= 3500` (after
> base64-encoding into JSON it will be ≤ ~4666 chars), otherwise wake-only. Conservative threshold
> to leave room for other fields. Keep `summary`/`detail` in payloads small to stay in the
> preferred case.
### 5.3 FCM (Android)
Use **FCM HTTP v1** with a **data-only message** (`"data": { … }`) so the app always handles
decryption (even in background), avoiding automatic display of an unencrypted `notification`.
Fields `ns`/`from`/`n`/`c` as above. Priority `high`.
### 5.4 Push Key Management
| Secret | Where | How |
|--------|-------|-----|
| APNs `.p8` (Apple) | **relay only** | AWS Secrets Manager in prod; `config/apns-key.json` (git-ignored) locally |
| FCM service account JSON (Google) | **relay only** | Secrets Manager / local file |
Never in the app, never in the plugin. At startup the relay loads secrets and generates:
- **APNs JWT** (ES256, valid 60 min) held in memory, **refreshed every ~30 min** (never more
than once every 20 min, per Apple's rules). No key on disk beyond the `.p8`.
- **FCM OAuth token** (from the service account) with auto-refresh.
Example APNs secret in Secrets Manager:
```json
{ "team_id":"ABC123DEFG", "key_id":"XYZ789ABCD",
"private_key":"-----BEGIN PRIVATE KEY-----\nMIGTA…\n-----END PRIVATE KEY-----" }
```
Minimal IAM for the ECS task:
```json
{ "Effect":"Allow", "Action":"secretsmanager:GetSecretValue",
"Resource":"arn:aws:secretsmanager:REGION:ACCOUNT:secret:skald/push-keys-*" }
```
---
## 6. Security Checklist
- [ ] **WSS mandatory**: reject plain `ws://`.
- [ ] Verify Ed25519 signature **before** any other logic; reject malformed input with `bad_request`.
- [ ] `namespace_id` recomputed from pubkey, **never** trusted from client input.
- [ ] Pairing token and tag comparisons in **constant-time** (`subtle`).
- [ ] `PairingStart` token **single-use** enforced atomically (`UPDATE … WHERE consumed=0`).
- [ ] Role gating: `client` only if `authorized`; `pairing` only if window open + token valid.
- [ ] **Rate-limit** per-IP on new connections and per-connection on messages (`governor`).
- [ ] Frame size limit 64 KiB (pre-auth + store-and-forward), 512 KiB (live channel post-auth);
`payload_too_large` + close on exceeded.
- [ ] **No content in logs**: log only `namespace_id`, truncated pubkeys, codes, counts, latencies.
**Never** `ciphertext`, `nonce`, full `device_token` (truncate/hash).
- [ ] `Authorize` shrink → close the revoked client's WS + **purge their queue** + forget `device_token`.
- [ ] 7-day GC for namespaces/queues.
---
## 7. Startup & Shutdown
1. `main.rs` loads config (CLI/env): port, DB path, push key source, thresholds.
2. Load push keys (Secrets Manager or file). Generate APNs JWT + FCM token (refresh task).
3. Initialise SQLite (migrations via `sqlx::migrate!`).
4. Start axum on `0.0.0.0:{port}`; route `GET /healthz` → 200; `GET /v1/ws` → upgrade.
5. **Graceful shutdown** on SIGTERM/SIGINT: stop accepting, drain WS connections, flush queue,
close DB.
### Logging
`main.rs` writes logs to both **stdout** and a file at **`logs/skald-relay.log`** (daily rotation
via `tracing-appender`), aligned with the main app. Log level controlled by `RUST_LOG`; default
`skald_relay_server=info,info`. In development:
```sh
RUST_LOG=skald_relay_server=debug # auth, routing, queue drain
RUST_LOG=skald_relay_server=trace # frame-level tracing
```
Invariant: **never log content** — only `namespace_id`, truncated pubkeys, codes, counts (see §6).
---
## 8. Deploy
| Aspect | v1 choice |
|--------|-----------|
| Compute | AWS ECS **Fargate**, **1 task** (§3 constraint) |
| Container | musl static, `FROM scratch`, ~7 MB |
| Storage | SQLite on **EFS** (persistent across restarts) |
| Push keys | Secrets Manager (`skald/push-keys`) |
| Domain/TLS | `relay.skaldagent.net` via ALB + ACM (free TLS) |
| Logs | CloudWatch (metrics only) |
| Cost | ~$5–10/month Fargate + ~$0.40 Secrets Manager |
```dockerfile
FROM clux/muslrust:stable AS build
COPY . /src
WORKDIR /src
RUN cargo build --release --target x86_64-unknown-linux-musl -p skald-relay-server --bin skald-relay-server
FROM scratch
COPY --from=build /src/target/x86_64-unknown-linux-musl/release/skald-relay-server /skald-relay-server
EXPOSE 8080
ENTRYPOINT ["/skald-relay-server"]
```
### Self-Hosting
Anyone can host their own relay: an Apple Developer Key ($99/year) for APNs (and/or a Firebase
project for FCM) is required. `docker compose` with local SQLite, or deploy on your own cloud.
The relay is open source and interoperable with any agent/app conforming to these documents.
---
## 9. Definition of Done
- [ ] `cargo build --release` produces a musl static binary.
- [ ] An agent can authenticate, create the namespace, start/stop pairing, authorize.
- [ ] A client can pair, then connect as `client` only after `authorize`.
- [ ] Messages routed live; store-and-forward + FIFO delivery on reconnect.
- [ ] Push content-in-push below threshold, wake-only above; APNs and (at least stub) FCM working.
- [ ] Revocation via `Authorize` shrink closes WS and purges queue.
- [ ] Live channel: `PeerOffline` sent correctly; no queue/push for `live=true` messages.
- [ ] Presence: `PresenceList` on `PresenceRequest`; `PresenceEvent` on connect/disconnect.
- [ ] Quotas/rate-limit active; no content in logs.
- [ ] 7-day GC verified.
- [ ] Relay never alters `nonce`/`ciphertext` (test: bytes identical in/out).
docs/relay/test-vectors.md
+243
View File
@@ -0,0 +1,243 @@
# Crypto Test Vectors — Interop
> Purpose: guarantee that **independent implementations** (relay/plugin in Rust, app in Swift,
> app in Kotlin) produce **the same bytes** from the same inputs. Without these vectors two
> "reasonable" implementations can silently diverge (KDF, byte order, AAD, nonce construction,
> plaintext framing) and never be able to decrypt each other's output.
>
> **Method (important):** the **source of truth** is the *reference generator* in §3 (Rust). The
> expected values in the tables MUST be produced by **running that tool** and then **committed**
> to this file. They are not hand-transcribed (manual transcription of crypto output causes
> errors). Every other implementation MUST reproduce those outputs exactly.
>
> **Framing:** the plaintext that is encrypted is not the raw JSON but a versioned envelope:
> `plaintext = version(0x01) ‖ comp(1B) ‖ payload(JSON)`. For payloads ≤ 1024 B, `comp = 0x00`
> (no compression). Vectors V14/V17 account for this framing — an implementor decrypting V14/V17
> must obtain the *framed* plaintext, then extract `plaintext[0]` (version), `plaintext[1]` (comp),
> and the payload.
Constants and encoding: [crypto.md §1](crypto.md), [index.md §5](index.md).
---
## 1. Fixed Inputs (deterministic)
All vectors start from these inputs. Bytes expressed in hex.
```
SEED_AGENT (32B) = 000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f
SEED_CLIENT (32B) = 202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f
CHALLENGE_NONCE (32B) = aabbccddeeff00112233445566778899aabbccddeeff00112233445566778899
COUNTER_AGENT_TO_CLIENT = 1 // u64
COUNTER_CLIENT_TO_AGENT = 1 // u64
PLAINTEXT_A2C (inbox_update, agent→client), exact UTF-8:
{"v":1,"kind":"inbox_update","id":"00000000-0000-4000-8000-000000000001","ts":1750000000000,"badge":1,"approvals":[{"request_id":"appr_test_1","tool_name":"send_email","agent_label":"Skald","summary":"Test","created_at":1750000000000}],"clarifications":[]}
PLAINTEXT_C2A (approval_response, client→agent), exact UTF-8:
{"v":1,"kind":"approval_response","id":"00000000-0000-4000-8000-000000000002","ts":1750000000000,"request_id":"appr_test_1","decision":"approved"}
```
> The two plaintext strings are **fixed** JSON (no spaces, no field reordering) **only for the
> vector**: in production JSON is not canonicalised (it is encrypted as a blob and re-parsed).
## 2. Vector Table
| # | Value | Definition | Expected (hex / base64) |
|----|-------|-------------|------------------------|
| V1 | `agent_x25519_priv` | HKDF(SEED_AGENT, salt=`skald-kdf-v1`, info=`x25519`, 32) | `<gen>` |
| V2 | `agent_x25519_pub` | X25519(V1, base) | `<gen>` |
| V3 | `agent_ed25519_priv` | HKDF(SEED_AGENT, salt=`skald-kdf-v1`, info=`ed25519`, 32) | `<gen>` |
| V4 | `agent_ed25519_pub` | Ed25519 pub from V3 | `<gen>` |
| V5 | `client_x25519_priv` | HKDF(SEED_CLIENT, …, info=`x25519`, 32) | `<gen>` |
| V6 | `client_x25519_pub` | X25519(V5, base) | `<gen>` |
| V7 | `client_ed25519_priv` | HKDF(SEED_CLIENT, …, info=`ed25519`, 32) | `<gen>` |
| V8 | `client_ed25519_pub` | Ed25519 pub from V7 | `<gen>` |
| V9 | `namespace_id` | hex(SHA256(`skald-namespace-v1` ‖ 0x00 ‖ V4)) | `<gen>` |
| V10 | `shared_secret` | X25519(V1, V6) **==** X25519(V5, V2) | `<gen>` |
| V11 | `aes_key` | HKDF(V10, salt=`skald-session-v1`, info=`aes-256-gcm`, 32) | `<gen>` |
| V12 | `nonce_a2c` | `00000001` ‖ u64_be(1) = 12B | `000000010000000000000001` |
| V13 | `aad_a2c` (96B) | `ns_raw ‖ V4 ‖ V8` (ns_raw = raw 32B of SHA256, NOT hex; from=agent, to=client) | `<gen>` |
| V14 | `sealed_a2c` | AES-256-GCM.seal(V11, V12, V13, **PT_FRAMED_A2C**) = ct‖tag | `<gen base64>` |
| V15 | `nonce_c2a` | `00000002` ‖ u64_be(1) (12B) | `000000020000000000000001` |
| V16 | `aad_c2a` (96B) | `ns_raw ‖ V8 ‖ V4` | `<gen>` |
| V17 | `sealed_c2a` | AES-256-GCM.seal(V11, V15, V16, **PT_FRAMED_C2A**) | `<gen base64>` |
| V18 | `auth_sig_client` | Ed25519_sign(V7, `skald-relay-auth-v1` ‖ 0x00 ‖ CHALLENGE_NONCE) | `<gen>` |
| | **PT_FRAMED_A2C** | `0x01 ‖ 0x00 ‖ PLAINTEXT_A2C` ([framing.md §1](framing.md)) — what is fed to AES-GCM for V14 | 258B, see §3 |
| | **PT_FRAMED_C2A** | `0x01 ‖ 0x00 ‖ PLAINTEXT_C2A` ([framing.md §1](framing.md)) — what is fed to AES-GCM for V17 | 148B, see §3 |
V12 and V15 are deterministic by construction (already filled in). All other `<gen>` values
must be filled by running the tool in §3.
## 3. Reference Generator (Rust)
Lives in `crates/skald-relay-common` as the `gen-vectors` binary.
```sh
cargo run -p skald-relay-common --bin gen-vectors
```
The generator uses the shared library (`skald_relay_common::crypto`). The Rust snippet below is
a reference for independent implementations (Swift/Kotlin).
```rust
// Framing (framing.md §1):
// plaintext = version(0x01) ‖ comp(1B) ‖ payload(JSON)
// comp=0x00 for payload ≤ 1024 B (no compression)
// What is encrypted is the FRAMED plaintext, not the raw JSON.
use hkdf::Hkdf; use sha2::{Sha256, Digest};
use ed25519_dalek::{SigningKey, Signer};
use x25519_dalek::{StaticSecret, PublicKey};
use aes_gcm::{Aes256Gcm, KeyInit, Nonce, aead::{Aead, Payload}};
use base64::{Engine, engine::general_purpose::STANDARD as B64};
fn hkdf(ikm: &[u8], salt: &[u8], info: &[u8]) -> [u8;32] {
let hk = Hkdf::<Sha256>::new(Some(salt), ikm);
let mut out = [0u8;32]; hk.expand(info, &mut out).unwrap(); out
}
fn derive(seed: &[u8;32]) -> (StaticSecret, [u8;32], SigningKey, [u8;32]) {
let x = StaticSecret::from(hkdf(seed, b"skald-kdf-v1", b"x25519"));
let xp = PublicKey::from(&x).to_bytes();
let e = SigningKey::from_bytes(&hkdf(seed, b"skald-kdf-v1", b"ed25519"));
let ep = e.verifying_key().to_bytes();
(x, xp, e, ep)
}
fn frame_payload(payload: &[u8]) -> Vec<u8> {
let mut framed = vec![0x01u8]; // version
framed.push(0x00); // comp = none (payload < 1024B)
framed.extend_from_slice(payload);
framed
}
fn main() {
let seed_a: [u8;32] = (0u8..32).collect::<Vec<_>>().try_into().unwrap();
let seed_c: [u8;32] = (32u8..64).collect::<Vec<_>>().try_into().unwrap();
let (xa, xa_pub, ea, ea_pub) = derive(&seed_a);
let (xc, xc_pub, ec, ec_pub) = derive(&seed_c);
let mut h = Sha256::new();
h.update(b"skald-namespace-v1"); h.update([0u8]); h.update(ea_pub);
let ns_raw = h.finalize();
let ns_hex = hex::encode(ns_raw);
let s1 = xa.diffie_hellman(&PublicKey::from(xc_pub));
let s2 = xc.diffie_hellman(&PublicKey::from(xa_pub));
assert_eq!(s1.as_bytes(), s2.as_bytes(), "ECDH mismatch");
let aes_key = hkdf(s1.as_bytes(), b"skald-session-v1", b"aes-256-gcm");
let cipher = Aes256Gcm::new((&aes_key).into());
let mut n_a2c = [0u8;12]; n_a2c[..4].copy_from_slice(&[0,0,0,1]);
n_a2c[4..].copy_from_slice(&1u64.to_be_bytes());
let mut aad_a2c = Vec::new(); aad_a2c.extend_from_slice(&ns_raw);
aad_a2c.extend_from_slice(&ea_pub); aad_a2c.extend_from_slice(&ec_pub);
let pt_a2c = br#"{"v":1,"kind":"inbox_update","id":"00000000-0000-4000-8000-000000000001","ts":1750000000000,"badge":1,"approvals":[{"request_id":"appr_test_1","tool_name":"send_email","agent_label":"Skald","summary":"Test","created_at":1750000000000}],"clarifications":[]}"#;
let framed_a2c = frame_payload(pt_a2c);
let sealed_a2c = cipher.encrypt(Nonce::from_slice(&n_a2c),
Payload{ msg: &framed_a2c, aad: &aad_a2c }).unwrap();
let mut m = Vec::new(); m.extend_from_slice(b"skald-relay-auth-v1"); m.push(0);
m.extend_from_slice(&hex::decode("aabbccddeeff00112233445566778899aabbccddeeff00112233445566778899").unwrap());
let sig = ec.sign(&m);
println!("V2 agent_x25519_pub = {}", hex::encode(xa_pub));
println!("V4 agent_ed25519_pub = {}", hex::encode(ea_pub));
println!("V6 client_x25519_pub = {}", hex::encode(xc_pub));
println!("V8 client_ed25519_pub = {}", hex::encode(ec_pub));
println!("V9 namespace_id = {}", ns_hex);
println!("V10 shared_secret = {}", hex::encode(s1.as_bytes()));
println!("V11 aes_key = {}", hex::encode(aes_key));
println!("V13 aad_a2c = {}", hex::encode(&aad_a2c));
println!("V14 sealed_a2c (b64) = {}", B64.encode(&sealed_a2c));
println!("V18 auth_sig_client = {}", hex::encode(sig.to_bytes()));
println!("# PT_FRAMED_A2C = {}", hex::encode(&framed_a2c));
}
```
---
## 4. Canonical Outputs (committed once)
```
# Generated by `cargo run -p skald-relay-common --bin gen-vectors`
# Framing (framing.md §1): the bytes fed to AES-GCM are
# plaintext = version(0x01) ‖ comp(1B) ‖ payload(JSON).
# Below threshold (1024B), comp = 0x00. V14/V17 seal the FRAMED plaintext.
V1 agent_x25519_priv = 497a4febd79a47e0a0b9522273ef8db2588b113e3d58365e4462e0899b932495
V2 agent_x25519_pub = 4fcb9922300372851653f0d8a0d48855674b6f6095e3770273d212bcaf51bc64
V3 agent_ed25519_priv = 13b9de6a991a9d382dec70bdeb7d8b36327ebcb81a45fa7ac7829376a695f433
V4 agent_ed25519_pub = b3e202f4ac99fd9929da47df20adedd5b2598411a466a229f086eda3467ffa7b
V5 client_x25519_priv = 5cc48fd4f6fa941053037ba6b8b1ed1daad48764d0084670307d79c4809b28a8
V6 client_x25519_pub = fc472466d9013da9a50a49b6031cde99c1cfd11c87ee04fe4da952417a1f7337
V7 client_ed25519_priv= cbaabfd5b937657cf4e7964ba87c975401337f3ce0d27026a404f102bd7c68c8
V8 client_ed25519_pub = 12355ea750e60d6370ba6776037f25062f6c9450c5009669884895fd5b377a18
V9 namespace_id = f7d340d3c3f0b0052fa904ba60ebd38a0f7e7d10672ac80648991a2c632c9e58
V10 shared_secret = 66c51034dd6360b9cdddc495049463b0191d7f3bddce9ea6f2975c85d471540a
V11 aes_key = 74fb4ffcbbe069859cfb0790023811554dad328d9f4ac4a1d28077086e33a4e7
V12 nonce_a2c = 000000010000000000000001
V13 aad_a2c = f7d340d3c3f0b0052fa904ba60ebd38a0f7e7d10672ac80648991a2c632c9e58b3e202f4ac99fd9929da47df20adedd5b2598411a466a229f086eda3467ffa7b12355ea750e60d6370ba6776037f25062f6c9450c5009669884895fd5b377a18
V14 sealed_a2c (b64) = FrtkSke7RpPUAg24p1XPZpswSX3WoDv/Y2IUvvaahY5+2CcdHXKvyRhsdjqCVa7zVs9Y0a4SZ1a7ddsPKYPz0BX/Ur3nDOOwTySKaDqT8fca//XpJyVkd60TxbfZkILNejruBLX7y2he3OI6MYu2TrmgmUSrqqfJ6NX9Go5gaKoyenXoVKOY3NKuSNmIEyIzYEkZj8uImEgah9BG/6lI59a1LWfJDlgggFf5KWkoPJHHAHA4546aPFEk5iG+3WLcjq6yiiE0p/umsr5jG2AjnkvVWYpYe8paZ4sWy/HkIYkzo9zJAGnmvK9UBHJupZABSioeRYFW2WN6ierUHbp2WyQxYvcb0x/K73Lmp4hSg6DS3w==
V15 nonce_c2a = 000000020000000000000001
V16 aad_c2a = f7d340d3c3f0b0052fa904ba60ebd38a0f7e7d10672ac80648991a2c632c9e5812355ea750e60d6370ba6776037f25062f6c9450c5009669884895fd5b377a18b3e202f4ac99fd9929da47df20adedd5b2598411a466a229f086eda3467ffa7b
V17 sealed_c2a (b64) = WYOy3vzVD+DI6lZQ4atH8g2yPfcgSo9uNNsfkWUoRD+KXWaKlDaazN6AmYAM+S3tGEVimk1HedYUJ4QrzBZJYoeBUYSxiz7WpRnqgD9mumHp8GCypttt9+/FNc7tc/zLERvtW2GfsVJSKrs0MpKFTNCauoYLdFuKdWy/A2QykrZXlySbwaNXPnMOA3ApeEsPidPHutom7G6ksgSz0qhuceIbNt4=
V18 auth_sig_client = ae38491a1f25bb5fb11f0b17e3d344412bfc927461b6517e9a0ab6a64020054677f59490af026f34c81d9378d4daae4823109ca2d1afbf4ff00230a038270002
# Framed plaintexts (input to AES-GCM, framing.md §1):
PT_FRAMED_A2C = 01007b2276223a312c226b696e64223a22696e626f785f757064617465222c226964223a2230303030303030302d303030302d343030302d383030302d303030303030303030303031222c227473223a313735303030303030303030302c226261646765223a312c22617070726f76616c73223a5b7b22726571756573745f6964223a22617070725f746573745f31222c22746f6f6c5f6e616d65223a2273656e645f656d61696c222c226167656e745f6c6162656c223a22536b616c64222c2273756d6d617279223a2254657374222c22637265617465645f6174223a313735303030303030303030307d5d2c22636c6172696669636174696f6e73223a5b5d7d
PT_FRAMED_C2A = 01007b2276223a312c226b696e64223a22617070726f76616c5f726573706f6e7365222c226964223a2230303030303030302d303030302d343030302d383030302d303030303030303030303032222c227473223a313735303030303030303030302c22726571756573745f6964223a22617070725f746573745f31222c226465636973696f6e223a22617070726f766564227d
# framed_a2c[:2] = 0100 (version=01, comp=00 = none for <1024 B)
# framed_c2a[:2] = 0100 (version=01, comp=00 = none for <1024 B)
# PT_FRAMED_A2C.len = 258 (PLAINTEXT_A2C.len + 2 framing header bytes)
# PT_FRAMED_C2A.len = 148 (PLAINTEXT_C2A.len + 2 framing header bytes)
```
Once committed, these values are **immutable**. If they change after a library update, it is a
**bug** (likely a KDF/encoding/framing divergence): investigate, do not blindly update.
> **Interop invariant:** the relay's `verify_strict` MUST accept signatures produced by the iOS
> client. Verified by cross-compat tests in
> `crates/skald-relay-server/src/auth.rs::tests::challenge_verifies_cryptokit_signature` and
> `SkaldInboxTests/SkaldInboxTests.swift::testAuthSignatureCrossCompatWithDalek`.
---
## 5. Swift Verification (CryptoKit)
Unit test in the app: derive from `SEED_AGENT`/`SEED_CLIENT` and **assert** equality with §4.
```swift
func testInteropVectors() throws {
let seedA = Data((0..<32).map { UInt8($0) })
let (signA, agreeA) = deriveKeys(seed: seedA) // crypto.md §3
XCTAssertEqual(agreeA.publicKey.rawRepresentation.hex, "<V2>")
XCTAssertEqual(signA.publicKey.rawRepresentation.hex, "<V4>")
let seedC = Data((32..<64).map { UInt8($0) })
let (signC, agreeC) = deriveKeys(seed: seedC)
let shared = try agreeC.sharedSecretFromKeyAgreement(with: agreeA.publicKey)
let key = shared.hkdfDerivedSymmetricKey(using: SHA256.self,
salt: Data("skald-session-v1".utf8),
sharedInfo: Data("aes-256-gcm".utf8), outputByteCount: 32)
// Decrypt: strip framing header from the decrypted bytes, compare with PLAINTEXT_A2C
// open(sealed=base64(<V14>), nonce=<V12>, aad=<V13>) → plaintext_framed
// plaintext_framed[2:] == PLAINTEXT_A2C
}
```
If **even one** vector does not match, the app will not be interoperable: fix it before
continuing.
---
## 6. Interop Checklist (for each implementation)
- [ ] V2/V4/V6/V8: same pubkeys from seed → **identical KDF derivation**.
- [ ] V9: same `namespace_id` → **correct domain + byte order**.
- [ ] V10: ECDH symmetric and equal → **correct X25519** (no ed25519-as-x25519).
- [ ] V11: same `aes_key` → **correct session HKDF**.
- [ ] V14/V17: mutually decryptable → **correct nonce(DIR‖counter) + AAD + GCM**.
- [ ] V14/V17: decrypted framed plaintext starts with `0x01 0x00`, remainder == PLAINTEXT_*
→ **framing.md §1 implemented correctly**.
- [ ] V18: valid and reproducible signature → **correct auth domain separation**.
- [ ] Cross-language round-trip: app decrypts a `sealed` produced by the Rust plugin and vice versa.
docs/secrets.md
+138
View File
@@ -0,0 +1,138 @@
# Secrets Store
Centralised key-value store for sensitive tokens and credentials (API keys,
HuggingFace tokens, etc.) that need to be shared across plugins and tools
without appearing in `config.yml` or plugin configs.
---
## Architecture
```text
crates/core-api/src/secrets.rs
— SecretsApi trait (full CRUD: get, set, delete, list_keys)
— require() (helper: get or bail with helpful error message)
src/secrets.rs
— SecretsStore (implements SecretsApi over SQLite)
```
`SecretsStore` holds an `Arc<SqlitePool>` and issues direct SQL queries — no
in-memory cache, no state. It is cheap to clone (just clones the pool Arc).
---
## Trait API (crates/core-api)
```rust
// core_api::secrets
#[async_trait]
pub trait SecretsApi: Send + Sync {
async fn get(&self, key: &str) -> Option<String>;
async fn set(&self, key: &str, value: &str) -> Result<()>;
async fn delete(&self, key: &str) -> Result<()>;
async fn list_keys(&self) -> Vec<String>; // never returns values
}
// Convenience: returns the value or an anyhow error with instructions.
pub async fn require(secrets: &Arc<dyn SecretsApi>, key: &str) -> Result<String>;
```
---
## Access points
| Location | Field | Use |
|---|---|---|
| `Skald` | `secrets: Arc<SecretsStore>` | Agent tools, REST API handlers |
| `PluginContext` | `secrets: Arc<dyn SecretsApi>` | Plugin start/reload (read or write) |
Plugins read secrets at startup (e.g. to pass a token to a subprocess). The
agent writes secrets via its tools. Neither needs to depend on the main crate.
---
## Usage from a plugin
```rust
use core_api::secrets;
// require() fails with a helpful message if the secret is absent.
let token = secrets::require(&ctx.secrets, "HUGGINGFACE_TOKEN").await?;
// Or a soft check:
if let Some(token) = ctx.secrets.get("MY_API_KEY").await {
// use token
}
```
---
## Agent tools
Two built-in tools let the agent manage secrets without exposing values:
| Tool | Parameters | Behaviour |
|---|---|---|
| `set_secret` | `key: string`, `value: string\|null` | Sets the secret. Empty string or null **deletes** the key. |
| `list_secrets` | `pattern?: string` | Returns keys that exist. Optional glob filter (e.g. `GOOGLE_*`). Never returns values. |
The agent can check whether a key is set by calling `list_secrets("KEY_NAME")` — if the key is absent from the result it has not been configured yet.
## Usage from Rust code
Agent tools receive `Arc<dyn SecretsApi>` from `Skald`:
```rust
skald.secrets.set("HUGGINGFACE_TOKEN", &value).await?;
skald.secrets.delete("OLD_KEY").await?;
let keys = skald.secrets.list_keys().await; // safe to log
```
---
## Well-known keys
| Key | Used by |
|-----|---------|
| `HUGGINGFACE_TOKEN` | `plugin-tts-orpheus-3b` — passed as `HF_TOKEN` env var to the Python subprocess |
Add new rows here when a plugin or tool introduces a new well-known secret key.
---
## DB: secrets table
```sql
CREATE TABLE secrets (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
)
```
Values are stored in plain text — same protection level as the rest of the
SQLite database. Do not commit the DB file.
---
## Security notes
- `list_keys()` never returns values — safe to log or surface to the agent.
- `get()` and `set()` return/accept the raw value — never log these.
- Keys are case-sensitive uppercase by convention (`HUGGINGFACE_TOKEN`).
- **The `secrets/` folder is distinct from this store.** Some credentials live on disk under
a cwd-relative `secrets/` directory (e.g. OAuth tokens written by MCP servers). The
filesystem read tools (`read_file`, `grep_files`, `list_files`, `search_file`,
`get_ast_outline`) are **denied** access to `secrets/` via seeded approval rules, so their
contents never reach the LLM context. See [approval/index.md](approval/index.md). External
MCP server processes read those token files directly and are unaffected.
---
## When to Update This File
- A new well-known secret key is introduced
- Access patterns change (new tool, new plugin using secrets)
- `secrets` table schema changes
docs/self-rewriting.md
+77
View File
@@ -0,0 +1,77 @@
# Self-Rewriting
## Restart Mechanism
The `restart` tool has **two modes**, branched on the `desktop` cargo feature:
| Mode | Behaviour | Triggers |
|---|---|---|
| **Headless** (`cargo run`, `run.sh`, Docker) | `libc::_exit(-1)` → exit code `255`. `run.sh` detects `255` and re-executes `cargo run`, which recompiles changed source files and relaunches. The supervisor is the only thing that can rebuild the binary. | The agent edited `src/**/*.rs` or `Cargo.toml` and wants the new code loaded. |
| **Desktop** (`--features desktop`, Tauri bundle) | `AppHandle::cleanup_before_exit()` (Tauri-side teardown) → `Command::new(current_exe).spawn()` → `std::process::exit(0)`. The same read-only bundled binary is relaunched — **no rebuild** (there is no source tree to rebuild from). | The user/agent wants to apply `config.yml` / DB changes that are only read at startup. Self-modification of `src/**/*.rs` has no effect in a bundle. |
The headless exit code `255` (`-1`) deliberately uses `libc::_exit()` rather
than `std::process::exit()` to skip C `atexit` handlers (e.g. Metal GPU cleanup
in `whisper-rs`, which would crash with SIGABRT and produce `134` instead of
`255`).
See [desktop.md](desktop.md) for the desktop-bundle architecture and the
`desktop::app_handle()` OnceLock that lets the tool reach the `AppHandle`.
---
## run.sh Exit Codes
| Exit code | Meaning | run.sh action |
|---|---|---|
| `0` | Graceful shutdown — SIGINT (Ctrl+C) **or** SIGTERM, both trapped in `main.rs` | Stop loop, exit 0 |
| `255` | Restart requested (`exit(-1)`) | `cargo run` again (recompile) |
| `143` | SIGTERM with no handler (`128+15`) — no longer reachable; see note | Stop loop, propagate code |
| other | Unexpected error (e.g. `101` panic) | Stop loop, propagate code |
`main.rs` traps **both** SIGINT and SIGTERM (`wait_for_shutdown_signal`) and runs the graceful shutdown path, so an external `kill` now exits `0` and logs `signal=SIGTERM` instead of dying silently with code `143`. To force a restart, use the `restart` tool (exit `255`) — never `kill` the process.
> The `run.sh` supervisor and exit-code table above apply only to **headless
> mode**. In desktop mode there is no supervisor; the Tauri process manages its
> own lifecycle and exit code `0` simply terminates the app.
---
## Safe Self-Modification Workflow
1. **Read** the relevant source files with `read_file` before making any changes.
2. **Edit** source files (`edit_file`, `write_file`, etc.).
3. **Check**: `execute_cmd` with command `cargo check 2>&1`. Inspect output.
4. **Fix** any compiler errors. Repeat steps 2–3 until clean.
5. **Restart**: call the `restart` tool only after a clean `cargo check`. The app rebuilds and relaunches automatically.
Never skip the `cargo check` step. A broken build will crash the supervisor loop with a non-zero non-255 exit code, stopping the app entirely.
---
## Requires Restart vs Does Not
| Change | Restart required? |
|---|---|
| `src/**/*.rs` | **Yes** |
| `Cargo.toml` / `Cargo.lock` | **Yes** |
| `agents/*/AGENT.md` | No — read at request time |
| `agents/*/meta.json` | No — read at request time |
| `config.yml` | No — read at startup only; take effect on next restart |
| `data/memory/**` | No — read at request time |
| `docs/**` | No |
---
## Risk Points
- **Never call `restart` mid-approval flow.** If a `PendingWrite` is waiting for user input, calling `restart` drops the `oneshot` sender, which unblocks the handler with an `Err` — the approval is cancelled and the tool call is aborted. Wait for the approval to resolve first.
- **Always check build before restart.** A compilation failure with `cargo run` returns a non-255 exit code, causing `run.sh` to stop the loop rather than retry.
- **`execute_cmd` requires user approval.** The user must approve the shell command in the UI before it executes.
---
## When to Update This File
- The restart mechanism or exit codes change
- The safe-modification workflow gains or loses a step
- New file types are added that do/don't require a restart
docs/session.md
+480
View File
@@ -0,0 +1,480 @@
# Session & Message Handling
**RunContext** (approval policy, system prompt injection, file-write pre-authorization, working directory) is documented separately — see [run-context.md](run-context.md).
---
## ChatSessionHandler Fields
| Field | Type | Purpose |
| --- | --- | --- |
| `session_id` | `i64` | DB session identifier |
| `db` | `Arc<SqlitePool>` | Persistent storage |
| `llm_manager` | `Arc<LlmManager>` | Resolves which LLM client to use |
| `max_history_messages` | `usize` | Max messages kept in context when compaction is disabled; ignored when compaction is configured |
| `max_tool_rounds` | `usize` | Max LLM rounds per turn before `Exhausted` |
| `max_tool_result_chars` | `Option<usize>` | When set, tool results from previous turns that exceed this char count are replaced with a placeholder in the LLM context. DB content is unchanged. See [Tool Result Hiding](#tool-result-hiding). |
| `agent_id` | `String` | Agent owning this session (default: `"main"`) |
| `tools` | `Arc<ToolRegistry>` | Built-in tools |
| `mcp` | `Arc<McpManager>` | MCP tools |
| `approval` | `Arc<ApprovalManager>` | Central approval service (rules + pending registry) |
| `event_bus` | `Arc<ChatEventBus>` | Publishes completed turns (user + assistant) to the in-process event bus |
| `question_registry` | `Arc<Mutex<HashMap<i64, oneshot::Sender<String>>>>` | Pending `ask_user_clarification` channels |
| `processing` | `Mutex<()>` | Prevents concurrent `handle_message` / `resume_turn` calls |
| `current_cancel` | `std::sync::Mutex<CancellationToken>` | Cancellation scope for the in-flight turn. A fresh token is minted per user message / resume and a clone is threaded by value through the whole recursive call tree; `cancel()` cancels the stored token. Never reset mid-turn → a `/stop` is sticky across sub-agent recursion |
---
## build_agent_config
Private helper called by both `handle_message` and `resume_turn` to avoid duplicating the LLM-resolution and tool-assembly logic.
1. Load `meta.json` for the current `agent_id` (scope, strength).
2. Resolve LLM client key via `LlmManager::resolve(client_name, scope, strength)`.
3. Build `base_tool_defs`: built-in tools + `call_agent` + `update_scratchpad`. **MCP tools are no longer included here** — they are resolved dynamically in `all_tool_defs()` each round based on `active_mcp_grants`.
4. Load session MCP grants from `session_mcp_grants` DB → populate `active_mcp_grants`. If `enabled_mcp_servers` override is provided, merge those names in-memory without touching the DB.
5. Inject `activate_tools` as an `InterfaceTool` (session-scoped, `stack_id = None`). Skipped if `enabled_mcp_servers` override is active.
6. **RunContext system prompt injection**: read `RunContext.extra_system_prompt()` and append its result to `extra_system_dynamic` (the dynamic tail system message, injected after conversation history, not cached). If both the caller-provided `extra_system_dynamic` and the RunContext fragments are non-empty, they are joined with `"\n\n"`.
7. Return `AgentRunConfig { ..., mcp: Arc<McpManager>, active_mcp_grants }`.
---
## handle_message Flow
1. Acquire `processing` mutex (blocks if another message is being processed).
2. Mint a fresh `CancellationToken`, store it in `current_cancel`, and thread a clone by value through `run_agent_turn` (and the sub-agent recursion).
3. **Memory context** — call `memory_manager.query_context(session_id, user_message)` for **all** sessions (including cron and tic). If a string is returned it is stored as `extra_system_dynamic` — **not** merged into `extra_system_context`. It will be injected as a dynamic tail system message after the conversation history (see *Context Building*). Only the write path filters by `is_interactive`/`is_ephemeral`.
4. Call `build_agent_config(client_name, enabled_mcp_servers, extra_system_static, extra_system_dynamic, interface_tools)` → `AgentRunConfig`. This also calls `memory_manager.tools()` and stores them in `AgentRunConfig::memory_tools`.
5. Get or create the active `chat_sessions_stack` frame.
6. Check for orphaned user message (see below) and mark it `failed` if found.
7. Append the user message to `chat_history` (with `is_synthetic` and optional `metadata` persisted via `append_with_metadata`); capture the returned `user_message_id`. `metadata` (type `MessageMetadata` in `core-api`) carries file attachments for web/mobile/Telegram messages; `content` stays the clean typed text. For non-synthetic messages, emit a `UserMessage { message_id, content, attachments }` event right after the append — the telnet-style echo that makes the bubble appear (clients never render the message optimistically).
8. Call `resume_pending_tools(stack_id, &config, &tx)` — re-gates and executes any `pending` tool calls left from an interrupted session.
9. Call `run_agent_turn(stack_id, &config, &tx, pending_input)` and await outcome. `pending_input: Option<Arc<dyn PendingUserInput>>` is the source's inbox handle for [live mid-turn injection](#mid-turn-injection) — `Some` for interactive web/mobile turns, `None` for cron/tic.
10. On `Final`: send `Done` event (and `Truncated` if applicable); then publish **two events** to `ChatEventBus` — one `User` event (with `is_synthetic` from the caller) and one `Assistant` event (with all `tool_calls` collected during the turn).
11. On `Cancelled`: send `Error` event ("interrupted by user"); return `Err("Turn cancelled by user")`. Background runners (cron, tickets) see the `Err` and record the job as `"failed"`. The WS handler logs at INFO level (not ERROR) when it detects this error string, since the client already received the error event.
12. On `Exhausted`: send `Error` event (tool round limit exceeded); return `Err(...)`. Background runners (cron, tickets) see the `Err` and record the job as `"failed"`. Interactive WS sessions already received the `ServerEvent::Error`; the returned `Err` is logged by the WS handler.
`is_synthetic` is a parameter of `handle_message`. It is `true` for TicManager ticks (system-generated messages injected as user turns), `false` for all real user input. Additionally, `ChatHub::notification_consumer` injects synthetic **Assistant** messages with `is_synthetic = true` containing the `read_notification` tool call and reasoning trace — these are not user turns, but share the same flag for UI filtering. The flag is **persisted** to `chat_history.is_synthetic` so that the UI history API (`GET /api/sessions/:id`) can filter those rows out on page reload — synthetic messages never appear in the conversation visible to the user. They are still included in the LLM context (via `build_openai_messages`) so the assistant can see what it previously said in response to a notification.
### Session detail debug view
`GET /api/sessions/:id` returns the full session tree in a debug-friendly format. Unlike the live message API, this endpoint:
- **Includes** synthetic user messages (marked `is_synthetic: true` in the JSON)
- **Includes** `reasoning_content` on assistant / thinking messages
- Returns session metadata (`source`, `agent_id`, `is_interactive`, `is_ephemeral`, `created_at`)
Response shape:
```json
{
"session": { "id": 42, "source": "tic", "agent_id": "main", "is_interactive": false, "is_ephemeral": true, "created_at": "…" },
"messages": [
{ "kind": "user", "content": "…", "is_synthetic": true, "created_at": "…" },
{ "kind": "thinking", "content": "…", "reasoning": "…|null", "created_at": "…", "input_tokens": N, "output_tokens": N },
{ "kind": "assistant", "content": "…", "reasoning": "…|null", "created_at": "…", "input_tokens": N, "output_tokens": N },
{ "kind": "tool", "name": "…", "arguments": {}, "status": "done|error|pending", "result": "…" },
{ "kind": "agent", "agent_id": "…", "depth": N },
{ "kind": "agent_end", "agent_id": "…", "depth": N }
]
}
```
The frontend `<session-detail-page>` renders this at hash `#session/{id}`. The detail page includes a **Back** button that calls `history.back()`. The page is not linked directly in the sidebar but is fully functional when the hash is set directly, or when navigated to from the TIC Sessions page.
### Session list API
`GET /api/sessions?source=tic&page=1&per_page=20` — paginated list of sessions, optionally filtered by source.
Response shape:
```json
{
"items": [
{ "id": 42, "source": "tic", "agent_id": "main", "is_ephemeral": true, "is_interactive": false,
"created_at": "…", "message_count": 7, "last_message_at": "…" }
],
"total": 100, "page": 1, "per_page": 20
}
```
The `<tic-sessions-page>` component renders this at hash `#tic` (linked from the sidebar under **TIC Sessions**). Each row is clickable and navigates to `#session/{id}`.
---
## resume_turn Flow
Called by `ChatHub::resume()` (routed through the global event bus) when the client sends `{"type":"resume"}`, and by `inject_async_result` after an async task finishes. Continues without appending a new user message. It is **not** part of the normal synchronous sub-agent path (that is plain recursion in `dispatch_sub_agent`); `resume_turn` exists for app-restart recovery of an active child stack, async result injection, and the WS resume message.
1. Acquire `processing` mutex.
2. Mint a fresh `CancellationToken` (a resume is a new unit of work — it must not inherit a stale cancellation, but a `/stop` during the resume still cancels this token) and store it in `current_cancel`.
3. Call `build_agent_config(...)` → `AgentRunConfig`.
3b. **`reap_interrupted_parallel_batches`** — the linear cascade below assumes one active frame per depth, which an interrupted [parallel sub-agent batch](#parallel-sub-agent-batches) violates. Detect a batch by ≥2 active `chat_sessions_stack` frames sharing a depth (impossible for a linear stack), then — accepting the single-user tolerance for restart loss — fail each such frame's spawning tool call and terminate the frame, from the shallowest multi-frame depth downward. The parent is left with a fully-resolved tool-call set and the normal cascade resumes it. A lone interrupted sub-agent (one frame at its depth) is left untouched.
4. Get the active `chat_sessions_stack` frame — if none exists, return immediately.
5. Call `resume_pending_tools(stack_id)`.
6. **Seed the cascade**: if no pending tools were found AND the deepest active frame's last assistant message has no associated tool calls (pure-text final response), that frame's own turn is already complete. Two sub-cases:
- **Root frame** (no `parent_tool_call_id`) → nothing to do, return immediately.
- **Child frame** (has a `parent_tool_call_id`) → its result was produced but **never propagated** to the parent (e.g. the turn task died right after the child finished — see below). Seed `current_outcome` from the existing final message (its `content`) **without re-running the LLM**, and enter the cascade so the parent's tool call is completed and the parent continues. *(Skipping this case unconditionally — as an earlier version did — left the parent wedged forever: tool call stuck `running`, child frame never terminated, main agent never resumed.)*
Otherwise (last assistant message *does* have tool calls, e.g. a `task_completed` injected asynchronously), call `run_agent_turn(stack_id, …, None)` — resume never does live user-message injection.
7. **Cascade loop**: while the current stack has a `parent_tool_call_id`, complete/fail the parent's tool call, terminate the child stack, and run `run_agent_turn` on the parent stack. Repeat until reaching the root (depth = 0). Handles both restart recovery of an active child stack and the "completed-child never propagated" repair seeded in step 6.
8. At root: same `Final` / `Cancelled` / `Exhausted` handling as `handle_message`.
---
## resume_pending_tools
Called at the start of `handle_message` (and by the REST endpoint after a manual resolve). Finds any `running`/`pending` tool calls left from a previous interrupted session, re-runs them through the approval gate, executes approved ones, and rejects denied ones — so `run_agent_turn` sees complete history and can continue cleanly. Takes the turn's `token` so a `/stop` during resume cancels cleanly.
It shares the same collaborators as the live loop: `run_approval_gate` (so resume applies the **RunContext fast-path** and the **auto-deny** short-circuit identically — previously only the live loop did) and `record_tool_outcome` (so `resume` does not accumulate `ToolCallEvent`s nor re-emit `FileChanged`, which are live-turn concerns — it passes `None` for both).
**Rehydration = re-run from intent.** Each pending row is `(name, args, status)`; the live future was never serialized. The tool is reconstructed with the same `build_execution(name, args) → ToolExecution` used by the live loop and re-run from the start via `drive_execution`. This uniformly covers registry / memory / image / interface / MCP tools (previously only memory + registry were handled). `cancelled`/`rejected` rows are terminal and are **not** re-run.
`restart` is handled as a special case: it marks the call `done` in the DB before calling `std::process::exit(-1)`.
---
## AgentFlowSignal
`AgentFlowSignal` (`src/core/session/handler/mod.rs`) is a typed `pub(super)` enum used by internal dispatch methods to communicate control-flow outcomes through `anyhow::Error` without sentinel structs:
| Variant | Emitted by | Handled in |
| --- | --- | --- |
| `QuestionChannelClosed` | `dispatch_ask_user_clarification` (WS dropped) | `llm_loop.rs` → returns `TurnOutcome::Cancelled`; `resume.rs` → aborts resume |
Dispatch checks it with a single `downcast_ref::<AgentFlowSignal>()`.
---
## run_agent_turn Inner Loop
Called recursively via `Box::pin` to support async recursion without stack overflow.
`run_agent_turn` is a **thin round orchestrator**: each round's real work is delegated to focused collaborators (each an `impl ChatSessionHandler` block in its own file under `src/core/session/handler/`), so the loop reads as a sequence of named steps rather than one large function. The collaborators are shared with `resume_pending_tools`, so a live turn and a resume gate/execute/record identically.
| Collaborator | File | Responsibility |
| --- | --- | --- |
| `TurnEmitter` | `emitter.rs` | Typed, fire-and-forget seam over the per-turn `mpsc::Sender<ServerEvent>` — one semantic method per event (`tool_done`, `thinking`, …). All turn events flow through it. |
| `call_llm_round` → `RoundLlm` | `llm_call.rs` | One LLM call for the round + automatic model fallback (retries, `ModelFallback`/`LlmFailed`, message rebuild on `prompt_cache` change). Mutates `cur_name`/`cur_llm`/`messages` in place. |
| `handle_tool_call` → `CallFlow` | `llm_loop.rs` | Handles one tool call end-to-end (persist row → `ToolStart` → gate → restart → dispatch → record). Returns `Continue` or `End(outcome)`. |
| `effective_args` | `dispatch.rs` | Applies the RunContext working directory to a call's args (relative `path` → absolute, inject `workdir` for `execute_cmd`). |
| `run_approval_gate` → `GateOutcome` | `gate.rs` | The approval decision + human-approval flow (see [Approval Gate](#approval-gate)). |
| `execute_tool_call` → `DispatchResult` | `dispatch.rs` | Routes an approved call to the right executor (special non-cancellable paths + the unified cancellable `ToolExecution` path). |
| `record_tool_outcome` → `RecordFlow` | `outcome.rs` | Persists a tool outcome and emits `ToolDone`/`ToolError`/`ToolCancelled`. |
Takes the per-turn `token: &CancellationToken` by value-clone from the caller, plus `pending_input: Option<&Arc<dyn PendingUserInput>>` (see [Mid-turn injection](#mid-turn-injection)). For each round (up to `max_tool_rounds`):
1. Check `token.is_cancelled()` — return `Cancelled` immediately if set.
1b. **Mid-turn injection**: if `pending_input` is `Some`, `drain_user()` and append each queued message as its own `user` row + emit a `UserMessage` echo. These rows are read by `build_openai_messages()` in this same round, so the model sees them immediately. `None` for sub-agents / resume / non-interactive runners.
2. `build_openai_messages()` — reconstruct full context from DB.
3. `call_llm_round(...)` — one LLM call wrapped in `tokio::select!` against `token.cancelled()` (a `/stop` aborts the in-flight request → `RoundLlm::Cancelled` → `Cancelled`), with automatic model fallback on retriable errors. Returns `RoundLlm::{Turn, Cancelled, Failed}`.
4. On `LlmTurn::Message` — persist assistant message, return `Final` (with all `tool_calls` accumulated across rounds).
5. On `LlmTurn::ToolCalls` — persist the assistant "thinking" message (emit `Thinking` if non-empty). If the response is a **homogeneous batch** — `≥2` calls that are *all* synchronous sub-agents (`is_sync_sub_agent`) — dispatch them concurrently via `handle_sub_agent_batch` (see [Parallel sub-agent batches](#parallel-sub-agent-batches)); otherwise fall back to the sequential loop below: for each call (checking `token.is_cancelled()` before each one) call `handle_tool_call`, which:
- Records the tool call in `chat_llm_tools` (status: `pending`) and emits `ToolStart` (with original LLM-provided args, before WD injection).
- Computes `effective_args` — **working directory injection**: if `RunContext.effective_working_dir()` is set, resolve relative `path` args to absolute and inject `workdir` into `execute_cmd` args (if the LLM didn't already set one).
- Runs `run_approval_gate` on `effective_args` (see [Approval Gate](#approval-gate)). `GateOutcome::Rejected` → the gate already marked the row `rejected` and emitted `ToolRejected` → `CallFlow::Continue` (skip); `GateOutcome::ChannelClosed` → `CallFlow::End(Cancelled)`.
- Handles `restart` inline (mark `done`, emit `ToolDone`, `libc::_exit(-1)`).
- Dispatches via `execute_tool_call`. **Special, non-cancellable paths** return a plain `Result<String>`: sync sub-agent (`execute_task` mode=sync / `execute_subtask`) → `dispatch_sub_agent` (recursive, inline); `update_scratchpad`/`write_todos`; `ask_user_clarification` → emit `AgentQuestion`, await answer; `task_completed` stub. **Everything else** (built-in registry incl. `execute_cmd`, memory/image tools, MCP, interface tools) goes through the **unified cancellable path**: `build_execution(name, args) → ToolExecution`, driven by `drive_execution(exec, token)`. See [Tool execution lifecycle](tools.md#tool-execution-lifecycle). If the clarification WS channel closed while awaiting an answer, `execute_tool_call` returns `DispatchResult::AbortPending` → the tool stays `pending` (not recorded) and the turn ends `Cancelled` so resume re-asks it.
- Records the outcome via `record_tool_outcome`: `Completed` → `ToolDone`, status `done` (+ `FileChanged` for file-write tools); `Failed` → `ToolError`, status `failed`; `Cancelled` (a `/stop` hit the tool mid-flight) → `ToolCancelled`, status `cancelled`, and `handle_tool_call` returns `CallFlow::End(Cancelled)`. The execution's `stop()` was called (e.g. dropping the work future kills an `execute_cmd` child via `kill_on_drop`), so the tool aborts **immediately** instead of running to completion.
6. Loop back — next round rebuilds context with tool results included.
7. If all rounds exhausted: return `Exhausted`.
A sync sub-agent runs via `dispatch_sub_agent`, which awaits `run_agent_turn` recursively in the **same task** (same `processing` lock, same `token` clone) and returns the child's result as the parent tool call's result. Because parent and child share the token, a `/stop` that cancels a running child also stops the parent at its next check — no `WaitingChild` / task-spawn / resume cascade involved.
### Parallel sub-agent batches
When a single assistant response emits **≥2** synchronous sub-agent calls and *nothing else*, `handle_sub_agent_batch` (in `llm_loop.rs`) runs them concurrently instead of one-by-one. It is a three-phase restructuring of the same seams `handle_tool_call` uses, so the sequential path is left untouched as the fallback for every other shape (a lone call, or any mix with regular/side-effecting tools).
- **Phase 1 (sequential, in call order):** `chat_llm_tools::append` allocates each call's row id and emits `ToolStart`. The row id is what the LLM uses to reconstruct tool-result order (`ORDER BY id ASC`), so allocating them up front — before any concurrency — is what preserves ordering regardless of which sub-agent finishes first.
- **Phase 2 (concurrent, bounded):** the approval gate + `execute_tool_call` (→ `dispatch_sub_agent`) for all calls run through `futures::stream::…buffer_unordered(max_parallel_subagents)` (default `4`, config `llm.max_parallel_subagents`, `1` = sequential). Each future writes only to its own child stack frame + tool_call_id and borrows `&self`/`config`/`token`/`tx` — no shared mutable state between siblings. The shared cancellation token means a `/stop` (or one cancelled sibling) stops the others.
- **Phase 3 (sequential, in call order):** `record_tool_outcome` persists each result and appends to `all_tool_calls` in the original call order, so completion order never leaks into history or the event stream.
Siblings share the session-scoped scratchpad blackboard: concurrent writes to the *same* key are last-writer-wins by design (write distinct keys to avoid clobbering). Restart recovery of an interrupted batch is handled by `reap_interrupted_parallel_batches` (see [resume_turn Flow](#resume_turn-flow)).
### Mid-turn injection
A user can send a message while a turn is still running, and the agent picks it up at its next round boundary — without `/stop` and without waiting for the whole turn to finish.
- `run_agent_turn` receives `pending_input: Option<&Arc<dyn PendingUserInput>>` (the source's inbox handle from `ChatHub`). It is `Some` **only** for the root interactive turn; sub-agents (`dispatch_sub_agent`), `resume_turn`, and non-interactive runners (cron, tic) pass `None`.
- At the top of each round (step 1b above), the turn drains the inbox and appends each queued message as its **own** `chat_history` `user` row, then emits a `UserMessage` event (telnet-style echo — see [frontend.md](frontend.md)). The round boundary is the only safe ordering point: the previous round's assistant message + tool results are all persisted, so a trailing `user` row is well-ordered.
- It does **not** interrupt the in-flight LLM call or tool, and does **not** reset the round budget. Messages that arrive after the turn's last boundary stay queued and seed the next turn.
- `MessageBuilder` later merges consecutive non-failed `user`/`agent` rows into one `role:user` (see *Context Building*), so several injected messages read as one clean user turn for the LLM while the DB keeps each message distinct.
- A `/stop` clears the inbox, so queued-but-not-yet-injected messages are dropped, never persisted, never echoed. See [chat-hub.md](chat-hub.md) for the inbox/consumer side.
---
## Approval Gate
The gate is `ApprovalManager.check(session_id, category, agent_id, source, tool_name, args)` → `GateResult`.
**Evaluation order:**
1. Hardcoded exception: file-write tools targeting a path that starts with `memory/` → `Allow` (always auto-approved).
2. Rules from the `approval_rules` table, sorted by `priority ASC` (lower = evaluated first). First match wins.
3. **Session bypass** (in-memory, not persisted): if the result would be `Require` and an active bypass exists for this `session_id` whose `scope` matches (All, Category, or McpServer), convert to `Allow`. `Deny` is never bypassed.
4. No match → `Allow` (default-open policy).
**Default rules** (seeded at startup if the table is empty):
`execute_cmd`, `restart`, `write_file`, `edit_file`, `insert_at_line`, `replace_lines` → `require`
**Session bypass** is activated by the **human** (not the LLM) from the **Agent Inbox** UI or via the REST endpoint. Each bypass entry targets a `BypassScope`:
| Scope | What it covers |
| ----- | -------------- |
| `All` | Every tool regardless of category |
| `Category(ToolCategory)` | Only tools with the given registered category (e.g. `Filesystem`, `Shell`) |
| `McpServer(String)` | Only tools from the named MCP server (matched by the `mcp__<server>__` prefix) |
The bypass state lives in `ApprovalManager::session_bypasses` (`Mutex<HashMap<i64, Vec<ApprovalBypass>>>`). `check()` receives `session_id`, `category`, and `tool_name`. Expired entries are pruned lazily on each `check()` call. All entries for a session are cleared when `cancel_for_session()` is called (WS disconnect). The state is **never persisted** — it is reset on app restart.
**`run_approval_gate` (`gate.rs`)** wraps the whole decision + human-approval flow and returns a `GateOutcome`, so `run_agent_turn` and `resume_pending_tools` gate identically. It first calls `ApprovalManager.check(...)`, then applies the **RunContext fast-path** (relax `Require` → `Allow` for a pre-authorized file read/write path; never overrides a `Deny`), then:
- `Allow` → `GateOutcome::Proceed`.
- `Deny` → mark tool call `rejected`, emit `ToolRejected`, `GateOutcome::Rejected`.
- `Require`:
1. If the session has `auto_deny_approvals` set (headless runners that cannot answer, e.g. TIC), mark `rejected` + emit `ToolRejected` → `GateOutcome::Rejected` (no blocking).
2. Otherwise mark the row `pending`, register a `oneshot` channel via `ApprovalManager.register(...)` → `(request_id, rx)`.
3. Call `emit_approval_event(em, request_id, tool_call_id, name, args)` (emits through the [`TurnEmitter`](#run_agent_turn-inner-loop)), which selects the event type:
- **file-write tools** (`write_file`, `edit_file`, `insert_at_line`, `replace_lines`): read current file + compute predicted result concurrently → `PendingWrite { old_content, new_content }`. Falls back to `ApprovalRequired` if the diff cannot be computed.
- **`execute_cmd`**: `PendingWrite` with `path = "$ execute_cmd"`, `new_content = "$ <command>"`.
- **`restart`**: `PendingWrite` with restart description.
- **everything else**: `ApprovalRequired { tool_name, arguments }`.
4. Await `rx`:
- `Approved` → `GateOutcome::Proceed`.
- `Rejected { note }` → mark tool call `rejected` with the reason, emit `ToolRejected` → `GateOutcome::Rejected`. The saved reason — including the user's justification — is surfaced to the LLM as the tool-result content on the next request (see [MessageBuilder](#context-building)). Every reject surface (copilot WS, Agent Inbox, REST `/sessions` and `/inbox`, mobile, Telegram) passes the **raw** user note; the canonical message string is built in one place by `ApprovalDecision::rejection_message(note)` → `"User rejected this tool call. Reason: <note>"` (or `"User rejected this tool call."` when the note is empty), so wording stays consistent and no surface-specific prefix leaks into the LLM context.
- Channel closed (WS disconnected) → `GateOutcome::ChannelClosed`, which the caller maps to `TurnOutcome::Cancelled` (live) / aborts the resume.
---
## MessageBuilder
`build_openai_messages` is now a thin wrapper that delegates to `MessageBuilder` (`src/core/session/handler/message_builder.rs`). `MessageBuilder` is a self-contained struct with no reference to `ChatSessionHandler`:
```rust
pub struct MessageBuilder {
pub pool: Arc<SqlitePool>,
pub session_id: i64,
pub mcp: Arc<McpManager>,
pub datetime_config: DatetimeConfig,
pub max_history_messages: usize,
pub max_tool_result_chars: Option<usize>,
pub compactor: Option<Arc<ContextCompactor>>,
}
```
This allows the message-building logic to be tested in isolation with an in-memory SQLite database (no full `ChatSessionHandler` required). `ChatSessionHandler::build_openai_messages` constructs a `MessageBuilder` from its own fields and delegates.
---
## Context Building
`build_openai_messages` (backed by `MessageBuilder::build`) assembles the message array in the following order, optimised for prefix KV caching:
### 1. Static system message
Contents: AGENT.md + `inject_memory` files + `extra_system_static` (e.g. Telegram format rules) + MCP list.
**Runtime substitutions**: after assembling the static content, `MessageBuilder::build` applies `system_substitutions` — each entry replaces the `__KEY__` sentinel with the provided value. These sentinels originate from `<!-- KEY -->` directives in AGENT.md (resolved by `agents::resolve_includes`).
When `cache_hints = true` (Anthropic models via OpenRouter), the content is wrapped in a `cache_control: ephemeral` block so the provider caches it as a KV prefix. For all other providers this message is a plain string that never changes turn-to-turn, so the provider's own automatic prefix cache (if any) hits on it.
### 2. Scratchpad system message *(if non-empty)*
The session scratchpad emitted as a separate `[system]` message **before** the conversation. Kept isolated from the static block so a mid-turn `update_scratchpad` call only invalidates this small message, not the large cacheable prefix.
**Async sub-tasks** share the parent session's scratchpad: when a task is launched with `kind='async'`, its handler is initialised with `scratchpad_session_id = parent_session_id`. All reads and writes via `update_scratchpad` are then scoped to the parent session instead of the task's own isolated session, so 5 parallel async tasks launched by the same parent all read/write the same shared pad.
### 3. Compaction summary system message *(if present)*
See *Context Compaction*.
### 4. Conversation history
`chat_history` for the stack. When compaction is **disabled**, the list is truncated to `max_history_messages` (oldest dropped first). When compaction is **enabled**, `max_history_messages` has no effect — the compactor owns the token budget and truncating by count would silently discard history that should be summarised instead. For a user/agent row that carries attachments in its `metadata` column, the builder appends an `[SYSTEM INFO]` block (`core_api::message_meta::attachments_block`, path-only) to that turn's content **on the fly** — the block is never persisted; `content` stays the clean typed text and the UI renders the same `metadata.attachments` as chips. **Consecutive non-failed `user`/`agent` rows are coalesced into a single `role:user` message** (their contents joined by blank lines, attachment blocks preserved) — `for_stack` already excludes `failed` rows. This is what keeps the LLM context clean when several messages were stored as distinct rows, e.g. injected back-to-back mid-turn (see [Mid-turn injection](#mid-turn-injection)). Each assistant entry with tool calls in `chat_llm_tools` is reconstructed with a `tool_calls` array and one `tool` result message per call. The tool-result content is derived from the call's terminal status:
| Status | LLM-visible `tool` content |
| ------ | -------------------------- |
| `done` | the saved `result` |
| `failed` | `Error: <result>` |
| `rejected` | the saved reason (e.g. `User rejected this tool call. Reason: <note>`) — the human's justification reaches the LLM verbatim |
| `cancelled` | the saved note (a `/stop` cancellation) |
| `pending` / `running` (interrupted by a crash or lost connection) | `Error: tool call was interrupted (connection lost before user approval). Please retry the operation.` |
Tool result hiding (see below) is applied to results from previous turns.
### 5. Dynamic tail system message
Contains `extra_system_dynamic` (e.g. Honcho long-term memories, retrieved fresh each turn) followed by a date/time/OS/working-directory block:
- **Date/time** — formatted in the effective timezone (the `datetime.timezone` config value if set, otherwise the OS timezone via `iana-time-zone`); the IANA name is shown alongside the offset, e.g. `2026-06-17T21:20:00+01:00 (Europe/Rome)`.
- **Operating system** — type + version via `os_info` (e.g. `Mac OS 15.5.0 [64-bit]`), computed once and cached.
- **Working directory** — the session's effective WD, followed by a note that filesystem tools and `execute_cmd` use it for relative paths (no need to `cd`).
Placed **after** the conversation so the stable prefix (messages 1–4) is never invalidated by per-turn changes. The model's recency-biased attention also ensures it reads fresh user context immediately before generating its response.
### 6. Tail reminder system message *(if provided)*
Short anti-drift reminder (e.g. Telegram HTML format rules) at the very end.
---
## Tool Result Hiding
Controlled by `max_tool_result_chars` in `config.yml` (`llm.max_tool_result_chars`).
When set, `build_openai_messages` calls `maybe_hide_tool_result` for every tool result it reconstructs. The replacement happens **only when all three conditions hold**:
1. The result belongs to a **previous turn** — i.e. the assistant message that produced it appears before the last user/agent message in the (truncated) history.
2. `max_tool_result_chars` is `Some(n)`.
3. The result string exceeds `n` characters.
When all three are true, the content sent to the LLM is replaced with:
```text
[Tool response for `<tool_name>` hidden: response was N chars, exceeding the L-char limit. Call the tool again if you need this information.]
```
**What is never affected:**
- The database row — always retains the original content.
- The frontend — always displays the full result.
- Tool results from the **current turn** — always shown in full, regardless of size, so the LLM can work with them within the same turn.
**Current-turn boundary detection:** the last `User` or `Agent` role entry in the truncated history marks the start of the current turn. Any assistant message at a lower index is from a previous turn.
### Scratchpad injection format
```xml
<scratchpad>
<!-- Temporary notes shared by all agents in this session (including async sub-tasks). Not persisted across sessions. -->
<note key="db_url">postgres://localhost/mydb</note>
<note key="main_struct">src/session/handler/mod.rs</note>
</scratchpad>
```
Only injected when the `session_scratchpad` table has at least one row for the session. For async sub-tasks the `session_id` used here is the parent's (see above).
---
## TurnOutcome Enum
| Variant | Meaning |
| --- | --- |
| `Final { content, message_id, input_tokens, output_tokens, truncated, tool_calls }` | LLM produced a final text response; `tool_calls` carries all `ToolCallEvent`s from all rounds |
| `Cancelled` | The turn's `CancellationToken` was cancelled (`/stop`), or WS closed during approval. `handle_message` returns `Ok(())`. |
| `Exhausted` | All `max_tool_rounds` used without a final message. `handle_message` returns `Err(...)` so background runners record the job as `"failed"`. |
---
## Session Cancellation via System Bus
Forceful task termination (e.g. the kill-task API) goes through the system bus to avoid direct coupling between the HTTP layer and the session internals.
**Flow**:
1. `POST /api/cron/jobs/{id}/kill` reads `running_session_id` from the DB and emits `SystemEvent::SessionCancelled { session_id }` on the system bus. Returns 202 immediately.
2. A background subscriber started in `Skald::new()` receives the event and calls `ChatSessionManager::cancel_session(session_id)`.
3. `cancel_session` — operates only on handlers already in the `active` map (no side-effectful creation for an unknown session):
- `handler.cancel()` — cancels the `CancellationToken`; LLM calls and `execute_cmd` unblock via `tokio::select!`.
- `handler.cancel_pending_approvals()` — drops the `oneshot::Sender` for every pending approval of that session; `approve_rx.await` returns `Err`, which the loop interprets as `TurnOutcome::Cancelled`.
- `handler.cancel_pending_questions()` — same for clarification channels; `rx.await` returns `Err(QuestionChannelClosed)`, which also yields `TurnOutcome::Cancelled`.
4. `handle_message` returns `Err("Turn cancelled by user")` → `run_job` records the job run as `"failed"`.
This means kill works correctly even when the task is blocked on `ask_user_clarification` or waiting for human approval — both unblock the moment `cancel_session` drops their sender channels.
---
## Concurrency Constraint
Only one `handle_message` / `resume_turn` call can run per `ChatSessionHandler` at a time. The `processing: Mutex<()>` is held for the entire duration. A second call blocks until the first completes or is cancelled.
Note that callers don't reach `handle_message` directly: `ChatHub` serializes user messages **per source** through a single-consumer inbox *before* the `processing` lock, and messages that arrive during an in-flight turn are **injected into that turn** at a round boundary (not queued as a separate turn). So in practice the `processing` lock is rarely contended for interactive sources — see [chat-hub.md](chat-hub.md).
Synchronous sub-agents run **inline in the same task** as the parent (plain recursion in `dispatch_sub_agent`), so the single `processing` lock covers the whole parent+child tree — one user message is one logical critical section. A [parallel sub-agent batch](#parallel-sub-agent-batches) still runs within that one critical section and one task: `handle_sub_agent_batch` drives the concurrent children on the current task's async runtime (via `buffer_unordered`, not `tokio::spawn`), so they share the same `processing` guard and cancellation token — the concurrency is between the children, not between top-level turns. (Asynchronous tasks — `execute_task` mode=async — are a separate mechanism: a new ephemeral session driven by the cron runner, whose result is later injected via `inject_async_result` → `resume_turn`.)
---
## Orphaned Message Handling
If the last message in history has `role = User` or `role = Agent` (no following assistant message), the previous turn was cancelled before the LLM responded. That message is marked `status = failed` and excluded from the context sent to the LLM, preventing user→assistant alternation errors.
---
## AgentRunConfig
Built once per `handle_message` call and passed by reference through the entire agent/sub-agent recursion.
| Field | Purpose |
| --- | --- |
| `agent_id` | ID of the current agent |
| `client_name` | Resolved LLM client key |
| `depth` | Recursion depth: 0 = root, 1+ = sub-agent |
| `base_tool_defs` | Built-in tool definitions only (no MCP — those come from `all_tool_defs()` dynamically) |
| `extra_system` | Optional extra system context (set to `None` for sub-agents) |
| `system_substitutions` | `HashMap<String, String>` — named substitutions applied to the system prompt at build time. Each entry replaces `__KEY__` sentinels in the prompt text. |
| `interface_tools` | Interface-specific tools. For sub-agents contains only `activate_tools`; all other interface tools are dropped |
| `memory_tools` | Memory backend tools (inherited by sub-agents) |
| `mcp` | `Arc<McpManager>` — used by `all_tool_defs()` to resolve MCP tools dynamically |
| `active_mcp_grants` | `Arc<RwLock<HashSet<String>>>` — granted tool groups. Holds MCP server names and/or the reserved keyword `"config"` (which unlocks `config_tool_defs`). Re-read on every round so `activate_tools` in round N makes tools visible in round N+1. Root: session-scoped (from `session_mcp_grants` DB). Sub-agents: stack-scoped (from `stack_mcp_grants` DB), starts empty |
| `config_tool_defs` | `Vec<Value>` — built-in `Config`-category tool defs (the lazy `config` group). Appended by `all_tool_defs()` only when `active_mcp_grants` contains `"config"`. Pre-filtered (interactive-only / approval) by `build_agent_config` |
### `all_tool_defs()` — dynamic group resolution
Called on every LLM round. Returns `base_tool_defs` + MCP tools for currently-granted servers (re-queried from `McpManager` using `active_mcp_grants`) + `config_tool_defs` when the `config` group is granted + memory tools + interface tools.
This means that calling `activate_tools` in round N makes those tools available to the LLM starting from round N+1 of the **same turn** — no cross-turn delay.
Uniqueness is guaranteed by construction, not by a dedup pass: built-in tools have distinct names by registry construction, MCP tools are namespaced `mcp__{server}__{tool}`, and sub-agent inheritance cannot re-introduce a name because `for_sub_agent()` strips the per-level augmentations it re-derives (see below).
### `for_sub_agent()`
Derives a child config: inherits `base_tool_defs` (after filtering), `memory_tools`, and `mcp`; starts with **empty** `active_mcp_grants`; clears `interface_tools`; increments `depth`.
It drops two categories from the inherited `base_tool_defs`:
- **`root_agent_only` tools** (registry flag) — never exposed to sub-agents.
- **Per-level augmentations** that the config builders re-derive for every agent: `ask_user_clarification` and `execute_subtask`. Stripping them here makes `dispatch_sub_agent` the **single owner** of sub-agent augmentation, so a name can never be both inherited and re-added — the OpenAI-compat APIs reject non-unique tool names with HTTP 400, and this eliminates that failure mode structurally (no dedup pass needed).
`dispatch_sub_agent` then:
1. Replaces the empty `active_mcp_grants` arc with one pre-populated from `stack_mcp_grants` DB (restart recovery).
2. Appends `sub_agents_only` tools, `ask_user_clarification`, and (below the depth limit) `execute_subtask` to `base_tool_defs` — each present exactly once, since the inherited copy was stripped by `for_sub_agent()`.
3. Injects `activate_tools` (stack-scoped, `stack_id = Some(child.id)`) as the only interface tool.
---
## ask_user_clarification Flow
Available to every agent except hidden `system` agents (e.g. TIC), at any depth.
1. An agent calls `ask_user_clarification(question)`.
2. `run_agent_turn` intercepts it before ToolRegistry dispatch.
3. A `oneshot` channel is registered in `question_registry` keyed by `request_id`.
4. `AgentQuestion { request_id, question }` event is emitted to the frontend.
5. Execution is suspended until the client sends `{"type":"answer_question","request_id":<N>,"answer":"..."}`.
6. `resolve_question()` unblocks the channel; the answer is returned as the tool result.
7. On WS disconnect: `cancel_pending_questions()` drops all senders, causing the await to return `Err`, which propagates as a tool error.
---
## WS Resume Event Routing
When the client sends `{"type":"resume"}`, the WS handler calls `ChatHub::resume(&source)` which:
1. Finds the session handler for `source`.
2. Spawns a task running `handler.resume_turn(...)` with an mpsc sender.
3. Bridges every event from that sender to the **global broadcast bus** (tagged with the session's `source`).
All WS connections for the same source (including newly reconnected ones) receive the events via their global bus subscription. This avoids the previous design where events went to a local mpsc channel and were silently lost if the client reconnected while `resume_turn` was in flight.
### Running-state on (re)connect
A turn runs on a detached task and survives a page reload (closing the WS just `return`s from the socket loop — it does **not** cancel the turn). To let a reloaded client restore its SEND→STOP button, the WS handler — right after subscribing to the global bus — sends a `TurnRunning { running }` event to that socket, where `running = ChatSessionHandler::is_processing()` (a `try_lock` on the `processing` mutex, held for the whole turn). Because the send happens after subscribing, a turn that finishes immediately after still delivers its `Done` via the bus, which resets the client's state. The client also flips to "running" on any live streaming event (`thinking` / `tool_start` / `agent_start` / `pending_write` / `approval_required`) as a fallback. Note: with synchronous sub-agents now running recursively, the `processing` lock is held continuously for the whole parent+child tree, so `is_processing()` is a reliable signal.
---
## When to Update This File
- `needs_approval()` rules change (new tool added, path exemption modified)
- The tool-calling loop gains new behavior (new event type, new cancellation path)
- `build_openai_messages` changes (new context injected, truncation logic modified)
- `AgentRunConfig` fields change
- `build_agent_config` changes (new default tool added, resolution logic modified)
docs/session/index.md
+11
View File
@@ -0,0 +1,11 @@
# Session & Message Handling
Session management and the LLM message loop.
## Files
- [run-context.md](run-context.md) — RunContext: permissions, system prompt, file authorization (single source of truth)
- [session.md](../session.md) — ChatSessionHandler lifecycle, message flow, tool dispatch
- [llm-loop.md](llm-loop.md) — (TODO: split from session.md) Core LLM loop: handle_message, resume_turn, run_agent_turn
See [../index.md#session--llm-loop](../index.md#session--llm-loop) for navigation to related files (compaction, memory, event bus).
docs/session/run-context.md
+241
View File
@@ -0,0 +1,241 @@
# RunContext — Session Permissions & Configuration
**Single source of truth for RunContext.** Consolidates resolution, fields, usage, and API.
---
## Overview
Each session can have an active **RunContext** that controls:
- **Approval policy** — which permission group (`security_group`) applies to tool calls
- **System prompt injection** — dynamic prompt fragments per session
- **File-write pre-authorization** — paths auto-approved for writes (`allow_fs_writes`)
- **File-read auto-allow** — paths auto-approved for reads (working dir + `docs/` + `skills/` + `allow_fs_reads` + anything writable)
- **Working directory** — effective CWD for tool calls and file operations
`RunContext` is a JSON blob stored in the DB:
- `chat_sessions.run_context` — interactive web/mobile sessions
- `scheduled_jobs.run_context` — cron tasks
- `projects.run_context` — project-level defaults
- `project_tickets.run_context` — ticket-level overrides
---
## Fields
| Field | Type | Default | Purpose |
|-------|------|---------|---------|
| `security_group` | `Option<String>` | `null` | Permission group ID for approval rule lookup. Rules in this group take precedence over `"default"`. |
| `system_prompt` | `Vec<String>` | `[]` | Prompt fragments injected as dynamic system context every turn (joined with `"\n\n"`). |
| `allow_fs_writes` | `Vec<String>` | `[]` | Paths pre-authorized for file writes. Resolved against the working dir; recursive directory prefix match (no globs). Entries are also readable (write implies read). |
| `allow_fs_reads` | `Vec<String>` | `[]` | Extra **read-only** grants, beyond the working dir / `docs/` / `skills/` (always-safe baseline) and `allow_fs_writes`. Same prefix-match semantics. |
| `working_directory` | `Option<String>` | `null` | Effective WD for tool calls; `null` = Skald's process cwd. |
---
## Applicative Methods
`RunContext` exposes these methods (the session handler is agnostic to internal fields):
```rust
rc.tool_group_id() -> Option<&str> // for approval rule lookup
rc.extra_system_prompt() -> Option<String> // joins system_prompt with "\n\n"
rc.effective_working_dir() -> PathBuf // configured path or process cwd
rc.is_write_allowed(path) -> bool // pre-auth check for file writes
rc.is_read_allowed(path) -> bool // pre-auth check for file reads
```
Both `is_*_allowed` canonicalize the path first (resolving `..` and symlinks via
`tools::fs::canonicalize_for_policy`) so traversal/symlink escapes cannot widen a grant.
---
## Resolution at Session Creation
**Order of precedence** (`ChatSessionManager::create_session` or `ChatHub::provision_session`):
1. **Explicit `run_context` parameter** — JSON blob passed at session creation. Persisted in DB immediately so the handler reads it at construction.
2. **Config-driven defaults** — from `config.yml` (per-source or per-agent), or TIC's `tic.run_context` key.
3. **None** — all RunContext methods return zero values (`tool_group_id()` → `None`, `is_write_allowed()` → `false`, `effective_working_dir()` → process cwd).
The `RunContext` is stored in `ChatSessionHandler::run_context` (`RwLock<Option<RunContext>>`). The handler **reads it once at construction** and **never directly accesses its internal fields** — only calls applicative methods.
### Session Handler Usage
| Method | Used for |
|--------|----------|
| `tool_group_id()` | Approval rule lookup (passed to `ApprovalManager::check()`) |
| `extra_system_prompt()` | Injected as dynamic system tail in `build_agent_config` (see [llm-loop.md](llm-loop.md)) |
| `is_write_allowed(path)` | Fast-path for file write tools; upgrades a `Require` decision to `Allow` |
| `is_read_allowed(path)` | Fast-path for file read tools; upgrades a `Require` decision to `Allow` |
| `effective_working_dir()` | WD injection for file tools and `execute_cmd` |
---
## Runtime Update
**Endpoint:** `POST /api/sessions/{id}/run_context`
**Body:** Full `RunContext` JSON (or `null` to clear):
```json
{
"security_group": "cron_restrictive",
"system_prompt": ["Always reply in English.", "Use metric units."],
"allow_fs_writes": ["data/output", "logs/*"],
"working_directory": "/projects/skald"
}
```
**Effects:**
- Updates `chat_sessions.run_context` in DB
- If the handler is live in memory, calls `handler.set_run_context()` immediately (no restart needed)
- Changes take effect on the next turn
---
## Approval Gate Integration
Rules are scoped to **permission groups** (`tool_permission_groups` table). A session's `RunContext.security_group` field references a group; rules in that group take precedence over `"default"`.
### Evaluation Chain
```
chat_session.run_context (JSON blob)
└─► RunContext.tool_group_id() → e.g. "cron_restrictive"
│
├─ rules WHERE group_id = "cron_restrictive" ← evaluated first
└─ rules WHERE group_id = "default" ← fallback
```
**Default behavior:** If a session has no `run_context` or the blob has no `security_group`, only `"default"` group rules apply.
The `"default"` group is seeded automatically at startup and **cannot be deleted**. Its rules can be freely edited.
See [approval/index.md](../approval/index.md) for rule evaluation and pattern matching.
---
## Filesystem fast-paths and gate precedence
`RunContext` pre-authorizes filesystem access **without a human approval prompt**, but it
**never overrides an explicit `Deny`**. The gate in `llm_loop.rs` evaluates in this order:
```
1. ApprovalManager::check() → Allow | Deny | Require
2. if Require and is_file_read_tool → rc.is_read_allowed(path) ? Allow : Require
if Require and is_file_write_tool → rc.is_write_allowed(path) ? Allow : Require
```
So a `Deny` (e.g. the seeded `secrets/` rule) always wins, and the fast-path only relaxes
a `Require` to `Allow` — the same semantics as a session bypass. When the session has no
`RunContext`, a default one is used (working dir = process cwd), so `docs/`/`skills/` and the
cwd are still auto-readable.
### Writes — `allow_fs_writes`
Pre-authorizes writes to the listed paths. Resolved against the working dir; matching is a
recursive directory prefix (canonicalized — no globs). `memory/` is always writable via a
separate hardcoded exception in the approval engine.
**Common presets:** `["data"]` (project output), `["logs", "tmp"]` (temporary files).
### Reads — auto-allow roots
Reads are auto-allowed (no prompt) for, in order:
1. the **working directory** itself,
2. its **`docs/`** and **`skills/`** subtrees (always-safe baseline),
3. every **`allow_fs_reads`** entry (read-only grants),
4. everything in **`allow_fs_writes`** (write implies read).
Anything else falls back to the approval rules of the `security_group`. Under a
`require`-default group this means reads outside these roots prompt for approval.
The built-in **`default`** group is itself require-by-default (its final catch-all is
`require *`; see [approval/index.md](../approval/index.md)), so these auto-allow roots are
load-bearing even for the default group — not just for explicitly-restrictive custom groups.
> **`secrets/` is denied.** The approval engine seeds `deny` rules for the read tools on
> `secrets` / `secrets/*` (see [approval/index.md](../approval/index.md)). Because `Deny`
> is evaluated first and is non-bypassable, `secrets/` stays unreadable even though it sits
> inside the auto-read working dir. The recursive read tools (`grep_files`, `list_files`)
> additionally skip any `secrets` directory during traversal, so a search rooted higher up
> cannot leak secret values.
---
## Project Integration
Projects can set default `RunContext` for all interactive and ticket chats under that project:
- **Project-level:** `POST /api/projects/{id}/run_context` → stored in `projects.run_context`
- **Ticket override:** `POST /api/projects/{project_id}/tickets/{ticket_id}/run_context` → stored in `project_tickets.run_context`
Ticket override **takes precedence** over project-level when a ticket chat is opened.
See [projects.md](../projects.md) for project lifecycle and `build_runtime_run_context`.
---
## Example Scenarios
### Scenario 1: Cron job with restricted permissions
**Config:**
```json
{
"security_group": "cron_restrictive",
"allow_fs_writes": ["logs/*"],
"working_directory": "/tmp"
}
```
**Effect:**
- All tool calls evaluated against `cron_restrictive` group rules (typically more restrictive)
- File writes to `logs/*` bypass approval entirely
- File operations use `/tmp` as working directory
### Scenario 2: Project ticket with context injection
**Config:**
```json
{
"system_prompt": ["You are fixing ticket #42: DB migration failure. Current logs are in `logs/migration.log`."],
"working_directory": "/projects/skald",
"allow_fs_writes": ["data/migrations/", "logs/*"]
}
```
**Effect:**
- Extra context prepended to system prompt every turn
- Ticket-scoped working directory for `execute_cmd`
- Migration files and logs can be written without approval
### Scenario 3: Interactive session (default)
**Config:**
```json
{
"security_group": null,
"system_prompt": [],
"allow_fs_writes": [],
"working_directory": null
}
```
**Effect:**
- Default approval group rules apply
- No extra system prompt
- All file writes require approval (if rule triggers)
- Reads of the cwd, `docs/` and `skills/` are auto-allowed; `secrets/` is denied; other reads follow the default group rules
- Uses Skald's process CWD
---
## When to Update This File
- Adding a new field to `RunContext`
- Changing resolution order or approval gate behavior
- Adding new pre-authorization scenarios
- Documenting config best practices
docs/skills.md
+45
View File
@@ -0,0 +1,45 @@
# Skills System
Skills are reusable capability packages that extend what the agent can do without modifying the core source code.
## Structure
```
skills/
index.md ← registry of all available skills
<skill-name>/
SKILL.md ← documentation: purpose, usage, script API
<script>.py ← one or more Python scripts
```
## How the agent uses skills
1. `skills/index.md` is injected into the agent's system prompt automatically as a
`<skills_index>` block, so every agent discovers available skills without reading it
explicitly. Controlled by the `inject_skills` meta.json flag (default `true`; see
[agents.md](agents.md)) — set it to `false` for background agents that don't need skills.
The block is skipped silently when no skills are installed.
2. It reads the relevant `SKILL.md` to understand how to invoke the script.
3. It runs the script via a shell command (e.g. `python3 skills/pdf/scripts/...`).
4. It uses the script's stdout as the result.
> **Path note:** the injected `<skills_index>` shows the index path relative to the session's
> working directory when it lives under it, absolute otherwise. In a project chat (working
> directory = project root) it shows as absolute, since skills live under Skald's own cwd.
> Invoking a skill from a project session still needs care — `execute_cmd` runs with the project
> working directory, so scripts referenced cwd-relative (`python3 skills/...`) won't resolve; use
> an absolute path or `cd` to Skald's root.
## Adding a skill
1. Create `skills/<name>/SKILL.md` — document purpose, required inputs, expected output, and example invocation.
2. Add the Python script(s) alongside it.
3. Register the skill in `skills/index.md` by adding a row to the table.
No code changes or restarts are required — the agent discovers skills at runtime by reading the index.
## Conventions
- Scripts must be runnable with `python3` and accept arguments from the command line.
- Scripts should write their result to stdout and errors to stderr, exiting with code `0` on success.
- Keep each script focused on one task. Compose multiple skills via the agent, not within a single script.
docs/tools.md
+355
View File
@@ -0,0 +1,355 @@
# Tools
## Tool Trait
```rust
pub trait Tool: Send + Sync {
fn name(&self) -> &str;
fn description(&self) -> &str;
fn parameters_schema(&self) -> Value; // JSON Schema object
fn describe(&self, args, length) -> String { name } // default impl — UI/notification label
fn target_path(&self, _args: &Value) -> Option<String> { None } // default impl — file this call opens, if any
fn execute(&self, _args: Value) -> Result<String> { /* default: Err */ }
fn execute_async<'a>(&'a self, args: Value) -> Pin<Box<dyn Future<Output = Result<String>> + Send + 'a>>;
fn run<'a>(&'a self, args: Value) -> Box<dyn ToolExecution + 'a> { /* default: SimpleExecution(execute_async) */ }
fn category(&self) -> ToolCategory; // access-control grouping
fn sub_agents_only(&self) -> bool { false } // default impl — visible only to sub-agents (depth > 0)
fn root_agent_only(&self) -> bool { false } // default impl — visible only to root agent (depth == 0)
fn openai_definition(&self) -> Value { ... } // default impl, rarely overridden
}
```
`Tool` is the **definition** (the catalogue entry in `ToolRegistry`); a single
live invocation is a `ToolExecution` produced by `run()`. See
[Tool execution lifecycle](#tool-execution-lifecycle) below.
**Two execution paths:**
- **Sync tools** implement `execute(&self, args)` only. The default `execute_async` wraps it in a ready future — no changes needed.
- **Async tools** (e.g. `image_generate`, `image_generate_providers_list`) implement `execute_async` directly and omit `execute`. Do NOT use `block_in_place` — override `execute_async` instead.
The dispatcher in `llm_loop.rs` drives every tool through `Tool::run(args) → ToolExecution`, so sync and async tools are dispatched uniformly (and cancellably).
---
## Tool execution lifecycle
`Tool` (definition) and `ToolExecution` (a single in-flight invocation) are split
on the `Command → spawn() → Child` pattern. `Tool::run(args)` starts one
execution and returns a handle that owns its state and implements its own stop.
Defined in [`crates/core-api/src/tool.rs`](../crates/core-api/src/tool.rs).
```rust
pub trait ToolExecution: Send + Sync {
fn state(&self) -> ToolExecutionState;
fn wait<'a>(&'a self) -> Pin<Box<dyn Future<Output = ExecutionOutcome> + Send + 'a>>;
fn stop<'a>(&'a self) -> Pin<Box<dyn Future<Output = ()> + Send + 'a>> { /* default: no-op */ }
}
```
**States** (`ToolExecutionState`, richer than the persisted status string):
| State | Meaning | DB `status` |
| --- | --- | --- |
| `Pending` | created, not yet started (transient, not persisted alone) | `running` |
| `AwaitingApproval` | blocked on a human approve/clarification | `pending` |
| `Running` | actively executing | `running` |
| `Completed` | finished OK | `done` |
| `Failed` | tool/runtime error | `failed` |
| `Cancelled` | stopped by `/stop` — **not** an error | `cancelled` |
| `Rejected` | denied by policy/human — **not** an error | `rejected` |
`Cancelled` and `Rejected` are deliberately distinct from `Failed` so a stop or a
policy denial never pollutes error metrics. `wait()` only ever returns the three
terminal `ExecutionOutcome`s (`Completed` / `Failed` / `Cancelled`); the
approval-phase states are owned by the session driver.
**Purity.** A `ToolExecution` never touches the DB or the WebSocket. The session
driver (`ChatSessionHandler`) mirrors its state to `chat_llm_tools` and emits the
`ToolStart`/`ToolDone`/`ToolError`/`ToolCancelled`/`ToolRejected` events.
**`SimpleExecution`** is the default handle: a work future + a stop-token. `wait`
races the two, so `stop()` (or the driver dropping `wait`) drops the work future,
aborting the in-flight I/O — enough to make `/stop` responsive for **every**
I/O-bound tool with zero per-tool code (this includes `execute_cmd`, whose
`kill_on_drop(true)` child dies when the future is dropped).
**`drive_execution(exec, cancel_token)`** is the generic driver: it runs `wait`
and, when the turn's `/stop` token fires, calls `exec.stop()` once. Used by both
the live loop (`llm_loop.rs`) and `resume_pending_tools` (`resume.rs`).
**Bespoke `stop()` (extension point).** Tools that must tear down *remote* work
override `run()` to return their own `ToolExecution` whose `stop()` does more than
drop the future — e.g. a ComfyUI image tool POSTing `/interrupt` so the server
stops generating too. Dropping the future already frees the client; a bespoke
`stop()` propagates the cancellation to the far side. (Not yet wired for any
built-in tool — the default covers responsive `/stop` today.)
**Rehydration = re-run from intent.** A persisted tool call is `(name, args, status)`;
the live future is never serialized. `resume_pending_tools` reconstructs an
execution via `build_execution(name, args)` and re-runs it from the start — it
does not resume a checkpoint.
**`sub_agents_only`**: if a tool returns `true`, it is excluded from the root agent's tool list and only added to sub-agent configs (depth ≥ 1) in `dispatch_sub_agent`. Default is `false`.
**`root_agent_only`**: if a tool returns `true`, it is included in the root agent's tool list but filtered out from sub-agent configs in `AgentRunConfig::for_sub_agent()`. Default is `false`.
Both flags are mutually exclusive — a tool should never return `true` for both. If it does, it will be invisible to all agents.
---
## ToolCategory
Every tool declares a `ToolCategory`, used for access-control filtering and audit:
| Variant | Used by |
| --- | --- |
| `Filesystem` | File read/write tools (`read_file`, `write_file`, `edit_file`, …) |
| `Shell` | `execute_cmd`, `restart` |
| `Subagent` | `execute_task` / `execute_subtask` (synthetic — not in registry) |
| `Introspection` | `list_items`, `image_generate_providers_list` |
| `Config` | `register_mcp`, `delete_mcp`, `toggle_item`, `execute_task` (InterfaceTool, interactive only), `delete_cron_job`, `configure_plugin`, `image_generate`, `set_secret`, `list_secrets` |
---
## ToolRegistry
`HashMap<String, Arc<dyn Tool>>` with four public methods:
| Method | Purpose |
| --- | --- |
| `register(tool)` | Insert tool keyed by `tool.name()` |
| `openai_definitions()` | Returns definitions for root-agent tools (excludes `sub_agents_only`) |
| `openai_definitions_sub_agents_only()` | Returns definitions for tools where `sub_agents_only() == true` |
| `root_agent_only_names()` | Returns names of all tools where `root_agent_only() == true` — used by `for_sub_agent()` to filter |
| `list_all()` | Returns `(name, description)` for all registered tools (sorted) |
| `category_of(name)` | Returns `Option<ToolCategory>` for a registered tool; `None` for MCP/interface/unknown tools |
| `dispatch(name, args)` | Executes tool by name to a `Result<String>`; errors on unknown name (used by the REST resolve endpoint) |
| `run(name, args)` | Starts a `ToolExecution` for a registered tool; `None` for unknown names (MCP/interface handled by the caller). The cancellable dispatch path. |
| `describe_call(name, args, length)` | Returns a human-readable label for any tool call (including non-registry tools). Falls back to `name` for unknown tools. |
---
## ToolCatalog
`ToolCatalog` (`src/core/tool_catalog.rs`) is a unified façade wrapping `ToolRegistry` + `McpManager`:
| Method | Purpose |
| --- | --- |
| `list_all() -> AllTools` | Returns all built-in tools (registry), a small **static list** of core tools injected outside the registry (`synthetic_tools()`: `execute_task`, `execute_subtask`, `update_scratchpad`, `ask_user_clarification`, `write_todos`, `activate_tools`, `notify`, `show_file_to_user`, `image_generate`), and MCP tools as a single `AllTools { built_in, mcp }` struct. Used by `GET /api/approval/tools`. |
| `describe_call(name, args, length) -> String` | Pass-through to `ToolRegistry::describe_call()`. |
`AllTools` and `ToolInfo` are `#[derive(Serialize)]` — the frontend handler can return `Json<AllTools>` directly.
### Dynamically-injected tools & discovery
Many tools reach the LLM **outside** the `ToolRegistry` — `InterfaceTool` closures
(`write_todos`, `notify`, `show_file_to_user`, `activate_tools`), plugin tools
(Telegram `send_voice_message`/`send_attachment`, honcho `memory_*`) and provider
tools (`image_generate`). The `synthetic_tools()` static list above eagerly
surfaces the *core-owned* ones so they can be pre-configured in the Security-groups
UI before first use; plugin/provider names are intentionally left out so core
stays decoupled from them.
To cover the rest without a hand-maintained mirror, [`ToolDiscovery`](../src/core/tool_discovery.rs)
taps the single point where the tool array is assembled — `AgentRunConfig::all_tool_defs()`,
observed each round from `llm_loop.rs` — and upserts every offered tool into the
`known_tools` table (in-memory seen-set guard → DB write only for new names, off
the turn's critical path). `GET /api/approval/tools` merges `known_tools` into the
response so any tool that has been offered at least once becomes gate-able. This
**cannot drift** from what is really offered and needs no per-tool/per-plugin
wiring. See [approval docs](approval/index.md#alltools-response-get-apiapprovaltools).
---
## Tool Name Constants
All system tool names are centralised in `src/core/tools/tool_names.rs` as `pub const` strings. Import with `use crate::tools::tool_names as tn;`.
| Constant | Value |
| --- | --- |
| `tn::EXECUTE_TASK` | `"execute_task"` |
| `tn::EXECUTE_SUBTASK` | `"execute_subtask"` |
| `tn::RESTART` | `"restart"` |
| `tn::UPDATE_SCRATCHPAD` | `"update_scratchpad"` |
| `tn::WRITE_TODOS` | `"write_todos"` |
| `tn::ASK_USER_CLARIFICATION` | `"ask_user_clarification"` |
| `tn::ACTIVATE_TOOLS` | `"activate_tools"` |
| `tn::NOTIFY` | `"notify"` |
| `tn::READ_NOTIFICATION` | `"read_notification"` |
| `tn::EXECUTE_CMD` | `"execute_cmd"` |
| `tn::SHOW_FILE_TO_USER` | `"show_file_to_user"` |
**Rule:** never hardcode these strings in new code — always use the constants. This ensures that a rename is a single-file change and that typos produce a compile error rather than a silent dispatch miss.
---
## Registration Pattern
All tools are registered in `src/main.rs` before `ChatSessionManager` is built.
**Not in ToolRegistry — synthetic tools intercepted in `run_agent_turn`:**
- `execute_task` — interactive-session sub-agent/task launcher (modes: `cron`, `sync`=inline sub-agent, `async`=background). Intercepted in `run_agent_turn` only for `mode=sync`; `cron`/`async` go through the normal cancellable path as InterfaceTools
- `execute_subtask` — background-session sub-agent launcher (sync-only). Injected as an InterfaceTool in cron/async sessions in place of `execute_task`
- `update_scratchpad` — writes to `session_scratchpad` table; **shared** blackboard injected into every agent in the session; available to all agents
- `write_todos` — **stateless** private task list (TodoWrite-style: the agent re-sends the whole list with statuses on every call); available to all agents. Unlike the scratchpad it is **not** persisted and **not** shared: the formatted checklist lives only in the calling agent's own tool-result history, which is per-stack, so sub-agents and the caller never see it. Handled by `dispatch_write_todos` (`agent_dispatch.rs`); no DB table involved
- `ask_user_clarification` — pauses and asks the user a question; available to every agent except hidden `system` agents (e.g. TIC). Routing depends on session type:
- **Interactive sessions** (web, Telegram): available at any depth (root `chat` agent included); emits `ServerEvent::AgentQuestion` inline (and registers with `ClarificationManager`), so the user can answer either from the chat or the Agent Inbox
- **Background sessions** (cron): available at any depth; registers with `ClarificationManager` only, visible in Agent Inbox; agent suspends until answered
- `activate_tools` — activates **tool groups** on demand (lazy loading): MCP server names and/or the reserved `config` group (the built-in `Config`-category tools). Injected as an `InterfaceTool` in `build_agent_config` (root, session-scoped) and in `dispatch_sub_agent` (sub-agents, stack-scoped). See [Lazy `config` tool group](#lazy-config-tool-group)
- `notify` — queues a structured notification (`Notification`: `{source, event_type, summary, event_time, refs}`) to the home conversation via `ChatHub`; **injected as an `InterfaceTool` by the caller** (`TicManager` for TIC, `TaskManager` for background task agents); not in ToolRegistry so ordinary agents cannot call it
- `show_file_to_user` — opens a file in the user's UI. Emits `ServerEvent::OpenFile`, which the SPA routes to the [file viewer](frontend.md#file-viewer) (every kind, HTML included). Supported formats in the file viewer: Markdown, source code, plain text, raster images (PNG/JPG/GIF/WebP/…), SVG, PDF, LaTeX (`.tex` / `.latex` — compiled to PDF automatically on the server), and HTML (`.html` / `.htm` — rendered live in an origin-isolated `<iframe>`, toggleable to source). **For a LaTeX document the agent should pass the `.tex` source, not a pre-built `.pdf`**: only the `.tex` path triggers server-side compilation, dependency-aware caching, and dependency watching (see [LaTeX compile & cache](frontend.md#latex-compile--cache)). A raw `.pdf` is served statically — never recompiled, dependencies not watched — so editing a source fragment leaves the rendered `.pdf` stale. The tool description enforces this guidance. **Injected as an `InterfaceTool` only for SPA clients** in `ws.rs` (sources `web` + `mobile`); the Telegram plugin goes through a separate handler and never receives it — its analogue is `send_attachment`. Built by `tools::show_file::make_tool(hub, source)`; the path emitted to the frontend is normalised by `fs::relativize_for_display` (relative to the project root when inside it, absolute otherwise)
**Also not in ToolRegistry:**
- MCP tools — injected dynamically per-request via `McpManager::tools()`
---
## Tool Visibility Filtering (permission groups)
Tools are filtered out of the LLM's tool list when the effective approval rule for the session's **permission group** marks them `Deny`. The group comes from the session's run context (or the built-in `"default"` group). This replaces the removed per-agent `allow_tools` whitelist — see [approval/index.md](approval/index.md).
**MCP tools are never filtered here** — they pass through regardless of the group. The Approval gate governs MCP tool execution.
Filtering happens in `src/core/session/handler/config.rs` (depth 0) and `agent_dispatch.rs` (sub-agents) after assembling `base_tool_defs` (registry + synthetic tools), before extending with MCP tools. The same interactive-only and approval-visibility filters are applied to `config_tool_defs` (the lazy `config` group), so activating `config` never bypasses a group's `Deny` rules.
---
## Built-in Tool Catalogue
| Tool name | Module | Category | Approval | Sub-agents only |
| --- | --- | --- | --- | --- |
| `list_files` | `tools::fs` | Filesystem | No | No |
| `read_file` | `tools::fs` | Filesystem | No | No |
| `write_file` | `tools::fs` | Filesystem | Yes (non-memory/) | No |
| `edit_file` | `tools::fs` | Filesystem | Yes (non-memory/) | No |
| `insert_at_line` | `tools::fs` | Filesystem | Yes (non-memory/) | No |
| `replace_lines` | `tools::fs` | Filesystem | Yes (non-memory/) | No |
| `search_file` | `tools::fs` | Filesystem | No | No |
| `grep_files` | `tools::fs` | Filesystem | No | No |
| `get_ast_outline` | `tools::ast_outline` | Filesystem | No | No |
| `execute_cmd` | `tools::exec` | Shell | **Always** | No |
| `restart` | `tools::restart` | Shell | **Always** | No |
| `list_items` | `tools::list_items` | Introspection | No | Merged listing for `type` ∈ {mcp, plugins, cron, agents}. For `agents`, each entry carries `id`, `name`, `description`, optional `instructions` (how to call the agent well, present only when set in `meta.json`), and optional `client`. |
| `register_mcp` | `tools::register_mcp` | Config | No | No |
| `delete_mcp` | `tools::register_mcp` | Config | No | Permanently unregisters + disconnects an MCP server (destructive; kept separate from `toggle_item` like `delete_cron_job`) |
| `toggle_item` | `tools::toggle_item` | Config | No | Merged enable/disable for `kind` ∈ {mcp, plugin, cron} |
| `execute_task` | InterfaceTool (not in registry) | Config | No | Interactive sessions only; `session_id` and `run_context_id` captured in closure at tool-build time; tasks inherit the parent RunContext |
| `execute_subtask` | InterfaceTool (injected in run_job) | — | No | Background sessions only (sync sub-tasks); inherits `run_context_id` from the parent job |
| `read_agent_result` | synthetic | — | No | Interactive only; always returns not_ready; real delivery is async synthetic message |
| `delete_cron_job` | `tools::cron_jobs` | Config | No | No |
| `configure_plugin` | `tools::configure_plugin` | Config | No | No |
| `set_secret` | `tools::set_secret` | Config | No | No |
| `list_secrets` | `tools::list_secrets` | Config | No | No |
| `read_notification` | `tools::read_notification` | Introspection | No | Root only (depth == 0) |
| `image_generate_providers_list` | `tools::image_generate` | Introspection | No | No |
| `image_generate` | `tools::image_generate` | Config | No | No |
| `update_scratchpad` | synthetic | — | No | No |
| `write_todos` | synthetic (stateless) | — | No | No — private per-stack list; not shared with sub-agents or caller |
| `ask_user_clarification` | synthetic | — | No | No — all agents except `system` (TIC) |
| `activate_tools` | synthetic (per-session) | Config | No | No |
---
### Lazy `config` tool group
The built-in **`Config`-category** registry tools are **not** part of the always-on tool set. Like MCP tools, they are lazy-loaded on demand: the LLM calls `activate_tools(["config"])` to bring them into context from the next round onward.
- **Affected tools** (registered in `ToolRegistry` with `ToolCategory::Config`): `set_secret`, `list_secrets`, `register_mcp`, `delete_mcp`, `configure_plugin`, `delete_cron_job`, `toggle_item`.
- **Mechanism**: `build_agent_config` builds the base set with `ToolRegistry::openai_definitions_excluding_config()` and carries the config defs separately as `AgentRunConfig.config_tool_defs` (from `openai_definitions_config_only()`). `all_tool_defs()` appends them only when `active_mcp_grants` contains the reserved `"config"` string. The grant persists in `session_mcp_grants` / `stack_mcp_grants` exactly like an MCP server name. See [mcp.md → Config tool group](mcp.md#config-tool-group).
- **Not affected**: `image_generate` / `image_generate_providers_list` come from the image-generator manager (not the `ToolRegistry`), so despite `image_generate`'s `Config` category it stays always-on. `list_items` is `Introspection`, so listing MCP/plugins/cron stays available without activating `config`; only the mutating `toggle_item` is gated.
- **Discoverability**: a short static hint lives in `agents/main/AGENT.md` and `agents/project-coordinator/AGENT.md`. The `<!-- MCP_LIST -->` block stays MCP-only.
---
### Key Parameter Notes (recent additions)
| Tool | New parameters | Notes |
| --- | --- | --- |
| `execute_cmd` | `workdir` (absolute path), `timeout` (1–600 s, default 120) | Output truncated at 100 KB. Description tells LLM to use dedicated tools (`read_file`, `grep_files`, etc.) instead of shell equivalents. **Audit:** every command is logged at `info` (`execute_cmd: running shell command`, fields `command`/`workdir`/`timeout_secs`) before running, so auto-approved commands (approval bypass active) are still traceable. |
| `edit_file` | `replace_all` (bool, default false) | Replaces every occurrence when true; otherwise requires unique match. Description tells LLM to use instead of `sed`/`awk`. |
| `grep_files` | `output_mode` (`content`/`files_only`/`count`), `context_lines` (0–10), `offset` (pagination) | Description tells LLM to use instead of `grep`/`rg`. Result paths are relative to the queried directory (stripped of the resolved root), consistent with `list_files`. |
| `get_ast_outline` | `path` | Returns top-level definitions (functions, classes, structs, methods) without bodies. **tree-sitter 0.26** backend for: `.py .js .mjs .ts .tsx .go .java .c .h .cpp .cc .hpp .swift .lua .rb .sh .ex .exs .json .yaml .yml .html .css`. **syn** backend for `.rs`. Text/regex fallback for `.kt .toml .md .sql` (grammar crates incompatible with tree-sitter 0.26 at time of writing). |
---
## Tool Display Labels
Every `Tool` implementation can override `describe(&self, args: &Value, length: ToolDescriptionLength) -> String` to produce a compact human-readable label shown in the UI and on Telegram instead of the raw tool name.
| Length | Max chars | Example |
| --- | --- | --- |
| `Short` | 60 | `execute_cmd \`git\`` |
| `Full` | 120 | `execute_cmd \`git commit -m "feat: ..."\`` |
Constants `MAX_LABEL_SHORT` and `MAX_LABEL_FULL` are defined in `src/core/tools/mod.rs`. `truncate_label(s, max)` truncates at char boundary appending `…`.
The default implementation returns `self.name()`, so all tools work without implementing `describe`. Built-in tools (fs, exec) have explicit implementations; MCP and plugin tools fall back to the tool name.
`ToolRegistry::describe_call(name, args, length)` is the single call-site used by `llm_loop.rs`, `resume.rs`, and the `/api/{source}/messages` history endpoint. It also handles synthetic tools (`call_agent`) that are not in the registry.
Labels are emitted in `ServerEvent::ToolStart` as `label_short` and `label_full` and included in history responses so the frontend always has them.
## Clickable Target Path
A tool can override `target_path(&self, args: &Value) -> Option<String>` to advertise a **single viewable file** the call acts on. The single-file fs tools (`read_file`, `write_file`, `edit_file`, `insert_at_line`, `replace_lines`, `search_file`) return their `path` argument via the `fs::path_arg` helper; directory tools (`list_files`, `grep_files`) and every other tool return `None` (the default).
`ToolRegistry::target_path(name, args)` is the registry-level accessor, mirroring `describe_call`. Its result is emitted in `ServerEvent::ToolStart` as the optional `path` field (omitted when `None`) and included in `/api/{source}/messages` history items. The frontend renders that path as a link that opens the [file viewer](frontend.md#file-viewer) via `window.openFile(path)`.
`show_file_to_user` is an `InterfaceTool` (not in the registry, so it has no `Tool::target_path`). Because its whole purpose is to open a file, both `describe_call` and `target_path` special-case it **inline**: the label becomes `` show_file_to_user `path` `` and the same raw `path` arg is returned, so the frontend's `renderLabel` makes the path in the label clickable with no extra wiring.
The sub-agent delegation tools — `execute_task`, `execute_subtask`, and the legacy `run_subtask` alias — are likewise `InterfaceTool`s outside the registry, so they have no `Tool::describe`. `describe_call` special-cases them **inline** via `describe_sub_agent_call`, building a label from the call's `agent_id` + `description` (falling back to `title`, then to the bare name) so the UI/Telegram shows what is being delegated instead of the raw tool name. For `execute_task` the `mode` is shown as a single compact emoji appended to the tool name: sync → ⚡, async → 🚀, cron → 📅. Example: `execute_task ⚡ → software-engineer: Refactor auth module`.
---
## FS Path Resolution
`tools::fs::resolve(path)`:
- If path starts with `/` → used as absolute path
- Otherwise → resolved relative to CWD (project root when running via `run.sh`)
Paths starting with `memory/` bypass the approval gate for write tools.
### Security-aware canonicalization
`tools::fs::canonicalize_for_policy(path, base)` resolves a path to its canonical absolute
form (resolving `..` and symlinks of the longest existing ancestor) for security
prefix-matching. It is shared by the RunContext fast-paths (`is_write_allowed` /
`is_read_allowed`) and `approval::normalize_path`, so traversal/symlink tricks like
`docs/../secrets/x` cannot evade an allow grant or a deny rule. `path_under(child, base)`
does the component-wise prefix test.
### Read auto-allow & `secrets/` deny
Read tools (`read_file`, `grep_files`, `list_files`, `search_file`, `get_ast_outline`) are
auto-allowed without a prompt when the path is under the working dir, `docs/`, `skills/`,
`allow_fs_reads`, or `allow_fs_writes` (the RunContext read fast-path). The `secrets/`
directory is denied for these tools via seeded `deny` rules; the recursive ones
(`grep_files`, `list_files`) additionally list `secrets` in their `SKIP_DIRS` so a search
rooted higher up never descends into it. See [approval/index.md](approval/index.md) and
[session/run-context.md](session/run-context.md).
---
## Adding a Tool
1. Create a struct in `src/core/tools/` (new file or existing module).
2. `impl Tool` for the struct — include `fn category()`.
3. Register in `src/main.rs`: `tool_registry.register(MyTool::new(...))`.
4. If the tool should only be visible to certain agent depths, implement `sub_agents_only()` or `root_agent_only()` instead of using `InterfaceTool` injection.
5. If the tool needs `ChatHub`, a per-session resource, or should only be visible to specific callers, do **not** add it to `ToolRegistry` — implement it as an `InterfaceTool` and inject it at the call site (see `tools::notify::make_tool`).
6. If the tool needs user approval before executing, add an `approval_rules` row (or let the admin add one). The approval gate (`ApprovalManager::check`) is rule-driven — no code change required unless the default-open policy is not suitable.
7. Update this doc (catalogue table).
---
## When to Update This File
- A tool is added, removed, or renamed
- The approval rules for a tool change
- The `Tool` trait gains or loses a method
- `ToolCategory` gains a new variant
- The tool visibility (permission-group) filtering logic changes