Nightly Build / build (push) Successful in 9s
The routing table said "before you touch one of these areas, open its file", and the closing line explained why a pointer is not a summary. Neither survives contact with a change that looks trivial: the diagnosis feels complete after a grep, the edit is one line, and the file never gets opened. That is how the narrow-page bug in the previous commit was nearly shipped as a one-line addition to the very enumeration that was the defect. State the missing half. "The fix is obvious" is what triggers the rule, not what excuses you from it, because a dev-doc is not a description of the code — it is the rules and traps the code cannot state about itself, and grepping the source finds what the code does, never what you must not do to it. Add the two consequences that make it cheap to comply: the same-change update rule means the file has to be opened regardless, so opening it first is free and is the only moment it can still change what gets built; and the file is read whole, since the paragraph that saves you is not the one matching the grep. Give the write-side standing rule its read half explicitly, where it was only ever phrased as an obligation to type into the file.
240 lines
34 KiB
Markdown
240 lines
34 KiB
Markdown
|
|
# Skald (project-family) — codebase guide
|
|
|
|
Rust async web app (Tokio + Axum). Runs as a local chat server with LLM tool-calling and a sub-agent system.
|
|
|
|
> **Never `git commit` unless explicitly asked.** Staging, building, running and testing are fine on your own initiative; creating a commit is not. Do the work, leave it in the working tree, and let the user commit — or ask them to — even when a commit looks like the obvious next step.
|
|
>
|
|
> **Commit messages must be in English.**
|
|
|
|
## How this documentation is organized
|
|
|
|
Four places. **Only this file is loaded into your context automatically** — the rest you open on demand.
|
|
|
|
- **`CLAUDE.md`** (this file) — the rules whose blast radius is the whole repo (the commit rule, the production/schema constraint, domain neutrality, the event-bus rule, the crate boundaries), plus the map of the code. Keep it that way: the mechanism of one subsystem does not belong here.
|
|
- **`dev-docs/*.md`** — one subsystem each: how it works, and which traps have already been paid for. Indexed in [`dev-docs/README.md`](dev-docs/README.md). **Standing rule: a change to a subsystem updates its dev-doc in the same change** — same reason as `docs/` and `CHANGELOG.md`, see [Documentation](#documentation).
|
|
- **`blueprint/project-family.md`** — the design document and source of truth, referenced by section number (§0.1 neutrality, §2 threat model, §4/§5.1 crypto + database layout, §6 filesystem, §7 MCP, §9 unlock, §11 `UserManager`, §12 auth schema, §13 reports, §14/§15 connectors, §16 LLM privacy tiers, §17 sequencing, §19). **Gitignored and not under version control.** Read it before any architectural work, and never assume a section says what you remember.
|
|
- **`docs/`** — *not* developer documentation: it is written for the in-app LLM and mounted read-only into every user's container. See [Documentation](#documentation).
|
|
|
|
Code that lives outside this repo but that a change here can break is listed under [Sibling repositories](#sibling-repositories).
|
|
|
|
**Before you touch one of these areas, open its file — every time, before the first edit:**
|
|
|
|
| You are touching | Read |
|
|
| ---- | ---- |
|
|
| login, sessions, `UserManager` / `UserContext`, per-user DB encryption, what boot unlocks | [`dev-docs/users-auth-and-boot.md`](dev-docs/users-auth-and-boot.md) |
|
|
| any table or accessor under `db/`, the registry vs owner bucket split, memory notes, reports | [`dev-docs/database.md`](dev-docs/database.md) |
|
|
| `container/`, the fs-tools, mounts, path routing, skills, the memory signposts | [`dev-docs/filesystem-and-containers.md`](dev-docs/filesystem-and-containers.md) |
|
|
| projects, shared folders, `<file-explorer>`, the `#files` page | [`dev-docs/projects-and-files.md`](dev-docs/projects-and-files.md) |
|
|
| `crates/agent-loop/`, `loop_adapters/`, `session/handler/`, sub-agents, cancellation, recovery, the approval gate | [`dev-docs/agent-loop.md`](dev-docs/agent-loop.md) |
|
|
| compaction, the history window, the cached system-prompt prefix | [`dev-docs/context-and-compaction.md`](dev-docs/context-and-compaction.md) |
|
|
| LLM clients, `providers.yaml`, retriability, request logging, token streaming, attachments | [`dev-docs/llm-stack.md`](dev-docs/llm-stack.md) |
|
|
| MCP runtimes, connectors, marketplace installs, OAuth, device/QR login | [`dev-docs/mcp-connectors.md`](dev-docs/mcp-connectors.md) |
|
|
| plugin visibility, per-user plugin config, plugin HTTP routers and web pages | [`dev-docs/plugins.md`](dev-docs/plugins.md) |
|
|
| anything grantable (a plugin, a connector) and who receives it by default | [`dev-docs/default-access.md`](dev-docs/default-access.md) |
|
|
| event triage, the memory lints, the conversation review, their scheduler | [`dev-docs/system-agents.md`](dev-docs/system-agents.md) |
|
|
| anything under `web/` — components, chat tabs, routing, i18n, theme, the security-group picker | [`dev-docs/frontend.md`](dev-docs/frontend.md) |
|
|
|
|
A pointer is not a summary. If the table sends you to a file, that file is where the decision was recorded and why the obvious alternative was rejected — inferring it from this one instead is how a trap already paid for gets stepped on twice.
|
|
|
|
**Reading it is not conditional on the size of the change, and "the fix is obvious" is what triggers the rule, not what excuses you from it.** A one-line CSS edit, a renamed field, a typo in a label — those are exactly the changes made without opening anything, because the diagnosis felt complete after a grep. It wasn't: a `dev-docs` file is not a description of the code, it is the **rules and traps the code cannot state about itself** — invariants whose violation compiles cleanly and fails silently, a helper that must be called synchronously and looks identical to the one that must not, an enumeration that is load-bearing, the alternative that was already tried and reverted. Grepping the source finds *what* the code does; it cannot find *what you must not do to it*. Reconstructing that from the code later means reconstructing it from the one version that cannot explain itself.
|
|
|
|
Two practical consequences:
|
|
|
|
- **You will have to open the file anyway.** The [standing rule](#dev-docs) says a change to a subsystem updates its dev-doc *in the same change*. Opening it first costs nothing extra and is the only moment when what it says can still change what you build; opening it last reduces it to a place to type into.
|
|
- **Read the whole file, not the section you think you need.** They are short by design. The part that saves you is rarely the part matching your grep — it is two paragraphs away, in the trap you did not know existed.
|
|
|
|
The worked example is in [`dev-docs/frontend.md`](dev-docs/frontend.md): the Models → TTS page rendering 45px wide. The cause was not in the page but in a missing rule *about* the page, and the fix was not to add the missing name to a list but to delete the list — because a hand-maintained enumeration of element names fails silently, with no console error and no failed build. A grep found the symptom in three calls and would have shipped the one-line version of the fix.
|
|
|
|
## Sibling repositories
|
|
|
|
Three repositories are checked out **beside** this one, at the same level as its root. They are separate git repos — own history, own `CLAUDE.md`, own release cycle — and are not part of this Cargo workspace:
|
|
|
|
| Path | What it is | It concerns you when |
|
|
| ---- | ---- | ---- |
|
|
| `../marketplace` | The **Skald Connectors Marketplace**: the connector feed and every manifest in it. Its `CONNECTOR_MANIFEST_GUIDE.md` is the **authoritative authoring spec**; this repo deliberately keeps no copy, because two files with one name drift and the one sitting next to the connectors is the one an author actually reads. | you touch the manifest format, the feed schema, or anything `mcp::install` consumes. The spec is edited **there**, never restated here. |
|
|
| `../skald-circle-ios` | The iOS client (Swift): a remote control for an instance — chat, projects, files, approvals — end-to-end encrypted. Pairs through `crates/plugin-mobile-connector`. | you change that plugin's wire protocol, pairing flow or push payloads. |
|
|
| `../skald-circle-android` | The Android client (Kotlin/Gradle), same role as the iOS one. **Early stage** — the repo exists but has no commits yet. | same as above. |
|
|
|
|
**Do not edit them as a side effect of work done here.** The coupling that matters is `plugin-mobile-connector`: a shipped client cannot be recompiled by this repo's build, so a protocol change is a compatibility decision, not a refactor. When a change here breaks one of them, say so and let it get its own commit in its own repo.
|
|
|
|
## What this repository is
|
|
|
|
A **dedicated fork** of Skald, turning a single-user personal agent into a **multi-user assistant for a small trusted group** — positioned at families, but see the neutrality rule below.
|
|
|
|
The design lives in **`blueprint/project-family.md`** (see above) and is the source of truth for everything below.
|
|
|
|
Load-bearing decisions from that document:
|
|
|
|
- **Not upstreamable.** Nothing here needs to preserve Skald's schema or be portable back to it.
|
|
- **~~Greenfield~~ — no longer true. The instance is in production.** There are live users with data we cannot recreate, so the greenfield licence (restructure, rename, wipe, recreate) has expired: **every schema change now needs a versioning mechanism**, and "drop the box and re-run setup" stopped being an acceptable answer. Until that mechanism exists, the only safe change is an additive one through `db::ensure_column` (see [`dev-docs/database.md`](dev-docs/database.md)); anything that renames, drops, retypes or moves a column or table is **blocked** on building schema versioning first, not something to do carefully by hand. A user's `{userid}.db` is SQLCipher-encrypted and readable **only while they are logged in**, so a migration cannot be a boot-time sweep over every file — it has to run per user, at unlock, and be idempotent. Design for that when the time comes.
|
|
- **Dual memory**: a private per-user pool plus a shared pool. A user's private space is encrypted so that nobody else — the admin included — can read it *through normal use of the system*. Never claim "mathematically impossible": the honest promise is transparency plus verifiability (§3).
|
|
- **Threat model** (§2): the adversary is the **tempted admin**, who owns the box but does not recompile the binary or dump RAM. Do not design against a forensic attacker.
|
|
- **Roles are data, not enums** (§0.1): a `roles` table binds permission-group, run-context and data-handling attributes. "Children" is a seeded preset row, never a hardcoded type.
|
|
|
|
### Event-driven coupling — think in events, not calls
|
|
|
|
Three global broadcast buses — **never add a fourth without checking these first**:
|
|
|
|
| Bus | Cap | Events | File |
|
|
|-----|-----|--------|------|
|
|
| `ChatEventBus` | 256 | user message, assistant response, compaction done | `core-api/src/bus.rs` |
|
|
| `SystemEventBus` | 64 | provider (un)registered, config key updated, job completed, session cancelled, **user created/deleted/active-changed/mounts-changed**, **global connectors changed, connector reinstalled**, **report created** | `core-api/src/system_bus.rs` |
|
|
| `GlobalEvent` (per-user) | 512 | all `ServerEvent` variants → WS clients + inbox lifecycle | `core-api/src/events.rs` |
|
|
|
|
Plus internal `mpsc` queues: per-source `SourceInbox` (message serialization) and a central `notify` queue (background agents → user).
|
|
|
|
**The user-lifecycle reconciler** is the worked example of the rule. Creating a user, deleting one, deactivating one, or changing a shared-folder/project membership all need Docker work (provision, tear down, stop, recreate with new bind mounts); enabling or reinstalling a connector needs live runtimes re-snapshotted. None of the endpoints that make those changes touches `ContainerManager` or the refresh helpers: each announces `SystemEvent::User{Created,Deleted,ActiveChanged,MountsChanged}` / `McpGlobalServersChanged` / `ConnectorReinstalled` **after** its DB write, and one subscriber — `skald::wiring::spawn_user_lifecycle`, spawned post-construction because it reacts through `Skald`'s own accessors, holding only a `Weak` — does the reacting, sequentially and best-effort. Being off the response path matters for `ConnectorReinstalled` in particular: it re-copies files and restarts servers inside every live user's container, seconds of work the admin's install no longer waits on. The payoff is that a *future* endpoint granting membership cannot forget to remount, because remounting was never its job. Reactions never block the HTTP response, and a failure settles at the user's next login or at boot reconciliation.
|
|
|
|
**Where the bus stops: reconciliation rides it, authorization does not.** `SystemEventBus` is a lossy 64-slot broadcast whose contract is *"best-effort, settles at the next login"* — right for a stale mount, wrong for a revocation, where "settles later" *is* the failure. So deactivating or deleting a user splits in two: `Skald::revoke_user_runtime` runs **synchronously in the handler, before it responds** (revoke every session → evict + cancel the `UserContext` → `UserManager::lock`, in that order, so nothing is left querying a pool we then close and the DEK leaves RAM per §9), while only the container half — stop or remove — rides the bus. Before this, `active = 0` blocked the *next* login but left live sessions working: `login` checks the flag, `require_auth` only maps token → id. Same split for security groups (see the picker section in [`dev-docs/frontend.md`](dev-docs/frontend.md)) and for connectors, where the test is worth internalising because the call is literally the same function: `Skald::refresh_global_mcp_access` is **announced** (`McpGlobalServersChanged`) when a global connector is enabled or deleted — the first only makes something *appear*, the second is already enforced by `stop_server` — but **called directly** from `global_set_access` and `user_connectors_set`, where `set_access`/`set_for_user` *replace* a grant set and the refresh is what actually revokes. Both sync call-sites carry a `DELIBERATELY SYNCHRONOUS` comment, because they look identical to the announced ones. **Never put an access revocation on a bus.**
|
|
|
|
**Before you add a direct function call or a new import between two components, stop and ask:** is one component producing data another needs? If yes, add a variant to an existing bus and spawn a subscriber. Don't call `some_manager.log_thing(...)` from the producer — emit a `ThingHappened` event on `SystemEventBus` and let the manager subscribe.
|
|
|
|
**A new `mpsc::channel` or `broadcast::channel` is a code-review flag.** Nine times out of ten you want one of the three buses above. If you truly need a new one, be ready to explain why none of the existing three fits.
|
|
|
|
### The core is domain-neutral — this is a hard rule
|
|
|
|
"Family" is **positioning, not architecture**. Schema, engine, API, identifiers **and comments** must never contain `family`, `household`, `parent`, `child` or `minor`. A pivot to teams, small orgs or care settings must not require renaming anything.
|
|
|
|
| Domain concept | Technical primitive |
|
|
| ---- | ---- |
|
|
| the group | **implicit — it is the instance**. No group entity. Future multi-group ⇒ `tenant` / `workspace`, never `family` |
|
|
| shared memory | `memory/shared` |
|
|
| parent / admin | role `admin` |
|
|
| child / minor | a **data-driven role** defined by the admin |
|
|
| "the parent reads the child's data" | a generic **supervision edge** between users |
|
|
|
|
Domain words are allowed only in seed data, preset labels, UI copy and positioning.
|
|
|
|
### Current state
|
|
|
|
`UserManager` (§11) is **consumed**: login exists, the deny-by-default middleware is `src/frontend/api/guard.rs`, the first admin is created by `skald-setup`, and the per-user owner-bound runtime is `UserContext` (`crates/skald-core/src/skald/user_context.rs`) — resolved by `Skald::user_context` / the frontend's `require_context`, carrying its own `CancellationToken` so one user's loops can be stopped without touching anyone else's. Every frontend owner call-site routes through the per-user pool; **boot unlocks the databases that have no key and starts their runtimes**, so an instance works before anyone opens the SPA. The "owner-without-a-user" question resolved to **there isn't one**: every owner content belongs to a logged-in user, the admin included. The global owner-bound bundles (`Conversation`/`Tasks`: the "ownerless" `ChatSessionManager`, `ChatHub`, cron `TaskManager`) are still constructed but **inert** — their loops never spawn and nothing consumes their accessors; removing them is pending follow-on work (kept for now because `RunContextManager` shares the `Conversation` bundle and *is* used, being registry-backed). See blueprint §19, and [`dev-docs/users-auth-and-boot.md`](dev-docs/users-auth-and-boot.md) for why each of those pieces is shaped the way it is — the ordering of revocation, what a pool being open means, and why the auto-unlock is deliberately not on a lazy path.
|
|
|
|
Direction of travel, decided but not yet executed: strip the **power-user surface** (self-rewriting, arbitrary shell, dev-agent suite, ticket system) and move to a **binary-first** layout — the app is built once and run from a compiled binary, not executed from its own source tree.
|
|
|
|
## Workspace layout
|
|
|
|
The application core is the `skald-core` crate; the binaries are **shells** around it.
|
|
|
|
| Crate | Role |
|
|
| ---- | ---- |
|
|
| `crates/skald-core/` | Storage, identity, crypto, LLM stack, tools, MCP, sessions. Knows nothing about what runs it: no HTTP server and **no concrete plugin crate** — `PluginManager` only ever sees `Arc<dyn Plugin>` from `core-api` |
|
|
| `skald` (root, `src/`) | The server shell: `main.rs`, the Axum `frontend/`, `config.rs`. Constructs the plugin list and hands it to `Skald::new`. Runs headless as a background daemon under the `run.sh` supervisor |
|
|
| `crates/skald-setup/` | Guided first-run setup — a terminal shell over `skald-core`. Creates the first admin and seeds the instance through the **shared seam `skald_core::setup::initialize_instance`** (apply the chosen seed profile → `register_user(admin)` → set default locale) — the *same* function the web setup calls, so the two shells can't drift. Asks profile, interface language, whether to encrypt — default yes — and password. A separate binary so the server never links TTY-prompt deps, and so a future GUI installer is a third shell over the same seam. `run.sh` runs it before the server loop; it prompts only when `users` is empty **and** stdin is a terminal, otherwise a no-op. `--check` reports readiness by exit code (0 done, 1 needed) |
|
|
| `crates/core-api/` | The contracts both sides share: `Plugin`, `Tool`, event buses, provider types |
|
|
|
|
Two rules keep the boundary real, and both are enforced by the compiler:
|
|
|
|
- **The core never names a plugin.** A plugin contributes tools through `Plugin::tools(self: Arc<Self>)` — the sibling of `http_router()` — so nothing in the core has to downcast to a concrete type. Naming one would drag every plugin in the tree into the core, including a C build via `plugin-transcribe-whisper-local`.
|
|
- **The core never learns about the process shell.** There is no in-core restart hook — the former `restart` tool and its `tools::restart::set_restart_handler` seam were removed. The only coupling to the supervisor is now the `run.sh` exit-code protocol (exit `255` ⇒ re-exec the same binary by path), a seam no code currently triggers (kept for a future admin-driven restart). The live expression of this principle is `skald_core::boot`, which emits startup lines each shell renders (`src/boot_format.rs` here).
|
|
|
|
`skald_core::boot` emits curated startup lines on the `boot` tracing target; each shell decides how to render them (`src/boot_format.rs` here). The core says what happened, never how it looks.
|
|
|
|
## Key modules
|
|
|
|
| Path | Role |
|
|
| ---- | ---- |
|
|
| `src/main.rs` | Thin entry point: tracing → `Skald::new` → `WebFrontend::start` → shutdown. Builds a tokio runtime and blocks on `async_main`, which runs the backend until a SIGINT/SIGTERM. Exposes `run_backend()` / `shutdown_backend()` |
|
|
| `crates/skald-core/src/skald/` | `Skald` — headless application core. `mod.rs` (struct + staged `new()` / `shutdown()`), `runtime.rs` (cross-cutting `Runtime` context), `bundles.rs` (8 domain bundles + `build()`), `wiring.rs` (`wire()` + `spawn_background()`), `supervisor.rs` (`TaskSupervisor`), `accessors.rs` (per-manager accessor facade — the API surface the frontend uses) |
|
|
| `crates/agent-loop/` | **The LLM loop itself, as a standalone crate**: kernel (round loop, fallback, tool fan-out), `LoopManager`, `HistoryStore`, projection (history→wire), `DelegateTool` (sub-agents), `recovery.rs` (restart), `compaction.rs`, plus the shipped model clients (`models/`). Knows nothing about Skald — [`dev-docs/agent-loop.md`](dev-docs/agent-loop.md) |
|
|
| `crates/skald-core/src/loop_adapters/` | Skald's side of that crate's traits: history store, model selector, approval gate, tool set + bridges, agent catalog, event translator, projection knobs, async executor. This is where "how Skald does it" lives |
|
|
| `crates/skald-core/src/session/handler/` | What is left of the session layer: `mod.rs` (`ChatSessionHandler` + `handle_message`), `kernel_turn.rs` (the three loop entry points), `config.rs`, `interface_tools.rs`, `media.rs` |
|
|
| `crates/skald-core/src/session/manager.rs` | Creates/retrieves `ChatSessionHandler` per session |
|
|
| `crates/skald-core/src/chat_hub/` | `ChatHub`: broadcast events to all connected WS clients |
|
|
| `crates/skald-core/src/chat_event_bus.rs` | Global async bus for cross-session events |
|
|
| `crates/skald-core/src/agents.rs` | Discovers agents from `agents/*/`, loads meta + system prompt |
|
|
| `crates/skald-core/src/tools/` | Built-in tools: `exec` (**runs inside the caller's per-user Docker container**; the context-free `Tool::execute` errors, so nothing can run a command outside the sandbox), `list_agents`, `fs/*` (route `user-memory/`/`shared-memory/` to `memory_docs`, every other **physical** path through `ctx.fs`), `notify`, `ast_outline`, `image_generate`, MCP tools, plugin tools, cron tools — [`dev-docs/filesystem-and-containers.md`](dev-docs/filesystem-and-containers.md) |
|
|
| `crates/skald-core/src/container/` | `ContainerManager` (§6): per-user Docker containers — the execution sandbox. Docker is a **hard requirement**: `check_docker()` fails `Skald::new` (→ shell exits) if the daemon is unreachable. Builds the `skald-runtime` image, then `reconcile_all()` at boot ensures one running container `skald-{userid}` per active user. Shells the `docker` CLI (no client crate) — [`dev-docs/filesystem-and-containers.md`](dev-docs/filesystem-and-containers.md) |
|
|
| `crates/skald-core/src/tool_catalog.rs` | `ToolCatalog`: unified tool listing façade (wraps ToolRegistry + McpManager) |
|
|
| `crates/skald-core/src/events.rs` | `ServerEvent` enum streamed over WebSocket to the frontend |
|
|
| `crates/skald-core/src/db/` | sqlx SQLite: the registry/owner bucket split, the accessors, the memory and report stores — [`dev-docs/database.md`](dev-docs/database.md) |
|
|
| `crates/skald-core/src/users/` | `UserManager` (§11): user directory CRUD on `system.db`, credential check, and the map `userid → SqlitePool` of **unlocked** databases. The pool *is* the unlock token (§9). Knows nothing about cookies — [`dev-docs/users-auth-and-boot.md`](dev-docs/users-auth-and-boot.md) |
|
|
| `crates/skald-core/src/crypto/` | Envelope encryption (§4/§5.1): a random 256-bit DEK encrypts `{userid}.db`, sealed with AES-256-GCM under `Argon2id(password, salt)`; **the AEAD tag is the password verifier** — [`dev-docs/users-auth-and-boot.md`](dev-docs/users-auth-and-boot.md) |
|
|
| `src/config.rs` | Loads `config.yml`; LLM clients, strength, data root. All relative paths (db, logs, data, …) resolve against the launch cwd |
|
|
| `crates/skald-core/src/mcp/` | MCP runtimes + the `McpProvider` seam (§7): the shared host **global** runtime and the per-user **container** runtimes, unioned per session as `UserMcpView` — [`dev-docs/mcp-connectors.md`](dev-docs/mcp-connectors.md) |
|
|
| `crates/skald-core/src/plugin/` | Plugin system: discovery, enable/disable, tool registration, per-user access grants + per-user config — [`dev-docs/plugins.md`](dev-docs/plugins.md) |
|
|
| `crates/skald-core/src/cron/` | Scheduled job runner |
|
|
| `crates/skald-core/src/system_agents/` | The `SystemAgent` trait + `run_and_record` + the shared ephemeral-turn/run-context machinery, plus `registry()` (the one enumeration of the agents) and `memory_lint.rs` (the two lint agents) — [`dev-docs/system-agents.md`](dev-docs/system-agents.md) |
|
|
| `crates/skald-core/src/event_triage/` | `EventTriageManager`: one pass of the event-triage system agent for **one** user. No timer of its own — the instance-wide scheduler is `skald::wiring::spawn_system_agents` |
|
|
| `crates/skald-core/src/compactor.rs` | Context compaction **policy** — when to compact and with which model; the mechanics are `agent_loop::compaction`. Always constructed, because manual `/compact` must work with no config — [`dev-docs/context-and-compaction.md`](dev-docs/context-and-compaction.md) |
|
|
| `crates/skald-core/src/approval/` | Approval rules engine |
|
|
| `crates/skald-core/src/clarification/` | `ClarificationManager`: background-session question/answer |
|
|
| `crates/skald-core/src/elicitation/` | `ElicitationManager` + bridge: MCP server-initiated input (`elicitation/create`), surfaced in the Inbox; secrets never logged/persisted |
|
|
| `crates/skald-core/src/inbox.rs` | `Inbox`: unified façade for pending approvals + clarifications + elicitations (wraps ApprovalManager, ClarificationManager, ElicitationManager). The managers already emit the `*Requested`/`*Resolved` lifecycle events on the per-user bus; `ws.rs` forwards them to every connected client of that user regardless of `source`, so the web UI updates live (see `sidebar.js` row) |
|
|
| `crates/skald-core/src/llm/` | LLM client abstraction (OpenAI-compat, Anthropic, Ollama…). OpenAI-compatible provider *types* are runtime data, not code: `providers/declared.rs` loads `providers.yaml` at boot (see [Config](#config)). Retriability, the `LoggingModel` decorator and request-log ownership — [`dev-docs/llm-stack.md`](dev-docs/llm-stack.md) |
|
|
| `crates/skald-core/src/transcribe/` | Transcription providers |
|
|
| `crates/skald-core/src/image_generate/` | Image generation providers |
|
|
| `crates/skald-core/src/memory/` | Agent memory tools |
|
|
| `crates/skald-core/src/skills/` | The skills index: pure functions over the two read-only trees (enumerate → parse frontmatter → render → digest). No state, no watcher — [`dev-docs/filesystem-and-containers.md`](dev-docs/filesystem-and-containers.md) |
|
|
| `src/frontend/mod.rs` | `WebFrontend`: wires router_factory, starts plugins, runs Axum |
|
|
| `src/frontend/server.rs` | Axum router, static file serving |
|
|
| `src/frontend/api/` | HTTP + WebSocket handlers — `State<Arc<Skald>>` |
|
|
| `web/components/` | Lit web components — [`dev-docs/frontend.md`](dev-docs/frontend.md) |
|
|
|
|
## Build & run
|
|
|
|
```sh
|
|
./build.sh # release build → bin/skald and bin/skald-setup (atomic install)
|
|
./build.sh -d # debug profile; extra args are forwarded to the server build
|
|
./run.sh # first-run setup, then the supervisor loop — never compiles
|
|
```
|
|
|
|
`build.sh` builds and installs **both** binaries; any forwarded args go to the server only.
|
|
|
|
`run.sh` resolves the server binary as `$SKALD_BIN` → `bin/skald` → `target/release/skald`, and warns when sources are newer than it. Before the loop it runs `skald-setup` (found next to the server, or `$SKALD_SETUP_BIN`); a non-zero exit there — a failed or cancelled wizard — stops `run.sh` before the server starts. Server exit `0` stops the loop, `255` re-executes, anything else propagates.
|
|
|
|
> In a **debug** build, Argon2id at 256 MiB is unoptimised and takes far longer than the ~1s of a release build — `skald-setup -d` will feel stuck at the password step. Use the release binary for anything interactive.
|
|
|
|
Tracing filter: `RUST_LOG=skald=debug,info`
|
|
|
|
## Config
|
|
|
|
Copy `default.config.yaml` → `config.yml`. Never commit `config.yml` (contains API keys).
|
|
|
|
`providers.yaml` (repo root, cwd-relative like `config.yml`) declares the **OpenAI-compatible LLM provider types** — endpoints, UI metadata, per-model JSON field mapping, id-glob enrichment rules, reasoning knobs. Loaded at boot by `llm::providers::declared`; edit + restart the process, no rebuild. An invalid entry is logged and skipped, never fatal; an `id` colliding with a native provider is skipped. Adding a new OpenAI-compatible provider is a YAML edit, not a Rust file. The shipped file is validated by a unit test (`declared::tests::shipped_providers_yaml_is_valid`).
|
|
|
|
## Python environment
|
|
|
|
Host-side Python runs from a local virtualenv at `.venv/` in the project root. `run.sh` creates it on first launch (using `uv` if available, otherwise `python3 -m venv`), installs `requirements.txt`, and prepends `.venv/bin` to `PATH` before starting the app, so every child process resolves `python3` to the venv. No manual activation needed.
|
|
|
|
**`requirements.txt` is for the two TTS plugins, and nothing else.** `plugin-tts-kokoro` and `plugin-tts-orpheus-3b` write an embedded server script to disk and spawn a bare `python3` on it — they have no dependency reconciler of their own, so their imports must be satisfied in the venv. The GPU/ML half of Orpheus (torch, transformers, snac, bitsandbytes, huggingface_hub) is split into `requirements-optional.txt`, installed by hand.
|
|
|
|
**A connector's deps never go in `requirements.txt`.** A connector ships its own `requirements.txt`/`package.json` and `mcp::install::ensure_installed` installs it into `.pydeps`/`node_modules` — inside the user's container for a per-user connector, beside the connector's files on the host for a global one (`ensure_installed_host`). Putting them in the root file would install them on every box for a connector nobody activated; this is what the file used to do for the since-deleted `scripts/` MCP servers.
|
|
|
|
**Python is optional**: with neither `uv` nor `python3` present the app starts normally; the TTS plugins fail to start and a host-run global connector has no interpreter to install its deps with. Per-user connectors are unaffected — they run in the container, which ships its own Python.
|
|
|
|
## Adding an agent
|
|
|
|
Create `agents/<id>/meta.json` and `agents/<id>/AGENT.md`. The agent is discovered at runtime (no restart needed for prompt edits). Optionally set `"client": "<name>"` in meta.json to pin a specific LLM.
|
|
|
|
## Restart
|
|
|
|
There is **no in-app restart** anymore. The agent-callable `restart` tool and its `set_restart_handler` seam were removed (blast radius = the whole box: it dropped every user's session and in-RAM DEK from one user's chat — a power-user leftover, out of place in the multi-user model). Nothing in the process now calls `libc::_exit(-1)`.
|
|
|
|
The supervisor protocol survives but is currently **unreachable in-app**: `run.sh` still re-executes the binary *by path* when it exits `255`, but no code produces that exit code. Restarting is therefore a manual/admin operation.
|
|
|
|
To pick up `config.yml` / `providers.yaml` / database changes (read only at startup), or to load new **code** (`./build.sh` installs the new binary via atomic rename): stop the server and let `run.sh` loop, or re-run `./run.sh`. A future admin-only restart action (endpoint/button gated by an admin capability) would re-use the `255 ⇒ re-exec` seam — it is intentionally kept for that.
|
|
|
|
> `run.bat` is still stale (`cargo run`) and must be fixed.
|
|
|
|
## Documentation
|
|
|
|
`docs/` is **not developer documentation** — it's written for the in-app LLM, not for a human reading the repo, and is mounted read-only into every user's container at `~/docs/` (see [`dev-docs/filesystem-and-containers.md`](dev-docs/filesystem-and-containers.md): `docs_host` on `UserFs`, `DOCS_DIR` in `container/mod.rs`). It explains the software's UX (plugins, and eventually agents/connectors/memory/roles/…) in plain terms, in English, so the assistant can help a non-technical user configure things instead of guessing. `docs/index.md` is the entry point (general index of feature pages); `docs/plugins/<plugin id>.md` covers each built-in plugin. The three `type: chat` agents (`assistant`, `kid`, `project-coordinator`) are told in their `AGENT.md` to read `docs/index.md` when a user asks how the software works. **Standing rule: every change that impacts the UX must update `docs/` in the same change** — a new/renamed feature page plus the `docs/index.md` index entry. It goes stale like any other doc, except users actually see this one.
|
|
|
|
### dev-docs
|
|
|
|
`dev-docs/*.md` carries the **third standing rule**, for the same reason as the other two: **a change to a subsystem updates that subsystem's dev-doc in the same change.** These files are the recorded rationale — what was tried, what broke, why the obvious alternative was rejected — and a rationale reconstructed later is reconstructed from the code, which is the one version that cannot explain itself. New subsystem ⇒ new file plus a row in [`dev-docs/README.md`](dev-docs/README.md) *and* in the routing table at the top of this file; if it does not appear in both, nobody will open it.
|
|
|
|
That rule has a **read half, and it is the half that gets skipped**: you do not edit a subsystem you have not read the dev-doc for — see [How this documentation is organized](#how-this-documentation-is-organized). Writing into a file you opened only at the end is bookkeeping; the file earns its cost only when it is read before the first edit.
|
|
|
|
Keep the split honest in the other direction too: a rule a change *anywhere* could violate belongs in `CLAUDE.md`, not in a dev-doc nobody loaded.
|
|
|
|
### The changelog
|
|
|
|
`CHANGELOG.md` (repo root) is the release history, and it carries the **twin standing rule**: every change a user or an operator would notice must add a bullet under `## [Unreleased]` **in the same change** — a feature, a behaviour change, a bug fix, a new config key, an image-tag bump. Same reason as `docs/`: written after the fact it is written from the diff, which is exactly the version nobody can use.
|
|
|
|
Format is [Keep a Changelog](https://keepachangelog.com): newest first, one `## [x.y.z] - YYYY-MM-DD` section per released version, bullets grouped under `Added` / `Changed` / `Fixed` / `Removed` / `Security`. The versions are the **workspace `Cargo.toml` version** — the same string `ci/verify-version.sh` gates a release PR on — so cutting a release is two edits in one commit: bump `version` in `Cargo.toml`, and rename `## [Unreleased]` to the version with today's date, leaving a fresh empty `Unreleased` above it. There are no git tags on this repo; the changelog *is* the record of what a given `v{version}` tarball contains.
|
|
|
|
Entries are written **for the person reading the release, not for the person who wrote the code**: say what changed for them, not which module moved — the commit message and the diff already hold that. Which is also the test for whether a bullet is owed at all: a refactor with no observable effect gets none, however large. Keep one bullet per user-visible thing, not one per commit, and fold a fix-on-top-of-an-unreleased-feature into that feature's bullet rather than listing a bug that never shipped. History before `0.2.0` is not covered — git is the record for it.
|
|
|