feat(users): UserManager with per-user SQLCipher, and extract skald-core crate

Two changes developed together in one session; they share the same module
structure (db/mod.rs, the core lib root) and only compile together, so they
land as one commit.

## UserManager + per-user encryption (§9/§11)

New `users::UserManager`: owns the system.db pool plus a map
`userid -> SqlitePool` of unlocked databases. The pool *is* the unlock token —
its connect options carry the DEK as SQLCipher's raw key, so an open pool means
the key is in RAM until restart and dropping it re-locks (§9). Knows nothing
about cookies.

New `crypto` module: envelope encryption. A random 256-bit DEK encrypts
`{userid}.db`; `users.database_password` holds it sealed with AES-256-GCM under
`Argon2id(password, salt)`. The AEAD tag is the password verifier — one
derivation both authenticates and yields the key, so encrypted users store no
second hash. Cleartext users store the Argon2id output directly, compared in
constant time. Argon2 runs in spawn_blocking behind a 2-permit semaphore
(256 MiB per derivation).

- SQLCipher via `libsqlite3-sys` `bundled-sqlcipher-vendored-openssl`, pinned
  <0.38 so it unifies with the one sqlx-sqlite links (a newer copy would apply
  the feature to a SQLite sqlx never uses). OpenSSL is vendored and static, so
  the binary stays self-contained.
- Schema split into `create_registry_tables` (instance-wide, no user key) and
  `create_owner_tables` (one owner's content, identical in every file). No FK in
  the owner bucket may reach the registry — enforced by a standalone test.
  Dropped `chat_history.model_db_id` (write-only, and the only registry-crossing
  key); moved `projects`/`project_tickets` into the owner bucket.
- Provisioning invariant: the file is written before the row, deleted after it,
  so a crash leaves an orphan file, never a user without a database. `open_db`
  never creates: a missing file is an error, not a silent empty database.

Not consumed yet: no login, call sites still use the shared system.db pool.

## Extract crates/skald-core

The headless core moves out of `src/` into its own crate; `skald` (server) and
the coming `skald-setup` are shells around it. Two dependencies on the shell
were inverted rather than dragged along, so the core names neither Tauri nor any
concrete plugin:

- `Plugin::tools(self: Arc<Self>)` — plugins contribute tools through this hook
  (sibling of `http_router`), so the core no longer downcasts to
  `MobileConnectorPlugin`.
- `tools::restart::set_restart_handler` — the desktop shell installs its
  teardown-and-respawn; the core defaults to the supervisor exit code. The core
  loses its `desktop` feature.
- `boot`'s stdout formatter moves to the binary (`src/boot_format.rs`); the core
  only emits tracing events.

All 79 core tests pass; the binary boots and serves in a clean directory, and
the mobile-connector tools still register through the new hook.
This commit is contained in:
2026-07-10 16:48:51 +01:00
parent 38494a85a9
commit 178a38357e
173 changed files with 2650 additions and 1106 deletions
+57 -31
View File
@@ -33,39 +33,58 @@ Domain words are allowed only in seed data, preset labels, UI copy and positioni
### Current state ### Current state
Single-user Skald plus a `users` table (`src/core/db/users.rs`). Next step is `UserManager`11). The multi-user split of the database has not started; `Runtime` still carries one shared `Arc<SqlitePool>`. `UserManager` (§11) exists and works `crates/skald-core/src/users/mod.rs`, with real per-user SQLCipher encryption4). It is **not consumed yet**: there is no login, and `Runtime` still hands every call site the one shared `Arc<SqlitePool>` on `system.db`, so chats still land in that file's owner tables. The next step is migrating those call sites to `pool_of`, and only then deciding where the owner-without-a-user lives (see blueprint §19).
Direction of travel, decided but not yet executed: strip the **power-user surface** (self-rewriting, arbitrary shell, dev-agent suite, ticket system) and move to a **binary-first** layout — the app is built once and run from a compiled binary, not executed from its own source tree. Direction of travel, decided but not yet executed: strip the **power-user surface** (self-rewriting, arbitrary shell, dev-agent suite, ticket system) and move to a **binary-first** layout — the app is built once and run from a compiled binary, not executed from its own source tree.
## Workspace layout
The application core is the `skald-core` crate; the binaries are **shells** around it.
| Crate | Role |
| ---- | ---- |
| `crates/skald-core/` | Storage, identity, crypto, LLM stack, tools, MCP, sessions. Knows nothing about what runs it: no Tauri, no HTTP server, and **no concrete plugin crate**`PluginManager` only ever sees `Arc<dyn Plugin>` from `core-api` |
| `skald` (root, `src/`) | The server shell: `main.rs`, the Axum `frontend/`, the Tauri `desktop/`, `config.rs`. Constructs the plugin list and hands it to `Skald::new` |
| `crates/core-api/` | The contracts both sides share: `Plugin`, `Tool`, event buses, provider types |
Two rules keep the boundary real, and both are enforced by the compiler:
- **The core never names a plugin.** A plugin contributes tools through `Plugin::tools(self: Arc<Self>)` — the sibling of `http_router()` — so nothing in the core has to downcast to a concrete type. Naming one would drag every plugin in the tree into the core, including a C build via `plugin-transcribe-whisper-local`.
- **The core never learns about the process shell.** The `restart` tool defaults to the supervisor protocol (`exit(-1)`); a shell with different needs installs `tools::restart::set_restart_handler` at startup. The Tauri shell installs teardown-and-respawn there. This is why `skald-core` has no `desktop` feature.
`skald_core::boot` emits curated startup lines on the `boot` tracing target; each shell decides how to render them (`src/boot_format.rs` here). The core says what happened, never how it looks.
## Key modules ## Key modules
| Path | Role | | Path | Role |
| ---- | ---- | | ---- | ---- |
| `src/main.rs` | Thin entry point: tracing → `Skald::new``WebFrontend::start` → shutdown. Branches on the `desktop` feature: under `--features desktop` enters `desktop::run()` (Tauri event loop) instead of blocking on a tokio runtime. Exposes `run_backend()` / `shutdown_backend()` shared by both entry points | | `src/main.rs` | Thin entry point: tracing → `Skald::new``WebFrontend::start` → shutdown. Branches on the `desktop` feature: under `--features desktop` enters `desktop::run()` (Tauri event loop) instead of blocking on a tokio runtime. Exposes `run_backend()` / `shutdown_backend()` shared by both entry points |
| `src/desktop/mod.rs` | Tauri shell — **only compiled under `--features desktop`**. Builds the system-tray icon + menu (`Open` / `Quit`), creates the main `WebviewWindow` (URL = `http://127.0.0.1:{config.port}`), spawns the backend on Tauri's shared tokio runtime, handles graceful shutdown. Holds the `OnceLock<AppHandle>` that the `restart` tool reads from. See [docs/desktop.md](docs/desktop.md) | | `src/desktop/mod.rs` | Tauri shell — **only compiled under `--features desktop`**. Builds the system-tray icon + menu (`Open` / `Quit`), creates the main `WebviewWindow` (URL = `http://127.0.0.1:{config.port}`), spawns the backend on Tauri's shared tokio runtime, handles graceful shutdown. Holds the `OnceLock<AppHandle>`, and installs the core's restart handler. See [docs/desktop.md](docs/desktop.md) |
| `src/core/skald/` | `Skald` — headless application core. `mod.rs` (struct + staged `new()` / `shutdown()`), `runtime.rs` (cross-cutting `Runtime` context), `bundles.rs` (8 domain bundles + `build()`), `wiring.rs` (`wire()` + `spawn_background()`), `supervisor.rs` (`TaskSupervisor`), `accessors.rs` (per-manager accessor facade — the API surface the frontend uses) | | `crates/skald-core/src/skald/` | `Skald` — headless application core. `mod.rs` (struct + staged `new()` / `shutdown()`), `runtime.rs` (cross-cutting `Runtime` context), `bundles.rs` (8 domain bundles + `build()`), `wiring.rs` (`wire()` + `spawn_background()`), `supervisor.rs` (`TaskSupervisor`), `accessors.rs` (per-manager accessor facade — the API surface the frontend uses) |
| `src/core/session/handler/` | Core LLM loop — `mod.rs`, `llm_loop.rs` (`run_agent_turn`), `agent_dispatch.rs`, `dispatcher.rs`, `approval.rs`, `resume.rs`, `messages.rs`, `config.rs`, `interface_tools.rs` | | `crates/skald-core/src/session/handler/` | Core LLM loop — `mod.rs`, `llm_loop.rs` (`run_agent_turn`), `agent_dispatch.rs`, `dispatcher.rs`, `approval.rs`, `resume.rs`, `messages.rs`, `config.rs`, `interface_tools.rs` |
| `src/core/session/manager.rs` | Creates/retrieves `ChatSessionHandler` per session | | `crates/skald-core/src/session/manager.rs` | Creates/retrieves `ChatSessionHandler` per session |
| `src/core/chat_hub/` | `ChatHub`: broadcast events to all connected WS clients | | `crates/skald-core/src/chat_hub/` | `ChatHub`: broadcast events to all connected WS clients |
| `src/core/chat_event_bus.rs` | Global async bus for cross-session events | | `crates/skald-core/src/chat_event_bus.rs` | Global async bus for cross-session events |
| `src/core/agents.rs` | Discovers agents from `agents/*/`, loads meta + system prompt | | `crates/skald-core/src/agents.rs` | Discovers agents from `agents/*/`, loads meta + system prompt |
| `src/core/tools/` | Built-in tools: `exec`, `restart`, `list_agents`, `fs/*`, `notify`, `ast_outline`, `image_generate`, MCP tools, plugin tools, cron tools | | `crates/skald-core/src/tools/` | Built-in tools: `exec`, `restart`, `list_agents`, `fs/*`, `notify`, `ast_outline`, `image_generate`, MCP tools, plugin tools, cron tools |
| `src/core/tool_catalog.rs` | `ToolCatalog`: unified tool listing façade (wraps ToolRegistry + McpManager) | | `crates/skald-core/src/tool_catalog.rs` | `ToolCatalog`: unified tool listing façade (wraps ToolRegistry + McpManager) |
| `src/core/events.rs` | `ServerEvent` enum streamed over WebSocket to the frontend | | `crates/skald-core/src/events.rs` | `ServerEvent` enum streamed over WebSocket to the frontend |
| `src/core/db/` | sqlx SQLite — see below | | `crates/skald-core/src/db/` | sqlx SQLite — see below |
| `crates/skald-core/src/users/` | `UserManager` (§11): user directory CRUD on `system.db`, credential check, and the map `userid → SqlitePool` of **unlocked** databases. The pool *is* the unlock token — its connect options carry the DEK as SQLCipher's raw key, so an open pool means the key is in RAM (§9) and dropping it re-locks. Knows nothing about cookies: whatever maps an HTTP session to a user id sits above it |
| `crates/skald-core/src/crypto/` | Envelope encryption (§4/§5.1). A random 256-bit DEK encrypts `{userid}.db`; `users.database_password` holds it sealed with AES-256-GCM under `Argon2id(password, salt)`. **The AEAD tag is the password verifier** — one derivation both authenticates and yields the key, and no second hash sits in the admin-readable DB. Cleartext users store the Argon2id output directly, compared constant-time. Argon2 runs in `spawn_blocking` behind a 2-permit semaphore (256 MiB per derivation) |
| `src/config.rs` | Loads `config.yml`; LLM clients, strength/use_cases, data root. Also hosts `bootstrap_data_dir()` — under the `desktop` feature, relocates the process cwd to a per-user data dir when running inside a `.app` bundle (no-op in dev mode and headless mode) | | `src/config.rs` | Loads `config.yml`; LLM clients, strength/use_cases, data root. Also hosts `bootstrap_data_dir()` — under the `desktop` feature, relocates the process cwd to a per-user data dir when running inside a `.app` bundle (no-op in dev mode and headless mode) |
| `src/core/mcp/` | MCP client manager (connects to external MCP servers) | | `crates/skald-core/src/mcp/` | MCP client manager (connects to external MCP servers) |
| `src/core/plugin/` | Plugin system: discovery, enable/disable, tool registration | | `crates/skald-core/src/plugin/` | Plugin system: discovery, enable/disable, tool registration |
| `src/core/cron/` | Scheduled job runner | | `crates/skald-core/src/cron/` | Scheduled job runner |
| `src/core/compactor.rs` | Context compaction (summarises history when token budget exceeded) | | `crates/skald-core/src/compactor.rs` | Context compaction (summarises history when token budget exceeded) |
| `src/core/approval/` | Approval rules engine | | `crates/skald-core/src/approval/` | Approval rules engine |
| `src/core/clarification/` | `ClarificationManager`: background-session question/answer | | `crates/skald-core/src/clarification/` | `ClarificationManager`: background-session question/answer |
| `src/core/elicitation/` | `ElicitationManager` + bridge: MCP server-initiated input (`elicitation/create`), surfaced in the Inbox; secrets never logged/persisted | | `crates/skald-core/src/elicitation/` | `ElicitationManager` + bridge: MCP server-initiated input (`elicitation/create`), surfaced in the Inbox; secrets never logged/persisted |
| `src/core/inbox.rs` | `Inbox`: unified façade for pending approvals + clarifications + elicitations (wraps ApprovalManager, ClarificationManager, ElicitationManager) | | `crates/skald-core/src/inbox.rs` | `Inbox`: unified façade for pending approvals + clarifications + elicitations (wraps ApprovalManager, ClarificationManager, ElicitationManager) |
| `src/core/llm/` | LLM client abstraction (OpenAI-compat, Anthropic, Ollama…) | | `crates/skald-core/src/llm/` | LLM client abstraction (OpenAI-compat, Anthropic, Ollama…) |
| `src/core/transcribe/` | Transcription providers | | `crates/skald-core/src/transcribe/` | Transcription providers |
| `src/core/image_generate/` | Image generation providers | | `crates/skald-core/src/image_generate/` | Image generation providers |
| `src/core/memory/` | Agent memory tools | | `crates/skald-core/src/memory/` | Agent memory tools |
| `src/frontend/mod.rs` | `WebFrontend`: wires router_factory, starts plugins, runs Axum | | `src/frontend/mod.rs` | `WebFrontend`: wires router_factory, starts plugins, runs Axum |
| `src/frontend/server.rs` | Axum router, static file serving | | `src/frontend/server.rs` | Axum router, static file serving |
| `src/frontend/api/` | HTTP + WebSocket handlers — `State<Arc<Skald>>` | | `src/frontend/api/` | HTTP + WebSocket handlers — `State<Arc<Skald>>` |
@@ -73,11 +92,18 @@ Direction of travel, decided but not yet executed: strip the **power-user surfac
## DB tables (sqlx SQLite) ## DB tables (sqlx SQLite)
One file, `database/system.db` — the path is a constant (`core::db::SYSTEM_DB_PATH`), **not** configurable. `init_pool` creates the directory; SQLite only creates the file. Per-user `database/{userid}.db` files are the next step (§5.1). `database/system.db` — the path is a constant (`core::db::SYSTEM_DB_PATH`), **not** configurable. `init_system_pool` creates the directory; SQLite only creates the file. Per-user files are `database/{userid}.db`, created by `UserManager::register_user` and encrypted with SQLCipher.
`users`, `chat_sessions`, `chat_sessions_stack`, `chat_history`, `chat_llm_tools`, `chat_summaries`, `llm_requests`, `llm_providers`, `llm_models`, `transcribe_models`, `tts_models`, `image_generate_models`, `scheduled_jobs`, `job_runs`, `mcp_servers`, `mcp_events`, `plugins`, `approval_rules`, `tool_permission_groups`, `sources`, `secrets`, `config`, `session_scratchpad`, `session_mcp_grants`, `stack_mcp_grants`, `known_tools`, `projects`, `project_tickets` The schema is split into two buckets (§5.1), and the split is the point:
`users` (`src/core/db/users.rs`) holds the directory plus auth material. It lives in the system DB, which the box owner can read, so it must never store anything that derives a user's key. `Credentials` is an enum mirroring the table's `CHECK`: an encrypted user carries a **wrapped DEK** (whose AEAD tag *is* the password verifier — hence no `password_hash`); a cleartext user carries an ordinary verifier, or none. `User` is deliberately not `Serialize` and its `Debug` redacts key material — use `User::summary()` for anything leaving the process. `role_id` has no foreign key yet: sqlx enables `PRAGMA foreign_keys`, so referencing the not-yet-existing `roles` table would fail every insert. - **`create_registry_tables`** — instance-wide, readable without any user key: `users`, `llm_providers`, `llm_models`, `transcribe_models`, `tts_models`, `image_generate_models`, `plugins`, `approval_rules`, `tool_permission_groups`, `config`, `known_tools`, `llm_requests`.
- **`create_owner_tables`** — one owner's content, **identical schema in every file that has it**: `chat_sessions`, `chat_sessions_stack`, `chat_history`, `chat_llm_tools`, `chat_summaries`, `session_scratchpad`, `session_mcp_grants`, `stack_mcp_grants`, `scheduled_jobs`, `job_runs`, `mcp_servers`, `mcp_events`, `sources`, `secrets`, `projects`, `project_tickets`.
**No foreign key in the owner bucket may point at a registry table.** SQLite cannot enforce a key across files, not even through `ATTACH`, and sqlx turns on `PRAGMA foreign_keys`: the `CREATE TABLE` succeeds and every `INSERT` fails. `db::tests::owner_tables_stand_alone_with_foreign_keys_on` enforces this by running the owner schema against a database holding nothing else, then inserting a row into each table. Two keys crossed and were fixed: `chat_history.model_db_id` (dropped — write-only, and `llm_requests.model_name` already records the model) and `project_tickets.job_id` (fixed by moving `projects`/`project_tickets` into the owner bucket).
`system.db` currently gets **both** bucket functions, because nothing has migrated to per-user pools yet. That is transitional.
`users` (`crates/skald-core/src/db/users.rs`) holds the directory plus auth material. It lives in the system DB, which the box owner can read, so it must never store anything that derives a user's key. `Credentials` is an enum mirroring the table's `CHECK`: an encrypted user carries a **wrapped DEK** (whose AEAD tag *is* the password verifier — hence no `password_hash`); a cleartext user carries an ordinary verifier, or none. `User` is deliberately not `Serialize` and its `Debug` redacts key material — use `User::summary()` for anything leaving the process. `role_id` has no foreign key yet: sqlx enables `PRAGMA foreign_keys`, so referencing the not-yet-existing `roles` table would fail every insert.
## Sub-agent system ## Sub-agent system
@@ -102,18 +128,18 @@ The rule engine `ApprovalManager::check` returns `Allow`/`Deny`/`Require` per to
Resolution is **source-agnostic**: the WS + Inbox paths resolve by `request_id`; the inline chat card resolves by the durable `tool_call_id` via `POST /api/tools/:tool_call_id/resolve` (`resolve_tool` in `src/frontend/api/sessions.rs`), which derives the owning session from the tool call's own stack row — never a hardcoded source. Live pending cards fire the `oneshot`; post-restart they execute directly on the owning session. See `docs/approval/`. Resolution is **source-agnostic**: the WS + Inbox paths resolve by `request_id`; the inline chat card resolves by the durable `tool_call_id` via `POST /api/tools/:tool_call_id/resolve` (`resolve_tool` in `src/frontend/api/sessions.rs`), which derives the owning session from the tool call's own stack row — never a hardcoded source. Live pending cards fire the `oneshot`; post-restart they execute directly on the owning session. See `docs/approval/`.
**Tool visibility in the Security-groups UI** (`GET /api/approval/tools`): tools injected outside the `ToolRegistry` (interface/plugin/provider tools) would otherwise be un-configurable. `ToolCatalog::list_all()` covers registry tools + a static `synthetic_tools()` list of core interface tools; everything else is captured by `src/core/tool_discovery.rs` (`ToolDiscovery`), which taps `all_tool_defs()` in `llm_loop.rs` each round and upserts every offered tool into the `known_tools` table (in-memory seen-set guard → background DB write). `list_tools` merges `known_tools` (deduped, `category: "dynamic"`) so any tool offered at least once becomes gate-able. Drift-proof by construction; core never hardcodes plugin tool names. **Tool visibility in the Security-groups UI** (`GET /api/approval/tools`): tools injected outside the `ToolRegistry` (interface/plugin/provider tools) would otherwise be un-configurable. `ToolCatalog::list_all()` covers registry tools + a static `synthetic_tools()` list of core interface tools; everything else is captured by `crates/skald-core/src/tool_discovery.rs` (`ToolDiscovery`), which taps `all_tool_defs()` in `llm_loop.rs` each round and upserts every offered tool into the `known_tools` table (in-memory seen-set guard → background DB write). `list_tools` merges `known_tools` (deduped, `category: "dynamic"`) so any tool offered at least once becomes gate-able. Drift-proof by construction; core never hardcodes plugin tool names.
## Restart ## Restart
`restart` **no longer rebuilds anything** — neither mode compiles. `restart` **no longer rebuilds anything** — neither mode compiles.
- **Headless** (default): `restart` calls `libc::_exit(-1)` (= exit code 255); `run.sh` re-executes the same binary *by path*. - **Headless** (default): no handler installed, so `restart` calls `libc::_exit(-1)` (= exit code 255); `run.sh` re-executes the same binary *by path*.
- **Desktop** (`--features desktop`): Tauri cleanup + respawn of the bundled binary + `exit(0)`. - **Desktop** (`--features desktop`): the Tauri shell installs a handler via `tools::restart::set_restart_handler` cleanup + respawn of the bundled binary + `exit(0)`. The core does not know Tauri exists.
Use it to pick up `config.yml` / database changes, which are only read at startup. To load new **code**: `./build.sh`, then restart — the supervisor picks up the new binary on the next loop, since `build.sh` installs it with an atomic rename. Use it to pick up `config.yml` / database changes, which are only read at startup. To load new **code**: `./build.sh`, then restart — the supervisor picks up the new binary on the next loop, since `build.sh` installs it with an atomic rename.
> `restart`'s tool description in `src/core/tools/restart.rs` still tells the model it triggers a `cargo build`. That is stale and must be fixed, along with `run.bat` (still `cargo run`). > `run.bat` is still stale (`cargo run`) and must be fixed.
## Build & run ## Build & run
Generated
+100 -22
View File
@@ -111,6 +111,18 @@ dependencies = [
"object", "object",
] ]
[[package]]
name = "argon2"
version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3c3610892ee6e0cbce8ae2700349fcf8f98adb0dbfbee85aec3c9179d29cc072"
dependencies = [
"base64ct",
"blake2",
"cpufeatures 0.2.17",
"password-hash",
]
[[package]] [[package]]
name = "arrayvec" name = "arrayvec"
version = "0.7.6" version = "0.7.6"
@@ -2990,11 +3002,12 @@ dependencies = [
[[package]] [[package]]
name = "libsqlite3-sys" name = "libsqlite3-sys"
version = "0.30.1" version = "0.37.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2e99fb7a497b1e3339bc746195567ed8d3e24945ecd636e3619d20b9de9e9149" checksum = "b1f111c8c41e7c61a49cd34e44c7619462967221a6443b0ec299e0ac30cfb9b1"
dependencies = [ dependencies = [
"cc", "cc",
"openssl-sys",
"pkg-config", "pkg-config",
"vcpkg", "vcpkg",
] ]
@@ -3620,6 +3633,28 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7c87def4c32ab89d880effc9e097653c8da5d6ef28e6b539d313baaacfbafcbe" checksum = "7c87def4c32ab89d880effc9e097653c8da5d6ef28e6b539d313baaacfbafcbe"
[[package]]
name = "openssl-src"
version = "300.6.1+3.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "46eb8fb9fb3b61ce1c0f8a026c4c1a0714d3a9e138e7fbde78753ce2babc3846"
dependencies = [
"cc",
]
[[package]]
name = "openssl-sys"
version = "0.9.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b47e7e6bb2c38cd930d25a23b40fa52e068c10e85f3e03a7f5ba5aaca5713695"
dependencies = [
"cc",
"libc",
"openssl-src",
"pkg-config",
"vcpkg",
]
[[package]] [[package]]
name = "option-ext" name = "option-ext"
version = "0.2.0" version = "0.2.0"
@@ -3696,6 +3731,17 @@ dependencies = [
"windows-link 0.2.1", "windows-link 0.2.1",
] ]
[[package]]
name = "password-hash"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "346f04948ba92c43e8469c1ee6736c7563d71012b17d40745260fe106aac2166"
dependencies = [
"base64ct",
"rand_core 0.6.4",
"subtle",
]
[[package]] [[package]]
name = "paste" name = "paste"
version = "1.0.15" version = "1.0.15"
@@ -4299,7 +4345,7 @@ version = "0.13.5"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "be769465445e8c1474e9c5dac2018218498557af32d9ed057325ec9a41ae81bf" checksum = "be769465445e8c1474e9c5dac2018218498557af32d9ed057325ec9a41ae81bf"
dependencies = [ dependencies = [
"heck 0.5.0", "heck 0.4.1",
"itertools 0.14.0", "itertools 0.14.0",
"log", "log",
"multimap", "multimap",
@@ -5396,22 +5442,16 @@ dependencies = [
"anyhow", "anyhow",
"async-trait", "async-trait",
"axum", "axum",
"base64 0.22.1",
"chrono", "chrono",
"chrono-tz",
"core-api", "core-api",
"cron",
"dirs 5.0.1", "dirs 5.0.1",
"futures", "futures",
"glob",
"honcho-client", "honcho-client",
"iana-time-zone",
"indexmap 2.14.0", "indexmap 2.14.0",
"libc", "libc",
"llm-client", "llm-client",
"mcp-client", "mcp-client",
"notify", "notify",
"os_info",
"plugin-comfyui", "plugin-comfyui",
"plugin-elevenlabs", "plugin-elevenlabs",
"plugin-honcho", "plugin-honcho",
@@ -5421,18 +5461,13 @@ dependencies = [
"plugin-transcribe-whisper-local", "plugin-transcribe-whisper-local",
"plugin-tts-kokoro", "plugin-tts-kokoro",
"plugin-tts-orpheus-3b", "plugin-tts-orpheus-3b",
"proc-macro2",
"quote",
"rand 0.10.1",
"regex",
"reqwest 0.13.4", "reqwest 0.13.4",
"rustls", "rustls",
"serde", "serde",
"serde_json", "serde_json",
"serde_yaml", "serde_yaml",
"sha2 0.10.9", "skald-core",
"sqlx", "sqlx",
"syn 2.0.117",
"tauri", "tauri",
"tauri-build", "tauri-build",
"tokio", "tokio",
@@ -5442,6 +5477,47 @@ dependencies = [
"tracing", "tracing",
"tracing-appender", "tracing-appender",
"tracing-subscriber", "tracing-subscriber",
]
[[package]]
name = "skald-core"
version = "0.1.0"
dependencies = [
"aes-gcm",
"anyhow",
"argon2",
"async-trait",
"axum",
"base64 0.22.1",
"chrono",
"chrono-tz",
"core-api",
"cron",
"futures",
"glob",
"honcho-client",
"iana-time-zone",
"indexmap 2.14.0",
"libc",
"libsqlite3-sys",
"llm-client",
"mcp-client",
"notify",
"os_info",
"proc-macro2",
"quote",
"rand 0.10.1",
"regex",
"reqwest 0.13.4",
"serde",
"serde_json",
"sha2 0.10.9",
"sqlx",
"subtle",
"syn 2.0.117",
"tokio",
"tokio-util",
"tracing",
"tree-sitter", "tree-sitter",
"tree-sitter-bash", "tree-sitter-bash",
"tree-sitter-c", "tree-sitter-c",
@@ -5459,6 +5535,8 @@ dependencies = [
"tree-sitter-swift", "tree-sitter-swift",
"tree-sitter-typescript", "tree-sitter-typescript",
"tree-sitter-yaml", "tree-sitter-yaml",
"uuid",
"zeroize",
] ]
[[package]] [[package]]
@@ -6389,7 +6467,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [ dependencies = [
"fastrand", "fastrand",
"getrandom 0.4.2", "getrandom 0.3.4",
"once_cell", "once_cell",
"rustix", "rustix",
"windows-sys 0.61.2", "windows-sys 0.61.2",
@@ -7721,9 +7799,9 @@ checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be"
[[package]] [[package]]
name = "uuid" name = "uuid"
version = "1.23.1" version = "1.23.4"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ddd74a9687298c6858e9b88ec8935ec45d22e8fd5e6394fa1bd4e99a87789c76" checksum = "bf80a72845275afea99e7f2b434723d3bc7e38470fcd1c7ed39a599c73319a53"
dependencies = [ dependencies = [
"getrandom 0.4.2", "getrandom 0.4.2",
"js-sys", "js-sys",
@@ -8903,18 +8981,18 @@ dependencies = [
[[package]] [[package]]
name = "zeroize" name = "zeroize"
version = "1.8.2" version = "1.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0" checksum = "e13c156562582aa81c60cb29407084cdb54c4164760106ab78e6c5b0858cf64e"
dependencies = [ dependencies = [
"zeroize_derive", "zeroize_derive",
] ]
[[package]] [[package]]
name = "zeroize_derive" name = "zeroize_derive"
version = "1.4.3" version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85a5b4158499876c763cb03bc4e49185d3cccbabb15b33c627f7884f43db852e" checksum = "3c50655cbb0fe3fc43170059e702f1ce5e19b84cec58dc87b037a09935c2f328"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
+11 -33
View File
@@ -1,6 +1,7 @@
[workspace] [workspace]
members = [ members = [
".", ".",
"crates/skald-core",
"crates/honcho-client", "crates/honcho-client",
"crates/llm-client", "crates/llm-client",
"crates/core-api", "crates/core-api",
@@ -45,6 +46,8 @@ embedded-tailscale = ["plugin-tailscale-remote/remote-tailscale"]
tauri-build = { version = "2", optional = true , features = [] } tauri-build = { version = "2", optional = true , features = [] }
[dependencies] [dependencies]
skald-core = { path = "crates/skald-core" }
axum = { version = "0.8", features = ["ws", "multipart"] } axum = { version = "0.8", features = ["ws", "multipart"] }
tokio = { version = "1.52.3", features = ["full"] } tokio = { version = "1.52.3", features = ["full"] }
tokio-util = { version = "0.7", features = ["rt"] } tokio-util = { version = "0.7", features = ["rt"] }
@@ -57,10 +60,14 @@ anyhow = "1"
sqlx = { version = "0.9.0", features = ["runtime-tokio", "sqlite"] } sqlx = { version = "0.9.0", features = ["runtime-tokio", "sqlite"] }
reqwest = { version = "0.13.4", default-features = false, features = ["rustls-no-provider", "charset", "http2", "system-proxy", "json", "multipart"] } reqwest = { version = "0.13.4", default-features = false, features = ["rustls-no-provider", "charset", "http2", "system-proxy", "json", "multipart"] }
# rustls is pinned as a direct dependency solely to select the crypto provider: # rustls is pinned as a direct dependency solely to select the crypto provider:
# `ring` instead of the default `aws-lc-rs`. This keeps TLS off OpenSSL/aws-lc # `ring` instead of the default `aws-lc-rs`, avoiding aws-lc's cmake/NASM build.
# (no libssl link, no cmake/NASM build), which is what makes a fully static musl # Every reqwest client uses `rustls-no-provider`, so exactly one process-wide
# binary practical. Every reqwest client uses `rustls-no-provider`, so exactly # provider is installed in main() before any TLS handshake.
# one process-wide provider is installed in main() before any TLS handshake. #
# TLS therefore never touches OpenSSL. libcrypto is nonetheless in the tree now,
# vendored and statically linked for SQLCipher (see `libsqlite3-sys` in
# crates/skald-core) — so the binary stays self-contained, but "no OpenSSL
# anywhere" is no longer true.
rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12", "logging"] } rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12", "logging"] }
async-trait = "0.1" async-trait = "0.1"
serde_json = "1" serde_json = "1"
@@ -69,36 +76,7 @@ tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] } tracing-subscriber = { version = "0.3", features = ["env-filter"] }
tracing-appender = "0.2" tracing-appender = "0.2"
chrono = { version = "0.4", default-features = false, features = ["clock", "std"] } chrono = { version = "0.4", default-features = false, features = ["clock", "std"] }
chrono-tz = "0.10"
iana-time-zone = "0.1"
os_info = "3"
cron = "0.16"
rand = "0.10.1"
syn = { version = "2", features = ["full"] }
quote = "1"
proc-macro2 = { version = "1", features = ["span-locations"] }
tree-sitter = "0.26"
tree-sitter-python = "0.25"
tree-sitter-javascript = "0.25"
tree-sitter-typescript = "0.23"
tree-sitter-go = "0.25"
tree-sitter-java = "0.23"
tree-sitter-c = "0.24"
tree-sitter-cpp = "0.23"
tree-sitter-swift = "0.7"
tree-sitter-lua = "0.5"
tree-sitter-ruby = "0.23"
tree-sitter-bash = "0.25"
tree-sitter-elixir = "0.3"
tree-sitter-json = "0.24"
tree-sitter-yaml = "0.7"
tree-sitter-html = "0.23"
tree-sitter-css = "0.25"
regex = "1"
glob = "0.3"
libc = "0.2" libc = "0.2"
base64 = "0.22"
sha2 = "0.10"
notify = "8" notify = "8"
honcho-client = { path = "crates/honcho-client" } honcho-client = { path = "crates/honcho-client" }
llm-client = { path = "crates/llm-client" } llm-client = { path = "crates/llm-client" }
+13
View File
@@ -87,6 +87,19 @@ pub trait Plugin: Send + Sync {
/// unaffected. /// unaffected.
fn http_router(&self) -> Option<axum::Router> { None } fn http_router(&self) -> Option<axum::Router> { None }
/// Tools this plugin contributes to the registry — the sibling of
/// [`Plugin::http_router`].
///
/// The receiver is `Arc<Self>` because the tools a plugin builds usually
/// call back into it, so it must hand them its own handle. Without this
/// hook the core has to name concrete plugin crates in order to downcast
/// them, and ends up depending on every plugin in the tree.
///
/// Called once while the tool registry is built, *before* the plugin's
/// runloop starts: the tools must tolerate being invoked while their plugin
/// is stopped. Default: no tools.
fn tools(self: Arc<Self>) -> Vec<Arc<dyn crate::tool::Tool>> { Vec::new() }
/// Returns a [`Memory`] backend if this plugin provides one. /// Returns a [`Memory`] backend if this plugin provides one.
fn memory(&self) -> Option<Arc<dyn Memory>> { None } fn memory(&self) -> Option<Arc<dyn Memory>> { None }
@@ -316,6 +316,13 @@ impl Plugin for MobileConnectorPlugin {
Some(router::build(Arc::clone(&self.inner))) Some(router::build(Arc::clone(&self.inner)))
} }
/// Control tools (plugin.md §11). They close over the plugin itself as a
/// `RelayAgent` and call into it lazily, so building them before the runloop
/// starts is fine — they fail gracefully while it is stopped.
fn tools(self: Arc<Self>) -> Vec<Arc<dyn core_api::tool::Tool>> {
crate::tools::mobile_tools(self)
}
fn as_any(&self) -> &dyn std::any::Any { self } fn as_any(&self) -> &dyn std::any::Any { self }
fn as_arc_any(self: Arc<Self>) -> Arc<dyn std::any::Any + Send + Sync> { self } fn as_arc_any(self: Arc<Self>) -> Arc<dyn std::any::Any + Send + Sync> { self }
} }
+82
View File
@@ -0,0 +1,82 @@
[package]
name = "skald-core"
version = "0.1.0"
edition = "2024"
# The headless application core: database + crypto + identity, the LLM stack,
# tools, MCP, plugins-as-a-registry, sessions.
#
# Deliberately knows nothing about the process shell around it. No Tauri, no
# concrete plugin crates (it only ever sees `Arc<dyn Plugin>` from `core-api`),
# no `axum` server — `skald` and `skald-setup` are both consumers.
[dependencies]
axum = { version = "0.8", features = ["ws", "multipart"] }
tokio = { version = "1.52.3", features = ["full"] }
tokio-util = { version = "0.7", features = ["rt"] }
futures = "0.3"
serde = { version = "1", features = ["derive"] }
anyhow = "1"
sqlx = { version = "0.9.0", features = ["runtime-tokio", "sqlite"] }
# Per-user database encryption. `sqlx-sqlite` already builds `libsqlite3-sys`;
# naming it here only adds a feature, and Cargo's feature unification turns the
# one SQLite the process links into SQLCipher. A database opened without
# `PRAGMA key` still behaves as plain SQLite, so `system.db` is unaffected.
# `-vendored-openssl` compiles libcrypto from source instead of linking the
# system one, so the binary stays self-contained (see b08a13c). It costs a C
# build (perl + make) — the trade the blueprint §4 accepts explicitly.
#
# Pinned below 0.38 on purpose: `sqlx-sqlite` 0.9 requires ">=0.30.1, <0.38.0".
# A newer release would resolve to a *second* copy of the crate, and the
# SQLCipher feature would then apply to a SQLite that sqlx never links. 0.37.0
# is the newest version that still unifies.
libsqlite3-sys = { version = "0.37", features = ["bundled-sqlcipher-vendored-openssl"] }
# 0.6 is still a release candidate; stay on the stable line.
argon2 = "0.5"
# Held at 0.10 to match `skald-relay-common`, so the binary links one AES-GCM
# rather than two. Bumping to 0.11 means bumping the mobile E2E code with it.
aes-gcm = "0.10"
zeroize = { version = "1", features = ["derive"] }
subtle = "2"
uuid = { version = "1", features = ["v4"] }
reqwest = { version = "0.13.4", default-features = false, features = ["rustls-no-provider", "charset", "http2", "system-proxy", "json", "multipart"] }
async-trait = "0.1"
serde_json = "1"
indexmap = { version = "2", features = ["serde"] }
tracing = "0.1"
chrono = { version = "0.4", default-features = false, features = ["clock", "std"] }
chrono-tz = "0.10"
iana-time-zone = "0.1"
os_info = "3"
cron = "0.16"
rand = "0.10.1"
syn = { version = "2", features = ["full"] }
quote = "1"
proc-macro2 = { version = "1", features = ["span-locations"] }
tree-sitter = "0.26"
tree-sitter-python = "0.25"
tree-sitter-javascript = "0.25"
tree-sitter-typescript = "0.23"
tree-sitter-go = "0.25"
tree-sitter-java = "0.23"
tree-sitter-c = "0.24"
tree-sitter-cpp = "0.23"
tree-sitter-swift = "0.7"
tree-sitter-lua = "0.5"
tree-sitter-ruby = "0.23"
tree-sitter-bash = "0.25"
tree-sitter-elixir = "0.3"
tree-sitter-json = "0.24"
tree-sitter-yaml = "0.7"
tree-sitter-html = "0.23"
tree-sitter-css = "0.25"
regex = "1"
glob = "0.3"
libc = "0.2"
base64 = "0.22"
sha2 = "0.10"
notify = "8"
honcho-client = { path = "../honcho-client" }
llm-client = { path = "../llm-client" }
core-api = { path = "../core-api" }
mcp-client = { path = "../mcp-client" }
@@ -2,7 +2,7 @@ use anyhow::{Context, Result};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use tracing::{debug, trace, warn}; use tracing::{debug, trace, warn};
use crate::config::LlmStrength; use core_api::provider::LlmStrength;
const AGENTS_DIR: &str = "agents"; const AGENTS_DIR: &str = "agents";
@@ -46,11 +46,11 @@ use sqlx::SqlitePool;
use tokio::sync::{broadcast, Mutex, oneshot}; use tokio::sync::{broadcast, Mutex, oneshot};
use tracing::{debug, error, info, warn}; use tracing::{debug, error, info, warn};
use crate::core::pending_registry::PendingRegistry; use crate::pending_registry::PendingRegistry;
use crate::core::session::handler::ApprovalDecision; use crate::session::handler::ApprovalDecision;
use crate::core::tools::tool_names as tn; use crate::tools::tool_names as tn;
use crate::core::tools::ToolCategory; use crate::tools::ToolCategory;
use crate::core::events::{GlobalEvent, ServerEvent}; use crate::events::{GlobalEvent, ServerEvent};
// ── Public types ────────────────────────────────────────────────────────────── // ── Public types ──────────────────────────────────────────────────────────────
@@ -437,7 +437,7 @@ impl ApprovalManager {
args: &Value, args: &Value,
group_id: Option<&str>, group_id: Option<&str>,
) -> GateResult { ) -> GateResult {
let rules = match crate::core::db::approval_rules::list_for_group(&self.db, group_id).await { let rules = match crate::db::approval_rules::list_for_group(&self.db, group_id).await {
Ok(r) => r, Ok(r) => r,
Err(e) => { Err(e) => {
// Fail closed: this is a default-closed security gate (see module docs). // Fail closed: this is a default-closed security gate (see module docs).
@@ -754,7 +754,7 @@ impl ApprovalManager {
group_id: &str, group_id: &str,
tool_name: &str, tool_name: &str,
) -> Option<RuleAction> { ) -> Option<RuleAction> {
let rules = crate::core::db::approval_rules::list_for_group(&self.db, Some(group_id)) let rules = crate::db::approval_rules::list_for_group(&self.db, Some(group_id))
.await .await
.unwrap_or_default(); .unwrap_or_default();
for rule in &rules { for rule in &rules {
@@ -768,7 +768,7 @@ impl ApprovalManager {
// ── Rule management ─────────────────────────────────────────────────────── // ── Rule management ───────────────────────────────────────────────────────
pub async fn list_rules(&self) -> Result<Vec<ApprovalRule>> { pub async fn list_rules(&self) -> Result<Vec<ApprovalRule>> {
crate::core::db::approval_rules::list(&self.db).await crate::db::approval_rules::list(&self.db).await
} }
pub async fn add_rule(&self, r: NewApprovalRule) -> Result<i64> { pub async fn add_rule(&self, r: NewApprovalRule) -> Result<i64> {
@@ -776,11 +776,11 @@ impl ApprovalManager {
let group = r.group_id.as_deref().unwrap_or("default"); let group = r.group_id.as_deref().unwrap_or("default");
self.ensure_no_conflicting_catch_all(group, &r.action, None).await?; self.ensure_no_conflicting_catch_all(group, &r.action, None).await?;
} }
crate::core::db::approval_rules::insert(&self.db, r).await crate::db::approval_rules::insert(&self.db, r).await
} }
pub async fn delete_rule(&self, id: i64) -> Result<()> { pub async fn delete_rule(&self, id: i64) -> Result<()> {
crate::core::db::approval_rules::delete(&self.db, id).await crate::db::approval_rules::delete(&self.db, id).await
} }
pub async fn update_rule(&self, id: i64, r: NewApprovalRule) -> Result<()> { pub async fn update_rule(&self, id: i64, r: NewApprovalRule) -> Result<()> {
@@ -788,7 +788,7 @@ impl ApprovalManager {
let group = r.group_id.as_deref().unwrap_or("default"); let group = r.group_id.as_deref().unwrap_or("default");
self.ensure_no_conflicting_catch_all(group, &r.action, Some(id)).await?; self.ensure_no_conflicting_catch_all(group, &r.action, Some(id)).await?;
} }
crate::core::db::approval_rules::update(&self.db, id, r).await crate::db::approval_rules::update(&self.db, id, r).await
} }
/// Rejects creating/updating a `*` catch-all when the target group already has a /// Rejects creating/updating a `*` catch-all when the target group already has a
@@ -916,11 +916,11 @@ fn rule_matches(rule: &ApprovalRule, agent_id: &str, source: &str, tool_name: &s
/// These back the "File System" permission panel, where each path row is one rule row. /// These back the "File System" permission panel, where each path row is one rule row.
pub(crate) fn tool_pattern_matches(pattern: &str, tool_name: &str) -> bool { pub(crate) fn tool_pattern_matches(pattern: &str, tool_name: &str) -> bool {
match pattern { match pattern {
"@fs_read" => crate::core::tools::is_file_read_tool(tool_name), "@fs_read" => crate::tools::is_file_read_tool(tool_name),
"@fs_write" => crate::core::tools::is_file_write_tool(tool_name), "@fs_write" => crate::tools::is_file_write_tool(tool_name),
"@fs_any" => { "@fs_any" => {
crate::core::tools::is_file_read_tool(tool_name) crate::tools::is_file_read_tool(tool_name)
|| crate::core::tools::is_file_write_tool(tool_name) || crate::tools::is_file_write_tool(tool_name)
} }
_ => pattern_matches(pattern, tool_name), _ => pattern_matches(pattern, tool_name),
} }
@@ -981,7 +981,7 @@ fn mcp_server_from_tool_name(name: &str) -> Option<String> {
pub(crate) fn normalize_path(path: &str) -> String { pub(crate) fn normalize_path(path: &str) -> String {
let cwd = std::env::current_dir().unwrap_or_default(); let cwd = std::env::current_dir().unwrap_or_default();
let cwd = std::fs::canonicalize(&cwd).unwrap_or(cwd); let cwd = std::fs::canonicalize(&cwd).unwrap_or(cwd);
let canon = crate::core::tools::fs::canonicalize_for_policy(path, &cwd); let canon = crate::tools::fs::canonicalize_for_policy(path, &cwd);
if let Ok(rel) = canon.strip_prefix(&cwd) { if let Ok(rel) = canon.strip_prefix(&cwd) {
return rel.to_string_lossy().into_owned(); return rel.to_string_lossy().into_owned();
} }
@@ -1059,7 +1059,7 @@ mod tests {
let path = std::env::temp_dir().join(format!("skald_fs_test_{}.db", std::process::id())); let path = std::env::temp_dir().join(format!("skald_fs_test_{}.db", std::process::id()));
let path_str = path.to_string_lossy().to_string(); let path_str = path.to_string_lossy().to_string();
let _ = std::fs::remove_file(&path); let _ = std::fs::remove_file(&path);
let pool = crate::core::db::init_pool(&path_str).await.expect("init_pool"); let pool = crate::db::init_system_pool(&path_str).await.expect("init_system_pool");
let db = Arc::new(pool); let db = Arc::new(pool);
// The `default` permission group is a FK target for approval_rules.group_id. // The `default` permission group is a FK target for approval_rules.group_id.
+53
View File
@@ -0,0 +1,53 @@
//! Curated, human-readable bootstrap progress printed to **stdout**.
//!
//! At runtime stdout is silent — only the file log (`logs/skald.log`) records
//! events. During startup, though, it is useful to see at a glance how the app
//! is configured and how it is coming up. These helpers emit a small, ordered
//! set of lines on the dedicated `boot` tracing target.
//!
//! Rendering belongs to whoever owns the process: each shell installs a stdout
//! layer filtered on [`TARGET`] and formats the lines as it likes (`skald`'s
//! `BootFormat` strips timestamps and paints failures red). The core only says
//! *what* happened, never how it looks — which is also why nothing here depends
//! on `tracing-subscriber`.
//!
//! The same lines land in the log file (they pass the normal `EnvFilter`), so
//! they double as a high-level startup trace. Glyphs are baked into the message
//! on purpose, so the file keeps the same readable shape.
use std::fmt;
use tracing::{info, warn};
/// Tracing target for curated bootstrap lines shown on stdout.
pub const TARGET: &str = "boot";
/// Top-level title (no glyph), e.g. `skald v0.5 — starting`.
pub fn title(msg: impl fmt::Display) {
info!(target: TARGET, "{}", msg);
}
/// A phase header, e.g. ` Plugins — 6 active, 1 failed`.
pub fn section(msg: impl fmt::Display) {
info!(target: TARGET, " {}", msg);
}
/// A successful item under a phase.
pub fn ok(msg: impl fmt::Display) {
info!(target: TARGET, " ✓ {}", msg);
}
/// An item that exists but is inactive (e.g. a disabled plugin).
pub fn off(msg: impl fmt::Display) {
info!(target: TARGET, " ○ {}", msg);
}
/// A failed item (rendered in red on stdout; logged at WARN in the file).
pub fn fail(msg: impl fmt::Display) {
warn!(target: TARGET, " ✗ {}", msg);
}
/// The final "app is up" line, e.g. `✅ Ready — http://localhost:8080`.
pub fn ready(msg: impl fmt::Display) {
info!(target: TARGET, "✅ {}", msg);
}
@@ -12,14 +12,14 @@ use tracing::{error, info, warn};
mod inbox; mod inbox;
use inbox::{QueuedMessage, SourceInbox, build_unit, drain_leading_user}; use inbox::{QueuedMessage, SourceInbox, build_unit, drain_leading_user};
use crate::core::approval::ApprovalManager; use crate::approval::ApprovalManager;
use crate::core::cron::TaskManager; use crate::cron::TaskManager;
use crate::core::db::{chat_history, chat_llm_tools, chat_sessions, chat_sessions_stack, config, sources}; use crate::db::{chat_history, chat_llm_tools, chat_sessions, chat_sessions_stack, config, sources};
use crate::core::events::{GlobalEvent, ServerEvent}; use crate::events::{GlobalEvent, ServerEvent};
use crate::core::notification::Notification; use crate::notification::Notification;
use crate::core::session::handler::{ChatSessionHandler, InterfaceTool, PendingMsg, PendingUserInput}; use crate::session::handler::{ChatSessionHandler, InterfaceTool, PendingMsg, PendingUserInput};
use crate::core::session::manager::ChatSessionManager; use crate::session::manager::ChatSessionManager;
use crate::core::tools::tool_names as tn; use crate::tools::tool_names as tn;
pub use core_api::chat_hub::{ChatHubApi, ModelCommandOutcome, SendMessageOptions}; pub use core_api::chat_hub::{ChatHubApi, ModelCommandOutcome, SendMessageOptions};
@@ -196,7 +196,7 @@ impl ChatHub {
let mut interface_tools = opts.interface_tools; let mut interface_tools = opts.interface_tools;
if let Some(task_mgr) = self.task_mgr.get() { if let Some(task_mgr) = self.task_mgr.get() {
interface_tools.push( interface_tools.push(
crate::core::tools::cron_jobs::build_execute_task_interface_tool( crate::tools::cron_jobs::build_execute_task_interface_tool(
Arc::clone(task_mgr), Arc::clone(task_mgr),
session_id, session_id,
run_context_json, run_context_json,
@@ -245,7 +245,7 @@ impl ChatHub {
&self, &self,
source_id: &str, source_id: &str,
agent_id: &str, agent_id: &str,
run_context: Option<&crate::core::run_context::RunContext>, run_context: Option<&crate::run_context::RunContext>,
reset: bool, reset: bool,
) -> anyhow::Result<i64> { ) -> anyhow::Result<i64> {
// A reset discards the current session; drop any messages queued for it. // A reset discards the current session; drop any messages queued for it.
@@ -372,7 +372,7 @@ impl ChatHub {
let mut tools = Vec::new(); let mut tools = Vec::new();
if let Some(task_mgr) = self.task_mgr.get() { if let Some(task_mgr) = self.task_mgr.get() {
let run_context_json = handler.run_context_json().await; let run_context_json = handler.run_context_json().await;
tools.push(crate::core::tools::cron_jobs::build_execute_task_interface_tool( tools.push(crate::tools::cron_jobs::build_execute_task_interface_tool(
Arc::clone(task_mgr), Arc::clone(task_mgr),
session_id, session_id,
run_context_json, run_context_json,
@@ -402,7 +402,7 @@ impl ChatHub {
/// The next LLM turn will start with no MCP servers activated. /// The next LLM turn will start with no MCP servers activated.
pub async fn reset_mcp(&self, source_id: &str) -> anyhow::Result<()> { pub async fn reset_mcp(&self, source_id: &str) -> anyhow::Result<()> {
let session_id = self.get_or_create_session(source_id, "main").await?; let session_id = self.get_or_create_session(source_id, "main").await?;
crate::core::db::session_mcp_grants::revoke_all(&self.db, session_id).await?; crate::db::session_mcp_grants::revoke_all(&self.db, session_id).await?;
info!(source_id, session_id, "ChatHub: MCP grants reset"); info!(source_id, session_id, "ChatHub: MCP grants reset");
Ok(()) Ok(())
} }
@@ -15,7 +15,7 @@ use serde_json::Value;
use sqlx::SqlitePool; use sqlx::SqlitePool;
use tracing::warn; use tracing::warn;
use crate::core::db::llm_requests; use crate::db::llm_requests;
use super::{ChatOptions, ChatResponse, ChatbotClient, LlmRawMeta, LlmTurn, Message}; use super::{ChatOptions, ChatResponse, ChatbotClient, LlmRawMeta, LlmTurn, Message};
@@ -6,8 +6,8 @@ use serde::Serialize;
use tokio::sync::{broadcast, oneshot}; use tokio::sync::{broadcast, oneshot};
use tracing::info; use tracing::info;
use crate::core::events::{GlobalEvent, ServerEvent}; use crate::events::{GlobalEvent, ServerEvent};
use crate::core::pending_registry::PendingRegistry; use crate::pending_registry::PendingRegistry;
#[derive(Debug, Clone, Serialize)] #[derive(Debug, Clone, Serialize)]
pub struct PendingClarificationInfo { pub struct PendingClarificationInfo {
@@ -49,11 +49,11 @@ use serde_json::json;
use sqlx::SqlitePool; use sqlx::SqlitePool;
use tracing::{debug, info, warn}; use tracing::{debug, info, warn};
use crate::core::chat_event_bus::{ChatEventBus, CompactionEvent}; use crate::chat_event_bus::{ChatEventBus, CompactionEvent};
use crate::core::chatbot::ChatOptions; use crate::chatbot::ChatOptions;
use crate::core::config::CompactionConfig; use crate::config::CompactionConfig;
use crate::core::db::{chat_history, chat_llm_tools, chat_summaries}; use crate::db::{chat_history, chat_llm_tools, chat_summaries};
use crate::core::llm::LlmManager; use crate::llm::LlmManager;
// ── Compaction constants (ported from Hermes context_compressor.py) ────────── // ── Compaction constants (ported from Hermes context_compressor.py) ──────────
// //
@@ -322,8 +322,8 @@ impl ContextCompactor {
})?; })?;
let summary_text = match turn { let summary_text = match turn {
crate::core::chatbot::LlmTurn::Message(resp) => resp.content, crate::chatbot::LlmTurn::Message(resp) => resp.content,
crate::core::chatbot::LlmTurn::ToolCalls { content, .. } => { crate::chatbot::LlmTurn::ToolCalls { content, .. } => {
warn!(stack_id, "compactor: unexpected tool calls in summary response, using content"); warn!(stack_id, "compactor: unexpected tool calls in summary response, using content");
content content
} }
@@ -12,10 +12,10 @@ use tracing::{error, info};
use core_api::system_bus::{SystemEvent, SystemEventBus}; use core_api::system_bus::{SystemEvent, SystemEventBus};
use crate::core::chat_hub::ChatHub; use crate::chat_hub::ChatHub;
use crate::core::db::chat_sessions; use crate::db::chat_sessions;
use crate::core::db::scheduled_jobs::{self, ScheduledJob}; use crate::db::scheduled_jobs::{self, ScheduledJob};
use crate::core::session::manager::ChatSessionManager; use crate::session::manager::ChatSessionManager;
pub struct TaskManager { pub struct TaskManager {
pool: Arc<SqlitePool>, pool: Arc<SqlitePool>,
@@ -183,7 +183,7 @@ impl TaskManager {
if agent_id.trim().is_empty() { if agent_id.trim().is_empty() {
anyhow::bail!("agent_id is required — specify which task agent runs this task (no default)"); anyhow::bail!("agent_id is required — specify which task agent runs this task (no default)");
} }
crate::core::agents::load_task_meta(agent_id)?; crate::agents::load_task_meta(agent_id)?;
Ok(()) Ok(())
} }
@@ -449,7 +449,7 @@ async fn run_job(
"cron" => { "cron" => {
if let Some(hub) = hub { if let Some(hub) = hub {
let outcome = final_response.as_deref().unwrap_or("(no output)"); let outcome = final_response.as_deref().unwrap_or("(no output)");
hub.notify(crate::core::notification::Notification { hub.notify(crate::notification::Notification {
source: "cron".into(), source: "cron".into(),
event_type: "cron_result".into(), event_type: "cron_result".into(),
summary: format!( summary: format!(
@@ -496,7 +496,7 @@ async fn run_job(
}); });
if let Some(hub) = hub { if let Some(hub) = hub {
hub.notify(crate::core::notification::Notification { hub.notify(crate::notification::Notification {
source: "cron".into(), source: "cron".into(),
event_type: "cron_error".into(), event_type: "cron_error".into(),
summary: format!( summary: format!(
@@ -525,14 +525,14 @@ async fn inject_async_result(
result: &str, result: &str,
) { ) {
// Resolve source_id from the parent session row. // Resolve source_id from the parent session row.
let source_id = match crate::core::db::chat_sessions::find_by_id(pool, parent_session_id).await { let source_id = match crate::db::chat_sessions::find_by_id(pool, parent_session_id).await {
Ok(Some(s)) => s.source, Ok(Some(s)) => s.source,
Ok(None) => { error!("inject_async_result: session {parent_session_id} not found"); return; } Ok(None) => { error!("inject_async_result: session {parent_session_id} not found"); return; }
Err(e) => { error!("inject_async_result: DB error: {e}"); return; } Err(e) => { error!("inject_async_result: DB error: {e}"); return; }
}; };
// Get the active stack for the parent session. // Get the active stack for the parent session.
let stack = match crate::core::db::chat_sessions_stack::active_for_session(pool, parent_session_id).await { let stack = match crate::db::chat_sessions_stack::active_for_session(pool, parent_session_id).await {
Ok(Some(s)) => s, Ok(Some(s)) => s,
Ok(None) => { error!("inject_async_result: no active stack for session {parent_session_id}"); return; } Ok(None) => { error!("inject_async_result: no active stack for session {parent_session_id}"); return; }
Err(e) => { error!("inject_async_result: stack lookup failed: {e}"); return; } Err(e) => { error!("inject_async_result: stack lookup failed: {e}"); return; }
@@ -544,8 +544,8 @@ async fn inject_async_result(
Let me process the result via task_completed.", Let me process the result via task_completed.",
task_title, task_title,
); );
let assistant_id = match crate::core::db::chat_history::append( let assistant_id = match crate::db::chat_history::append(
pool, stack.id, &crate::core::db::chat_history::Role::Assistant, pool, stack.id, &crate::db::chat_history::Role::Assistant,
"", true, Some(&reasoning), "", true, Some(&reasoning),
).await { ).await {
Ok(id) => id, Ok(id) => id,
@@ -559,7 +559,7 @@ async fn inject_async_result(
"result": result, "result": result,
})).unwrap_or_else(|_| "{}".to_string()); })).unwrap_or_else(|_| "{}".to_string());
let tool_call_id = match crate::core::db::chat_llm_tools::append( let tool_call_id = match crate::db::chat_llm_tools::append(
pool, assistant_id, "task_completed", pool, assistant_id, "task_completed",
&serde_json::json!({"task_id": task_id}).to_string(), &serde_json::json!({"task_id": task_id}).to_string(),
).await { ).await {
@@ -567,7 +567,7 @@ async fn inject_async_result(
Err(e) => { error!("inject_async_result: append tool call failed: {e}"); return; } Err(e) => { error!("inject_async_result: append tool call failed: {e}"); return; }
}; };
if let Err(e) = crate::core::db::chat_llm_tools::complete(pool, tool_call_id, &result_json, "string").await { if let Err(e) = crate::db::chat_llm_tools::complete(pool, tool_call_id, &result_json, "string").await {
error!("inject_async_result: complete tool call failed: {e}"); return; error!("inject_async_result: complete tool call failed: {e}"); return;
} }
@@ -580,15 +580,15 @@ async fn inject_async_result(
/// Builds the `execute_subtask` InterfaceTool injected into background sessions. /// Builds the `execute_subtask` InterfaceTool injected into background sessions.
/// Background tasks can only run synchronous sub-tasks — no cron or async. /// Background tasks can only run synchronous sub-tasks — no cron or async.
fn build_execute_subtask_tool(task_mgr: Arc<TaskManager>, run_context: Option<String>) -> crate::core::session::handler::InterfaceTool { fn build_execute_subtask_tool(task_mgr: Arc<TaskManager>, run_context: Option<String>) -> crate::session::handler::InterfaceTool {
use crate::core::session::handler::{InterfaceTool, ToolFuture}; use crate::session::handler::{InterfaceTool, ToolFuture};
use serde_json::json; use serde_json::json;
InterfaceTool { InterfaceTool {
definition: json!({ definition: json!({
"type": "function", "type": "function",
"function": { "function": {
"name": crate::core::tools::tool_names::EXECUTE_SUBTASK, "name": crate::tools::tool_names::EXECUTE_SUBTASK,
"description": "Run a synchronous sub-task and return its result. Blocks until the sub-task completes.", "description": "Run a synchronous sub-task and return its result. Blocks until the sub-task completes.",
"parameters": { "parameters": {
"type": "object", "type": "object",
@@ -633,7 +633,7 @@ async fn record_job_run(
response: Option<&str>, response: Option<&str>,
error: Option<&str>, error: Option<&str>,
) -> Result<()> { ) -> Result<()> {
crate::core::db::job_runs::insert( crate::db::job_runs::insert(
pool, job_id, Some(session_id), pool, job_id, Some(session_id),
started_at, completed_at, duration_ms, started_at, completed_at, duration_ms,
status, response, error, status, response, error,
+363
View File
@@ -0,0 +1,363 @@
//! Envelope encryption for the per-user databases (blueprint §4 / §5.1).
//!
//! A user's database is encrypted by SQLCipher under a random 256-bit **DEK**.
//! The DEK is never stored in the clear: `users.database_password` holds it
//! sealed with AES-256-GCM under a **KEK** derived from the password with
//! Argon2id.
//!
//! That seal *is* the password verifier. Opening it either yields the DEK — in
//! which case the password was right — or fails the AEAD tag, which means a
//! wrong password, cleanly distinct from a corrupt file. Storing a second hash
//! of the same password beside the seal would only hand an offline attacker an
//! easier target than the seal itself, so encrypted users have no
//! `password_hash` at all.
//!
//! Cleartext users have no database key to bind a verifier to, so they store the
//! Argon2id output directly and it is compared in constant time. Its
//! crackability is harmless: their database is readable by the box owner by
//! design.
//!
//! Nothing here invents a construction — it composes Argon2id, AES-256-GCM and a
//! CSPRNG. Changing the password re-seals the same DEK; the database itself is
//! never re-encrypted.
use std::fmt::{self, Write as _};
use std::sync::LazyLock;
use aes_gcm::aead::Aead;
use aes_gcm::{Aes256Gcm, KeyInit, Nonce};
use anyhow::{Context, Result, anyhow, bail};
use argon2::{Algorithm, Argon2, Params, Version};
use rand::Rng as _;
use serde::{Deserialize, Serialize};
use subtle::ConstantTimeEq;
use tokio::sync::Semaphore;
use zeroize::{Zeroize, ZeroizeOnDrop};
/// 256-bit keys throughout: AES-256-GCM for the seal, raw SQLCipher key for the
/// database.
const KEY_LEN: usize = 32;
const NONCE_LEN: usize = 12;
const TAG_LEN: usize = 16;
/// Sealed layout: `nonce ‖ ciphertext ‖ tag`.
const SEALED_LEN: usize = NONCE_LEN + KEY_LEN + TAG_LEN;
pub const SALT_LEN: usize = 16;
/// The only algorithm we accept. Stored per row so it can change later without a
/// migration, but a row asking for anything else is a bug, not a fallback.
const ALGO: &str = "argon2id";
/// Argon2id at 256 MiB is a memory bomb if it runs unbounded: every concurrent
/// login would allocate its own arena. Two at a time keeps a login responsive
/// while capping the peak at ~2× `KdfParams::m`.
///
/// Deliberately a module-level invariant rather than a caller's responsibility —
/// a forgotten permit is a memory incident, not a style problem.
static KDF_GATE: LazyLock<Semaphore> = LazyLock::new(|| Semaphore::new(2));
// ── KDF parameters ────────────────────────────────────────────────────────────
/// Serialized into `users.kdf_params`. Not secret: calibrated on the box when the
/// user is created, and kept per row so raising the cost later needs no migration.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct KdfParams {
pub algo: String,
/// Memory cost, in KiB.
pub m: u32,
/// Time cost (passes).
pub t: u32,
/// Parallelism (lanes).
pub p: u32,
}
impl Default for KdfParams {
/// 256 MiB, ~1s on the weakest box we target.
///
/// Between the OWASP baseline (64 MiB) and the 512 MiB1 GiB of §5.1: four
/// times the cost for an attacker with a GPU, while two concurrent logins
/// peak at 512 MiB rather than 2 GiB and never reach swap.
fn default() -> Self {
Self { algo: ALGO.to_string(), m: 262_144, t: 3, p: 1 }
}
}
impl KdfParams {
pub fn to_json(&self) -> Result<String> {
serde_json::to_string(self).context("serializing kdf_params")
}
pub fn from_json(s: &str) -> Result<Self> {
let p: Self = serde_json::from_str(s).context("parsing kdf_params")?;
if p.algo != ALGO {
bail!("unsupported kdf algorithm: {}", p.algo);
}
Ok(p)
}
/// Cheap parameters, tests only: the real ones cost ~1s and 256 MiB per
/// derivation, which a test suite pays on every register and every login.
#[cfg(test)]
pub fn fast() -> Self {
Self { algo: ALGO.to_string(), m: 64, t: 1, p: 1 }
}
}
// ── Keys ──────────────────────────────────────────────────────────────────────
/// Data Encryption Key: the raw SQLCipher key for one user's database. Random,
/// never derived from the password, so changing the password does not re-encrypt
/// anything.
#[derive(Clone, Zeroize, ZeroizeOnDrop)]
pub struct Dek([u8; KEY_LEN]);
/// Key Encryption Key: `Argon2id(password, salt)`. Seals the [`Dek`], and for a
/// cleartext user its raw bytes *are* the stored verifier.
#[derive(Clone, Zeroize, ZeroizeOnDrop)]
pub struct Kek([u8; KEY_LEN]);
impl Dek {
pub fn random() -> Self {
let mut k = [0u8; KEY_LEN];
rand::rng().fill_bytes(&mut k);
Self(k)
}
/// The `PRAGMA key` value for a raw 256-bit key.
///
/// The double quotes are **part of the value**: sqlx pastes it verbatim into
/// `PRAGMA key = {value};`, and SQLCipher only treats the argument as a raw
/// key when it parses as the blob literal `x'…'`. Given anything else it
/// runs its own KDF over the bytes instead, silently deriving a *different*
/// key from our hex digits — and the database would open, just not the one
/// we meant.
///
/// The returned string is key material. It must never be logged, and it
/// outlives this call inside the pool's connect options.
pub fn to_pragma(&self) -> String {
// `"x'` + 64 hex digits + `'"`
let mut s = String::with_capacity(5 + 2 * KEY_LEN);
s.push_str("\"x'");
for b in self.0 {
let _ = write!(s, "{b:02x}");
}
s.push_str("'\"");
s
}
}
impl Kek {
/// The verifier stored in `users.password_hash` for a cleartext user.
pub fn as_verifier(&self) -> &[u8] {
&self.0
}
}
// Hand-written so a stray `{:?}` — a tracing span, an error context, a panic —
// cannot print key material. Same reasoning as `db::users::Credentials`.
impl fmt::Debug for Dek {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str("Dek(<redacted>)")
}
}
impl fmt::Debug for Kek {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str("Kek(<redacted>)")
}
}
pub fn random_salt() -> Vec<u8> {
let mut s = vec![0u8; SALT_LEN];
rand::rng().fill_bytes(&mut s);
s
}
// ── Derivation ────────────────────────────────────────────────────────────────
/// `Argon2id(password, salt)` → 256-bit key.
///
/// Runs on a blocking thread — it burns a core and 256 MiB for about a second,
/// which would stall a tokio worker — and behind [`KDF_GATE`].
pub async fn derive_kek(password: &str, salt: &[u8], params: &KdfParams) -> Result<Kek> {
let _permit = KDF_GATE.acquire().await.context("kdf gate closed")?;
let mut password = password.to_owned();
let salt = salt.to_vec();
let params = params.clone();
let out = tokio::task::spawn_blocking(move || {
let result = derive_blocking(password.as_bytes(), &salt, &params);
password.zeroize();
result
})
.await
.context("argon2 task panicked")?;
out
}
fn derive_blocking(password: &[u8], salt: &[u8], params: &KdfParams) -> Result<Kek> {
if params.algo != ALGO {
bail!("unsupported kdf algorithm: {}", params.algo);
}
let p = Params::new(params.m, params.t, params.p, Some(KEY_LEN))
.map_err(|e| anyhow!("invalid argon2 params: {e}"))?;
let mut key = [0u8; KEY_LEN];
Argon2::new(Algorithm::Argon2id, Version::V0x13, p)
.hash_password_into(password, salt, &mut key)
.map_err(|e| anyhow!("argon2 derivation failed: {e}"))?;
Ok(Kek(key))
}
/// Constant-time comparison of a freshly derived key against a stored verifier.
/// `subtle` returns "not equal" for a length mismatch rather than short-circuiting.
pub fn verify(derived: &Kek, stored: &[u8]) -> bool {
derived.0.ct_eq(stored).into()
}
// ── Envelope ──────────────────────────────────────────────────────────────────
/// Why a seal did not open. The distinction is the whole point: a failed tag is
/// an authentication answer, a malformed blob is a broken row.
#[derive(Debug, PartialEq, Eq)]
pub enum KeyError {
/// The AEAD tag did not verify. This *is* the wrong-password signal.
WrongPassword,
Malformed(&'static str),
}
impl fmt::Display for KeyError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
KeyError::WrongPassword => f.write_str("wrong password"),
KeyError::Malformed(w) => write!(f, "malformed wrapped key: {w}"),
}
}
}
impl std::error::Error for KeyError {}
/// Seals `dek` under `kek`. Output is `nonce ‖ ciphertext ‖ tag`, stored as-is in
/// `users.database_password`.
pub fn wrap_dek(kek: &Kek, dek: &Dek) -> Result<Vec<u8>> {
let cipher = Aes256Gcm::new_from_slice(&kek.0).map_err(|e| anyhow!("bad kek length: {e}"))?;
let mut nonce = [0u8; NONCE_LEN];
rand::rng().fill_bytes(&mut nonce);
let sealed = cipher
.encrypt(Nonce::from_slice(&nonce), dek.0.as_slice())
.map_err(|_| anyhow!("AEAD seal failed"))?;
let mut out = Vec::with_capacity(SEALED_LEN);
out.extend_from_slice(&nonce);
out.extend_from_slice(&sealed);
Ok(out)
}
/// Opens a seal produced by [`wrap_dek`]. A failed tag returns
/// [`KeyError::WrongPassword`] — one Argon2id pass answers "is the password
/// right" *and* hands back the key.
pub fn unwrap_dek(kek: &Kek, sealed: &[u8]) -> Result<Dek, KeyError> {
if sealed.len() != SEALED_LEN {
return Err(KeyError::Malformed("unexpected length"));
}
let (nonce, body) = sealed.split_at(NONCE_LEN);
let cipher =
Aes256Gcm::new_from_slice(&kek.0).map_err(|_| KeyError::Malformed("bad kek length"))?;
let mut plain = cipher
.decrypt(Nonce::from_slice(nonce), body)
.map_err(|_| KeyError::WrongPassword)?;
if plain.len() != KEY_LEN {
plain.zeroize();
return Err(KeyError::Malformed("unexpected plaintext length"));
}
let mut dek = [0u8; KEY_LEN];
dek.copy_from_slice(&plain);
plain.zeroize();
Ok(Dek(dek))
}
#[cfg(test)]
mod tests {
use super::*;
async fn kek(password: &str, salt: &[u8]) -> Kek {
derive_kek(password, salt, &KdfParams::fast()).await.unwrap()
}
#[tokio::test]
async fn seal_round_trips_and_the_tag_is_the_password_check() {
let salt = random_salt();
let dek = Dek::random();
let sealed = wrap_dek(&kek("correct horse", &salt).await, &dek).unwrap();
assert_eq!(sealed.len(), SEALED_LEN);
let opened = unwrap_dek(&kek("correct horse", &salt).await, &sealed).unwrap();
assert_eq!(opened.0, dek.0, "the same DEK must come back out");
// The whole auth story: a wrong password is a failed AEAD tag, and it is
// reported as such — not as a corrupt blob.
let err = unwrap_dek(&kek("wrong horse", &salt).await, &sealed).unwrap_err();
assert_eq!(err, KeyError::WrongPassword);
}
#[tokio::test]
async fn a_truncated_seal_is_malformed_not_a_wrong_password() {
let salt = random_salt();
let sealed = wrap_dek(&kek("pw", &salt).await, &Dek::random()).unwrap();
let err = unwrap_dek(&kek("pw", &salt).await, &sealed[..SEALED_LEN - 1]).unwrap_err();
assert!(matches!(err, KeyError::Malformed(_)), "got {err:?}");
}
#[tokio::test]
async fn the_salt_separates_identical_passwords() {
let a = kek("same", &random_salt()).await;
let b = kek("same", &random_salt()).await;
assert_ne!(a.0, b.0, "a per-user salt must defuse rainbow tables");
}
#[tokio::test]
async fn verify_accepts_the_right_password_and_rejects_a_short_verifier() {
let salt = random_salt();
let stored = kek("pw", &salt).await;
assert!(verify(&kek("pw", &salt).await, stored.as_verifier()));
assert!(!verify(&kek("nope", &salt).await, stored.as_verifier()));
// A length mismatch must not panic, and must not compare equal.
assert!(!verify(&kek("pw", &salt).await, &stored.as_verifier()[..16]));
}
#[test]
fn pragma_is_a_quoted_blob_literal() {
let dek = Dek([0xAB; KEY_LEN]);
let p = dek.to_pragma();
assert!(p.starts_with("\"x'") && p.ends_with("'\""), "got {p}");
assert_eq!(p.len(), 5 + 2 * KEY_LEN, "\"x' + 64 hex digits + '\"");
assert!(p.contains("abab"), "lowercase hex");
}
#[test]
fn debug_never_prints_key_material() {
let dek = Dek([0xDE; KEY_LEN]);
let kek = Kek([0xAD; KEY_LEN]);
assert_eq!(format!("{dek:?}"), "Dek(<redacted>)");
assert_eq!(format!("{kek:?}"), "Kek(<redacted>)");
assert!(!format!("{dek:?}").contains("dede"), "no raw key bytes");
assert!(!format!("{kek:?}").contains("adad"), "no raw key bytes");
}
#[test]
fn kdf_params_round_trip_and_reject_foreign_algorithms() {
let p = KdfParams::default();
assert_eq!(p.m, 262_144);
assert_eq!(KdfParams::from_json(&p.to_json().unwrap()).unwrap(), p);
let pbkdf2 = r#"{"algo":"pbkdf2","m":1,"t":1,"p":1}"#;
assert!(KdfParams::from_json(pbkdf2).is_err(), "only argon2id is accepted");
}
}
@@ -1,7 +1,7 @@
use anyhow::Result; use anyhow::Result;
use sqlx::SqlitePool; use sqlx::SqlitePool;
use crate::core::approval::{ApprovalRule, NewApprovalRule, RuleAction}; use crate::approval::{ApprovalRule, NewApprovalRule, RuleAction};
type RawRow = (i64, Option<String>, Option<String>, String, Option<String>, String, Option<String>, i64, Option<String>); type RawRow = (i64, Option<String>, Option<String>, String, Option<String>, String, Option<String>, i64, Option<String>);
@@ -191,15 +191,6 @@ pub async fn for_stack_all(
rows.into_iter().map(row_to_message).collect() rows.into_iter().map(row_to_message).collect()
} }
pub async fn set_model_db_id(pool: &SqlitePool, id: i64, model_db_id: i64) -> anyhow::Result<()> {
sqlx::query("UPDATE chat_history SET model_db_id = ? WHERE id = ?")
.bind(model_db_id)
.bind(id)
.execute(pool)
.await?;
Ok(())
}
/// Ok messages for a stack frame whose id is strictly greater than `after_id`, /// Ok messages for a stack frame whose id is strictly greater than `after_id`,
/// ordered chronologically. Used by `build_openai_messages` when a compaction /// ordered chronologically. Used by `build_openai_messages` when a compaction
/// summary exists: only the "raw" messages after the summary boundary are loaded. /// summary exists: only the "raw" messages after the summary boundary are loaded.
@@ -164,7 +164,7 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn active_all_reflects_parallel_siblings() { async fn active_all_reflects_parallel_siblings() {
let path = temp_db_path("stack-parallel"); let path = temp_db_path("stack-parallel");
let pool = crate::core::db::init_pool(&path).await.unwrap(); let pool = crate::db::init_system_pool(&path).await.unwrap();
let sid = 1; let sid = 1;
// `session_id` has a FK to chat_sessions (sqlx enables foreign_keys). // `session_id` has a FK to chat_sessions (sqlx enables foreign_keys).
sqlx::query("INSERT INTO chat_sessions (id) VALUES (?)") sqlx::query("INSERT INTO chat_sessions (id) VALUES (?)")
@@ -1,5 +1,5 @@
//! `known_tools` — every tool ever offered to the LLM, captured at injection //! `known_tools` — every tool ever offered to the LLM, captured at injection
//! time by [`crate::core::tool_discovery::ToolDiscovery`]. //! time by [`crate::tool_discovery::ToolDiscovery`].
//! //!
//! This is the drift-proof half of tool visibility: instead of maintaining a //! This is the drift-proof half of tool visibility: instead of maintaining a
//! parallel list of "all tools", we record what is actually assembled into the //! parallel list of "all tools", we record what is actually assembled into the
@@ -77,7 +77,7 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn upsert_is_idempotent_and_updates_metadata() { async fn upsert_is_idempotent_and_updates_metadata() {
let path = temp_db_path("known-tools"); let path = temp_db_path("known-tools");
let pool = crate::core::db::init_pool(&path).await.unwrap(); let pool = crate::db::init_system_pool(&path).await.unwrap();
upsert(&pool, "send_voice_message", "v1", Some(r#"{"type":"object"}"#)).await.unwrap(); upsert(&pool, "send_voice_message", "v1", Some(r#"{"type":"object"}"#)).await.unwrap();
upsert(&pool, "send_voice_message", "v2", None).await.unwrap(); upsert(&pool, "send_voice_message", "v2", None).await.unwrap();
@@ -13,7 +13,7 @@ use tokio::task::JoinHandle;
use tokio_util::sync::CancellationToken; use tokio_util::sync::CancellationToken;
use tracing::{info, warn}; use tracing::{info, warn};
use crate::core::config::LlmRequestsLogConfig; use crate::config::LlmRequestsLogConfig;
/// Spawns the retention/cleanup loop for the `llm_requests` table. /// Spawns the retention/cleanup loop for the `llm_requests` table.
/// ///
@@ -1,7 +1,7 @@
//! DB operations for the `llm_requests` table. //! DB operations for the `llm_requests` table.
//! //!
//! Every `chat_with_tools` call is logged here by the //! Every `chat_with_tools` call is logged here by the
//! [`crate::core::chatbot::logging::LoggingChatbotClient`] wrapper. //! [`crate::chatbot::logging::LoggingChatbotClient`] wrapper.
//! Rows are retained for `llm.request_log.retention_days` days (default 14). //! Rows are retained for `llm.request_log.retention_days` days (default 14).
use anyhow::Result; use anyhow::Result;
@@ -21,147 +21,138 @@ pub mod stack_mcp_grants;
pub mod tool_permission_groups; pub mod tool_permission_groups;
pub mod users; pub mod users;
use anyhow::{Context, Result}; use std::path::{Path, PathBuf};
use sqlx::{SqlitePool, sqlite::{SqliteConnectOptions, SqliteJournalMode, SqliteSynchronous}};
use std::str::FromStr; use std::str::FromStr;
use std::time::Duration; use std::time::Duration;
/// System database: instance-wide state, shared by every user. use anyhow::{Context, Result};
/// use sqlx::{SqlitePool, sqlite::{SqliteConnectOptions, SqliteJournalMode, SqliteSynchronous}};
/// Fixed path — not configurable. Sibling `database/{userid}.db` files hold
/// per-user content; keeping them in one directory makes backup, export and use crate::crypto::Dek;
/// per-user erasure a matter of files rather than tables.
/// Every database file lives here, so backup, export and per-user erasure are a
/// matter of files rather than tables.
pub const DATABASE_DIR: &str = "database";
/// System database: instance-wide state, shared by every user. Fixed path — not
/// configurable.
pub const SYSTEM_DB_PATH: &str = "database/system.db"; pub const SYSTEM_DB_PATH: &str = "database/system.db";
pub async fn init_pool(path: &str) -> Result<SqlitePool> { /// `{dir}/{userid}.db`. Keyed by the opaque user id and never the username, so a
/// rename never has to touch the file. `dir` is a parameter rather than the
/// [`DATABASE_DIR`] constant so nothing depends on the process's working
/// directory — tests in particular.
pub fn user_db_path(dir: &Path, user_id: &str) -> PathBuf {
dir.join(format!("{user_id}.db"))
}
/// The `-wal` and `-shm` sidecars SQLite keeps beside a database in WAL mode.
/// Erasing a user means erasing these too.
pub fn user_db_sidecars(path: &Path) -> [PathBuf; 2] {
let ext = |suffix: &str| {
let mut p = path.to_path_buf().into_os_string();
p.push(suffix);
PathBuf::from(p)
};
[ext("-wal"), ext("-shm")]
}
fn ensure_parent(path: &Path) -> Result<()> {
// `create_if_missing` creates the file, never its parent directory. // `create_if_missing` creates the file, never its parent directory.
if let Some(parent) = std::path::Path::new(path).parent() if let Some(parent) = path.parent()
&& !parent.as_os_str().is_empty() && !parent.as_os_str().is_empty()
{ {
std::fs::create_dir_all(parent) std::fs::create_dir_all(parent)
.with_context(|| format!("failed to create database directory {}", parent.display()))?; .with_context(|| format!("failed to create database directory {}", parent.display()))?;
} }
let opts = SqliteConnectOptions::from_str(path)? Ok(())
.create_if_missing(true) }
// WAL lets readers run alongside a single writer, and `busy_timeout`
// makes a writer *wait* for the lock instead of failing immediately with fn tuned(opts: SqliteConnectOptions) -> SqliteConnectOptions {
// SQLITE_BUSY ("database is locked"). Without these, concurrent writers // WAL lets readers run alongside a single writer, and `busy_timeout` makes a
// e.g. the mobile-connector persisting its E2E `send_counter` while the // writer *wait* for the lock instead of failing immediately with SQLITE_BUSY
// chat loop / cron write history — abort mid-operation, which silently // ("database is locked"). Without these, concurrent writers — e.g. the
// drops outbound mobile messages (inbox_update never reaches the device). // mobile-connector persisting its E2E `send_counter` while the chat loop /
.journal_mode(SqliteJournalMode::Wal) // cron write history — abort mid-operation, which silently drops outbound
// mobile messages (inbox_update never reaches the device).
opts.journal_mode(SqliteJournalMode::Wal)
.synchronous(SqliteSynchronous::Normal) .synchronous(SqliteSynchronous::Normal)
.busy_timeout(Duration::from_secs(5)); .busy_timeout(Duration::from_secs(5))
}
/// Connect options for one user's database.
///
/// When `key` is present the pool is a SQLCipher pool: sqlx puts `PRAGMA key`
/// ahead of every other pragma, so the page cipher is armed before `journal_mode`
/// touches the file. Without a key the same code opens an ordinary SQLite file —
/// which is exactly what a cleartext user gets.
///
/// The returned value carries the raw key. **Never** `{:?}` it: `SqliteConnectOptions`
/// prints its pragmas, so a debug format anywhere near this would write the DEK,
/// in hex, into the logs.
fn user_options(path: &Path, key: Option<&Dek>, create: bool) -> SqliteConnectOptions {
let opts = SqliteConnectOptions::new().filename(path).create_if_missing(create);
let opts = match key {
Some(dek) => opts.pragma("key", dek.to_pragma()),
None => opts,
};
tuned(opts)
}
/// Forces a page read so a wrong key fails here, at open time, rather than at the
/// first unrelated query. SQLCipher answers "file is not a database" when the key
/// does not decrypt the header.
async fn probe(pool: &SqlitePool) -> Result<()> {
sqlx::query_scalar::<_, i64>("SELECT count(*) FROM sqlite_master")
.fetch_one(pool)
.await
.context("database did not open — wrong key, or not a database")?;
Ok(())
}
/// The system database: registry tables, plus — for now — the owner tables.
///
/// Owner tables live here **transitionally**. Nothing has been migrated to
/// per-user pools yet, so sessions, history and jobs still land in `system.db`.
/// When the call sites move to `UserManager::pool_of`, this second call goes
/// away and the owner-without-a-user gets a file of its own.
pub async fn init_system_pool(path: &str) -> Result<SqlitePool> {
ensure_parent(Path::new(path))?;
let opts = tuned(SqliteConnectOptions::from_str(path)?.create_if_missing(true));
let pool = SqlitePool::connect_with(opts).await?; let pool = SqlitePool::connect_with(opts).await?;
create_tables(&pool).await?; create_registry_tables(&pool).await?;
create_owner_tables(&pool).await?;
crate::boot::section("Database initialised".to_string()); crate::boot::section("Database initialised".to_string());
Ok(pool) Ok(pool)
} }
async fn create_tables(pool: &SqlitePool) -> Result<()> { /// Provisions `database/{userid}.db` and lays down the owner schema.
sqlx::query( ///
"CREATE TABLE IF NOT EXISTS chat_sessions ( /// Only this function may create a user's database. Login goes through
id INTEGER PRIMARY KEY AUTOINCREMENT, /// [`open_user_pool`], which refuses to create anything: a missing file there is
title TEXT, /// data loss, and creating a fresh empty one under the right password would hide
source TEXT NOT NULL DEFAULT 'web', /// it instead of reporting it.
agent_id TEXT NOT NULL DEFAULT 'main', pub async fn create_user_pool(path: &Path, key: Option<&Dek>) -> Result<SqlitePool> {
is_interactive INTEGER NOT NULL DEFAULT 1, ensure_parent(path)?;
is_ephemeral INTEGER NOT NULL DEFAULT 0, let pool = SqlitePool::connect_with(user_options(path, key, true)).await?;
run_context TEXT, probe(&pool).await?;
created_at TEXT NOT NULL DEFAULT (datetime('now')) create_owner_tables(&pool).await?;
)", Ok(pool)
) }
.execute(pool)
.await?;
sqlx::query( /// Opens an existing user database. Never creates one — see [`create_user_pool`].
"CREATE TABLE IF NOT EXISTS chat_sessions_stack ( pub async fn open_user_pool(path: &Path, key: Option<&Dek>) -> Result<SqlitePool> {
id INTEGER PRIMARY KEY AUTOINCREMENT, let pool = SqlitePool::connect_with(user_options(path, key, false)).await?;
session_id INTEGER NOT NULL REFERENCES chat_sessions(id), probe(&pool).await?;
agent_id TEXT NOT NULL DEFAULT 'main', Ok(pool)
agent_prompt TEXT, }
depth INTEGER NOT NULL DEFAULT 0,
parent_tool_call_id INTEGER,
terminated_at TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query( // ── Registry tables ───────────────────────────────────────────────────────────
"CREATE TABLE IF NOT EXISTS chat_history ( //
id INTEGER PRIMARY KEY AUTOINCREMENT, // Instance-wide, readable without any user key: the directory you must open
session_stack_id INTEGER NOT NULL REFERENCES chat_sessions_stack(id), // before you know who exists. Nothing here is scoped to one user.
role TEXT NOT NULL CHECK(role IN ('user', 'assistant', 'agent')),
content TEXT NOT NULL DEFAULT '',
status TEXT NOT NULL DEFAULT 'ok' CHECK(status IN ('ok', 'failed')),
input_tokens INTEGER,
output_tokens INTEGER,
duration_ms INTEGER,
model_db_id INTEGER REFERENCES llm_models(id),
is_synthetic INTEGER NOT NULL DEFAULT 0,
reasoning_content TEXT,
cost REAL,
metadata TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS chat_llm_tools (
id INTEGER PRIMARY KEY AUTOINCREMENT,
message_id INTEGER NOT NULL REFERENCES chat_history(id),
name TEXT NOT NULL,
arguments TEXT,
result TEXT,
status TEXT NOT NULL DEFAULT 'running' CHECK(status IN ('running', 'pending', 'done', 'failed', 'cancelled', 'rejected')),
result_type TEXT NOT NULL DEFAULT 'string' CHECK(result_type IN ('string', 'json')),
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE INDEX IF NOT EXISTS idx_stack_session ON chat_sessions_stack(session_id)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE INDEX IF NOT EXISTS idx_history_stack ON chat_history(session_stack_id)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE INDEX IF NOT EXISTS idx_tools_message ON chat_llm_tools(message_id)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS mcp_servers (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL UNIQUE,
transport TEXT NOT NULL DEFAULT 'stdio',
command TEXT,
args_json TEXT,
env_json TEXT,
url TEXT,
api_key TEXT,
description TEXT,
friendly_name TEXT,
enabled INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
async fn create_registry_tables(pool: &SqlitePool) -> Result<()> {
sqlx::query( sqlx::query(
"CREATE TABLE IF NOT EXISTS llm_providers ( "CREATE TABLE IF NOT EXISTS llm_providers (
id INTEGER PRIMARY KEY AUTOINCREMENT, id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -205,82 +196,6 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
.execute(pool) .execute(pool)
.await?; .await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS scheduled_jobs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
title TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
cron TEXT NOT NULL,
prompt TEXT NOT NULL,
agent_id TEXT NOT NULL DEFAULT 'main',
session_id INTEGER REFERENCES chat_sessions(id),
enabled INTEGER NOT NULL DEFAULT 1,
last_run_at TEXT,
next_run_at TEXT,
single_run INTEGER NOT NULL DEFAULT 0,
running_session_id INTEGER,
kind TEXT NOT NULL DEFAULT 'cron',
parent_session_id INTEGER REFERENCES chat_sessions(id),
run_context TEXT,
running_since TEXT,
origin_ref TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS plugins (
id TEXT PRIMARY KEY,
enabled INTEGER NOT NULL DEFAULT 0,
config TEXT NOT NULL DEFAULT '{}',
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS session_scratchpad (
session_id INTEGER NOT NULL REFERENCES chat_sessions(id),
key TEXT NOT NULL,
value TEXT NOT NULL,
PRIMARY KEY (session_id, key)
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tool_permission_groups (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS approval_rules (
id INTEGER PRIMARY KEY AUTOINCREMENT,
agent_id TEXT,
source TEXT,
tool_pattern TEXT NOT NULL,
action TEXT NOT NULL DEFAULT 'require'
CHECK(action IN ('require', 'allow', 'deny')),
note TEXT,
priority INTEGER NOT NULL DEFAULT 100,
path_pattern TEXT,
group_id TEXT REFERENCES tool_permission_groups(id),
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query( sqlx::query(
"CREATE TABLE IF NOT EXISTS transcribe_models ( "CREATE TABLE IF NOT EXISTS transcribe_models (
id INTEGER PRIMARY KEY AUTOINCREMENT, id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -332,10 +247,40 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
.await?; .await?;
sqlx::query( sqlx::query(
"CREATE TABLE IF NOT EXISTS sources ( "CREATE TABLE IF NOT EXISTS plugins (
id TEXT PRIMARY KEY, id TEXT PRIMARY KEY,
active_session_id INTEGER REFERENCES chat_sessions(id), enabled INTEGER NOT NULL DEFAULT 0,
updated_at TEXT NOT NULL DEFAULT (datetime('now')) config TEXT NOT NULL DEFAULT '{}',
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS tool_permission_groups (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS approval_rules (
id INTEGER PRIMARY KEY AUTOINCREMENT,
agent_id TEXT,
source TEXT,
tool_pattern TEXT NOT NULL,
action TEXT NOT NULL DEFAULT 'require'
CHECK(action IN ('require', 'allow', 'deny')),
note TEXT,
priority INTEGER NOT NULL DEFAULT 100,
path_pattern TEXT,
group_id TEXT REFERENCES tool_permission_groups(id),
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)", )",
) )
.execute(pool) .execute(pool)
@@ -351,63 +296,26 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
.execute(pool) .execute(pool)
.await?; .await?;
// Every tool ever offered to the LLM, recorded by `ToolDiscovery` at
// injection time. Lets the approval / Security-groups UI list and gate tools
// that are injected dynamically outside the `ToolRegistry` (interface tools,
// plugin tools, provider tools).
sqlx::query( sqlx::query(
"CREATE TABLE IF NOT EXISTS secrets ( "CREATE TABLE IF NOT EXISTS known_tools (
key TEXT PRIMARY KEY, name TEXT PRIMARY KEY,
value TEXT NOT NULL, description TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL DEFAULT (datetime('now')), schema TEXT,
updated_at TEXT NOT NULL DEFAULT (datetime('now')) first_seen INTEGER NOT NULL DEFAULT (strftime('%s','now')),
)", last_seen INTEGER NOT NULL DEFAULT (strftime('%s','now'))
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS mcp_events (
id INTEGER PRIMARY KEY AUTOINCREMENT,
source TEXT NOT NULL,
method TEXT NOT NULL,
payload TEXT NOT NULL,
processed INTEGER NOT NULL DEFAULT 0,
processed_at TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE INDEX IF NOT EXISTS idx_mcp_events_pending
ON mcp_events (processed, created_at)
WHERE processed = 0",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS session_mcp_grants (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id INTEGER NOT NULL,
mcp_name TEXT NOT NULL,
granted_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE(session_id, mcp_name)
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS stack_mcp_grants (
id INTEGER PRIMARY KEY AUTOINCREMENT,
stack_id INTEGER NOT NULL,
mcp_name TEXT NOT NULL,
granted_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE(stack_id, mcp_name)
)", )",
) )
.execute(pool) .execute(pool)
.await?; .await?;
// Spend/telemetry metadata. Stays in the registry so cost charts run without
// decrypting anything: the admin sees how much, when and which model — never
// what was said. `session_id` / `stack_id` are bare integers, not foreign
// keys, precisely because the rows they point at live in another file.
sqlx::query( sqlx::query(
"CREATE TABLE IF NOT EXISTS llm_requests ( "CREATE TABLE IF NOT EXISTS llm_requests (
id INTEGER PRIMARY KEY AUTOINCREMENT, id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -437,6 +345,140 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
.execute(pool) .execute(pool)
.await?; .await?;
// User directory + auth material. Read before every login, so it lives in
// the registry — which means it must never hold anything that derives a
// user's key: `database_password` is the DEK sealed under a key derived
// from the password, useless without it.
//
// `role_id` has no `REFERENCES roles(id)` yet: sqlx turns on
// `PRAGMA foreign_keys`, so pointing at a table that does not exist would
// make every INSERT fail. The constraint lands with the `roles` table.
sqlx::query(
"CREATE TABLE IF NOT EXISTS users (
id TEXT PRIMARY KEY,
username TEXT NOT NULL UNIQUE,
display_name TEXT,
role_id TEXT NOT NULL,
encrypted INTEGER NOT NULL,
kdf_params TEXT,
kdf_salt BLOB,
database_password BLOB,
password_hash BLOB,
active INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
CHECK (
(encrypted = 1 AND database_password IS NOT NULL AND password_hash IS NULL)
OR (encrypted = 0 AND database_password IS NULL)
)
)",
)
.execute(pool)
.await?;
Ok(())
}
// ── Owner tables ──────────────────────────────────────────────────────────────
//
// One owner's content. The schema is identical in every file that has it — only
// the file says who owns the rows — so this runs verbatim against `system.db`
// and against each `database/{userid}.db`.
//
// **No foreign key here may point at a registry table.** SQLite cannot enforce a
// key across files, not even through `ATTACH`, and sqlx turns on
// `PRAGMA foreign_keys`: the `CREATE TABLE` would succeed and every `INSERT`
// would fail. `create_owner_tables_stand_alone` in the tests guards this by
// running the schema against a file that has nothing else in it.
pub async fn create_owner_tables(pool: &SqlitePool) -> Result<()> {
sqlx::query(
"CREATE TABLE IF NOT EXISTS chat_sessions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
title TEXT,
source TEXT NOT NULL DEFAULT 'web',
agent_id TEXT NOT NULL DEFAULT 'main',
is_interactive INTEGER NOT NULL DEFAULT 1,
is_ephemeral INTEGER NOT NULL DEFAULT 0,
run_context TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS chat_sessions_stack (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id INTEGER NOT NULL REFERENCES chat_sessions(id),
agent_id TEXT NOT NULL DEFAULT 'main',
agent_prompt TEXT,
depth INTEGER NOT NULL DEFAULT 0,
parent_tool_call_id INTEGER,
terminated_at TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
// `model_db_id` used to point at `llm_models(id)`. It was write-only — never
// selected, never joined — and it was the one key that crossed into the
// registry. Which model answered is already recorded in
// `llm_requests.model_name`.
sqlx::query(
"CREATE TABLE IF NOT EXISTS chat_history (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_stack_id INTEGER NOT NULL REFERENCES chat_sessions_stack(id),
role TEXT NOT NULL CHECK(role IN ('user', 'assistant', 'agent')),
content TEXT NOT NULL DEFAULT '',
status TEXT NOT NULL DEFAULT 'ok' CHECK(status IN ('ok', 'failed')),
input_tokens INTEGER,
output_tokens INTEGER,
duration_ms INTEGER,
is_synthetic INTEGER NOT NULL DEFAULT 0,
reasoning_content TEXT,
cost REAL,
metadata TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS chat_llm_tools (
id INTEGER PRIMARY KEY AUTOINCREMENT,
message_id INTEGER NOT NULL REFERENCES chat_history(id),
name TEXT NOT NULL,
arguments TEXT,
result TEXT,
status TEXT NOT NULL DEFAULT 'running' CHECK(status IN ('running', 'pending', 'done', 'failed', 'cancelled', 'rejected')),
result_type TEXT NOT NULL DEFAULT 'string' CHECK(result_type IN ('string', 'json')),
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE INDEX IF NOT EXISTS idx_stack_session ON chat_sessions_stack(session_id)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE INDEX IF NOT EXISTS idx_history_stack ON chat_history(session_stack_id)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE INDEX IF NOT EXISTS idx_tools_message ON chat_llm_tools(message_id)",
)
.execute(pool)
.await?;
sqlx::query( sqlx::query(
"CREATE TABLE IF NOT EXISTS chat_summaries ( "CREATE TABLE IF NOT EXISTS chat_summaries (
id INTEGER PRIMARY KEY AUTOINCREMENT, id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -456,6 +498,66 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
.execute(pool) .execute(pool)
.await?; .await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS session_scratchpad (
session_id INTEGER NOT NULL REFERENCES chat_sessions(id),
key TEXT NOT NULL,
value TEXT NOT NULL,
PRIMARY KEY (session_id, key)
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS session_mcp_grants (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id INTEGER NOT NULL,
mcp_name TEXT NOT NULL,
granted_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE(session_id, mcp_name)
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS stack_mcp_grants (
id INTEGER PRIMARY KEY AUTOINCREMENT,
stack_id INTEGER NOT NULL,
mcp_name TEXT NOT NULL,
granted_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE(stack_id, mcp_name)
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS scheduled_jobs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
title TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
cron TEXT NOT NULL,
prompt TEXT NOT NULL,
agent_id TEXT NOT NULL DEFAULT 'main',
session_id INTEGER REFERENCES chat_sessions(id),
enabled INTEGER NOT NULL DEFAULT 1,
last_run_at TEXT,
next_run_at TEXT,
single_run INTEGER NOT NULL DEFAULT 0,
running_session_id INTEGER,
kind TEXT NOT NULL DEFAULT 'cron',
parent_session_id INTEGER REFERENCES chat_sessions(id),
run_context TEXT,
running_since TEXT,
origin_ref TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query( sqlx::query(
"CREATE TABLE IF NOT EXISTS job_runs ( "CREATE TABLE IF NOT EXISTS job_runs (
id INTEGER PRIMARY KEY AUTOINCREMENT, id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -481,6 +583,68 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
.execute(pool) .execute(pool)
.await?; .await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS mcp_servers (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL UNIQUE,
transport TEXT NOT NULL DEFAULT 'stdio',
command TEXT,
args_json TEXT,
env_json TEXT,
url TEXT,
api_key TEXT,
description TEXT,
friendly_name TEXT,
enabled INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS mcp_events (
id INTEGER PRIMARY KEY AUTOINCREMENT,
source TEXT NOT NULL,
method TEXT NOT NULL,
payload TEXT NOT NULL,
processed INTEGER NOT NULL DEFAULT 0,
processed_at TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE INDEX IF NOT EXISTS idx_mcp_events_pending
ON mcp_events (processed, created_at)
WHERE processed = 0",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS sources (
id TEXT PRIMARY KEY,
active_session_id INTEGER REFERENCES chat_sessions(id),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query(
"CREATE TABLE IF NOT EXISTS secrets (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
sqlx::query( sqlx::query(
"CREATE TABLE IF NOT EXISTS projects ( "CREATE TABLE IF NOT EXISTS projects (
id INTEGER PRIMARY KEY AUTOINCREMENT, id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -516,52 +680,111 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
.execute(pool) .execute(pool)
.await?; .await?;
// Every tool ever offered to the LLM, recorded by `ToolDiscovery` at
// injection time. Lets the approval / Security-groups UI list and gate tools
// that are injected dynamically outside the `ToolRegistry` (interface tools,
// plugin tools, provider tools). See docs/approval + docs/tools.
sqlx::query(
"CREATE TABLE IF NOT EXISTS known_tools (
name TEXT PRIMARY KEY,
description TEXT NOT NULL DEFAULT '',
schema TEXT,
first_seen INTEGER NOT NULL DEFAULT (strftime('%s','now')),
last_seen INTEGER NOT NULL DEFAULT (strftime('%s','now'))
)",
)
.execute(pool)
.await?;
// User directory + auth material. Read before every login, so it lives in
// the system DB — which means it must never hold anything that derives a
// user's key: `database_password` is the DEK sealed under a key derived
// from the password, useless without it.
//
// `role_id` has no `REFERENCES roles(id)` yet: sqlx turns on
// `PRAGMA foreign_keys`, so pointing at a table that does not exist would
// make every INSERT fail. The constraint lands with the `roles` table.
sqlx::query(
"CREATE TABLE IF NOT EXISTS users (
id TEXT PRIMARY KEY,
username TEXT NOT NULL UNIQUE,
display_name TEXT,
role_id TEXT NOT NULL,
encrypted INTEGER NOT NULL,
kdf_params TEXT,
kdf_salt BLOB,
database_password BLOB,
password_hash BLOB,
active INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
CHECK (
(encrypted = 1 AND database_password IS NOT NULL AND password_hash IS NULL)
OR (encrypted = 0 AND database_password IS NULL)
)
)",
)
.execute(pool)
.await?;
Ok(()) Ok(())
} }
#[cfg(test)]
mod tests {
use super::*;
fn temp_dir(tag: &str) -> PathBuf {
let nanos = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos();
let mut p = std::env::temp_dir();
p.push(format!("skald-db-{tag}-{}-{nanos}", std::process::id()));
p
}
/// The guardrail the file split would have given for free.
///
/// `create_owner_tables` runs here against a database that holds *nothing
/// else* — no `llm_models`, no `users`. Every table then takes a row. Any
/// foreign key that reaches into a registry table passes `CREATE TABLE` and
/// dies on the `INSERT`, right here, instead of lying dormant until a real
/// user logs in and writes to their own file.
#[tokio::test]
async fn owner_tables_stand_alone_with_foreign_keys_on() {
let dir = temp_dir("owner-standalone");
let path = dir.join("owner.db");
let pool = create_user_pool(&path, None).await.unwrap();
let (fk,): (i64,) = sqlx::query_as("PRAGMA foreign_keys")
.fetch_one(&pool).await.unwrap();
assert_eq!(fk, 1, "the guardrail is meaningless without FK enforcement");
let one = |q: &'static str| sqlx::query(q).execute(&pool);
one("INSERT INTO chat_sessions (id, title) VALUES (1, 't')").await.unwrap();
one("INSERT INTO chat_sessions_stack (id, session_id) VALUES (1, 1)").await.unwrap();
one("INSERT INTO chat_history (id, session_stack_id, role, content) VALUES (1, 1, 'user', 'hi')")
.await.unwrap();
one("INSERT INTO chat_llm_tools (message_id, name) VALUES (1, 'exec')").await.unwrap();
one("INSERT INTO chat_summaries (stack_id, content, covers_up_to_message_id) VALUES (1, 's', 1)")
.await.unwrap();
one("INSERT INTO session_scratchpad (session_id, key, value) VALUES (1, 'k', 'v')").await.unwrap();
one("INSERT INTO session_mcp_grants (session_id, mcp_name) VALUES (1, 'm')").await.unwrap();
one("INSERT INTO stack_mcp_grants (stack_id, mcp_name) VALUES (1, 'm')").await.unwrap();
one("INSERT INTO scheduled_jobs (id, title, cron, prompt, session_id) VALUES (1, 't', '* * * * *', 'p', 1)")
.await.unwrap();
one("INSERT INTO job_runs (job_id, started_at, status) VALUES (1, 'now', 'completed')").await.unwrap();
one("INSERT INTO mcp_servers (name) VALUES ('srv')").await.unwrap();
one("INSERT INTO mcp_events (source, method, payload) VALUES ('s', 'm', '{}')").await.unwrap();
one("INSERT INTO sources (id, active_session_id) VALUES ('web', 1)").await.unwrap();
one("INSERT INTO secrets (key, value) VALUES ('k', 'v')").await.unwrap();
one("INSERT INTO projects (id, name, path) VALUES (1, 'p', '/tmp')").await.unwrap();
one("INSERT INTO project_tickets (project_id, title, job_id) VALUES (1, 't', 1)").await.unwrap();
pool.close().await;
let _ = std::fs::remove_dir_all(&dir);
}
/// A cleartext user's database is an ordinary SQLite file; an encrypted one
/// is unreadable without the key, and does not even carry SQLite's header.
#[tokio::test]
async fn an_encrypted_database_is_opaque_without_its_key() {
let dir = temp_dir("opaque");
let clear_path = dir.join("clear.db");
let enc_path = dir.join("enc.db");
let clear = create_user_pool(&clear_path, None).await.unwrap();
clear.close().await;
let dek = Dek::random();
let enc = create_user_pool(&enc_path, Some(&dek)).await.unwrap();
sqlx::query("INSERT INTO chat_sessions (title) VALUES ('secret')")
.execute(&enc).await.unwrap();
enc.close().await;
let magic = b"SQLite format 3\0";
assert_eq!(&std::fs::read(&clear_path).unwrap()[..16], magic);
assert_ne!(&std::fs::read(&enc_path).unwrap()[..16], magic,
"an encrypted file must not advertise itself as SQLite");
assert!(open_user_pool(&enc_path, None).await.is_err(), "no key must not open it");
assert!(open_user_pool(&enc_path, Some(&Dek::random())).await.is_err(), "wrong key must not open it");
// ...and the right key still does, with the row intact.
let reopened = open_user_pool(&enc_path, Some(&dek)).await.unwrap();
let (title,): (String,) = sqlx::query_as("SELECT title FROM chat_sessions")
.fetch_one(&reopened).await.unwrap();
assert_eq!(title, "secret");
reopened.close().await;
let _ = std::fs::remove_dir_all(&dir);
}
/// Login must never conjure a database. A missing file is data loss and has
/// to be reported, not papered over with a fresh empty one.
#[tokio::test]
async fn open_never_creates_a_database() {
let dir = temp_dir("no-create");
std::fs::create_dir_all(&dir).unwrap();
let path = dir.join("ghost.db");
assert!(open_user_pool(&path, None).await.is_err());
assert!(open_user_pool(&path, Some(&Dek::random())).await.is_err());
assert!(!path.exists(), "open_user_pool must not have created the file");
let _ = std::fs::remove_dir_all(&dir);
}
}
@@ -358,7 +358,7 @@ pub async fn delete(pool: &SqlitePool, id: &str) -> Result<()> {
mod tests { mod tests {
use super::*; use super::*;
/// Nested on purpose: also covers `init_pool` creating the parent directory. /// Nested on purpose: also covers `init_system_pool` creating the parent directory.
fn temp_db_path(tag: &str) -> String { fn temp_db_path(tag: &str) -> String {
let mut p = std::env::temp_dir(); let mut p = std::env::temp_dir();
let nanos = std::time::SystemTime::now() let nanos = std::time::SystemTime::now()
@@ -392,11 +392,11 @@ mod tests {
} }
#[tokio::test] #[tokio::test]
async fn init_pool_creates_the_database_directory() { async fn init_system_pool_creates_the_database_directory() {
let path = temp_db_path("users-mkdir"); let path = temp_db_path("users-mkdir");
assert!(!std::path::Path::new(&path).parent().unwrap().exists()); assert!(!std::path::Path::new(&path).parent().unwrap().exists());
let pool = crate::core::db::init_pool(&path).await.unwrap(); let pool = crate::db::init_system_pool(&path).await.unwrap();
assert!(std::path::Path::new(&path).exists(), "system.db must exist under a fresh database/"); assert!(std::path::Path::new(&path).exists(), "system.db must exist under a fresh database/");
pool.close().await; pool.close().await;
@@ -406,7 +406,7 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn encrypted_user_round_trips_and_keeps_the_wrapped_dek() { async fn encrypted_user_round_trips_and_keeps_the_wrapped_dek() {
let path = temp_db_path("users-enc"); let path = temp_db_path("users-enc");
let pool = crate::core::db::init_pool(&path).await.unwrap(); let pool = crate::db::init_system_pool(&path).await.unwrap();
insert(&pool, "u-1", "ada", Some("Ada"), "admin", &encrypted()).await.unwrap(); insert(&pool, "u-1", "ada", Some("Ada"), "admin", &encrypted()).await.unwrap();
@@ -429,7 +429,7 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn cleartext_user_round_trips_with_and_without_a_verifier() { async fn cleartext_user_round_trips_with_and_without_a_verifier() {
let path = temp_db_path("users-clear"); let path = temp_db_path("users-clear");
let pool = crate::core::db::init_pool(&path).await.unwrap(); let pool = crate::db::init_system_pool(&path).await.unwrap();
insert(&pool, "u-1", "kid", None, "children", &cleartext()).await.unwrap(); insert(&pool, "u-1", "kid", None, "children", &cleartext()).await.unwrap();
insert(&pool, "u-2", "kiosk", None, "children", &Credentials::Cleartext(None)).await.unwrap(); insert(&pool, "u-2", "kiosk", None, "children", &Credentials::Cleartext(None)).await.unwrap();
@@ -455,7 +455,7 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn set_credentials_rewraps_and_migrates() { async fn set_credentials_rewraps_and_migrates() {
let path = temp_db_path("users-rewrap"); let path = temp_db_path("users-rewrap");
let pool = crate::core::db::init_pool(&path).await.unwrap(); let pool = crate::db::init_system_pool(&path).await.unwrap();
insert(&pool, "u-1", "ada", None, "admin", &encrypted()).await.unwrap(); insert(&pool, "u-1", "ada", None, "admin", &encrypted()).await.unwrap();
@@ -485,7 +485,7 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn check_constraint_rejects_impossible_rows() { async fn check_constraint_rejects_impossible_rows() {
let path = temp_db_path("users-check"); let path = temp_db_path("users-check");
let pool = crate::core::db::init_pool(&path).await.unwrap(); let pool = crate::db::init_system_pool(&path).await.unwrap();
// encrypted without a wrapped DEK // encrypted without a wrapped DEK
let err = sqlx::query( let err = sqlx::query(
@@ -7,7 +7,7 @@
//! secret it carries) flows straight back to the server's stdin — it is **never** //! secret it carries) flows straight back to the server's stdin — it is **never**
//! logged, broadcast in an event, or written to the DB. //! logged, broadcast in an event, or written to the DB.
//! //!
//! Mirrors [`crate::core::clarification`], but with the `accept`/`decline`/`cancel` //! Mirrors [`crate::clarification`], but with the `accept`/`decline`/`cancel`
//! outcome and a `sensitive` flag that elicitation needs and clarification lacks. //! outcome and a `sensitive` flag that elicitation needs and clarification lacks.
use std::sync::Arc; use std::sync::Arc;
@@ -23,8 +23,8 @@ use tracing::{debug, info};
use mcp_client::{ElicitationAction, ElicitationHandler, ElicitationReply, ElicitationRequest}; use mcp_client::{ElicitationAction, ElicitationHandler, ElicitationReply, ElicitationRequest};
use crate::core::events::{GlobalEvent, ServerEvent}; use crate::events::{GlobalEvent, ServerEvent};
use crate::core::pending_registry::PendingRegistry; use crate::pending_registry::PendingRegistry;
/// How long the user has to answer an elicitation before we reply `cancel`. /// How long the user has to answer an elicitation before we reply `cancel`.
/// Independent of any secret-cache TTL the MCP server keeps in its own RAM. /// Independent of any secret-cache TTL the MCP server keeps in its own RAM.
@@ -21,10 +21,10 @@ use tracing::{info, warn};
use core_api::image_generate::ImageGenerateRegistry; use core_api::image_generate::ImageGenerateRegistry;
use crate::core::llm::LlmProviderRecord; use crate::llm::LlmProviderRecord;
use crate::core::llm::db as llm_db; use crate::llm::db as llm_db;
use crate::core::provider::ProviderRegistry; use crate::provider::ProviderRegistry;
use crate::core::tools::Tool; use crate::tools::Tool;
use super::{ImageGenerate, ImageGenerateInfo, ImageGenerateModelInfo, ImageGenerateModelRecord}; use super::{ImageGenerate, ImageGenerateInfo, ImageGenerateModelInfo, ImageGenerateModelRecord};
use super::db as image_db; use super::db as image_db;
@@ -231,8 +231,8 @@ impl ImageGeneratorManager {
} }
drop(state); drop(state);
vec![ vec![
Arc::new(crate::core::tools::image_generate::ImageGenerateProvidersList { mgr: Arc::clone(&self) }) as Arc<dyn Tool>, Arc::new(crate::tools::image_generate::ImageGenerateProvidersList { mgr: Arc::clone(&self) }) as Arc<dyn Tool>,
Arc::new(crate::core::tools::image_generate::ImageGenerateTool { mgr: Arc::clone(&self) }) as Arc<dyn Tool>, Arc::new(crate::tools::image_generate::ImageGenerateTool { mgr: Arc::clone(&self) }) as Arc<dyn Tool>,
] ]
} }
@@ -10,10 +10,10 @@ use core_api::inbox::{
}; };
use core_api::tool::ToolDescriptionLength; use core_api::tool::ToolDescriptionLength;
use crate::core::approval::{ApprovalManager, PendingApprovalInfo}; use crate::approval::{ApprovalManager, PendingApprovalInfo};
use crate::core::clarification::{ClarificationManager, PendingClarificationInfo}; use crate::clarification::{ClarificationManager, PendingClarificationInfo};
use crate::core::elicitation::{ElicitationManager, ElicitationOutcome, PendingElicitationInfo}; use crate::elicitation::{ElicitationManager, ElicitationOutcome, PendingElicitationInfo};
use crate::core::tools::ToolRegistry; use crate::tools::ToolRegistry;
#[derive(Serialize)] #[derive(Serialize)]
pub struct InboxItems { pub struct InboxItems {
@@ -597,29 +597,43 @@ OUTPUT hello.log
} }
} }
/// Two real files to hash. These used to be `file!()` and `Cargo.toml`,
/// which quietly stopped resolving when this module moved into its own
/// crate: `file!()` is relative to the workspace root while a test's working
/// directory is the package root. Owning the fixtures keeps the test about
/// hashing.
fn hash_fixture(tag: &str) -> (PathBuf, PathBuf, PathBuf) {
let nanos = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos();
let dir = std::env::temp_dir().join(format!("skald-hash-{tag}-{}-{nanos}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let (a, b) = (dir.join("a.tex"), dir.join("b.tex"));
std::fs::write(&a, b"\\documentclass{article}").unwrap();
std::fs::write(&b, b"\\usepackage{amsmath}").unwrap();
(dir, a, b)
}
#[tokio::test] #[tokio::test]
async fn composite_hash_is_deterministic_for_same_contents() { async fn composite_hash_is_deterministic_for_same_contents() {
// Use the test file itself as a stand-in dependency: it exists on disk let (dir, a, _b) = hash_fixture("deterministic");
// and its contents are stable for the duration of the test. let deps = vec![a];
let me = Path::new(file!());
let deps_a = vec![me.to_path_buf()];
let deps_b = vec![me.to_path_buf()];
assert_eq!( assert_eq!(
composite_hash_of(&deps_a).await.unwrap(), composite_hash_of(&deps).await.unwrap(),
composite_hash_of(&deps_b).await.unwrap() composite_hash_of(&deps).await.unwrap()
); );
let _ = std::fs::remove_dir_all(&dir);
} }
#[tokio::test] #[tokio::test]
async fn composite_hash_is_order_independent() { async fn composite_hash_is_order_independent() {
let me = Path::new(file!()); let (dir, a, b) = hash_fixture("order");
let cargo = Path::new("Cargo.toml"); let forwards = vec![a.clone(), b.clone()];
let a = vec![me.to_path_buf(), cargo.to_path_buf()]; let backwards = vec![b, a];
let b = vec![cargo.to_path_buf(), me.to_path_buf()];
assert_eq!( assert_eq!(
composite_hash_of(&a).await.unwrap(), composite_hash_of(&forwards).await.unwrap(),
composite_hash_of(&b).await.unwrap() composite_hash_of(&backwards).await.unwrap()
); );
let _ = std::fs::remove_dir_all(&dir);
} }
#[tokio::test] #[tokio::test]
@@ -1,3 +1,11 @@
//! The headless Skald core: storage, identity, LLM stack, tools, sessions.
//!
//! Nothing here knows what runs it. The process shell — HTTP server, desktop
//! webview, setup wizard — lives in the crates that depend on this one. Concrete
//! plugins are never named: `plugin::PluginManager` only ever sees
//! `Arc<dyn Plugin>`, constructed by the consumer and handed to `Skald::new`.
pub mod boot;
pub mod config; pub mod config;
pub mod config_store; pub mod config_store;
pub mod skald; pub mod skald;
@@ -9,6 +17,7 @@ pub mod chatbot;
pub mod clarification; pub mod clarification;
pub mod command; pub mod command;
pub mod compactor; pub mod compactor;
pub mod crypto;
pub mod elicitation; pub mod elicitation;
pub mod cron; pub mod cron;
pub mod db; pub mod db;
@@ -35,3 +44,4 @@ pub mod tool_discovery;
pub mod tools; pub mod tools;
pub mod transcribe; pub mod transcribe;
pub mod tts; pub mod tts;
pub mod users;
@@ -1,7 +1,7 @@
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use sqlx::SqlitePool; use sqlx::SqlitePool;
use crate::config::LlmStrength; use core_api::provider::LlmStrength;
use super::{LlmModelRecord, LlmProviderRecord}; use super::{LlmModelRecord, LlmProviderRecord};
// ── Provider rows ───────────────────────────────────────────────────────────── // ── Provider rows ─────────────────────────────────────────────────────────────
@@ -124,11 +124,13 @@ pub async fn insert_model(pool: &SqlitePool, r: &LlmModelRecord) -> Result<i64>
let extra_params = r.extra_params.as_ref().map(|v| v.to_string()); let extra_params = r.extra_params.as_ref().map(|v| v.to_string());
let capabilities = serde_json::to_string(&r.capabilities)?; let capabilities = serde_json::to_string(&r.capabilities)?;
let reasoning = r.reasoning.as_ref().map(|v| v.to_string()); let reasoning = r.reasoning.as_ref().map(|v| v.to_string());
// A model row is never hard-deleted (llm_requests.model_db_id references it), // A model row is never hard-deleted, only soft-deleted via `removed_at`:
// only soft-deleted via `removed_at`. The unique identity is `name` (also the // history and telemetry name it, and `llm_requests` keeps only the model's
// resolution key), so re-adding a previously removed model with the same alias // name, so the row is what maps that name back to a provider. The unique
// would collide with the lingering soft-deleted row. Upsert on `name` so that // identity is `name` (also the resolution key), so re-adding a previously
// existing row is revived (removed_at cleared) and every field overwritten. // removed model with the same alias would collide with the lingering
// soft-deleted row. Upsert on `name` so that existing row is revived
// (removed_at cleared) and every field overwritten.
let id = sqlx::query_scalar::<_, i64>( let id = sqlx::query_scalar::<_, i64>(
"INSERT INTO llm_models (provider_id, model_id, name, strength, scope, is_default, priority, extra_params, "INSERT INTO llm_models (provider_id, model_id, name, strength, scope, is_default, priority, extra_params,
context_length, max_output_tokens, knowledge_cutoff, capabilities, reasoning) context_length, max_output_tokens, knowledge_cutoff, capabilities, reasoning)
@@ -8,10 +8,10 @@ use sqlx::SqlitePool;
use tokio::sync::RwLock; use tokio::sync::RwLock;
use tracing::{info, warn}; use tracing::{info, warn};
use crate::core::chatbot::ChatbotClient; use crate::chatbot::ChatbotClient;
use crate::core::chatbot::logging::{LoggingChatbotClient, LogSaveFlags}; use crate::chatbot::logging::{LoggingChatbotClient, LogSaveFlags};
use crate::config::LlmStrength; use core_api::provider::LlmStrength;
use crate::core::provider::{ApiProvider, ProviderRegistry, ReasoningMode}; use crate::provider::{ApiProvider, ProviderRegistry, ReasoningMode};
use super::providers::RemoteLlmModelInfo; use super::providers::RemoteLlmModelInfo;
use super::{ClientStatus, LlmEntry, LlmModelInfo, LlmModelRecord, LlmProviderInfo, LlmProviderRecord}; use super::{ClientStatus, LlmEntry, LlmModelInfo, LlmModelRecord, LlmProviderInfo, LlmProviderRecord};
@@ -4,8 +4,8 @@ pub mod providers;
use std::sync::Arc; use std::sync::Arc;
use crate::core::chatbot::ChatbotClient; use crate::chatbot::ChatbotClient;
use crate::core::provider::ServiceType; use crate::provider::ServiceType;
pub use core_api::provider::{LlmProviderRecord, LlmModelRecord, LlmStrength, ReasoningMode}; pub use core_api::provider::{LlmProviderRecord, LlmModelRecord, LlmStrength, ReasoningMode};
pub use manager::{LlmManager, sort_models_for_agent}; pub use manager::{LlmManager, sort_models_for_agent};
@@ -2,10 +2,10 @@ use std::sync::Arc;
use anyhow::{Context, Result, anyhow}; use anyhow::{Context, Result, anyhow};
use crate::core::chatbot::anthropic::AnthropicClient; use crate::chatbot::anthropic::AnthropicClient;
use crate::core::llm::{LlmModelRecord, LlmProviderRecord}; use crate::llm::{LlmModelRecord, LlmProviderRecord};
use crate::core::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning}; use crate::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType}; use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
pub struct AnthropicProvider { pub struct AnthropicProvider {
http: reqwest::Client, http: reqwest::Client,
@@ -2,10 +2,10 @@ use std::sync::Arc;
use anyhow::{Context, Result, anyhow}; use anyhow::{Context, Result, anyhow};
use crate::core::chatbot::openai::OpenAiClient; use crate::chatbot::openai::OpenAiClient;
use crate::core::llm::{LlmModelRecord, LlmProviderRecord}; use crate::llm::{LlmModelRecord, LlmProviderRecord};
use crate::core::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning}; use crate::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType}; use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
pub struct DeepSeekProvider { pub struct DeepSeekProvider {
http: reqwest::Client, http: reqwest::Client,
@@ -2,10 +2,10 @@ use std::sync::Arc;
use anyhow::{Result, anyhow}; use anyhow::{Result, anyhow};
use crate::core::chatbot::lm_studio::LmStudioClient; use crate::chatbot::lm_studio::LmStudioClient;
use crate::core::llm::{LlmModelRecord, LlmProviderRecord}; use crate::llm::{LlmModelRecord, LlmProviderRecord};
use crate::core::llm::providers::RemoteLlmModelInfo; use crate::llm::providers::RemoteLlmModelInfo;
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ServiceType}; use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ServiceType};
pub struct LmStudioProvider { pub struct LmStudioProvider {
http: reqwest::Client, http: reqwest::Client,
@@ -7,7 +7,7 @@ pub mod openrouter;
pub mod zai; pub mod zai;
// Re-export so existing code that uses `providers::ServiceType` / `providers::RemoteLlmModelInfo` keeps working. // Re-export so existing code that uses `providers::ServiceType` / `providers::RemoteLlmModelInfo` keeps working.
pub use crate::core::provider::ServiceType; pub use crate::provider::ServiceType;
pub use core_api::provider::RemoteLlmModelInfo; pub use core_api::provider::RemoteLlmModelInfo;
use core_api::provider::{ApiProvider, LlmModelRecord}; use core_api::provider::{ApiProvider, LlmModelRecord};
@@ -2,10 +2,10 @@ use std::sync::Arc;
use anyhow::{Result, anyhow}; use anyhow::{Result, anyhow};
use crate::core::chatbot::ollama::OllamaClient; use crate::chatbot::ollama::OllamaClient;
use crate::core::llm::{LlmModelRecord, LlmProviderRecord}; use crate::llm::{LlmModelRecord, LlmProviderRecord};
use crate::core::llm::providers::RemoteLlmModelInfo; use crate::llm::providers::RemoteLlmModelInfo;
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ServiceType}; use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ServiceType};
pub struct OllamaProvider { pub struct OllamaProvider {
http: reqwest::Client, http: reqwest::Client,
@@ -2,14 +2,14 @@ use std::sync::Arc;
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use crate::core::chatbot::openai::OpenAiClient; use crate::chatbot::openai::OpenAiClient;
use crate::core::llm::{LlmModelRecord, LlmProviderRecord}; use crate::llm::{LlmModelRecord, LlmProviderRecord};
use crate::core::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning}; use crate::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
use crate::core::transcribe::TranscribeModelRecord; use crate::transcribe::TranscribeModelRecord;
use crate::core::transcribe::openai_audio::OpenAiAudioTranscriber; use crate::transcribe::openai_audio::OpenAiAudioTranscriber;
use crate::core::tts::TtsModelRecord; use crate::tts::TtsModelRecord;
use crate::core::tts::openai_tts::OpenAiTtsSynthesiser; use crate::tts::openai_tts::OpenAiTtsSynthesiser;
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType}; use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
pub struct OpenAiProvider; pub struct OpenAiProvider;
@@ -58,7 +58,7 @@ impl ApiProvider for OpenAiProvider {
})()) })())
} }
fn build_tts(&self, record: &LlmProviderRecord, model: &TtsModelRecord) -> Option<Result<Arc<dyn crate::core::tts::TextToSpeech>>> { fn build_tts(&self, record: &LlmProviderRecord, model: &TtsModelRecord) -> Option<Result<Arc<dyn crate::tts::TextToSpeech>>> {
Some((|| { Some((|| {
let base_url = record.base_url.clone() let base_url = record.base_url.clone()
.unwrap_or_else(|| "https://api.openai.com/v1".to_string()); .unwrap_or_else(|| "https://api.openai.com/v1".to_string());
@@ -67,11 +67,11 @@ impl ApiProvider for OpenAiProvider {
Ok(Arc::new(OpenAiTtsSynthesiser::new( Ok(Arc::new(OpenAiTtsSynthesiser::new(
&model.name, base_url, api_key, &model.model_id, &model.name, base_url, api_key, &model.model_id,
model.voice_id.clone(), model.instructions.clone(), model.response_format.clone(), model.voice_id.clone(), model.instructions.clone(), model.response_format.clone(),
)) as Arc<dyn crate::core::tts::TextToSpeech>) )) as Arc<dyn crate::tts::TextToSpeech>)
})()) })())
} }
fn build_transcriber(&self, record: &LlmProviderRecord, model: &TranscribeModelRecord) -> Option<Result<Arc<dyn crate::core::transcribe::Transcribe>>> { fn build_transcriber(&self, record: &LlmProviderRecord, model: &TranscribeModelRecord) -> Option<Result<Arc<dyn crate::transcribe::Transcribe>>> {
Some((|| { Some((|| {
let base_url = record.base_url.clone() let base_url = record.base_url.clone()
.unwrap_or_else(|| "https://api.openai.com/v1".to_string()); .unwrap_or_else(|| "https://api.openai.com/v1".to_string());
@@ -79,7 +79,7 @@ impl ApiProvider for OpenAiProvider {
.with_context(|| format!("provider '{}': api_key required for open_ai", record.name))?; .with_context(|| format!("provider '{}': api_key required for open_ai", record.name))?;
Ok(Arc::new(OpenAiAudioTranscriber::new( Ok(Arc::new(OpenAiAudioTranscriber::new(
&model.name, base_url, api_key, &model.model_id, model.language.clone(), &model.name, base_url, api_key, &model.model_id, model.language.clone(),
)) as Arc<dyn crate::core::transcribe::Transcribe>) )) as Arc<dyn crate::transcribe::Transcribe>)
})()) })())
} }
@@ -2,16 +2,16 @@ use std::sync::Arc;
use anyhow::{Context, Result, anyhow}; use anyhow::{Context, Result, anyhow};
use crate::core::chatbot::openai::OpenAiClient; use crate::chatbot::openai::OpenAiClient;
use crate::core::image_generate::ImageGenerateModelRecord; use crate::image_generate::ImageGenerateModelRecord;
use crate::core::image_generate::openrouter_image::OpenRouterImageGenerator; use crate::image_generate::openrouter_image::OpenRouterImageGenerator;
use crate::core::llm::{LlmModelRecord, LlmProviderRecord}; use crate::llm::{LlmModelRecord, LlmProviderRecord};
use crate::core::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning}; use crate::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
use crate::core::transcribe::TranscribeModelRecord; use crate::transcribe::TranscribeModelRecord;
use crate::core::transcribe::openai_audio::OpenAiAudioTranscriber; use crate::transcribe::openai_audio::OpenAiAudioTranscriber;
use crate::core::tts::TtsModelRecord; use crate::tts::TtsModelRecord;
use crate::core::tts::openai_tts::OpenAiTtsSynthesiser; use crate::tts::openai_tts::OpenAiTtsSynthesiser;
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType}; use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
pub struct OpenRouterProvider { pub struct OpenRouterProvider {
http: reqwest::Client, http: reqwest::Client,
@@ -163,7 +163,7 @@ impl ApiProvider for OpenRouterProvider {
})()) })())
} }
fn build_tts(&self, record: &LlmProviderRecord, model: &TtsModelRecord) -> Option<Result<Arc<dyn crate::core::tts::TextToSpeech>>> { fn build_tts(&self, record: &LlmProviderRecord, model: &TtsModelRecord) -> Option<Result<Arc<dyn crate::tts::TextToSpeech>>> {
Some((|| { Some((|| {
let base_url = record.base_url.clone() let base_url = record.base_url.clone()
.unwrap_or_else(|| "https://openrouter.ai/api/v1".to_string()); .unwrap_or_else(|| "https://openrouter.ai/api/v1".to_string());
@@ -172,11 +172,11 @@ impl ApiProvider for OpenRouterProvider {
Ok(Arc::new(OpenAiTtsSynthesiser::new( Ok(Arc::new(OpenAiTtsSynthesiser::new(
&model.name, base_url, api_key, &model.model_id, &model.name, base_url, api_key, &model.model_id,
model.voice_id.clone(), model.instructions.clone(), model.response_format.clone(), model.voice_id.clone(), model.instructions.clone(), model.response_format.clone(),
)) as Arc<dyn crate::core::tts::TextToSpeech>) )) as Arc<dyn crate::tts::TextToSpeech>)
})()) })())
} }
fn build_transcriber(&self, record: &LlmProviderRecord, model: &TranscribeModelRecord) -> Option<Result<Arc<dyn crate::core::transcribe::Transcribe>>> { fn build_transcriber(&self, record: &LlmProviderRecord, model: &TranscribeModelRecord) -> Option<Result<Arc<dyn crate::transcribe::Transcribe>>> {
Some((|| { Some((|| {
let base_url = record.base_url.clone() let base_url = record.base_url.clone()
.unwrap_or_else(|| "https://openrouter.ai/api/v1".to_string()); .unwrap_or_else(|| "https://openrouter.ai/api/v1".to_string());
@@ -184,11 +184,11 @@ impl ApiProvider for OpenRouterProvider {
.with_context(|| format!("provider '{}': api_key required for openrouter", record.name))?; .with_context(|| format!("provider '{}': api_key required for openrouter", record.name))?;
Ok(Arc::new(OpenAiAudioTranscriber::new( Ok(Arc::new(OpenAiAudioTranscriber::new(
&model.name, base_url, api_key, &model.model_id, model.language.clone(), &model.name, base_url, api_key, &model.model_id, model.language.clone(),
)) as Arc<dyn crate::core::transcribe::Transcribe>) )) as Arc<dyn crate::transcribe::Transcribe>)
})()) })())
} }
fn build_image_generator(&self, record: &LlmProviderRecord, model: &ImageGenerateModelRecord) -> Option<Result<Arc<dyn crate::core::image_generate::ImageGenerate>>> { fn build_image_generator(&self, record: &LlmProviderRecord, model: &ImageGenerateModelRecord) -> Option<Result<Arc<dyn crate::image_generate::ImageGenerate>>> {
Some((|| { Some((|| {
let base_url = record.base_url.clone() let base_url = record.base_url.clone()
.unwrap_or_else(|| "https://openrouter.ai/api/v1".to_string()); .unwrap_or_else(|| "https://openrouter.ai/api/v1".to_string());
@@ -196,7 +196,7 @@ impl ApiProvider for OpenRouterProvider {
.with_context(|| format!("provider '{}': api_key required for openrouter", record.name))?; .with_context(|| format!("provider '{}': api_key required for openrouter", record.name))?;
Ok(Arc::new(OpenRouterImageGenerator::new( Ok(Arc::new(OpenRouterImageGenerator::new(
&model.name, base_url, api_key, &model.model_id, &model.name, base_url, api_key, &model.model_id,
)) as Arc<dyn crate::core::image_generate::ImageGenerate>) )) as Arc<dyn crate::image_generate::ImageGenerate>)
})()) })())
} }
@@ -2,10 +2,10 @@ use std::sync::Arc;
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use crate::core::chatbot::openai::OpenAiClient; use crate::chatbot::openai::OpenAiClient;
use crate::core::llm::{LlmModelRecord, LlmProviderRecord}; use crate::llm::{LlmModelRecord, LlmProviderRecord};
use crate::core::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning}; use crate::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType}; use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
/// Z.AI (Zhipu AI) — OpenAI-compatible GLM API. /// Z.AI (Zhipu AI) — OpenAI-compatible GLM API.
/// ///
@@ -11,7 +11,7 @@ use tokio::sync::mpsc;
use tokio_util::sync::CancellationToken; use tokio_util::sync::CancellationToken;
use tracing::{info, warn}; use tracing::{info, warn};
use crate::core::tools::ToolResult; use crate::tools::ToolResult;
pub use mcp_client::{ pub use mcp_client::{
ElicitationHandler, ElicitationHandler,
@@ -105,7 +105,7 @@ impl McpManager {
Some((source, payload)) => { Some((source, payload)) => {
let method = payload["method"].as_str().unwrap_or("unknown").to_string(); let method = payload["method"].as_str().unwrap_or("unknown").to_string();
let params = serde_json::to_string(&payload["params"]).unwrap_or_else(|_| "{}".to_string()); let params = serde_json::to_string(&payload["params"]).unwrap_or_else(|_| "{}".to_string());
match crate::core::db::mcp_events::insert(&pool, &source, &method, &params).await { match crate::db::mcp_events::insert(&pool, &source, &method, &params).await {
Ok(id) => info!("mcp_event stored: id={id} source={source} method={method}"), Ok(id) => info!("mcp_event stored: id={id} source={source} method={method}"),
Err(e) => warn!("mcp_events insert failed (source={source} method={method}): {e}"), Err(e) => warn!("mcp_events insert failed (source={source} method={method}): {e}"),
} }
@@ -116,7 +116,7 @@ impl McpManager {
} }
} }
fn cfg_from_row(row: &crate::core::db::mcp_servers::McpServerRow) -> McpServerConfig { fn cfg_from_row(row: &crate::db::mcp_servers::McpServerRow) -> McpServerConfig {
McpServerConfig { McpServerConfig {
name: row.name.clone(), name: row.name.clone(),
transport: match row.transport.as_str() { transport: match row.transport.as_str() {
@@ -155,7 +155,7 @@ impl McpManager {
} }
pub async fn initialize(&self) { pub async fn initialize(&self) {
let rows = match crate::core::db::mcp_servers::all_enabled(&self.pool).await { let rows = match crate::db::mcp_servers::all_enabled(&self.pool).await {
Ok(r) => r, Ok(r) => r,
Err(e) => { warn!("McpManager::initialize: failed to read DB: {e}"); return; } Err(e) => { warn!("McpManager::initialize: failed to read DB: {e}"); return; }
}; };
@@ -218,12 +218,12 @@ impl McpManager {
} }
} }
pub async fn register(&self, p: crate::core::db::mcp_servers::UpsertParams<'_>) -> Result<Vec<String>> { pub async fn register(&self, p: crate::db::mcp_servers::UpsertParams<'_>) -> Result<Vec<String>> {
let name = p.name.to_string(); let name = p.name.to_string();
crate::core::db::mcp_servers::upsert(&self.pool, p).await?; crate::db::mcp_servers::upsert(&self.pool, p).await?;
let rows = crate::core::db::mcp_servers::all_enabled(&self.pool).await?; let rows = crate::db::mcp_servers::all_enabled(&self.pool).await?;
let row = rows.into_iter().find(|r| r.name == name) let row = rows.into_iter().find(|r| r.name == name)
.ok_or_else(|| anyhow::anyhow!("register: server '{}' not found after upsert", name))?; .ok_or_else(|| anyhow::anyhow!("register: server '{}' not found after upsert", name))?;
let cfg = Self::cfg_from_row(&row); let cfg = Self::cfg_from_row(&row);
@@ -251,7 +251,7 @@ impl McpManager {
} }
pub async fn unregister(&self, name: &str) -> Result<()> { pub async fn unregister(&self, name: &str) -> Result<()> {
crate::core::db::mcp_servers::delete(&self.pool, name).await?; crate::db::mcp_servers::delete(&self.pool, name).await?;
self.servers.write().unwrap().remove(name); self.servers.write().unwrap().remove(name);
self.errors.write().unwrap().remove(name); self.errors.write().unwrap().remove(name);
self.descriptions.write().unwrap().remove(name); self.descriptions.write().unwrap().remove(name);
@@ -259,11 +259,11 @@ impl McpManager {
} }
pub async fn set_enabled(&self, name: &str, enabled: bool) -> Result<()> { pub async fn set_enabled(&self, name: &str, enabled: bool) -> Result<()> {
crate::core::db::mcp_servers::set_enabled(&self.pool, name, enabled).await crate::db::mcp_servers::set_enabled(&self.pool, name, enabled).await
} }
pub async fn list(&self) -> Result<Vec<McpServerInfo>> { pub async fn list(&self) -> Result<Vec<McpServerInfo>> {
let rows = crate::core::db::mcp_servers::all(&self.pool).await?; let rows = crate::db::mcp_servers::all(&self.pool).await?;
let servers = self.servers.read().unwrap(); let servers = self.servers.read().unwrap();
let errors = self.errors.read().unwrap(); let errors = self.errors.read().unwrap();
@@ -24,7 +24,7 @@ use tracing::{error, info};
pub use core_api::memory::Memory; pub use core_api::memory::Memory;
use crate::core::tools::Tool; use crate::tools::Tool;
// ── MemoryManager ───────────────────────────────────────────────────────────── // ── MemoryManager ─────────────────────────────────────────────────────────────
@@ -1,8 +1,8 @@
// The manager only ever handles `Arc<dyn Plugin>`. Naming a concrete plugin crate
// here would make the core depend on every plugin — and, through
// `plugin-transcribe-whisper-local`, on a C build — for no gain: the consumer
// constructs the plugin list and passes it to `Skald::new`.
pub use core_api::plugin::{Plugin, PluginContext, RouterFactory}; pub use core_api::plugin::{Plugin, PluginContext, RouterFactory};
pub use plugin_comfyui::ComfyUIPlugin;
#[cfg(feature = "whisper-local")]
pub use plugin_transcribe_whisper_local::WhisperLocalPlugin;
pub use plugin_telegram_bot::TelegramPlugin;
use std::collections::HashMap; use std::collections::HashMap;
use std::sync::{Arc, OnceLock}; use std::sync::{Arc, OnceLock};
@@ -19,8 +19,8 @@ use tokio::sync::Mutex;
use tokio::time::timeout; use tokio::time::timeout;
use tracing::{error, info, warn}; use tracing::{error, info, warn};
use crate::core::db::plugins as db; use crate::db::plugins as db;
use crate::core::skald::Skald; use crate::skald::Skald;
// ── Public plugin info (returned by list_items tool and REST API) ───────────── // ── Public plugin info (returned by list_items tool and REST API) ─────────────
@@ -238,7 +238,7 @@ impl PluginManager {
/// Toggle only the enabled flag, keeping existing config. /// Toggle only the enabled flag, keeping existing config.
pub async fn toggle(&self, id: &str, enabled: bool) -> Result<()> { pub async fn toggle(&self, id: &str, enabled: bool) -> Result<()> {
let row = db::get(&self.db, id).await? let row = db::get(&self.db, id).await?
.unwrap_or_else(|| crate::core::db::plugins::PluginRow { .unwrap_or_else(|| crate::db::plugins::PluginRow {
id: id.to_string(), id: id.to_string(),
enabled, enabled,
config: "{}".to_string(), config: "{}".to_string(),
@@ -329,6 +329,13 @@ impl PluginManager {
Ok(out) Ok(out)
} }
/// Every registered plugin, enabled or not. Lets the core ask each one for
/// its contributions (`Plugin::tools`, `Plugin::http_router`) without ever
/// naming a concrete plugin type.
pub fn all(&self) -> &[Arc<dyn Plugin>] {
&self.plugins
}
pub fn get_plugin_typed<T: Plugin + 'static>(&self, id: &str) -> Option<Arc<T>> { pub fn get_plugin_typed<T: Plugin + 'static>(&self, id: &str) -> Option<Arc<T>> {
self.plugins.iter() self.plugins.iter()
.find(|p| p.id() == id) .find(|p| p.id() == id)
@@ -5,8 +5,8 @@ use std::sync::Arc;
use anyhow::Result; use anyhow::Result;
use sqlx::SqlitePool; use sqlx::SqlitePool;
use crate::core::db::projects::{self, Project}; use crate::db::projects::{self, Project};
use crate::core::run_context::RunContext; use crate::run_context::RunContext;
pub struct ProjectManager { pub struct ProjectManager {
db: Arc<SqlitePool>, db: Arc<SqlitePool>,
@@ -7,9 +7,9 @@ use tracing::warn;
use core_api::system_bus::{SystemEvent, SystemEventBus}; use core_api::system_bus::{SystemEvent, SystemEventBus};
use crate::core::cron::TaskManager; use crate::cron::TaskManager;
use crate::core::db::{project_tickets, project_tickets::ProjectTicket, projects}; use crate::db::{project_tickets, project_tickets::ProjectTicket, projects};
use crate::core::run_context::RunContext; use crate::run_context::RunContext;
pub struct ProjectTicketManager { pub struct ProjectTicketManager {
db: Arc<SqlitePool>, db: Arc<SqlitePool>,
@@ -6,9 +6,9 @@ use serde::{Deserialize, Serialize};
use sqlx::SqlitePool; use sqlx::SqlitePool;
use tracing::info; use tracing::info;
pub use crate::core::db::tool_permission_groups::ToolPermissionGroup; pub use crate::db::tool_permission_groups::ToolPermissionGroup;
use crate::core::approval::{ApprovalManager, RuleAction}; use crate::approval::{ApprovalManager, RuleAction};
use crate::core::tools::fs::{canonicalize_for_policy, path_under}; use crate::tools::fs::{canonicalize_for_policy, path_under};
#[derive(Debug, Clone, Serialize, Deserialize, Default)] #[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct RunContext { pub struct RunContext {
@@ -113,7 +113,7 @@ impl RunContextManager {
/// Seeds the built-in "default" permission group and migrates legacy rules. /// Seeds the built-in "default" permission group and migrates legacy rules.
/// Safe to call at every startup (idempotent). /// Safe to call at every startup (idempotent).
pub async fn seed_defaults(&self) -> Result<()> { pub async fn seed_defaults(&self) -> Result<()> {
crate::core::db::tool_permission_groups::insert_or_ignore( crate::db::tool_permission_groups::insert_or_ignore(
&self.db, "default", "Default", Some("Built-in default permission group"), &self.db, "default", "Default", Some("Built-in default permission group"),
).await?; ).await?;
@@ -133,11 +133,11 @@ impl RunContextManager {
// ── ToolPermissionGroup CRUD ─────────────────────────────────────────────── // ── ToolPermissionGroup CRUD ───────────────────────────────────────────────
pub async fn list_groups(&self) -> Result<Vec<ToolPermissionGroup>> { pub async fn list_groups(&self) -> Result<Vec<ToolPermissionGroup>> {
crate::core::db::tool_permission_groups::list(&self.db).await crate::db::tool_permission_groups::list(&self.db).await
} }
pub async fn get_group(&self, id: &str) -> Result<Option<ToolPermissionGroup>> { pub async fn get_group(&self, id: &str) -> Result<Option<ToolPermissionGroup>> {
crate::core::db::tool_permission_groups::get(&self.db, id).await crate::db::tool_permission_groups::get(&self.db, id).await
} }
pub async fn create_group( pub async fn create_group(
@@ -149,7 +149,7 @@ impl RunContextManager {
if id == "default" { if id == "default" {
bail!("cannot create a permission group with reserved id 'default'"); bail!("cannot create a permission group with reserved id 'default'");
} }
crate::core::db::tool_permission_groups::insert(&self.db, id, name, description).await crate::db::tool_permission_groups::insert(&self.db, id, name, description).await
} }
pub async fn update_group( pub async fn update_group(
@@ -158,14 +158,14 @@ impl RunContextManager {
name: &str, name: &str,
description: Option<&str>, description: Option<&str>,
) -> Result<bool> { ) -> Result<bool> {
crate::core::db::tool_permission_groups::update(&self.db, id, name, description).await crate::db::tool_permission_groups::update(&self.db, id, name, description).await
} }
pub async fn delete_group(&self, id: &str) -> Result<bool> { pub async fn delete_group(&self, id: &str) -> Result<bool> {
if id == "default" { if id == "default" {
bail!("cannot delete the built-in 'default' permission group"); bail!("cannot delete the built-in 'default' permission group");
} }
crate::core::db::tool_permission_groups::delete(&self.db, id).await crate::db::tool_permission_groups::delete(&self.db, id).await
} }
/// Duplicates a permission group and all its rules atomically. /// Duplicates a permission group and all its rules atomically.
@@ -178,7 +178,7 @@ impl RunContextManager {
if new_id == "default" { if new_id == "default" {
bail!("cannot create a permission group with reserved id 'default'"); bail!("cannot create a permission group with reserved id 'default'");
} }
let source = crate::core::db::tool_permission_groups::get(&self.db, source_id).await? let source = crate::db::tool_permission_groups::get(&self.db, source_id).await?
.ok_or_else(|| anyhow::anyhow!("source group '{source_id}' not found"))?; .ok_or_else(|| anyhow::anyhow!("source group '{source_id}' not found"))?;
let mut tx = self.db.begin().await?; let mut tx = self.db.begin().await?;
@@ -1,4 +1,4 @@
use crate::core::provider::ServiceType; use crate::provider::ServiceType;
/// Light umbrella trait shared by all model managers. /// Light umbrella trait shared by all model managers.
/// Enables grouping managers generically (e.g. models-hub routing, diagnostics) /// Enables grouping managers generically (e.g. models-hub routing, diagnostics)
@@ -6,8 +6,8 @@ use tokio::sync::mpsc;
use tokio_util::sync::CancellationToken; use tokio_util::sync::CancellationToken;
use tracing::info; use tracing::info;
use crate::core::db::{chat_history, chat_llm_tools, chat_sessions_stack, scratchpad, stack_mcp_grants}; use crate::db::{chat_history, chat_llm_tools, chat_sessions_stack, scratchpad, stack_mcp_grants};
use crate::core::events::ServerEvent; use crate::events::ServerEvent;
use super::{ChatSessionHandler, MAX_AGENT_DEPTH, TurnOutcome}; use super::{ChatSessionHandler, MAX_AGENT_DEPTH, TurnOutcome};
use super::emitter::TurnEmitter; use super::emitter::TurnEmitter;
@@ -41,7 +41,7 @@ impl ChatSessionHandler {
// Only `task` agents are dispatchable: this rejects `chat` (e.g. `main`, // Only `task` agents are dispatchable: this rejects `chat` (e.g. `main`,
// `project-coordinator`) and `system` (e.g. `tic`) agents, and surfaces a // `project-coordinator`) and `system` (e.g. `tic`) agents, and surfaces a
// not-found error for unknown ids — all in one gate. // not-found error for unknown ids — all in one gate.
let target_meta = crate::core::agents::load_task_meta(target_id) let target_meta = crate::agents::load_task_meta(target_id)
.map_err(|e| anyhow::anyhow!("dispatch_sub_agent: {e}"))?; .map_err(|e| anyhow::anyhow!("dispatch_sub_agent: {e}"))?;
let parent_frame = chat_sessions_stack::find_by_id(pool, parent_stack_id).await? let parent_frame = chat_sessions_stack::find_by_id(pool, parent_stack_id).await?
@@ -91,7 +91,7 @@ impl ChatSessionHandler {
{ {
let group_id = self.tool_group_id().await; let group_id = self.tool_group_id().await;
let gid = group_id.as_deref().unwrap_or("default"); let gid = group_id.as_deref().unwrap_or("default");
let group_rules = crate::core::db::approval_rules::list_for_group( let group_rules = crate::db::approval_rules::list_for_group(
pool, Some(gid), pool, Some(gid),
).await.unwrap_or_default(); ).await.unwrap_or_default();
child_config.base_tool_defs.retain(|def| { child_config.base_tool_defs.retain(|def| {
@@ -107,7 +107,7 @@ impl ChatSessionHandler {
let mcp_clone = Arc::clone(&self.mcp); let mcp_clone = Arc::clone(&self.mcp);
let grants_clone = Arc::clone(&active_mcp_grants); let grants_clone = Arc::clone(&active_mcp_grants);
let activate_tool = crate::core::tools::activate_tools::ActivateTools { let activate_tool = crate::tools::activate_tools::ActivateTools {
pool: pool_clone, pool: pool_clone,
session_id, session_id,
stack_id: Some(stack_id), stack_id: Some(stack_id),
@@ -118,7 +118,7 @@ impl ChatSessionHandler {
child_config.interface_tools.push(InterfaceTool { child_config.interface_tools.push(InterfaceTool {
definition: activate_tools_tool_def(), definition: activate_tools_tool_def(),
handler: Arc::new(move |args| -> ToolFuture { handler: Arc::new(move |args| -> ToolFuture {
use crate::core::tools::Tool as _; use crate::tools::Tool as _;
let tool = Arc::clone(&activate_tool); let tool = Arc::clone(&activate_tool);
Box::pin(async move { Box::pin(async move {
tokio::task::spawn_blocking(move || tool.execute(args)) tokio::task::spawn_blocking(move || tool.execute(args))
@@ -3,7 +3,7 @@ use tracing::debug;
use super::ChatSessionHandler; use super::ChatSessionHandler;
use super::emitter::TurnEmitter; use super::emitter::TurnEmitter;
use crate::core::tools::{is_file_write_tool, tool_names as tn}; use crate::tools::{is_file_write_tool, tool_names as tn};
impl ChatSessionHandler { impl ChatSessionHandler {
/// Emits the appropriate frontend approval event for the given tool call. /// Emits the appropriate frontend approval event for the given tool call.
@@ -55,7 +55,7 @@ impl ChatSessionHandler {
/// Reads the current content of a file from disk (for diff generation in PendingWrite events). /// Reads the current content of a file from disk (for diff generation in PendingWrite events).
pub(super) async fn read_current_content(&self, path: &str) -> Option<String> { pub(super) async fn read_current_content(&self, path: &str) -> Option<String> {
let abs = crate::core::tools::fs::resolve(path).ok()?; let abs = crate::tools::fs::resolve(path).ok()?;
tokio::fs::read_to_string(&abs).await.ok() tokio::fs::read_to_string(&abs).await.ok()
} }
@@ -3,7 +3,7 @@ use std::sync::{Arc, RwLock};
use serde_json::Value; use serde_json::Value;
use crate::core::tools::tool_names as tn; use crate::tools::tool_names as tn;
use super::{ChatSessionHandler, update_scratchpad_tool_def, write_todos_tool_def}; use super::{ChatSessionHandler, update_scratchpad_tool_def, write_todos_tool_def};
use super::interface_tools::{AgentRunConfig, InterfaceTool, ToolFuture}; use super::interface_tools::{AgentRunConfig, InterfaceTool, ToolFuture};
@@ -47,7 +47,7 @@ impl ChatSessionHandler {
mut interface_tools: Vec<InterfaceTool>, mut interface_tools: Vec<InterfaceTool>,
system_substitutions: HashMap<String, String>, system_substitutions: HashMap<String, String>,
) -> anyhow::Result<AgentRunConfig> { ) -> anyhow::Result<AgentRunConfig> {
let meta = crate::core::agents::load_meta(&self.agent_id).ok(); let meta = crate::agents::load_meta(&self.agent_id).ok();
let (key, _) = self.llm_manager.resolve( let (key, _) = self.llm_manager.resolve(
client_name.as_deref(), client_name.as_deref(),
meta.as_ref().and_then(|m| m.scope.as_deref()), meta.as_ref().and_then(|m| m.scope.as_deref()),
@@ -68,7 +68,7 @@ impl ChatSessionHandler {
// Inbox alone. // Inbox alone.
let is_system = meta let is_system = meta
.as_ref() .as_ref()
.map(|m| m.agent_type == crate::core::agents::AgentType::System) .map(|m| m.agent_type == crate::agents::AgentType::System)
.unwrap_or(false); .unwrap_or(false);
if !is_system { if !is_system {
base_tool_defs.push(super::ask_user_clarification_tool_def()); base_tool_defs.push(super::ask_user_clarification_tool_def());
@@ -114,7 +114,7 @@ impl ChatSessionHandler {
{ {
let group_id = self.tool_group_id().await; let group_id = self.tool_group_id().await;
let gid = group_id.as_deref().unwrap_or("default"); let gid = group_id.as_deref().unwrap_or("default");
let group_rules = crate::core::db::approval_rules::list_for_group( let group_rules = crate::db::approval_rules::list_for_group(
&self.db, Some(gid), &self.db, Some(gid),
).await.unwrap_or_default(); ).await.unwrap_or_default();
let visible = |def: &Value| { let visible = |def: &Value| {
@@ -130,7 +130,7 @@ impl ChatSessionHandler {
// Load persisted session grants from DB (MCP server names and/or the reserved // Load persisted session grants from DB (MCP server names and/or the reserved
// `config` keyword), then inject `activate_tools` so the LLM can activate // `config` keyword), then inject `activate_tools` so the LLM can activate
// additional groups on demand. // additional groups on demand.
let persisted = crate::core::db::session_mcp_grants::list_for_session( let persisted = crate::db::session_mcp_grants::list_for_session(
&self.db, self.session_id, &self.db, self.session_id,
).await.unwrap_or_default(); ).await.unwrap_or_default();
@@ -143,7 +143,7 @@ impl ChatSessionHandler {
let mcp_clone = Arc::clone(&self.mcp); let mcp_clone = Arc::clone(&self.mcp);
let grants_clone = Arc::clone(&active_mcp_grants); let grants_clone = Arc::clone(&active_mcp_grants);
let activate_tool = crate::core::tools::activate_tools::ActivateTools { let activate_tool = crate::tools::activate_tools::ActivateTools {
pool: pool_clone, pool: pool_clone,
session_id, session_id,
stack_id: None, stack_id: None,
@@ -155,7 +155,7 @@ impl ChatSessionHandler {
interface_tools.push(InterfaceTool { interface_tools.push(InterfaceTool {
definition: activate_tools_tool_def(), definition: activate_tools_tool_def(),
handler: Arc::new(move |args| -> ToolFuture { handler: Arc::new(move |args| -> ToolFuture {
use crate::core::tools::Tool as _; use crate::tools::Tool as _;
let tool = Arc::clone(&activate_tool); let tool = Arc::clone(&activate_tool);
Box::pin(async move { Box::pin(async move {
tokio::task::spawn_blocking(move || tool.execute(args)) tokio::task::spawn_blocking(move || tool.execute(args))
@@ -10,8 +10,8 @@ use tokio::sync::mpsc;
use tokio_util::sync::CancellationToken; use tokio_util::sync::CancellationToken;
use tracing::warn; use tracing::warn;
use crate::core::events::ServerEvent; use crate::events::ServerEvent;
use crate::core::tools::{drive_execution, tool_names as tn, ExecutionOutcome, ToolResult}; use crate::tools::{drive_execution, tool_names as tn, ExecutionOutcome, ToolResult};
use super::ChatSessionHandler; use super::ChatSessionHandler;
use super::interface_tools::AgentRunConfig; use super::interface_tools::AgentRunConfig;
@@ -16,7 +16,7 @@ use tokio::sync::mpsc;
use core_api::message_meta::Attachment; use core_api::message_meta::Attachment;
use crate::core::events::ServerEvent; use crate::events::ServerEvent;
/// Borrows the per-turn event sender and emits typed [`ServerEvent`]s. /// Borrows the per-turn event sender and emits typed [`ServerEvent`]s.
pub(super) struct TurnEmitter<'a> { pub(super) struct TurnEmitter<'a> {
@@ -11,10 +11,10 @@ use std::sync::atomic::Ordering;
use serde_json::Value; use serde_json::Value;
use tracing::{info, warn}; use tracing::{info, warn};
use crate::core::approval::GateResult; use crate::approval::GateResult;
use crate::core::db::chat_llm_tools; use crate::db::chat_llm_tools;
use crate::core::run_context::RunContext; use crate::run_context::RunContext;
use crate::core::tools::{is_file_read_tool, is_file_write_tool}; use crate::tools::{is_file_read_tool, is_file_write_tool};
use super::{ApprovalDecision, ChatSessionHandler}; use super::{ApprovalDecision, ChatSessionHandler};
use super::emitter::TurnEmitter; use super::emitter::TurnEmitter;
@@ -3,9 +3,9 @@ use std::sync::{Arc, RwLock};
use serde_json::Value; use serde_json::Value;
use crate::core::mcp::McpManager; use crate::mcp::McpManager;
use crate::core::tools::Tool; use crate::tools::Tool;
use crate::core::tools::tool_names as tn; use crate::tools::tool_names as tn;
pub use core_api::interface_tool::{InterfaceTool, ToolFuture}; pub use core_api::interface_tool::{InterfaceTool, ToolFuture};
@@ -89,7 +89,7 @@ impl AgentRunConfig {
// MCP servers: include tools for the granted server names. // MCP servers: include tools for the granted server names.
let servers: Vec<String> = granted.iter() let servers: Vec<String> = granted.iter()
.filter(|n| n.as_str() != crate::core::tools::tool_names::CONFIG_GROUP) .filter(|n| n.as_str() != crate::tools::tool_names::CONFIG_GROUP)
.cloned() .cloned()
.collect(); .collect();
if !servers.is_empty() { if !servers.is_empty() {
@@ -101,7 +101,7 @@ impl AgentRunConfig {
} }
// `config` group: include the built-in Config-category tools on demand. // `config` group: include the built-in Config-category tools on demand.
if granted.contains(crate::core::tools::tool_names::CONFIG_GROUP) { if granted.contains(crate::tools::tool_names::CONFIG_GROUP) {
defs.extend(self.config_tool_defs.iter().cloned()); defs.extend(self.config_tool_defs.iter().cloned());
} }
@@ -12,8 +12,8 @@ use serde_json::Value;
use tokio_util::sync::CancellationToken; use tokio_util::sync::CancellationToken;
use tracing::{error, warn}; use tracing::{error, warn};
use crate::core::chatbot::{ChatOptions, LlmTurn}; use crate::chatbot::{ChatOptions, LlmTurn};
use crate::core::llm::{LlmEntry, LlmStrength}; use crate::llm::{LlmEntry, LlmStrength};
use super::ChatSessionHandler; use super::ChatSessionHandler;
use super::emitter::TurnEmitter; use super::emitter::TurnEmitter;
@@ -3,12 +3,12 @@ use tokio::sync::mpsc;
use tokio_util::sync::CancellationToken; use tokio_util::sync::CancellationToken;
use tracing::{debug, info, trace}; use tracing::{debug, info, trace};
use crate::core::tools::tool_names as tn; use crate::tools::tool_names as tn;
use crate::core::chat_event_bus::ToolCallEvent; use crate::chat_event_bus::ToolCallEvent;
use crate::core::chatbot::{LlmTurn, ToolCall}; use crate::chatbot::{LlmTurn, ToolCall};
use crate::core::db::{chat_history, chat_llm_tools}; use crate::db::{chat_history, chat_llm_tools};
use crate::core::events::ServerEvent; use crate::events::ServerEvent;
use crate::core::tools::{ use crate::tools::{
ExecutionOutcome, SimpleExecution, ToolDescriptionLength, ToolExecution, ToolResult, ExecutionOutcome, SimpleExecution, ToolDescriptionLength, ToolExecution, ToolResult,
}; };
use futures::stream::{self, StreamExt}; use futures::stream::{self, StreamExt};
@@ -67,7 +67,7 @@ impl ChatSessionHandler {
.ok_or_else(|| anyhow::anyhow!("LLM client '{}' not found", cur_name))?; .ok_or_else(|| anyhow::anyhow!("LLM client '{}' not found", cur_name))?;
// Scope/strength needed for fallback re-selection. // Scope/strength needed for fallback re-selection.
let meta = crate::core::agents::load_meta(&config.agent_id).ok(); let meta = crate::agents::load_meta(&config.agent_id).ok();
let req_scope = meta.as_ref().and_then(|m| m.scope.as_deref()).map(str::to_string); let req_scope = meta.as_ref().and_then(|m| m.scope.as_deref()).map(str::to_string);
let req_strength = meta.as_ref().and_then(|m| m.strength); let req_strength = meta.as_ref().and_then(|m| m.strength);
@@ -144,7 +144,6 @@ impl ChatSessionHandler {
pool, stack_id, &chat_history::Role::Assistant, &resp.content, false, pool, stack_id, &chat_history::Role::Assistant, &resp.content, false,
resp.reasoning_content.as_deref(), resp.reasoning_content.as_deref(),
).await?; ).await?;
chat_history::set_model_db_id(pool, message_id, cur_llm.model_db_id).await?;
if let (Some(i), Some(o)) = (resp.input_tokens, resp.output_tokens) { if let (Some(i), Some(o)) = (resp.input_tokens, resp.output_tokens) {
chat_history::set_usage(pool, message_id, i, o, 0, resp.cost).await?; chat_history::set_usage(pool, message_id, i, o, 0, resp.cost).await?;
} }
@@ -163,7 +162,6 @@ impl ChatSessionHandler {
pool, stack_id, &chat_history::Role::Assistant, &assistant_text, false, pool, stack_id, &chat_history::Role::Assistant, &assistant_text, false,
reasoning_content.as_deref(), reasoning_content.as_deref(),
).await?; ).await?;
chat_history::set_model_db_id(pool, message_id, cur_llm.model_db_id).await?;
if let (Some(i), Some(o)) = (input_tokens, output_tokens) { if let (Some(i), Some(o)) = (input_tokens, output_tokens) {
chat_history::set_usage(pool, message_id, i, o, 0, cost).await?; chat_history::set_usage(pool, message_id, i, o, 0, cost).await?;
} }
@@ -418,7 +416,7 @@ impl ChatSessionHandler {
return Some(tool.run(args)); return Some(tool.run(args));
} }
// MCP tools (`server::tool`). Clone the Arc so the work future is 'static. // MCP tools (`server::tool`). Clone the Arc so the work future is 'static.
if let Some((srv, mcp_tool)) = crate::core::mcp::parse_mcp_tool_name(name) { if let Some((srv, mcp_tool)) = crate::mcp::parse_mcp_tool_name(name) {
let mcp = std::sync::Arc::clone(&self.mcp); let mcp = std::sync::Arc::clone(&self.mcp);
let srv = srv.to_string(); let srv = srv.to_string();
let mcp_tool = mcp_tool.to_string(); let mcp_tool = mcp_tool.to_string();
@@ -4,11 +4,11 @@ use std::sync::Arc;
use serde_json::{Value, json}; use serde_json::{Value, json};
use sqlx::SqlitePool; use sqlx::SqlitePool;
use crate::core::compactor::{ContextCompactor, SUMMARY_PREFIX}; use crate::compactor::{ContextCompactor, SUMMARY_PREFIX};
use crate::core::config::DatetimeConfig; use crate::config::DatetimeConfig;
use crate::core::db::{chat_history, chat_llm_tools, chat_summaries}; use crate::db::{chat_history, chat_llm_tools, chat_summaries};
use crate::core::mcp::McpManager; use crate::mcp::McpManager;
use crate::core::tools::tool_names as tn; use crate::tools::tool_names as tn;
/// Registry of installed skills, relative to Skald's process cwd. Injected into agents /// Registry of installed skills, relative to Skald's process cwd. Injected into agents
/// that have `inject_skills` enabled (the default). /// that have `inject_skills` enabled (the default).
@@ -84,9 +84,9 @@ impl MessageBuilder {
let pool = &*self.pool; let pool = &*self.pool;
// ── 1. Static system message ────────────────────────────────────────── // ── 1. Static system message ──────────────────────────────────────────
let mut static_content = crate::core::agents::load_prompt(agent_id)?; let mut static_content = crate::agents::load_prompt(agent_id)?;
let meta = crate::core::agents::load_meta(agent_id)?; let meta = crate::agents::load_meta(agent_id)?;
if !meta.inject_memory.is_empty() { if !meta.inject_memory.is_empty() {
static_content.push_str( static_content.push_str(
"\n\n---\nThe following memory files have been loaded automatically. \ "\n\n---\nThe following memory files have been loaded automatically. \
@@ -154,7 +154,7 @@ impl MessageBuilder {
let mut out = vec![static_msg]; let mut out = vec![static_msg];
// ── 2. Scratchpad system message (before conversation) ──────────────── // ── 2. Scratchpad system message (before conversation) ────────────────
let scratch = crate::core::db::scratchpad::for_session(pool, self.session_id).await?; let scratch = crate::db::scratchpad::for_session(pool, self.session_id).await?;
if !scratch.is_empty() { if !scratch.is_empty() {
let mut s = String::from( let mut s = String::from(
"<scratchpad>\n \ "<scratchpad>\n \
@@ -409,7 +409,7 @@ impl MessageBuilder {
let wd = self.working_directory.clone() let wd = self.working_directory.clone()
.unwrap_or_else(|| std::env::current_dir().unwrap_or_default()); .unwrap_or_else(|| std::env::current_dir().unwrap_or_default());
let expanded = mem_path.replace("$WD", &wd.display().to_string()); let expanded = mem_path.replace("$WD", &wd.display().to_string());
let abs = crate::core::tools::fs::resolve(&expanded) let abs = crate::tools::fs::resolve(&expanded)
.unwrap_or_else(|_| std::path::PathBuf::from(&expanded)); .unwrap_or_else(|_| std::path::PathBuf::from(&expanded));
let display = match abs.strip_prefix(&wd) { let display = match abs.strip_prefix(&wd) {
Ok(rel) => rel.to_string_lossy().into_owned(), Ok(rel) => rel.to_string_lossy().into_owned(),
@@ -10,22 +10,22 @@ use tokio_util::sync::CancellationToken;
use tracing::{error, info, trace, warn}; use tracing::{error, info, trace, warn};
use crate::core::approval::ApprovalManager; use crate::approval::ApprovalManager;
use crate::core::run_context::RunContext; use crate::run_context::RunContext;
use crate::core::tools::tool_names as tn; use crate::tools::tool_names as tn;
use crate::core::chat_event_bus::{ChatEvent, ChatEventBus, ChatEventRole}; use crate::chat_event_bus::{ChatEvent, ChatEventBus, ChatEventRole};
use crate::core::clarification::ClarificationManager; use crate::clarification::ClarificationManager;
use crate::core::compactor::ContextCompactor; use crate::compactor::ContextCompactor;
use crate::core::config::DatetimeConfig; use crate::config::DatetimeConfig;
use crate::core::db::{chat_history, chat_sessions_stack}; use crate::db::{chat_history, chat_sessions_stack};
use crate::core::events::ServerEvent; use crate::events::ServerEvent;
use core_api::message_meta::MessageMetadata; use core_api::message_meta::MessageMetadata;
use crate::core::llm::LlmManager; use crate::llm::LlmManager;
use crate::core::mcp::McpManager; use crate::mcp::McpManager;
use crate::core::image_generate::ImageGeneratorManager; use crate::image_generate::ImageGeneratorManager;
use crate::core::memory::MemoryManager; use crate::memory::MemoryManager;
use crate::core::tool_discovery::ToolDiscovery; use crate::tool_discovery::ToolDiscovery;
use crate::core::tools::ToolRegistry; use crate::tools::ToolRegistry;
mod approval; mod approval;
mod agent_dispatch; mod agent_dispatch;
@@ -102,7 +102,7 @@ pub(super) enum TurnOutcome {
output_tokens: Option<u32>, output_tokens: Option<u32>,
truncated: bool, truncated: bool,
/// All tool calls executed during this turn, across all rounds. /// All tool calls executed during this turn, across all rounds.
tool_calls: Vec<crate::core::chat_event_bus::ToolCallEvent>, tool_calls: Vec<crate::chat_event_bus::ToolCallEvent>,
}, },
Cancelled, Cancelled,
Exhausted, Exhausted,
@@ -8,9 +8,9 @@
use serde_json::Value; use serde_json::Value;
use tracing::{debug, info, warn}; use tracing::{debug, info, warn};
use crate::core::chat_event_bus::ToolCallEvent; use crate::chat_event_bus::ToolCallEvent;
use crate::core::db::chat_llm_tools; use crate::db::chat_llm_tools;
use crate::core::tools::{is_file_write_tool, ExecutionOutcome}; use crate::tools::{is_file_write_tool, ExecutionOutcome};
use super::ChatSessionHandler; use super::ChatSessionHandler;
use super::emitter::TurnEmitter; use super::emitter::TurnEmitter;
@@ -52,7 +52,7 @@ impl ChatSessionHandler {
if is_file_write_tool(tool_name) if is_file_write_tool(tool_name)
&& let Some(p) = args["path"].as_str() && let Some(p) = args["path"].as_str()
{ {
em.file_changed(crate::core::approval::normalize_path(p)).await; em.file_changed(crate::approval::normalize_path(p)).await;
} }
acc.push(ToolCallEvent { acc.push(ToolCallEvent {
name: tool_name.to_string(), name: tool_name.to_string(),
@@ -3,9 +3,9 @@ use tokio::sync::mpsc;
use tokio_util::sync::CancellationToken; use tokio_util::sync::CancellationToken;
use tracing::{error, info, warn}; use tracing::{error, info, warn};
use crate::core::db::{chat_history, chat_llm_tools, chat_sessions_stack}; use crate::db::{chat_history, chat_llm_tools, chat_sessions_stack};
use crate::core::events::ServerEvent; use crate::events::ServerEvent;
use crate::core::tools::{ToolDescriptionLength, ToolResult, tool_names as tn}; use crate::tools::{ToolDescriptionLength, ToolResult, tool_names as tn};
use super::{ChatSessionHandler, TurnOutcome}; use super::{ChatSessionHandler, TurnOutcome};
use super::emitter::TurnEmitter; use super::emitter::TurnEmitter;
@@ -18,7 +18,7 @@ impl ChatSessionHandler {
/// Used by the REST `resolve` endpoint and by `resume_pending_tools`. /// Used by the REST `resolve` endpoint and by `resume_pending_tools`.
/// Does NOT update the DB — caller is responsible for `complete` / `fail`. /// Does NOT update the DB — caller is responsible for `complete` / `fail`.
pub async fn execute_tool(&self, name: &str, args: Value) -> anyhow::Result<ToolResult> { pub async fn execute_tool(&self, name: &str, args: Value) -> anyhow::Result<ToolResult> {
if let Some((srv, mcp_tool)) = crate::core::mcp::parse_mcp_tool_name(name) { if let Some((srv, mcp_tool)) = crate::mcp::parse_mcp_tool_name(name) {
return self.mcp.call(srv, mcp_tool, args).await; return self.mcp.call(srv, mcp_tool, args).await;
} }
self.tools.dispatch(name, args).await.map(ToolResult::Text) self.tools.dispatch(name, args).await.map(ToolResult::Text)
@@ -365,7 +365,7 @@ fn shallowest_parallel_depth(active: &[chat_sessions_stack::SessionStack]) -> Op
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::shallowest_parallel_depth; use super::shallowest_parallel_depth;
use crate::core::db::chat_sessions_stack::SessionStack; use crate::db::chat_sessions_stack::SessionStack;
fn frame(id: i64, depth: i64, parent: Option<i64>) -> SessionStack { fn frame(id: i64, depth: i64, parent: Option<i64>) -> SessionStack {
SessionStack { id, agent_id: "agent".into(), depth, parent_tool_call_id: parent } SessionStack { id, agent_id: "agent".into(), depth, parent_tool_call_id: parent }
@@ -4,19 +4,19 @@ use std::sync::Arc;
use sqlx::SqlitePool; use sqlx::SqlitePool;
use tokio::sync::Mutex; use tokio::sync::Mutex;
use crate::core::approval::ApprovalManager; use crate::approval::ApprovalManager;
use crate::core::chat_event_bus::ChatEventBus; use crate::chat_event_bus::ChatEventBus;
use crate::core::clarification::ClarificationManager; use crate::clarification::ClarificationManager;
use crate::core::compactor::ContextCompactor; use crate::compactor::ContextCompactor;
use crate::core::config::DatetimeConfig; use crate::config::DatetimeConfig;
use crate::core::db::{chat_sessions, chat_sessions_stack}; use crate::db::{chat_sessions, chat_sessions_stack};
use crate::core::llm::LlmManager; use crate::llm::LlmManager;
use crate::core::mcp::McpManager; use crate::mcp::McpManager;
use crate::core::image_generate::ImageGeneratorManager; use crate::image_generate::ImageGeneratorManager;
use crate::core::memory::MemoryManager; use crate::memory::MemoryManager;
use crate::core::run_context::{RunContext, RunContextManager}; use crate::run_context::{RunContext, RunContextManager};
use crate::core::tool_discovery::ToolDiscovery; use crate::tool_discovery::ToolDiscovery;
use crate::core::tools::ToolRegistry; use crate::tools::ToolRegistry;
use super::handler::ChatSessionHandler; use super::handler::ChatSessionHandler;
@@ -1,8 +1,12 @@
//! The logical API surface of `Skald`: one accessor per manager, named after the //! The logical API surface of `Skald`: one accessor per manager, named after the
//! historical field, delegating into the domain bundle that now owns it. This is //! historical field, delegating into the domain bundle that now owns it. This is
//! the intentional surface consumers (frontend handlers, plugin context) use — the //! the intentional surface consumers (frontend handlers, plugin context) use — the
//! bundles themselves stay internal. Promote this block to a `SkaldApi` trait if/ //! bundles themselves stay internal.
//! when `src/core/` is lifted into its own crate. //!
//! Now that the core is its own crate, this is a real boundary rather than a
//! convention: everything here is `pub` because the `skald` binary lives outside,
//! and everything not here is unreachable from it. Promote the block to a
//! `SkaldApi` trait if the shells ever need to mock it.
use std::sync::Arc; use std::sync::Arc;
@@ -13,43 +17,45 @@ use tokio_util::sync::CancellationToken;
use core_api::remote::RemoteAccess; use core_api::remote::RemoteAccess;
use core_api::system_bus::SystemEventBus; use core_api::system_bus::SystemEventBus;
use crate::core::approval::ApprovalManager; use crate::approval::ApprovalManager;
use crate::core::chat_event_bus::ChatEventBus; use crate::chat_event_bus::ChatEventBus;
use crate::core::chat_hub::ChatHub; use crate::chat_hub::ChatHub;
use crate::core::clarification::ClarificationManager; use crate::clarification::ClarificationManager;
use crate::core::command::LlmCommandManager; use crate::command::LlmCommandManager;
use crate::core::config_store::GlobalConfigManager; use crate::config_store::GlobalConfigManager;
use crate::core::cron::TaskManager; use crate::cron::TaskManager;
use crate::core::elicitation::ElicitationManager; use crate::elicitation::ElicitationManager;
use crate::core::image_generate::ImageGeneratorManager; use crate::image_generate::ImageGeneratorManager;
use crate::core::inbox::Inbox; use crate::inbox::Inbox;
use crate::core::latex::LatexCompiler; use crate::latex::LatexCompiler;
use crate::core::llm::LlmManager; use crate::llm::LlmManager;
use crate::core::location::LocationManager; use crate::location::LocationManager;
use crate::core::mcp::McpManager; use crate::mcp::McpManager;
use crate::core::memory::MemoryManager; use crate::memory::MemoryManager;
use crate::core::plugin::PluginManager; use crate::plugin::PluginManager;
use crate::core::projects::tickets::ProjectTicketManager; use crate::projects::tickets::ProjectTicketManager;
use crate::core::projects::ProjectManager; use crate::projects::ProjectManager;
use crate::core::provider::ProviderRegistry; use crate::provider::ProviderRegistry;
use crate::core::run_context::RunContextManager; use crate::run_context::RunContextManager;
use crate::core::secrets::SecretsStore; use crate::secrets::SecretsStore;
use crate::core::session::manager::ChatSessionManager; use crate::session::manager::ChatSessionManager;
use crate::core::tic::TicManager; use crate::tic::TicManager;
use crate::core::tool_catalog::ToolCatalog; use crate::tool_catalog::ToolCatalog;
use crate::core::tools::ToolRegistry; use crate::tools::ToolRegistry;
use crate::core::transcribe::TranscribeManager; use crate::transcribe::TranscribeManager;
use crate::core::tts::TtsManager; use crate::tts::TtsManager;
use crate::users::UserManager;
use super::Skald; use super::Skald;
impl Skald { impl Skald {
// Runtime / cross-cutting // Runtime / cross-cutting
pub(crate) fn db(&self) -> &Arc<SqlitePool> { &self.rt.db } pub fn db(&self) -> &Arc<SqlitePool> { &self.rt.db }
pub fn users(&self) -> &Arc<UserManager> { &self.rt.users }
pub fn config(&self) -> &Arc<GlobalConfigManager> { &self.rt.config } pub fn config(&self) -> &Arc<GlobalConfigManager> { &self.rt.config }
pub fn config_properties(&self) -> &[core_api::ConfigSet] { &self.rt.config_properties } pub fn config_properties(&self) -> &[core_api::ConfigSet] { &self.rt.config_properties }
pub(crate) fn system_bus(&self) -> &Arc<SystemEventBus> { &self.rt.system_bus } pub fn system_bus(&self) -> &Arc<SystemEventBus> { &self.rt.system_bus }
pub(crate) fn event_bus(&self) -> &Arc<ChatEventBus> { &self.rt.event_bus } pub fn event_bus(&self) -> &Arc<ChatEventBus> { &self.rt.event_bus }
pub fn shutdown_token(&self) -> &CancellationToken { &self.rt.shutdown_token } pub fn shutdown_token(&self) -> &CancellationToken { &self.rt.shutdown_token }
// Models // Models
@@ -14,35 +14,35 @@ use tracing::{debug, info, warn};
use core_api::remote::RemoteAccess; use core_api::remote::RemoteAccess;
use crate::core::approval::ApprovalManager; use crate::approval::ApprovalManager;
use crate::core::chat_hub::ChatHub; use crate::chat_hub::ChatHub;
use crate::core::clarification::ClarificationManager; use crate::clarification::ClarificationManager;
use crate::core::command::LlmCommandManager; use crate::command::LlmCommandManager;
use crate::core::compactor::ContextCompactor; use crate::compactor::ContextCompactor;
use crate::core::config::{CoreConfig, DatetimeConfig}; use crate::config::{CoreConfig, DatetimeConfig};
use crate::core::cron::TaskManager; use crate::cron::TaskManager;
use crate::core::elicitation::ElicitationManager; use crate::elicitation::ElicitationManager;
use crate::core::image_generate::ImageGeneratorManager; use crate::image_generate::ImageGeneratorManager;
use crate::core::inbox::Inbox; use crate::inbox::Inbox;
use crate::core::latex::LatexCompiler; use crate::latex::LatexCompiler;
use crate::core::llm::LlmManager; use crate::llm::LlmManager;
use crate::core::location::LocationManager; use crate::location::LocationManager;
use crate::core::mcp::McpManager; use crate::mcp::McpManager;
use crate::core::memory::MemoryManager; use crate::memory::MemoryManager;
use crate::core::plugin::PluginManager; use crate::plugin::PluginManager;
use crate::core::projects::tickets::ProjectTicketManager; use crate::projects::tickets::ProjectTicketManager;
use crate::core::projects::ProjectManager; use crate::projects::ProjectManager;
use crate::core::provider::ProviderRegistry; use crate::provider::ProviderRegistry;
use crate::core::run_context::RunContextManager; use crate::run_context::RunContextManager;
use crate::core::secrets::SecretsStore; use crate::secrets::SecretsStore;
use crate::core::session::handler::{DEFAULT_MAX_PARALLEL_SUBAGENTS, DEFAULT_MAX_TOOL_ROUNDS}; use crate::session::handler::{DEFAULT_MAX_PARALLEL_SUBAGENTS, DEFAULT_MAX_TOOL_ROUNDS};
use crate::core::session::manager::ChatSessionManager; use crate::session::manager::ChatSessionManager;
use crate::core::tic::TicManager; use crate::tic::TicManager;
use crate::core::tool_catalog::ToolCatalog; use crate::tool_catalog::ToolCatalog;
use crate::core::tool_discovery::ToolDiscovery; use crate::tool_discovery::ToolDiscovery;
use crate::core::tools::ToolRegistry; use crate::tools::ToolRegistry;
use crate::core::transcribe::TranscribeManager; use crate::transcribe::TranscribeManager;
use crate::core::tts::TtsManager; use crate::tts::TtsManager;
use tokio::sync::RwLock; use tokio::sync::RwLock;
@@ -62,18 +62,18 @@ pub(super) struct Models {
impl Models { impl Models {
pub(super) async fn build(rt: &Runtime, config: &CoreConfig) -> Result<Self> { pub(super) async fn build(rt: &Runtime, config: &CoreConfig) -> Result<Self> {
let mut provider_registry = ProviderRegistry::new(Arc::clone(&rt.system_bus)); let mut provider_registry = ProviderRegistry::new(Arc::clone(&rt.system_bus));
provider_registry.register_builtin(crate::core::llm::providers::openai::OpenAiProvider); provider_registry.register_builtin(crate::llm::providers::openai::OpenAiProvider);
provider_registry.register_builtin(crate::core::llm::providers::anthropic::AnthropicProvider::new()); provider_registry.register_builtin(crate::llm::providers::anthropic::AnthropicProvider::new());
provider_registry.register_builtin(crate::core::llm::providers::openrouter::OpenRouterProvider::new()); provider_registry.register_builtin(crate::llm::providers::openrouter::OpenRouterProvider::new());
provider_registry.register_builtin(crate::core::llm::providers::ollama::OllamaProvider::new()); provider_registry.register_builtin(crate::llm::providers::ollama::OllamaProvider::new());
provider_registry.register_builtin(crate::core::llm::providers::lm_studio::LmStudioProvider::new()); provider_registry.register_builtin(crate::llm::providers::lm_studio::LmStudioProvider::new());
provider_registry.register_builtin(crate::core::llm::providers::deepseek::DeepSeekProvider::new()); provider_registry.register_builtin(crate::llm::providers::deepseek::DeepSeekProvider::new());
provider_registry.register_builtin(crate::core::llm::providers::zai::ZaiProvider::new()); provider_registry.register_builtin(crate::llm::providers::zai::ZaiProvider::new());
let provider_registry = Arc::new(provider_registry); let provider_registry = Arc::new(provider_registry);
info!("provider registry ready ({} built-in providers)", provider_registry.all().len()); info!("provider registry ready ({} built-in providers)", provider_registry.all().len());
let log_flags = config.llm.requests_log.as_ref().filter(|r| r.enabled).map(|r| { let log_flags = config.llm.requests_log.as_ref().filter(|r| r.enabled).map(|r| {
use crate::core::chatbot::logging::LogSaveFlags; use crate::chatbot::logging::LogSaveFlags;
LogSaveFlags { LogSaveFlags {
request_payload: r.request_payload_save, request_payload: r.request_payload_save,
response_payload: r.response_payload_save, response_payload: r.response_payload_save,
@@ -214,35 +214,38 @@ impl Tools {
/// per interactive session by `ChatHub::send_message`. /// per interactive session by `ChatHub::send_message`.
pub(super) fn build(integrations: &Integrations, tasks: &Tasks, models: &Models) -> Self { pub(super) fn build(integrations: &Integrations, tasks: &Tasks, models: &Models) -> Self {
let mut tool_registry = ToolRegistry::new(); let mut tool_registry = ToolRegistry::new();
crate::core::tools::fs::register_all(&mut tool_registry); crate::tools::fs::register_all(&mut tool_registry);
tool_registry.register(crate::core::tools::ast_outline::AstOutline::new()); tool_registry.register(crate::tools::ast_outline::AstOutline::new());
tool_registry.register(crate::core::tools::exec::ExecuteCmd); tool_registry.register(crate::tools::exec::ExecuteCmd);
tool_registry.register(crate::core::tools::read_notification::ReadNotification); tool_registry.register(crate::tools::read_notification::ReadNotification);
tool_registry.register(crate::core::tools::restart::Restart); tool_registry.register(crate::tools::restart::Restart);
// Unified listing / toggling across mcp, plugins, cron (+ agents for list). // Unified listing / toggling across mcp, plugins, cron (+ agents for list).
tool_registry.register(crate::core::tools::list_items::ListItems::new( tool_registry.register(crate::tools::list_items::ListItems::new(
Arc::clone(&integrations.mcp), Arc::clone(&integrations.plugin_manager), Arc::clone(&tasks.cron))); Arc::clone(&integrations.mcp), Arc::clone(&integrations.plugin_manager), Arc::clone(&tasks.cron)));
tool_registry.register(crate::core::tools::toggle_item::ToggleItem::new( tool_registry.register(crate::tools::toggle_item::ToggleItem::new(
Arc::clone(&integrations.mcp), Arc::clone(&integrations.plugin_manager), Arc::clone(&tasks.cron))); Arc::clone(&integrations.mcp), Arc::clone(&integrations.plugin_manager), Arc::clone(&tasks.cron)));
tool_registry.register(crate::core::tools::register_mcp::RegisterMcp::new(Arc::clone(&integrations.mcp))); tool_registry.register(crate::tools::register_mcp::RegisterMcp::new(Arc::clone(&integrations.mcp)));
tool_registry.register(crate::core::tools::register_mcp::DeleteMcp::new(Arc::clone(&integrations.mcp))); tool_registry.register(crate::tools::register_mcp::DeleteMcp::new(Arc::clone(&integrations.mcp)));
tool_registry.register(crate::core::tools::cron_jobs::DeleteCronJob(Arc::clone(&tasks.cron))); tool_registry.register(crate::tools::cron_jobs::DeleteCronJob(Arc::clone(&tasks.cron)));
tool_registry.register(crate::core::tools::set_secret::SetSecret(Arc::clone(&models.secrets))); tool_registry.register(crate::tools::set_secret::SetSecret(Arc::clone(&models.secrets)));
tool_registry.register(crate::core::tools::list_secrets::ListSecrets(Arc::clone(&models.secrets))); tool_registry.register(crate::tools::list_secrets::ListSecrets(Arc::clone(&models.secrets)));
tool_registry.register(crate::core::tools::configure_plugin::ConfigurePlugin(Arc::clone(&integrations.plugin_manager))); tool_registry.register(crate::tools::configure_plugin::ConfigurePlugin(Arc::clone(&integrations.plugin_manager)));
// Mobile-connector control tools (plugin.md §11). The plugin Arc itself // Tools contributed by plugins (plugin.md §11), via `Plugin::tools()`.
// implements `RelayAgent`; we look it up and bind the tools to it. The tools // The core never names a plugin crate: each one hands over whatever tools
// call into the plugin lazily, so registering them before the plugin's // it wants, bound to its own handle. They are built before the plugins'
// runloop starts is fine (they fail gracefully while stopped). // runloops start, so they must tolerate being called while stopped.
if let Some(mc) = integrations.plugin_manager for plugin in integrations.plugin_manager.all() {
.get_plugin_typed::<plugin_mobile_connector::MobileConnectorPlugin>("mobile-connector") let id = plugin.id().to_string();
{ let tools = Arc::clone(plugin).tools();
let agent: Arc<dyn plugin_mobile_connector::RelayAgent> = mc; if tools.is_empty() {
for tool in plugin_mobile_connector::mobile_tools(agent) { continue;
}
let n = tools.len();
for tool in tools {
tool_registry.register_arc(tool); tool_registry.register_arc(tool);
} }
info!("mobile-connector tools registered"); info!(plugin = %id, count = n, "plugin tools registered");
} }
debug!("tool registry built"); debug!("tool registry built");
@@ -92,5 +92,8 @@ impl Skald {
self.rt.shutdown_token.cancel(); self.rt.shutdown_token.cancel();
self.rt.supervisor.join_all(tokio::time::Duration::from_secs(10)).await; self.rt.supervisor.join_all(tokio::time::Duration::from_secs(10)).await;
self.integrations.plugin_manager.stop_all().await; self.integrations.plugin_manager.stop_all().await;
// Last: every user key leaves RAM. A restarted box is opaque again until
// each user unlocks their own database (§9).
self.rt.users.lock_all().await;
} }
} }
@@ -17,13 +17,17 @@ use tracing::info;
use core_api::events::GlobalEvent; use core_api::events::GlobalEvent;
use core_api::system_bus::SystemEventBus; use core_api::system_bus::SystemEventBus;
use crate::core::chat_event_bus::ChatEventBus; use crate::chat_event_bus::ChatEventBus;
use crate::core::config_store::GlobalConfigManager; use crate::config_store::GlobalConfigManager;
use crate::users::UserManager;
use super::supervisor::TaskSupervisor; use super::supervisor::TaskSupervisor;
pub(super) struct Runtime { pub(super) struct Runtime {
/// The registry pool (`system.db`). Still the only pool anything reads or
/// writes: nothing has moved to per-user pools yet. `users` owns those.
pub(super) db: Arc<SqlitePool>, pub(super) db: Arc<SqlitePool>,
pub(super) users: Arc<UserManager>,
pub(super) config: Arc<GlobalConfigManager>, pub(super) config: Arc<GlobalConfigManager>,
pub(super) config_properties: Vec<core_api::ConfigSet>, pub(super) config_properties: Vec<core_api::ConfigSet>,
pub(super) system_bus: Arc<SystemEventBus>, pub(super) system_bus: Arc<SystemEventBus>,
@@ -41,6 +45,8 @@ impl Runtime {
pub(super) fn bootstrap(pool: Arc<SqlitePool>) -> Self { pub(super) fn bootstrap(pool: Arc<SqlitePool>) -> Self {
let config = Arc::new(GlobalConfigManager::new(Arc::clone(&pool))); let config = Arc::new(GlobalConfigManager::new(Arc::clone(&pool)));
let users = Arc::new(UserManager::new(Arc::clone(&pool)));
let system_bus = Arc::new(SystemEventBus::new()); let system_bus = Arc::new(SystemEventBus::new());
info!("system event bus ready"); info!("system event bus ready");
@@ -51,8 +57,9 @@ impl Runtime {
Runtime { Runtime {
db: pool, db: pool,
users,
config, config,
config_properties: vec![crate::core::tic::config_set()], config_properties: vec![crate::tic::config_set()],
system_bus, system_bus,
event_bus, event_bus,
global_tx, global_tx,

Some files were not shown because too many files have changed in this diff Show More