feat(users): UserManager with per-user SQLCipher, and extract skald-core crate
Two changes developed together in one session; they share the same module
structure (db/mod.rs, the core lib root) and only compile together, so they
land as one commit.
## UserManager + per-user encryption (§9/§11)
New `users::UserManager`: owns the system.db pool plus a map
`userid -> SqlitePool` of unlocked databases. The pool *is* the unlock token —
its connect options carry the DEK as SQLCipher's raw key, so an open pool means
the key is in RAM until restart and dropping it re-locks (§9). Knows nothing
about cookies.
New `crypto` module: envelope encryption. A random 256-bit DEK encrypts
`{userid}.db`; `users.database_password` holds it sealed with AES-256-GCM under
`Argon2id(password, salt)`. The AEAD tag is the password verifier — one
derivation both authenticates and yields the key, so encrypted users store no
second hash. Cleartext users store the Argon2id output directly, compared in
constant time. Argon2 runs in spawn_blocking behind a 2-permit semaphore
(256 MiB per derivation).
- SQLCipher via `libsqlite3-sys` `bundled-sqlcipher-vendored-openssl`, pinned
<0.38 so it unifies with the one sqlx-sqlite links (a newer copy would apply
the feature to a SQLite sqlx never uses). OpenSSL is vendored and static, so
the binary stays self-contained.
- Schema split into `create_registry_tables` (instance-wide, no user key) and
`create_owner_tables` (one owner's content, identical in every file). No FK in
the owner bucket may reach the registry — enforced by a standalone test.
Dropped `chat_history.model_db_id` (write-only, and the only registry-crossing
key); moved `projects`/`project_tickets` into the owner bucket.
- Provisioning invariant: the file is written before the row, deleted after it,
so a crash leaves an orphan file, never a user without a database. `open_db`
never creates: a missing file is an error, not a silent empty database.
Not consumed yet: no login, call sites still use the shared system.db pool.
## Extract crates/skald-core
The headless core moves out of `src/` into its own crate; `skald` (server) and
the coming `skald-setup` are shells around it. Two dependencies on the shell
were inverted rather than dragged along, so the core names neither Tauri nor any
concrete plugin:
- `Plugin::tools(self: Arc<Self>)` — plugins contribute tools through this hook
(sibling of `http_router`), so the core no longer downcasts to
`MobileConnectorPlugin`.
- `tools::restart::set_restart_handler` — the desktop shell installs its
teardown-and-respawn; the core defaults to the supervisor exit code. The core
loses its `desktop` feature.
- `boot`'s stdout formatter moves to the binary (`src/boot_format.rs`); the core
only emits tracing events.
All 79 core tests pass; the binary boots and serves in a clean directory, and
the mobile-connector tools still register through the new hook.
This commit is contained in:
@@ -33,39 +33,58 @@ Domain words are allowed only in seed data, preset labels, UI copy and positioni
|
||||
|
||||
### Current state
|
||||
|
||||
Single-user Skald plus a `users` table (`src/core/db/users.rs`). Next step is `UserManager` (§11). The multi-user split of the database has not started; `Runtime` still carries one shared `Arc<SqlitePool>`.
|
||||
`UserManager` (§11) exists and works — `crates/skald-core/src/users/mod.rs`, with real per-user SQLCipher encryption (§4). It is **not consumed yet**: there is no login, and `Runtime` still hands every call site the one shared `Arc<SqlitePool>` on `system.db`, so chats still land in that file's owner tables. The next step is migrating those call sites to `pool_of`, and only then deciding where the owner-without-a-user lives (see blueprint §19).
|
||||
|
||||
Direction of travel, decided but not yet executed: strip the **power-user surface** (self-rewriting, arbitrary shell, dev-agent suite, ticket system) and move to a **binary-first** layout — the app is built once and run from a compiled binary, not executed from its own source tree.
|
||||
|
||||
## Workspace layout
|
||||
|
||||
The application core is the `skald-core` crate; the binaries are **shells** around it.
|
||||
|
||||
| Crate | Role |
|
||||
| ---- | ---- |
|
||||
| `crates/skald-core/` | Storage, identity, crypto, LLM stack, tools, MCP, sessions. Knows nothing about what runs it: no Tauri, no HTTP server, and **no concrete plugin crate** — `PluginManager` only ever sees `Arc<dyn Plugin>` from `core-api` |
|
||||
| `skald` (root, `src/`) | The server shell: `main.rs`, the Axum `frontend/`, the Tauri `desktop/`, `config.rs`. Constructs the plugin list and hands it to `Skald::new` |
|
||||
| `crates/core-api/` | The contracts both sides share: `Plugin`, `Tool`, event buses, provider types |
|
||||
|
||||
Two rules keep the boundary real, and both are enforced by the compiler:
|
||||
|
||||
- **The core never names a plugin.** A plugin contributes tools through `Plugin::tools(self: Arc<Self>)` — the sibling of `http_router()` — so nothing in the core has to downcast to a concrete type. Naming one would drag every plugin in the tree into the core, including a C build via `plugin-transcribe-whisper-local`.
|
||||
- **The core never learns about the process shell.** The `restart` tool defaults to the supervisor protocol (`exit(-1)`); a shell with different needs installs `tools::restart::set_restart_handler` at startup. The Tauri shell installs teardown-and-respawn there. This is why `skald-core` has no `desktop` feature.
|
||||
|
||||
`skald_core::boot` emits curated startup lines on the `boot` tracing target; each shell decides how to render them (`src/boot_format.rs` here). The core says what happened, never how it looks.
|
||||
|
||||
## Key modules
|
||||
|
||||
| Path | Role |
|
||||
| ---- | ---- |
|
||||
| `src/main.rs` | Thin entry point: tracing → `Skald::new` → `WebFrontend::start` → shutdown. Branches on the `desktop` feature: under `--features desktop` enters `desktop::run()` (Tauri event loop) instead of blocking on a tokio runtime. Exposes `run_backend()` / `shutdown_backend()` shared by both entry points |
|
||||
| `src/desktop/mod.rs` | Tauri shell — **only compiled under `--features desktop`**. Builds the system-tray icon + menu (`Open` / `Quit`), creates the main `WebviewWindow` (URL = `http://127.0.0.1:{config.port}`), spawns the backend on Tauri's shared tokio runtime, handles graceful shutdown. Holds the `OnceLock<AppHandle>` that the `restart` tool reads from. See [docs/desktop.md](docs/desktop.md) |
|
||||
| `src/core/skald/` | `Skald` — headless application core. `mod.rs` (struct + staged `new()` / `shutdown()`), `runtime.rs` (cross-cutting `Runtime` context), `bundles.rs` (8 domain bundles + `build()`), `wiring.rs` (`wire()` + `spawn_background()`), `supervisor.rs` (`TaskSupervisor`), `accessors.rs` (per-manager accessor facade — the API surface the frontend uses) |
|
||||
| `src/core/session/handler/` | Core LLM loop — `mod.rs`, `llm_loop.rs` (`run_agent_turn`), `agent_dispatch.rs`, `dispatcher.rs`, `approval.rs`, `resume.rs`, `messages.rs`, `config.rs`, `interface_tools.rs` |
|
||||
| `src/core/session/manager.rs` | Creates/retrieves `ChatSessionHandler` per session |
|
||||
| `src/core/chat_hub/` | `ChatHub`: broadcast events to all connected WS clients |
|
||||
| `src/core/chat_event_bus.rs` | Global async bus for cross-session events |
|
||||
| `src/core/agents.rs` | Discovers agents from `agents/*/`, loads meta + system prompt |
|
||||
| `src/core/tools/` | Built-in tools: `exec`, `restart`, `list_agents`, `fs/*`, `notify`, `ast_outline`, `image_generate`, MCP tools, plugin tools, cron tools |
|
||||
| `src/core/tool_catalog.rs` | `ToolCatalog`: unified tool listing façade (wraps ToolRegistry + McpManager) |
|
||||
| `src/core/events.rs` | `ServerEvent` enum streamed over WebSocket to the frontend |
|
||||
| `src/core/db/` | sqlx SQLite — see below |
|
||||
| `src/desktop/mod.rs` | Tauri shell — **only compiled under `--features desktop`**. Builds the system-tray icon + menu (`Open` / `Quit`), creates the main `WebviewWindow` (URL = `http://127.0.0.1:{config.port}`), spawns the backend on Tauri's shared tokio runtime, handles graceful shutdown. Holds the `OnceLock<AppHandle>`, and installs the core's restart handler. See [docs/desktop.md](docs/desktop.md) |
|
||||
| `crates/skald-core/src/skald/` | `Skald` — headless application core. `mod.rs` (struct + staged `new()` / `shutdown()`), `runtime.rs` (cross-cutting `Runtime` context), `bundles.rs` (8 domain bundles + `build()`), `wiring.rs` (`wire()` + `spawn_background()`), `supervisor.rs` (`TaskSupervisor`), `accessors.rs` (per-manager accessor facade — the API surface the frontend uses) |
|
||||
| `crates/skald-core/src/session/handler/` | Core LLM loop — `mod.rs`, `llm_loop.rs` (`run_agent_turn`), `agent_dispatch.rs`, `dispatcher.rs`, `approval.rs`, `resume.rs`, `messages.rs`, `config.rs`, `interface_tools.rs` |
|
||||
| `crates/skald-core/src/session/manager.rs` | Creates/retrieves `ChatSessionHandler` per session |
|
||||
| `crates/skald-core/src/chat_hub/` | `ChatHub`: broadcast events to all connected WS clients |
|
||||
| `crates/skald-core/src/chat_event_bus.rs` | Global async bus for cross-session events |
|
||||
| `crates/skald-core/src/agents.rs` | Discovers agents from `agents/*/`, loads meta + system prompt |
|
||||
| `crates/skald-core/src/tools/` | Built-in tools: `exec`, `restart`, `list_agents`, `fs/*`, `notify`, `ast_outline`, `image_generate`, MCP tools, plugin tools, cron tools |
|
||||
| `crates/skald-core/src/tool_catalog.rs` | `ToolCatalog`: unified tool listing façade (wraps ToolRegistry + McpManager) |
|
||||
| `crates/skald-core/src/events.rs` | `ServerEvent` enum streamed over WebSocket to the frontend |
|
||||
| `crates/skald-core/src/db/` | sqlx SQLite — see below |
|
||||
| `crates/skald-core/src/users/` | `UserManager` (§11): user directory CRUD on `system.db`, credential check, and the map `userid → SqlitePool` of **unlocked** databases. The pool *is* the unlock token — its connect options carry the DEK as SQLCipher's raw key, so an open pool means the key is in RAM (§9) and dropping it re-locks. Knows nothing about cookies: whatever maps an HTTP session to a user id sits above it |
|
||||
| `crates/skald-core/src/crypto/` | Envelope encryption (§4/§5.1). A random 256-bit DEK encrypts `{userid}.db`; `users.database_password` holds it sealed with AES-256-GCM under `Argon2id(password, salt)`. **The AEAD tag is the password verifier** — one derivation both authenticates and yields the key, and no second hash sits in the admin-readable DB. Cleartext users store the Argon2id output directly, compared constant-time. Argon2 runs in `spawn_blocking` behind a 2-permit semaphore (256 MiB per derivation) |
|
||||
| `src/config.rs` | Loads `config.yml`; LLM clients, strength/use_cases, data root. Also hosts `bootstrap_data_dir()` — under the `desktop` feature, relocates the process cwd to a per-user data dir when running inside a `.app` bundle (no-op in dev mode and headless mode) |
|
||||
| `src/core/mcp/` | MCP client manager (connects to external MCP servers) |
|
||||
| `src/core/plugin/` | Plugin system: discovery, enable/disable, tool registration |
|
||||
| `src/core/cron/` | Scheduled job runner |
|
||||
| `src/core/compactor.rs` | Context compaction (summarises history when token budget exceeded) |
|
||||
| `src/core/approval/` | Approval rules engine |
|
||||
| `src/core/clarification/` | `ClarificationManager`: background-session question/answer |
|
||||
| `src/core/elicitation/` | `ElicitationManager` + bridge: MCP server-initiated input (`elicitation/create`), surfaced in the Inbox; secrets never logged/persisted |
|
||||
| `src/core/inbox.rs` | `Inbox`: unified façade for pending approvals + clarifications + elicitations (wraps ApprovalManager, ClarificationManager, ElicitationManager) |
|
||||
| `src/core/llm/` | LLM client abstraction (OpenAI-compat, Anthropic, Ollama…) |
|
||||
| `src/core/transcribe/` | Transcription providers |
|
||||
| `src/core/image_generate/` | Image generation providers |
|
||||
| `src/core/memory/` | Agent memory tools |
|
||||
| `crates/skald-core/src/mcp/` | MCP client manager (connects to external MCP servers) |
|
||||
| `crates/skald-core/src/plugin/` | Plugin system: discovery, enable/disable, tool registration |
|
||||
| `crates/skald-core/src/cron/` | Scheduled job runner |
|
||||
| `crates/skald-core/src/compactor.rs` | Context compaction (summarises history when token budget exceeded) |
|
||||
| `crates/skald-core/src/approval/` | Approval rules engine |
|
||||
| `crates/skald-core/src/clarification/` | `ClarificationManager`: background-session question/answer |
|
||||
| `crates/skald-core/src/elicitation/` | `ElicitationManager` + bridge: MCP server-initiated input (`elicitation/create`), surfaced in the Inbox; secrets never logged/persisted |
|
||||
| `crates/skald-core/src/inbox.rs` | `Inbox`: unified façade for pending approvals + clarifications + elicitations (wraps ApprovalManager, ClarificationManager, ElicitationManager) |
|
||||
| `crates/skald-core/src/llm/` | LLM client abstraction (OpenAI-compat, Anthropic, Ollama…) |
|
||||
| `crates/skald-core/src/transcribe/` | Transcription providers |
|
||||
| `crates/skald-core/src/image_generate/` | Image generation providers |
|
||||
| `crates/skald-core/src/memory/` | Agent memory tools |
|
||||
| `src/frontend/mod.rs` | `WebFrontend`: wires router_factory, starts plugins, runs Axum |
|
||||
| `src/frontend/server.rs` | Axum router, static file serving |
|
||||
| `src/frontend/api/` | HTTP + WebSocket handlers — `State<Arc<Skald>>` |
|
||||
@@ -73,11 +92,18 @@ Direction of travel, decided but not yet executed: strip the **power-user surfac
|
||||
|
||||
## DB tables (sqlx SQLite)
|
||||
|
||||
One file, `database/system.db` — the path is a constant (`core::db::SYSTEM_DB_PATH`), **not** configurable. `init_pool` creates the directory; SQLite only creates the file. Per-user `database/{userid}.db` files are the next step (§5.1).
|
||||
`database/system.db` — the path is a constant (`core::db::SYSTEM_DB_PATH`), **not** configurable. `init_system_pool` creates the directory; SQLite only creates the file. Per-user files are `database/{userid}.db`, created by `UserManager::register_user` and encrypted with SQLCipher.
|
||||
|
||||
`users`, `chat_sessions`, `chat_sessions_stack`, `chat_history`, `chat_llm_tools`, `chat_summaries`, `llm_requests`, `llm_providers`, `llm_models`, `transcribe_models`, `tts_models`, `image_generate_models`, `scheduled_jobs`, `job_runs`, `mcp_servers`, `mcp_events`, `plugins`, `approval_rules`, `tool_permission_groups`, `sources`, `secrets`, `config`, `session_scratchpad`, `session_mcp_grants`, `stack_mcp_grants`, `known_tools`, `projects`, `project_tickets`
|
||||
The schema is split into two buckets (§5.1), and the split is the point:
|
||||
|
||||
`users` (`src/core/db/users.rs`) holds the directory plus auth material. It lives in the system DB, which the box owner can read, so it must never store anything that derives a user's key. `Credentials` is an enum mirroring the table's `CHECK`: an encrypted user carries a **wrapped DEK** (whose AEAD tag *is* the password verifier — hence no `password_hash`); a cleartext user carries an ordinary verifier, or none. `User` is deliberately not `Serialize` and its `Debug` redacts key material — use `User::summary()` for anything leaving the process. `role_id` has no foreign key yet: sqlx enables `PRAGMA foreign_keys`, so referencing the not-yet-existing `roles` table would fail every insert.
|
||||
- **`create_registry_tables`** — instance-wide, readable without any user key: `users`, `llm_providers`, `llm_models`, `transcribe_models`, `tts_models`, `image_generate_models`, `plugins`, `approval_rules`, `tool_permission_groups`, `config`, `known_tools`, `llm_requests`.
|
||||
- **`create_owner_tables`** — one owner's content, **identical schema in every file that has it**: `chat_sessions`, `chat_sessions_stack`, `chat_history`, `chat_llm_tools`, `chat_summaries`, `session_scratchpad`, `session_mcp_grants`, `stack_mcp_grants`, `scheduled_jobs`, `job_runs`, `mcp_servers`, `mcp_events`, `sources`, `secrets`, `projects`, `project_tickets`.
|
||||
|
||||
**No foreign key in the owner bucket may point at a registry table.** SQLite cannot enforce a key across files, not even through `ATTACH`, and sqlx turns on `PRAGMA foreign_keys`: the `CREATE TABLE` succeeds and every `INSERT` fails. `db::tests::owner_tables_stand_alone_with_foreign_keys_on` enforces this by running the owner schema against a database holding nothing else, then inserting a row into each table. Two keys crossed and were fixed: `chat_history.model_db_id` (dropped — write-only, and `llm_requests.model_name` already records the model) and `project_tickets.job_id` (fixed by moving `projects`/`project_tickets` into the owner bucket).
|
||||
|
||||
`system.db` currently gets **both** bucket functions, because nothing has migrated to per-user pools yet. That is transitional.
|
||||
|
||||
`users` (`crates/skald-core/src/db/users.rs`) holds the directory plus auth material. It lives in the system DB, which the box owner can read, so it must never store anything that derives a user's key. `Credentials` is an enum mirroring the table's `CHECK`: an encrypted user carries a **wrapped DEK** (whose AEAD tag *is* the password verifier — hence no `password_hash`); a cleartext user carries an ordinary verifier, or none. `User` is deliberately not `Serialize` and its `Debug` redacts key material — use `User::summary()` for anything leaving the process. `role_id` has no foreign key yet: sqlx enables `PRAGMA foreign_keys`, so referencing the not-yet-existing `roles` table would fail every insert.
|
||||
|
||||
## Sub-agent system
|
||||
|
||||
@@ -102,18 +128,18 @@ The rule engine `ApprovalManager::check` returns `Allow`/`Deny`/`Require` per to
|
||||
|
||||
Resolution is **source-agnostic**: the WS + Inbox paths resolve by `request_id`; the inline chat card resolves by the durable `tool_call_id` via `POST /api/tools/:tool_call_id/resolve` (`resolve_tool` in `src/frontend/api/sessions.rs`), which derives the owning session from the tool call's own stack row — never a hardcoded source. Live pending cards fire the `oneshot`; post-restart they execute directly on the owning session. See `docs/approval/`.
|
||||
|
||||
**Tool visibility in the Security-groups UI** (`GET /api/approval/tools`): tools injected outside the `ToolRegistry` (interface/plugin/provider tools) would otherwise be un-configurable. `ToolCatalog::list_all()` covers registry tools + a static `synthetic_tools()` list of core interface tools; everything else is captured by `src/core/tool_discovery.rs` (`ToolDiscovery`), which taps `all_tool_defs()` in `llm_loop.rs` each round and upserts every offered tool into the `known_tools` table (in-memory seen-set guard → background DB write). `list_tools` merges `known_tools` (deduped, `category: "dynamic"`) so any tool offered at least once becomes gate-able. Drift-proof by construction; core never hardcodes plugin tool names.
|
||||
**Tool visibility in the Security-groups UI** (`GET /api/approval/tools`): tools injected outside the `ToolRegistry` (interface/plugin/provider tools) would otherwise be un-configurable. `ToolCatalog::list_all()` covers registry tools + a static `synthetic_tools()` list of core interface tools; everything else is captured by `crates/skald-core/src/tool_discovery.rs` (`ToolDiscovery`), which taps `all_tool_defs()` in `llm_loop.rs` each round and upserts every offered tool into the `known_tools` table (in-memory seen-set guard → background DB write). `list_tools` merges `known_tools` (deduped, `category: "dynamic"`) so any tool offered at least once becomes gate-able. Drift-proof by construction; core never hardcodes plugin tool names.
|
||||
|
||||
## Restart
|
||||
|
||||
`restart` **no longer rebuilds anything** — neither mode compiles.
|
||||
|
||||
- **Headless** (default): `restart` calls `libc::_exit(-1)` (= exit code 255); `run.sh` re-executes the same binary *by path*.
|
||||
- **Desktop** (`--features desktop`): Tauri cleanup + respawn of the bundled binary + `exit(0)`.
|
||||
- **Headless** (default): no handler installed, so `restart` calls `libc::_exit(-1)` (= exit code 255); `run.sh` re-executes the same binary *by path*.
|
||||
- **Desktop** (`--features desktop`): the Tauri shell installs a handler via `tools::restart::set_restart_handler` — cleanup + respawn of the bundled binary + `exit(0)`. The core does not know Tauri exists.
|
||||
|
||||
Use it to pick up `config.yml` / database changes, which are only read at startup. To load new **code**: `./build.sh`, then restart — the supervisor picks up the new binary on the next loop, since `build.sh` installs it with an atomic rename.
|
||||
|
||||
> `restart`'s tool description in `src/core/tools/restart.rs` still tells the model it triggers a `cargo build`. That is stale and must be fixed, along with `run.bat` (still `cargo run`).
|
||||
> `run.bat` is still stale (`cargo run`) and must be fixed.
|
||||
|
||||
## Build & run
|
||||
|
||||
|
||||
Generated
+100
-22
@@ -111,6 +111,18 @@ dependencies = [
|
||||
"object",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "argon2"
|
||||
version = "0.5.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3c3610892ee6e0cbce8ae2700349fcf8f98adb0dbfbee85aec3c9179d29cc072"
|
||||
dependencies = [
|
||||
"base64ct",
|
||||
"blake2",
|
||||
"cpufeatures 0.2.17",
|
||||
"password-hash",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "arrayvec"
|
||||
version = "0.7.6"
|
||||
@@ -2990,11 +3002,12 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "libsqlite3-sys"
|
||||
version = "0.30.1"
|
||||
version = "0.37.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2e99fb7a497b1e3339bc746195567ed8d3e24945ecd636e3619d20b9de9e9149"
|
||||
checksum = "b1f111c8c41e7c61a49cd34e44c7619462967221a6443b0ec299e0ac30cfb9b1"
|
||||
dependencies = [
|
||||
"cc",
|
||||
"openssl-sys",
|
||||
"pkg-config",
|
||||
"vcpkg",
|
||||
]
|
||||
@@ -3620,6 +3633,28 @@ version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7c87def4c32ab89d880effc9e097653c8da5d6ef28e6b539d313baaacfbafcbe"
|
||||
|
||||
[[package]]
|
||||
name = "openssl-src"
|
||||
version = "300.6.1+3.6.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "46eb8fb9fb3b61ce1c0f8a026c4c1a0714d3a9e138e7fbde78753ce2babc3846"
|
||||
dependencies = [
|
||||
"cc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "openssl-sys"
|
||||
version = "0.9.117"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b47e7e6bb2c38cd930d25a23b40fa52e068c10e85f3e03a7f5ba5aaca5713695"
|
||||
dependencies = [
|
||||
"cc",
|
||||
"libc",
|
||||
"openssl-src",
|
||||
"pkg-config",
|
||||
"vcpkg",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "option-ext"
|
||||
version = "0.2.0"
|
||||
@@ -3696,6 +3731,17 @@ dependencies = [
|
||||
"windows-link 0.2.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "password-hash"
|
||||
version = "0.5.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "346f04948ba92c43e8469c1ee6736c7563d71012b17d40745260fe106aac2166"
|
||||
dependencies = [
|
||||
"base64ct",
|
||||
"rand_core 0.6.4",
|
||||
"subtle",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "paste"
|
||||
version = "1.0.15"
|
||||
@@ -4299,7 +4345,7 @@ version = "0.13.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "be769465445e8c1474e9c5dac2018218498557af32d9ed057325ec9a41ae81bf"
|
||||
dependencies = [
|
||||
"heck 0.5.0",
|
||||
"heck 0.4.1",
|
||||
"itertools 0.14.0",
|
||||
"log",
|
||||
"multimap",
|
||||
@@ -5396,22 +5442,16 @@ dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
"axum",
|
||||
"base64 0.22.1",
|
||||
"chrono",
|
||||
"chrono-tz",
|
||||
"core-api",
|
||||
"cron",
|
||||
"dirs 5.0.1",
|
||||
"futures",
|
||||
"glob",
|
||||
"honcho-client",
|
||||
"iana-time-zone",
|
||||
"indexmap 2.14.0",
|
||||
"libc",
|
||||
"llm-client",
|
||||
"mcp-client",
|
||||
"notify",
|
||||
"os_info",
|
||||
"plugin-comfyui",
|
||||
"plugin-elevenlabs",
|
||||
"plugin-honcho",
|
||||
@@ -5421,18 +5461,13 @@ dependencies = [
|
||||
"plugin-transcribe-whisper-local",
|
||||
"plugin-tts-kokoro",
|
||||
"plugin-tts-orpheus-3b",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"rand 0.10.1",
|
||||
"regex",
|
||||
"reqwest 0.13.4",
|
||||
"rustls",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"serde_yaml",
|
||||
"sha2 0.10.9",
|
||||
"skald-core",
|
||||
"sqlx",
|
||||
"syn 2.0.117",
|
||||
"tauri",
|
||||
"tauri-build",
|
||||
"tokio",
|
||||
@@ -5442,6 +5477,47 @@ dependencies = [
|
||||
"tracing",
|
||||
"tracing-appender",
|
||||
"tracing-subscriber",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "skald-core"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"aes-gcm",
|
||||
"anyhow",
|
||||
"argon2",
|
||||
"async-trait",
|
||||
"axum",
|
||||
"base64 0.22.1",
|
||||
"chrono",
|
||||
"chrono-tz",
|
||||
"core-api",
|
||||
"cron",
|
||||
"futures",
|
||||
"glob",
|
||||
"honcho-client",
|
||||
"iana-time-zone",
|
||||
"indexmap 2.14.0",
|
||||
"libc",
|
||||
"libsqlite3-sys",
|
||||
"llm-client",
|
||||
"mcp-client",
|
||||
"notify",
|
||||
"os_info",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"rand 0.10.1",
|
||||
"regex",
|
||||
"reqwest 0.13.4",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2 0.10.9",
|
||||
"sqlx",
|
||||
"subtle",
|
||||
"syn 2.0.117",
|
||||
"tokio",
|
||||
"tokio-util",
|
||||
"tracing",
|
||||
"tree-sitter",
|
||||
"tree-sitter-bash",
|
||||
"tree-sitter-c",
|
||||
@@ -5459,6 +5535,8 @@ dependencies = [
|
||||
"tree-sitter-swift",
|
||||
"tree-sitter-typescript",
|
||||
"tree-sitter-yaml",
|
||||
"uuid",
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -6389,7 +6467,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
|
||||
dependencies = [
|
||||
"fastrand",
|
||||
"getrandom 0.4.2",
|
||||
"getrandom 0.3.4",
|
||||
"once_cell",
|
||||
"rustix",
|
||||
"windows-sys 0.61.2",
|
||||
@@ -7721,9 +7799,9 @@ checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be"
|
||||
|
||||
[[package]]
|
||||
name = "uuid"
|
||||
version = "1.23.1"
|
||||
version = "1.23.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ddd74a9687298c6858e9b88ec8935ec45d22e8fd5e6394fa1bd4e99a87789c76"
|
||||
checksum = "bf80a72845275afea99e7f2b434723d3bc7e38470fcd1c7ed39a599c73319a53"
|
||||
dependencies = [
|
||||
"getrandom 0.4.2",
|
||||
"js-sys",
|
||||
@@ -8903,18 +8981,18 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "zeroize"
|
||||
version = "1.8.2"
|
||||
version = "1.9.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0"
|
||||
checksum = "e13c156562582aa81c60cb29407084cdb54c4164760106ab78e6c5b0858cf64e"
|
||||
dependencies = [
|
||||
"zeroize_derive",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "zeroize_derive"
|
||||
version = "1.4.3"
|
||||
version = "1.5.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "85a5b4158499876c763cb03bc4e49185d3cccbabb15b33c627f7884f43db852e"
|
||||
checksum = "3c50655cbb0fe3fc43170059e702f1ce5e19b84cec58dc87b037a09935c2f328"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
|
||||
+11
-33
@@ -1,6 +1,7 @@
|
||||
[workspace]
|
||||
members = [
|
||||
".",
|
||||
"crates/skald-core",
|
||||
"crates/honcho-client",
|
||||
"crates/llm-client",
|
||||
"crates/core-api",
|
||||
@@ -45,6 +46,8 @@ embedded-tailscale = ["plugin-tailscale-remote/remote-tailscale"]
|
||||
tauri-build = { version = "2", optional = true , features = [] }
|
||||
|
||||
[dependencies]
|
||||
skald-core = { path = "crates/skald-core" }
|
||||
|
||||
axum = { version = "0.8", features = ["ws", "multipart"] }
|
||||
tokio = { version = "1.52.3", features = ["full"] }
|
||||
tokio-util = { version = "0.7", features = ["rt"] }
|
||||
@@ -57,10 +60,14 @@ anyhow = "1"
|
||||
sqlx = { version = "0.9.0", features = ["runtime-tokio", "sqlite"] }
|
||||
reqwest = { version = "0.13.4", default-features = false, features = ["rustls-no-provider", "charset", "http2", "system-proxy", "json", "multipart"] }
|
||||
# rustls is pinned as a direct dependency solely to select the crypto provider:
|
||||
# `ring` instead of the default `aws-lc-rs`. This keeps TLS off OpenSSL/aws-lc
|
||||
# (no libssl link, no cmake/NASM build), which is what makes a fully static musl
|
||||
# binary practical. Every reqwest client uses `rustls-no-provider`, so exactly
|
||||
# one process-wide provider is installed in main() before any TLS handshake.
|
||||
# `ring` instead of the default `aws-lc-rs`, avoiding aws-lc's cmake/NASM build.
|
||||
# Every reqwest client uses `rustls-no-provider`, so exactly one process-wide
|
||||
# provider is installed in main() before any TLS handshake.
|
||||
#
|
||||
# TLS therefore never touches OpenSSL. libcrypto is nonetheless in the tree now,
|
||||
# vendored and statically linked for SQLCipher (see `libsqlite3-sys` in
|
||||
# crates/skald-core) — so the binary stays self-contained, but "no OpenSSL
|
||||
# anywhere" is no longer true.
|
||||
rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12", "logging"] }
|
||||
async-trait = "0.1"
|
||||
serde_json = "1"
|
||||
@@ -69,36 +76,7 @@ tracing = "0.1"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
||||
tracing-appender = "0.2"
|
||||
chrono = { version = "0.4", default-features = false, features = ["clock", "std"] }
|
||||
chrono-tz = "0.10"
|
||||
iana-time-zone = "0.1"
|
||||
os_info = "3"
|
||||
cron = "0.16"
|
||||
rand = "0.10.1"
|
||||
syn = { version = "2", features = ["full"] }
|
||||
quote = "1"
|
||||
proc-macro2 = { version = "1", features = ["span-locations"] }
|
||||
tree-sitter = "0.26"
|
||||
tree-sitter-python = "0.25"
|
||||
tree-sitter-javascript = "0.25"
|
||||
tree-sitter-typescript = "0.23"
|
||||
tree-sitter-go = "0.25"
|
||||
tree-sitter-java = "0.23"
|
||||
tree-sitter-c = "0.24"
|
||||
tree-sitter-cpp = "0.23"
|
||||
tree-sitter-swift = "0.7"
|
||||
tree-sitter-lua = "0.5"
|
||||
tree-sitter-ruby = "0.23"
|
||||
tree-sitter-bash = "0.25"
|
||||
tree-sitter-elixir = "0.3"
|
||||
tree-sitter-json = "0.24"
|
||||
tree-sitter-yaml = "0.7"
|
||||
tree-sitter-html = "0.23"
|
||||
tree-sitter-css = "0.25"
|
||||
regex = "1"
|
||||
glob = "0.3"
|
||||
libc = "0.2"
|
||||
base64 = "0.22"
|
||||
sha2 = "0.10"
|
||||
notify = "8"
|
||||
honcho-client = { path = "crates/honcho-client" }
|
||||
llm-client = { path = "crates/llm-client" }
|
||||
|
||||
@@ -87,6 +87,19 @@ pub trait Plugin: Send + Sync {
|
||||
/// unaffected.
|
||||
fn http_router(&self) -> Option<axum::Router> { None }
|
||||
|
||||
/// Tools this plugin contributes to the registry — the sibling of
|
||||
/// [`Plugin::http_router`].
|
||||
///
|
||||
/// The receiver is `Arc<Self>` because the tools a plugin builds usually
|
||||
/// call back into it, so it must hand them its own handle. Without this
|
||||
/// hook the core has to name concrete plugin crates in order to downcast
|
||||
/// them, and ends up depending on every plugin in the tree.
|
||||
///
|
||||
/// Called once while the tool registry is built, *before* the plugin's
|
||||
/// runloop starts: the tools must tolerate being invoked while their plugin
|
||||
/// is stopped. Default: no tools.
|
||||
fn tools(self: Arc<Self>) -> Vec<Arc<dyn crate::tool::Tool>> { Vec::new() }
|
||||
|
||||
/// Returns a [`Memory`] backend if this plugin provides one.
|
||||
fn memory(&self) -> Option<Arc<dyn Memory>> { None }
|
||||
|
||||
|
||||
@@ -316,6 +316,13 @@ impl Plugin for MobileConnectorPlugin {
|
||||
Some(router::build(Arc::clone(&self.inner)))
|
||||
}
|
||||
|
||||
/// Control tools (plugin.md §11). They close over the plugin itself as a
|
||||
/// `RelayAgent` and call into it lazily, so building them before the runloop
|
||||
/// starts is fine — they fail gracefully while it is stopped.
|
||||
fn tools(self: Arc<Self>) -> Vec<Arc<dyn core_api::tool::Tool>> {
|
||||
crate::tools::mobile_tools(self)
|
||||
}
|
||||
|
||||
fn as_any(&self) -> &dyn std::any::Any { self }
|
||||
fn as_arc_any(self: Arc<Self>) -> Arc<dyn std::any::Any + Send + Sync> { self }
|
||||
}
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
[package]
|
||||
name = "skald-core"
|
||||
version = "0.1.0"
|
||||
edition = "2024"
|
||||
|
||||
# The headless application core: database + crypto + identity, the LLM stack,
|
||||
# tools, MCP, plugins-as-a-registry, sessions.
|
||||
#
|
||||
# Deliberately knows nothing about the process shell around it. No Tauri, no
|
||||
# concrete plugin crates (it only ever sees `Arc<dyn Plugin>` from `core-api`),
|
||||
# no `axum` server — `skald` and `skald-setup` are both consumers.
|
||||
|
||||
[dependencies]
|
||||
axum = { version = "0.8", features = ["ws", "multipart"] }
|
||||
tokio = { version = "1.52.3", features = ["full"] }
|
||||
tokio-util = { version = "0.7", features = ["rt"] }
|
||||
futures = "0.3"
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
anyhow = "1"
|
||||
sqlx = { version = "0.9.0", features = ["runtime-tokio", "sqlite"] }
|
||||
# Per-user database encryption. `sqlx-sqlite` already builds `libsqlite3-sys`;
|
||||
# naming it here only adds a feature, and Cargo's feature unification turns the
|
||||
# one SQLite the process links into SQLCipher. A database opened without
|
||||
# `PRAGMA key` still behaves as plain SQLite, so `system.db` is unaffected.
|
||||
# `-vendored-openssl` compiles libcrypto from source instead of linking the
|
||||
# system one, so the binary stays self-contained (see b08a13c). It costs a C
|
||||
# build (perl + make) — the trade the blueprint §4 accepts explicitly.
|
||||
#
|
||||
# Pinned below 0.38 on purpose: `sqlx-sqlite` 0.9 requires ">=0.30.1, <0.38.0".
|
||||
# A newer release would resolve to a *second* copy of the crate, and the
|
||||
# SQLCipher feature would then apply to a SQLite that sqlx never links. 0.37.0
|
||||
# is the newest version that still unifies.
|
||||
libsqlite3-sys = { version = "0.37", features = ["bundled-sqlcipher-vendored-openssl"] }
|
||||
# 0.6 is still a release candidate; stay on the stable line.
|
||||
argon2 = "0.5"
|
||||
# Held at 0.10 to match `skald-relay-common`, so the binary links one AES-GCM
|
||||
# rather than two. Bumping to 0.11 means bumping the mobile E2E code with it.
|
||||
aes-gcm = "0.10"
|
||||
zeroize = { version = "1", features = ["derive"] }
|
||||
subtle = "2"
|
||||
uuid = { version = "1", features = ["v4"] }
|
||||
reqwest = { version = "0.13.4", default-features = false, features = ["rustls-no-provider", "charset", "http2", "system-proxy", "json", "multipart"] }
|
||||
async-trait = "0.1"
|
||||
serde_json = "1"
|
||||
indexmap = { version = "2", features = ["serde"] }
|
||||
tracing = "0.1"
|
||||
chrono = { version = "0.4", default-features = false, features = ["clock", "std"] }
|
||||
chrono-tz = "0.10"
|
||||
iana-time-zone = "0.1"
|
||||
os_info = "3"
|
||||
cron = "0.16"
|
||||
rand = "0.10.1"
|
||||
syn = { version = "2", features = ["full"] }
|
||||
quote = "1"
|
||||
proc-macro2 = { version = "1", features = ["span-locations"] }
|
||||
tree-sitter = "0.26"
|
||||
tree-sitter-python = "0.25"
|
||||
tree-sitter-javascript = "0.25"
|
||||
tree-sitter-typescript = "0.23"
|
||||
tree-sitter-go = "0.25"
|
||||
tree-sitter-java = "0.23"
|
||||
tree-sitter-c = "0.24"
|
||||
tree-sitter-cpp = "0.23"
|
||||
tree-sitter-swift = "0.7"
|
||||
tree-sitter-lua = "0.5"
|
||||
tree-sitter-ruby = "0.23"
|
||||
tree-sitter-bash = "0.25"
|
||||
tree-sitter-elixir = "0.3"
|
||||
tree-sitter-json = "0.24"
|
||||
tree-sitter-yaml = "0.7"
|
||||
tree-sitter-html = "0.23"
|
||||
tree-sitter-css = "0.25"
|
||||
regex = "1"
|
||||
glob = "0.3"
|
||||
libc = "0.2"
|
||||
base64 = "0.22"
|
||||
sha2 = "0.10"
|
||||
notify = "8"
|
||||
honcho-client = { path = "../honcho-client" }
|
||||
llm-client = { path = "../llm-client" }
|
||||
core-api = { path = "../core-api" }
|
||||
mcp-client = { path = "../mcp-client" }
|
||||
@@ -2,7 +2,7 @@ use anyhow::{Context, Result};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use tracing::{debug, trace, warn};
|
||||
|
||||
use crate::config::LlmStrength;
|
||||
use core_api::provider::LlmStrength;
|
||||
|
||||
const AGENTS_DIR: &str = "agents";
|
||||
|
||||
@@ -46,11 +46,11 @@ use sqlx::SqlitePool;
|
||||
use tokio::sync::{broadcast, Mutex, oneshot};
|
||||
use tracing::{debug, error, info, warn};
|
||||
|
||||
use crate::core::pending_registry::PendingRegistry;
|
||||
use crate::core::session::handler::ApprovalDecision;
|
||||
use crate::core::tools::tool_names as tn;
|
||||
use crate::core::tools::ToolCategory;
|
||||
use crate::core::events::{GlobalEvent, ServerEvent};
|
||||
use crate::pending_registry::PendingRegistry;
|
||||
use crate::session::handler::ApprovalDecision;
|
||||
use crate::tools::tool_names as tn;
|
||||
use crate::tools::ToolCategory;
|
||||
use crate::events::{GlobalEvent, ServerEvent};
|
||||
|
||||
// ── Public types ──────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -437,7 +437,7 @@ impl ApprovalManager {
|
||||
args: &Value,
|
||||
group_id: Option<&str>,
|
||||
) -> GateResult {
|
||||
let rules = match crate::core::db::approval_rules::list_for_group(&self.db, group_id).await {
|
||||
let rules = match crate::db::approval_rules::list_for_group(&self.db, group_id).await {
|
||||
Ok(r) => r,
|
||||
Err(e) => {
|
||||
// Fail closed: this is a default-closed security gate (see module docs).
|
||||
@@ -754,7 +754,7 @@ impl ApprovalManager {
|
||||
group_id: &str,
|
||||
tool_name: &str,
|
||||
) -> Option<RuleAction> {
|
||||
let rules = crate::core::db::approval_rules::list_for_group(&self.db, Some(group_id))
|
||||
let rules = crate::db::approval_rules::list_for_group(&self.db, Some(group_id))
|
||||
.await
|
||||
.unwrap_or_default();
|
||||
for rule in &rules {
|
||||
@@ -768,7 +768,7 @@ impl ApprovalManager {
|
||||
// ── Rule management ───────────────────────────────────────────────────────
|
||||
|
||||
pub async fn list_rules(&self) -> Result<Vec<ApprovalRule>> {
|
||||
crate::core::db::approval_rules::list(&self.db).await
|
||||
crate::db::approval_rules::list(&self.db).await
|
||||
}
|
||||
|
||||
pub async fn add_rule(&self, r: NewApprovalRule) -> Result<i64> {
|
||||
@@ -776,11 +776,11 @@ impl ApprovalManager {
|
||||
let group = r.group_id.as_deref().unwrap_or("default");
|
||||
self.ensure_no_conflicting_catch_all(group, &r.action, None).await?;
|
||||
}
|
||||
crate::core::db::approval_rules::insert(&self.db, r).await
|
||||
crate::db::approval_rules::insert(&self.db, r).await
|
||||
}
|
||||
|
||||
pub async fn delete_rule(&self, id: i64) -> Result<()> {
|
||||
crate::core::db::approval_rules::delete(&self.db, id).await
|
||||
crate::db::approval_rules::delete(&self.db, id).await
|
||||
}
|
||||
|
||||
pub async fn update_rule(&self, id: i64, r: NewApprovalRule) -> Result<()> {
|
||||
@@ -788,7 +788,7 @@ impl ApprovalManager {
|
||||
let group = r.group_id.as_deref().unwrap_or("default");
|
||||
self.ensure_no_conflicting_catch_all(group, &r.action, Some(id)).await?;
|
||||
}
|
||||
crate::core::db::approval_rules::update(&self.db, id, r).await
|
||||
crate::db::approval_rules::update(&self.db, id, r).await
|
||||
}
|
||||
|
||||
/// Rejects creating/updating a `*` catch-all when the target group already has a
|
||||
@@ -916,11 +916,11 @@ fn rule_matches(rule: &ApprovalRule, agent_id: &str, source: &str, tool_name: &s
|
||||
/// These back the "File System" permission panel, where each path row is one rule row.
|
||||
pub(crate) fn tool_pattern_matches(pattern: &str, tool_name: &str) -> bool {
|
||||
match pattern {
|
||||
"@fs_read" => crate::core::tools::is_file_read_tool(tool_name),
|
||||
"@fs_write" => crate::core::tools::is_file_write_tool(tool_name),
|
||||
"@fs_read" => crate::tools::is_file_read_tool(tool_name),
|
||||
"@fs_write" => crate::tools::is_file_write_tool(tool_name),
|
||||
"@fs_any" => {
|
||||
crate::core::tools::is_file_read_tool(tool_name)
|
||||
|| crate::core::tools::is_file_write_tool(tool_name)
|
||||
crate::tools::is_file_read_tool(tool_name)
|
||||
|| crate::tools::is_file_write_tool(tool_name)
|
||||
}
|
||||
_ => pattern_matches(pattern, tool_name),
|
||||
}
|
||||
@@ -981,7 +981,7 @@ fn mcp_server_from_tool_name(name: &str) -> Option<String> {
|
||||
pub(crate) fn normalize_path(path: &str) -> String {
|
||||
let cwd = std::env::current_dir().unwrap_or_default();
|
||||
let cwd = std::fs::canonicalize(&cwd).unwrap_or(cwd);
|
||||
let canon = crate::core::tools::fs::canonicalize_for_policy(path, &cwd);
|
||||
let canon = crate::tools::fs::canonicalize_for_policy(path, &cwd);
|
||||
if let Ok(rel) = canon.strip_prefix(&cwd) {
|
||||
return rel.to_string_lossy().into_owned();
|
||||
}
|
||||
@@ -1059,7 +1059,7 @@ mod tests {
|
||||
let path = std::env::temp_dir().join(format!("skald_fs_test_{}.db", std::process::id()));
|
||||
let path_str = path.to_string_lossy().to_string();
|
||||
let _ = std::fs::remove_file(&path);
|
||||
let pool = crate::core::db::init_pool(&path_str).await.expect("init_pool");
|
||||
let pool = crate::db::init_system_pool(&path_str).await.expect("init_system_pool");
|
||||
let db = Arc::new(pool);
|
||||
|
||||
// The `default` permission group is a FK target for approval_rules.group_id.
|
||||
@@ -0,0 +1,53 @@
|
||||
//! Curated, human-readable bootstrap progress printed to **stdout**.
|
||||
//!
|
||||
//! At runtime stdout is silent — only the file log (`logs/skald.log`) records
|
||||
//! events. During startup, though, it is useful to see at a glance how the app
|
||||
//! is configured and how it is coming up. These helpers emit a small, ordered
|
||||
//! set of lines on the dedicated `boot` tracing target.
|
||||
//!
|
||||
//! Rendering belongs to whoever owns the process: each shell installs a stdout
|
||||
//! layer filtered on [`TARGET`] and formats the lines as it likes (`skald`'s
|
||||
//! `BootFormat` strips timestamps and paints failures red). The core only says
|
||||
//! *what* happened, never how it looks — which is also why nothing here depends
|
||||
//! on `tracing-subscriber`.
|
||||
//!
|
||||
//! The same lines land in the log file (they pass the normal `EnvFilter`), so
|
||||
//! they double as a high-level startup trace. Glyphs are baked into the message
|
||||
//! on purpose, so the file keeps the same readable shape.
|
||||
|
||||
use std::fmt;
|
||||
|
||||
use tracing::{info, warn};
|
||||
|
||||
/// Tracing target for curated bootstrap lines shown on stdout.
|
||||
pub const TARGET: &str = "boot";
|
||||
|
||||
/// Top-level title (no glyph), e.g. `skald v0.5 — starting`.
|
||||
pub fn title(msg: impl fmt::Display) {
|
||||
info!(target: TARGET, "{}", msg);
|
||||
}
|
||||
|
||||
/// A phase header, e.g. `› Plugins — 6 active, 1 failed`.
|
||||
pub fn section(msg: impl fmt::Display) {
|
||||
info!(target: TARGET, "› {}", msg);
|
||||
}
|
||||
|
||||
/// A successful item under a phase.
|
||||
pub fn ok(msg: impl fmt::Display) {
|
||||
info!(target: TARGET, " ✓ {}", msg);
|
||||
}
|
||||
|
||||
/// An item that exists but is inactive (e.g. a disabled plugin).
|
||||
pub fn off(msg: impl fmt::Display) {
|
||||
info!(target: TARGET, " ○ {}", msg);
|
||||
}
|
||||
|
||||
/// A failed item (rendered in red on stdout; logged at WARN in the file).
|
||||
pub fn fail(msg: impl fmt::Display) {
|
||||
warn!(target: TARGET, " ✗ {}", msg);
|
||||
}
|
||||
|
||||
/// The final "app is up" line, e.g. `✅ Ready — http://localhost:8080`.
|
||||
pub fn ready(msg: impl fmt::Display) {
|
||||
info!(target: TARGET, "✅ {}", msg);
|
||||
}
|
||||
@@ -12,14 +12,14 @@ use tracing::{error, info, warn};
|
||||
mod inbox;
|
||||
use inbox::{QueuedMessage, SourceInbox, build_unit, drain_leading_user};
|
||||
|
||||
use crate::core::approval::ApprovalManager;
|
||||
use crate::core::cron::TaskManager;
|
||||
use crate::core::db::{chat_history, chat_llm_tools, chat_sessions, chat_sessions_stack, config, sources};
|
||||
use crate::core::events::{GlobalEvent, ServerEvent};
|
||||
use crate::core::notification::Notification;
|
||||
use crate::core::session::handler::{ChatSessionHandler, InterfaceTool, PendingMsg, PendingUserInput};
|
||||
use crate::core::session::manager::ChatSessionManager;
|
||||
use crate::core::tools::tool_names as tn;
|
||||
use crate::approval::ApprovalManager;
|
||||
use crate::cron::TaskManager;
|
||||
use crate::db::{chat_history, chat_llm_tools, chat_sessions, chat_sessions_stack, config, sources};
|
||||
use crate::events::{GlobalEvent, ServerEvent};
|
||||
use crate::notification::Notification;
|
||||
use crate::session::handler::{ChatSessionHandler, InterfaceTool, PendingMsg, PendingUserInput};
|
||||
use crate::session::manager::ChatSessionManager;
|
||||
use crate::tools::tool_names as tn;
|
||||
|
||||
pub use core_api::chat_hub::{ChatHubApi, ModelCommandOutcome, SendMessageOptions};
|
||||
|
||||
@@ -196,7 +196,7 @@ impl ChatHub {
|
||||
let mut interface_tools = opts.interface_tools;
|
||||
if let Some(task_mgr) = self.task_mgr.get() {
|
||||
interface_tools.push(
|
||||
crate::core::tools::cron_jobs::build_execute_task_interface_tool(
|
||||
crate::tools::cron_jobs::build_execute_task_interface_tool(
|
||||
Arc::clone(task_mgr),
|
||||
session_id,
|
||||
run_context_json,
|
||||
@@ -245,7 +245,7 @@ impl ChatHub {
|
||||
&self,
|
||||
source_id: &str,
|
||||
agent_id: &str,
|
||||
run_context: Option<&crate::core::run_context::RunContext>,
|
||||
run_context: Option<&crate::run_context::RunContext>,
|
||||
reset: bool,
|
||||
) -> anyhow::Result<i64> {
|
||||
// A reset discards the current session; drop any messages queued for it.
|
||||
@@ -372,7 +372,7 @@ impl ChatHub {
|
||||
let mut tools = Vec::new();
|
||||
if let Some(task_mgr) = self.task_mgr.get() {
|
||||
let run_context_json = handler.run_context_json().await;
|
||||
tools.push(crate::core::tools::cron_jobs::build_execute_task_interface_tool(
|
||||
tools.push(crate::tools::cron_jobs::build_execute_task_interface_tool(
|
||||
Arc::clone(task_mgr),
|
||||
session_id,
|
||||
run_context_json,
|
||||
@@ -402,7 +402,7 @@ impl ChatHub {
|
||||
/// The next LLM turn will start with no MCP servers activated.
|
||||
pub async fn reset_mcp(&self, source_id: &str) -> anyhow::Result<()> {
|
||||
let session_id = self.get_or_create_session(source_id, "main").await?;
|
||||
crate::core::db::session_mcp_grants::revoke_all(&self.db, session_id).await?;
|
||||
crate::db::session_mcp_grants::revoke_all(&self.db, session_id).await?;
|
||||
info!(source_id, session_id, "ChatHub: MCP grants reset");
|
||||
Ok(())
|
||||
}
|
||||
@@ -15,7 +15,7 @@ use serde_json::Value;
|
||||
use sqlx::SqlitePool;
|
||||
use tracing::warn;
|
||||
|
||||
use crate::core::db::llm_requests;
|
||||
use crate::db::llm_requests;
|
||||
|
||||
use super::{ChatOptions, ChatResponse, ChatbotClient, LlmRawMeta, LlmTurn, Message};
|
||||
|
||||
@@ -6,8 +6,8 @@ use serde::Serialize;
|
||||
use tokio::sync::{broadcast, oneshot};
|
||||
use tracing::info;
|
||||
|
||||
use crate::core::events::{GlobalEvent, ServerEvent};
|
||||
use crate::core::pending_registry::PendingRegistry;
|
||||
use crate::events::{GlobalEvent, ServerEvent};
|
||||
use crate::pending_registry::PendingRegistry;
|
||||
|
||||
#[derive(Debug, Clone, Serialize)]
|
||||
pub struct PendingClarificationInfo {
|
||||
@@ -49,11 +49,11 @@ use serde_json::json;
|
||||
use sqlx::SqlitePool;
|
||||
use tracing::{debug, info, warn};
|
||||
|
||||
use crate::core::chat_event_bus::{ChatEventBus, CompactionEvent};
|
||||
use crate::core::chatbot::ChatOptions;
|
||||
use crate::core::config::CompactionConfig;
|
||||
use crate::core::db::{chat_history, chat_llm_tools, chat_summaries};
|
||||
use crate::core::llm::LlmManager;
|
||||
use crate::chat_event_bus::{ChatEventBus, CompactionEvent};
|
||||
use crate::chatbot::ChatOptions;
|
||||
use crate::config::CompactionConfig;
|
||||
use crate::db::{chat_history, chat_llm_tools, chat_summaries};
|
||||
use crate::llm::LlmManager;
|
||||
|
||||
// ── Compaction constants (ported from Hermes context_compressor.py) ──────────
|
||||
//
|
||||
@@ -322,8 +322,8 @@ impl ContextCompactor {
|
||||
})?;
|
||||
|
||||
let summary_text = match turn {
|
||||
crate::core::chatbot::LlmTurn::Message(resp) => resp.content,
|
||||
crate::core::chatbot::LlmTurn::ToolCalls { content, .. } => {
|
||||
crate::chatbot::LlmTurn::Message(resp) => resp.content,
|
||||
crate::chatbot::LlmTurn::ToolCalls { content, .. } => {
|
||||
warn!(stack_id, "compactor: unexpected tool calls in summary response, using content");
|
||||
content
|
||||
}
|
||||
@@ -12,10 +12,10 @@ use tracing::{error, info};
|
||||
|
||||
use core_api::system_bus::{SystemEvent, SystemEventBus};
|
||||
|
||||
use crate::core::chat_hub::ChatHub;
|
||||
use crate::core::db::chat_sessions;
|
||||
use crate::core::db::scheduled_jobs::{self, ScheduledJob};
|
||||
use crate::core::session::manager::ChatSessionManager;
|
||||
use crate::chat_hub::ChatHub;
|
||||
use crate::db::chat_sessions;
|
||||
use crate::db::scheduled_jobs::{self, ScheduledJob};
|
||||
use crate::session::manager::ChatSessionManager;
|
||||
|
||||
pub struct TaskManager {
|
||||
pool: Arc<SqlitePool>,
|
||||
@@ -183,7 +183,7 @@ impl TaskManager {
|
||||
if agent_id.trim().is_empty() {
|
||||
anyhow::bail!("agent_id is required — specify which task agent runs this task (no default)");
|
||||
}
|
||||
crate::core::agents::load_task_meta(agent_id)?;
|
||||
crate::agents::load_task_meta(agent_id)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -449,7 +449,7 @@ async fn run_job(
|
||||
"cron" => {
|
||||
if let Some(hub) = hub {
|
||||
let outcome = final_response.as_deref().unwrap_or("(no output)");
|
||||
hub.notify(crate::core::notification::Notification {
|
||||
hub.notify(crate::notification::Notification {
|
||||
source: "cron".into(),
|
||||
event_type: "cron_result".into(),
|
||||
summary: format!(
|
||||
@@ -496,7 +496,7 @@ async fn run_job(
|
||||
});
|
||||
|
||||
if let Some(hub) = hub {
|
||||
hub.notify(crate::core::notification::Notification {
|
||||
hub.notify(crate::notification::Notification {
|
||||
source: "cron".into(),
|
||||
event_type: "cron_error".into(),
|
||||
summary: format!(
|
||||
@@ -525,14 +525,14 @@ async fn inject_async_result(
|
||||
result: &str,
|
||||
) {
|
||||
// Resolve source_id from the parent session row.
|
||||
let source_id = match crate::core::db::chat_sessions::find_by_id(pool, parent_session_id).await {
|
||||
let source_id = match crate::db::chat_sessions::find_by_id(pool, parent_session_id).await {
|
||||
Ok(Some(s)) => s.source,
|
||||
Ok(None) => { error!("inject_async_result: session {parent_session_id} not found"); return; }
|
||||
Err(e) => { error!("inject_async_result: DB error: {e}"); return; }
|
||||
};
|
||||
|
||||
// Get the active stack for the parent session.
|
||||
let stack = match crate::core::db::chat_sessions_stack::active_for_session(pool, parent_session_id).await {
|
||||
let stack = match crate::db::chat_sessions_stack::active_for_session(pool, parent_session_id).await {
|
||||
Ok(Some(s)) => s,
|
||||
Ok(None) => { error!("inject_async_result: no active stack for session {parent_session_id}"); return; }
|
||||
Err(e) => { error!("inject_async_result: stack lookup failed: {e}"); return; }
|
||||
@@ -544,8 +544,8 @@ async fn inject_async_result(
|
||||
Let me process the result via task_completed.",
|
||||
task_title,
|
||||
);
|
||||
let assistant_id = match crate::core::db::chat_history::append(
|
||||
pool, stack.id, &crate::core::db::chat_history::Role::Assistant,
|
||||
let assistant_id = match crate::db::chat_history::append(
|
||||
pool, stack.id, &crate::db::chat_history::Role::Assistant,
|
||||
"", true, Some(&reasoning),
|
||||
).await {
|
||||
Ok(id) => id,
|
||||
@@ -559,7 +559,7 @@ async fn inject_async_result(
|
||||
"result": result,
|
||||
})).unwrap_or_else(|_| "{}".to_string());
|
||||
|
||||
let tool_call_id = match crate::core::db::chat_llm_tools::append(
|
||||
let tool_call_id = match crate::db::chat_llm_tools::append(
|
||||
pool, assistant_id, "task_completed",
|
||||
&serde_json::json!({"task_id": task_id}).to_string(),
|
||||
).await {
|
||||
@@ -567,7 +567,7 @@ async fn inject_async_result(
|
||||
Err(e) => { error!("inject_async_result: append tool call failed: {e}"); return; }
|
||||
};
|
||||
|
||||
if let Err(e) = crate::core::db::chat_llm_tools::complete(pool, tool_call_id, &result_json, "string").await {
|
||||
if let Err(e) = crate::db::chat_llm_tools::complete(pool, tool_call_id, &result_json, "string").await {
|
||||
error!("inject_async_result: complete tool call failed: {e}"); return;
|
||||
}
|
||||
|
||||
@@ -580,15 +580,15 @@ async fn inject_async_result(
|
||||
|
||||
/// Builds the `execute_subtask` InterfaceTool injected into background sessions.
|
||||
/// Background tasks can only run synchronous sub-tasks — no cron or async.
|
||||
fn build_execute_subtask_tool(task_mgr: Arc<TaskManager>, run_context: Option<String>) -> crate::core::session::handler::InterfaceTool {
|
||||
use crate::core::session::handler::{InterfaceTool, ToolFuture};
|
||||
fn build_execute_subtask_tool(task_mgr: Arc<TaskManager>, run_context: Option<String>) -> crate::session::handler::InterfaceTool {
|
||||
use crate::session::handler::{InterfaceTool, ToolFuture};
|
||||
use serde_json::json;
|
||||
|
||||
InterfaceTool {
|
||||
definition: json!({
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": crate::core::tools::tool_names::EXECUTE_SUBTASK,
|
||||
"name": crate::tools::tool_names::EXECUTE_SUBTASK,
|
||||
"description": "Run a synchronous sub-task and return its result. Blocks until the sub-task completes.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
@@ -633,7 +633,7 @@ async fn record_job_run(
|
||||
response: Option<&str>,
|
||||
error: Option<&str>,
|
||||
) -> Result<()> {
|
||||
crate::core::db::job_runs::insert(
|
||||
crate::db::job_runs::insert(
|
||||
pool, job_id, Some(session_id),
|
||||
started_at, completed_at, duration_ms,
|
||||
status, response, error,
|
||||
@@ -0,0 +1,363 @@
|
||||
//! Envelope encryption for the per-user databases (blueprint §4 / §5.1).
|
||||
//!
|
||||
//! A user's database is encrypted by SQLCipher under a random 256-bit **DEK**.
|
||||
//! The DEK is never stored in the clear: `users.database_password` holds it
|
||||
//! sealed with AES-256-GCM under a **KEK** derived from the password with
|
||||
//! Argon2id.
|
||||
//!
|
||||
//! That seal *is* the password verifier. Opening it either yields the DEK — in
|
||||
//! which case the password was right — or fails the AEAD tag, which means a
|
||||
//! wrong password, cleanly distinct from a corrupt file. Storing a second hash
|
||||
//! of the same password beside the seal would only hand an offline attacker an
|
||||
//! easier target than the seal itself, so encrypted users have no
|
||||
//! `password_hash` at all.
|
||||
//!
|
||||
//! Cleartext users have no database key to bind a verifier to, so they store the
|
||||
//! Argon2id output directly and it is compared in constant time. Its
|
||||
//! crackability is harmless: their database is readable by the box owner by
|
||||
//! design.
|
||||
//!
|
||||
//! Nothing here invents a construction — it composes Argon2id, AES-256-GCM and a
|
||||
//! CSPRNG. Changing the password re-seals the same DEK; the database itself is
|
||||
//! never re-encrypted.
|
||||
|
||||
use std::fmt::{self, Write as _};
|
||||
use std::sync::LazyLock;
|
||||
|
||||
use aes_gcm::aead::Aead;
|
||||
use aes_gcm::{Aes256Gcm, KeyInit, Nonce};
|
||||
use anyhow::{Context, Result, anyhow, bail};
|
||||
use argon2::{Algorithm, Argon2, Params, Version};
|
||||
use rand::Rng as _;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use subtle::ConstantTimeEq;
|
||||
use tokio::sync::Semaphore;
|
||||
use zeroize::{Zeroize, ZeroizeOnDrop};
|
||||
|
||||
/// 256-bit keys throughout: AES-256-GCM for the seal, raw SQLCipher key for the
|
||||
/// database.
|
||||
const KEY_LEN: usize = 32;
|
||||
const NONCE_LEN: usize = 12;
|
||||
const TAG_LEN: usize = 16;
|
||||
/// Sealed layout: `nonce ‖ ciphertext ‖ tag`.
|
||||
const SEALED_LEN: usize = NONCE_LEN + KEY_LEN + TAG_LEN;
|
||||
|
||||
pub const SALT_LEN: usize = 16;
|
||||
|
||||
/// The only algorithm we accept. Stored per row so it can change later without a
|
||||
/// migration, but a row asking for anything else is a bug, not a fallback.
|
||||
const ALGO: &str = "argon2id";
|
||||
|
||||
/// Argon2id at 256 MiB is a memory bomb if it runs unbounded: every concurrent
|
||||
/// login would allocate its own arena. Two at a time keeps a login responsive
|
||||
/// while capping the peak at ~2× `KdfParams::m`.
|
||||
///
|
||||
/// Deliberately a module-level invariant rather than a caller's responsibility —
|
||||
/// a forgotten permit is a memory incident, not a style problem.
|
||||
static KDF_GATE: LazyLock<Semaphore> = LazyLock::new(|| Semaphore::new(2));
|
||||
|
||||
// ── KDF parameters ────────────────────────────────────────────────────────────
|
||||
|
||||
/// Serialized into `users.kdf_params`. Not secret: calibrated on the box when the
|
||||
/// user is created, and kept per row so raising the cost later needs no migration.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct KdfParams {
|
||||
pub algo: String,
|
||||
/// Memory cost, in KiB.
|
||||
pub m: u32,
|
||||
/// Time cost (passes).
|
||||
pub t: u32,
|
||||
/// Parallelism (lanes).
|
||||
pub p: u32,
|
||||
}
|
||||
|
||||
impl Default for KdfParams {
|
||||
/// 256 MiB, ~1s on the weakest box we target.
|
||||
///
|
||||
/// Between the OWASP baseline (64 MiB) and the 512 MiB–1 GiB of §5.1: four
|
||||
/// times the cost for an attacker with a GPU, while two concurrent logins
|
||||
/// peak at 512 MiB rather than 2 GiB and never reach swap.
|
||||
fn default() -> Self {
|
||||
Self { algo: ALGO.to_string(), m: 262_144, t: 3, p: 1 }
|
||||
}
|
||||
}
|
||||
|
||||
impl KdfParams {
|
||||
pub fn to_json(&self) -> Result<String> {
|
||||
serde_json::to_string(self).context("serializing kdf_params")
|
||||
}
|
||||
|
||||
pub fn from_json(s: &str) -> Result<Self> {
|
||||
let p: Self = serde_json::from_str(s).context("parsing kdf_params")?;
|
||||
if p.algo != ALGO {
|
||||
bail!("unsupported kdf algorithm: {}", p.algo);
|
||||
}
|
||||
Ok(p)
|
||||
}
|
||||
|
||||
/// Cheap parameters, tests only: the real ones cost ~1s and 256 MiB per
|
||||
/// derivation, which a test suite pays on every register and every login.
|
||||
#[cfg(test)]
|
||||
pub fn fast() -> Self {
|
||||
Self { algo: ALGO.to_string(), m: 64, t: 1, p: 1 }
|
||||
}
|
||||
}
|
||||
|
||||
// ── Keys ──────────────────────────────────────────────────────────────────────
|
||||
|
||||
/// Data Encryption Key: the raw SQLCipher key for one user's database. Random,
|
||||
/// never derived from the password, so changing the password does not re-encrypt
|
||||
/// anything.
|
||||
#[derive(Clone, Zeroize, ZeroizeOnDrop)]
|
||||
pub struct Dek([u8; KEY_LEN]);
|
||||
|
||||
/// Key Encryption Key: `Argon2id(password, salt)`. Seals the [`Dek`], and for a
|
||||
/// cleartext user its raw bytes *are* the stored verifier.
|
||||
#[derive(Clone, Zeroize, ZeroizeOnDrop)]
|
||||
pub struct Kek([u8; KEY_LEN]);
|
||||
|
||||
impl Dek {
|
||||
pub fn random() -> Self {
|
||||
let mut k = [0u8; KEY_LEN];
|
||||
rand::rng().fill_bytes(&mut k);
|
||||
Self(k)
|
||||
}
|
||||
|
||||
/// The `PRAGMA key` value for a raw 256-bit key.
|
||||
///
|
||||
/// The double quotes are **part of the value**: sqlx pastes it verbatim into
|
||||
/// `PRAGMA key = {value};`, and SQLCipher only treats the argument as a raw
|
||||
/// key when it parses as the blob literal `x'…'`. Given anything else it
|
||||
/// runs its own KDF over the bytes instead, silently deriving a *different*
|
||||
/// key from our hex digits — and the database would open, just not the one
|
||||
/// we meant.
|
||||
///
|
||||
/// The returned string is key material. It must never be logged, and it
|
||||
/// outlives this call inside the pool's connect options.
|
||||
pub fn to_pragma(&self) -> String {
|
||||
// `"x'` + 64 hex digits + `'"`
|
||||
let mut s = String::with_capacity(5 + 2 * KEY_LEN);
|
||||
s.push_str("\"x'");
|
||||
for b in self.0 {
|
||||
let _ = write!(s, "{b:02x}");
|
||||
}
|
||||
s.push_str("'\"");
|
||||
s
|
||||
}
|
||||
}
|
||||
|
||||
impl Kek {
|
||||
/// The verifier stored in `users.password_hash` for a cleartext user.
|
||||
pub fn as_verifier(&self) -> &[u8] {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
// Hand-written so a stray `{:?}` — a tracing span, an error context, a panic —
|
||||
// cannot print key material. Same reasoning as `db::users::Credentials`.
|
||||
impl fmt::Debug for Dek {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
f.write_str("Dek(<redacted>)")
|
||||
}
|
||||
}
|
||||
|
||||
impl fmt::Debug for Kek {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
f.write_str("Kek(<redacted>)")
|
||||
}
|
||||
}
|
||||
|
||||
pub fn random_salt() -> Vec<u8> {
|
||||
let mut s = vec![0u8; SALT_LEN];
|
||||
rand::rng().fill_bytes(&mut s);
|
||||
s
|
||||
}
|
||||
|
||||
// ── Derivation ────────────────────────────────────────────────────────────────
|
||||
|
||||
/// `Argon2id(password, salt)` → 256-bit key.
|
||||
///
|
||||
/// Runs on a blocking thread — it burns a core and 256 MiB for about a second,
|
||||
/// which would stall a tokio worker — and behind [`KDF_GATE`].
|
||||
pub async fn derive_kek(password: &str, salt: &[u8], params: &KdfParams) -> Result<Kek> {
|
||||
let _permit = KDF_GATE.acquire().await.context("kdf gate closed")?;
|
||||
|
||||
let mut password = password.to_owned();
|
||||
let salt = salt.to_vec();
|
||||
let params = params.clone();
|
||||
|
||||
let out = tokio::task::spawn_blocking(move || {
|
||||
let result = derive_blocking(password.as_bytes(), &salt, ¶ms);
|
||||
password.zeroize();
|
||||
result
|
||||
})
|
||||
.await
|
||||
.context("argon2 task panicked")?;
|
||||
|
||||
out
|
||||
}
|
||||
|
||||
fn derive_blocking(password: &[u8], salt: &[u8], params: &KdfParams) -> Result<Kek> {
|
||||
if params.algo != ALGO {
|
||||
bail!("unsupported kdf algorithm: {}", params.algo);
|
||||
}
|
||||
let p = Params::new(params.m, params.t, params.p, Some(KEY_LEN))
|
||||
.map_err(|e| anyhow!("invalid argon2 params: {e}"))?;
|
||||
let mut key = [0u8; KEY_LEN];
|
||||
Argon2::new(Algorithm::Argon2id, Version::V0x13, p)
|
||||
.hash_password_into(password, salt, &mut key)
|
||||
.map_err(|e| anyhow!("argon2 derivation failed: {e}"))?;
|
||||
Ok(Kek(key))
|
||||
}
|
||||
|
||||
/// Constant-time comparison of a freshly derived key against a stored verifier.
|
||||
/// `subtle` returns "not equal" for a length mismatch rather than short-circuiting.
|
||||
pub fn verify(derived: &Kek, stored: &[u8]) -> bool {
|
||||
derived.0.ct_eq(stored).into()
|
||||
}
|
||||
|
||||
// ── Envelope ──────────────────────────────────────────────────────────────────
|
||||
|
||||
/// Why a seal did not open. The distinction is the whole point: a failed tag is
|
||||
/// an authentication answer, a malformed blob is a broken row.
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
pub enum KeyError {
|
||||
/// The AEAD tag did not verify. This *is* the wrong-password signal.
|
||||
WrongPassword,
|
||||
Malformed(&'static str),
|
||||
}
|
||||
|
||||
impl fmt::Display for KeyError {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
match self {
|
||||
KeyError::WrongPassword => f.write_str("wrong password"),
|
||||
KeyError::Malformed(w) => write!(f, "malformed wrapped key: {w}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for KeyError {}
|
||||
|
||||
/// Seals `dek` under `kek`. Output is `nonce ‖ ciphertext ‖ tag`, stored as-is in
|
||||
/// `users.database_password`.
|
||||
pub fn wrap_dek(kek: &Kek, dek: &Dek) -> Result<Vec<u8>> {
|
||||
let cipher = Aes256Gcm::new_from_slice(&kek.0).map_err(|e| anyhow!("bad kek length: {e}"))?;
|
||||
|
||||
let mut nonce = [0u8; NONCE_LEN];
|
||||
rand::rng().fill_bytes(&mut nonce);
|
||||
|
||||
let sealed = cipher
|
||||
.encrypt(Nonce::from_slice(&nonce), dek.0.as_slice())
|
||||
.map_err(|_| anyhow!("AEAD seal failed"))?;
|
||||
|
||||
let mut out = Vec::with_capacity(SEALED_LEN);
|
||||
out.extend_from_slice(&nonce);
|
||||
out.extend_from_slice(&sealed);
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Opens a seal produced by [`wrap_dek`]. A failed tag returns
|
||||
/// [`KeyError::WrongPassword`] — one Argon2id pass answers "is the password
|
||||
/// right" *and* hands back the key.
|
||||
pub fn unwrap_dek(kek: &Kek, sealed: &[u8]) -> Result<Dek, KeyError> {
|
||||
if sealed.len() != SEALED_LEN {
|
||||
return Err(KeyError::Malformed("unexpected length"));
|
||||
}
|
||||
let (nonce, body) = sealed.split_at(NONCE_LEN);
|
||||
|
||||
let cipher =
|
||||
Aes256Gcm::new_from_slice(&kek.0).map_err(|_| KeyError::Malformed("bad kek length"))?;
|
||||
|
||||
let mut plain = cipher
|
||||
.decrypt(Nonce::from_slice(nonce), body)
|
||||
.map_err(|_| KeyError::WrongPassword)?;
|
||||
|
||||
if plain.len() != KEY_LEN {
|
||||
plain.zeroize();
|
||||
return Err(KeyError::Malformed("unexpected plaintext length"));
|
||||
}
|
||||
|
||||
let mut dek = [0u8; KEY_LEN];
|
||||
dek.copy_from_slice(&plain);
|
||||
plain.zeroize();
|
||||
Ok(Dek(dek))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
async fn kek(password: &str, salt: &[u8]) -> Kek {
|
||||
derive_kek(password, salt, &KdfParams::fast()).await.unwrap()
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn seal_round_trips_and_the_tag_is_the_password_check() {
|
||||
let salt = random_salt();
|
||||
let dek = Dek::random();
|
||||
let sealed = wrap_dek(&kek("correct horse", &salt).await, &dek).unwrap();
|
||||
assert_eq!(sealed.len(), SEALED_LEN);
|
||||
|
||||
let opened = unwrap_dek(&kek("correct horse", &salt).await, &sealed).unwrap();
|
||||
assert_eq!(opened.0, dek.0, "the same DEK must come back out");
|
||||
|
||||
// The whole auth story: a wrong password is a failed AEAD tag, and it is
|
||||
// reported as such — not as a corrupt blob.
|
||||
let err = unwrap_dek(&kek("wrong horse", &salt).await, &sealed).unwrap_err();
|
||||
assert_eq!(err, KeyError::WrongPassword);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_truncated_seal_is_malformed_not_a_wrong_password() {
|
||||
let salt = random_salt();
|
||||
let sealed = wrap_dek(&kek("pw", &salt).await, &Dek::random()).unwrap();
|
||||
let err = unwrap_dek(&kek("pw", &salt).await, &sealed[..SEALED_LEN - 1]).unwrap_err();
|
||||
assert!(matches!(err, KeyError::Malformed(_)), "got {err:?}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn the_salt_separates_identical_passwords() {
|
||||
let a = kek("same", &random_salt()).await;
|
||||
let b = kek("same", &random_salt()).await;
|
||||
assert_ne!(a.0, b.0, "a per-user salt must defuse rainbow tables");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn verify_accepts_the_right_password_and_rejects_a_short_verifier() {
|
||||
let salt = random_salt();
|
||||
let stored = kek("pw", &salt).await;
|
||||
assert!(verify(&kek("pw", &salt).await, stored.as_verifier()));
|
||||
assert!(!verify(&kek("nope", &salt).await, stored.as_verifier()));
|
||||
// A length mismatch must not panic, and must not compare equal.
|
||||
assert!(!verify(&kek("pw", &salt).await, &stored.as_verifier()[..16]));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pragma_is_a_quoted_blob_literal() {
|
||||
let dek = Dek([0xAB; KEY_LEN]);
|
||||
let p = dek.to_pragma();
|
||||
assert!(p.starts_with("\"x'") && p.ends_with("'\""), "got {p}");
|
||||
assert_eq!(p.len(), 5 + 2 * KEY_LEN, "\"x' + 64 hex digits + '\"");
|
||||
assert!(p.contains("abab"), "lowercase hex");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn debug_never_prints_key_material() {
|
||||
let dek = Dek([0xDE; KEY_LEN]);
|
||||
let kek = Kek([0xAD; KEY_LEN]);
|
||||
assert_eq!(format!("{dek:?}"), "Dek(<redacted>)");
|
||||
assert_eq!(format!("{kek:?}"), "Kek(<redacted>)");
|
||||
assert!(!format!("{dek:?}").contains("dede"), "no raw key bytes");
|
||||
assert!(!format!("{kek:?}").contains("adad"), "no raw key bytes");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn kdf_params_round_trip_and_reject_foreign_algorithms() {
|
||||
let p = KdfParams::default();
|
||||
assert_eq!(p.m, 262_144);
|
||||
assert_eq!(KdfParams::from_json(&p.to_json().unwrap()).unwrap(), p);
|
||||
|
||||
let pbkdf2 = r#"{"algo":"pbkdf2","m":1,"t":1,"p":1}"#;
|
||||
assert!(KdfParams::from_json(pbkdf2).is_err(), "only argon2id is accepted");
|
||||
}
|
||||
}
|
||||
@@ -1,7 +1,7 @@
|
||||
use anyhow::Result;
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
use crate::core::approval::{ApprovalRule, NewApprovalRule, RuleAction};
|
||||
use crate::approval::{ApprovalRule, NewApprovalRule, RuleAction};
|
||||
|
||||
type RawRow = (i64, Option<String>, Option<String>, String, Option<String>, String, Option<String>, i64, Option<String>);
|
||||
|
||||
@@ -191,15 +191,6 @@ pub async fn for_stack_all(
|
||||
rows.into_iter().map(row_to_message).collect()
|
||||
}
|
||||
|
||||
pub async fn set_model_db_id(pool: &SqlitePool, id: i64, model_db_id: i64) -> anyhow::Result<()> {
|
||||
sqlx::query("UPDATE chat_history SET model_db_id = ? WHERE id = ?")
|
||||
.bind(model_db_id)
|
||||
.bind(id)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Ok messages for a stack frame whose id is strictly greater than `after_id`,
|
||||
/// ordered chronologically. Used by `build_openai_messages` when a compaction
|
||||
/// summary exists: only the "raw" messages after the summary boundary are loaded.
|
||||
+1
-1
@@ -164,7 +164,7 @@ mod tests {
|
||||
#[tokio::test]
|
||||
async fn active_all_reflects_parallel_siblings() {
|
||||
let path = temp_db_path("stack-parallel");
|
||||
let pool = crate::core::db::init_pool(&path).await.unwrap();
|
||||
let pool = crate::db::init_system_pool(&path).await.unwrap();
|
||||
let sid = 1;
|
||||
// `session_id` has a FK to chat_sessions (sqlx enables foreign_keys).
|
||||
sqlx::query("INSERT INTO chat_sessions (id) VALUES (?)")
|
||||
@@ -1,5 +1,5 @@
|
||||
//! `known_tools` — every tool ever offered to the LLM, captured at injection
|
||||
//! time by [`crate::core::tool_discovery::ToolDiscovery`].
|
||||
//! time by [`crate::tool_discovery::ToolDiscovery`].
|
||||
//!
|
||||
//! This is the drift-proof half of tool visibility: instead of maintaining a
|
||||
//! parallel list of "all tools", we record what is actually assembled into the
|
||||
@@ -77,7 +77,7 @@ mod tests {
|
||||
#[tokio::test]
|
||||
async fn upsert_is_idempotent_and_updates_metadata() {
|
||||
let path = temp_db_path("known-tools");
|
||||
let pool = crate::core::db::init_pool(&path).await.unwrap();
|
||||
let pool = crate::db::init_system_pool(&path).await.unwrap();
|
||||
|
||||
upsert(&pool, "send_voice_message", "v1", Some(r#"{"type":"object"}"#)).await.unwrap();
|
||||
upsert(&pool, "send_voice_message", "v2", None).await.unwrap();
|
||||
+1
-1
@@ -13,7 +13,7 @@ use tokio::task::JoinHandle;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
use tracing::{info, warn};
|
||||
|
||||
use crate::core::config::LlmRequestsLogConfig;
|
||||
use crate::config::LlmRequestsLogConfig;
|
||||
|
||||
/// Spawns the retention/cleanup loop for the `llm_requests` table.
|
||||
///
|
||||
@@ -1,7 +1,7 @@
|
||||
//! DB operations for the `llm_requests` table.
|
||||
//!
|
||||
//! Every `chat_with_tools` call is logged here by the
|
||||
//! [`crate::core::chatbot::logging::LoggingChatbotClient`] wrapper.
|
||||
//! [`crate::chatbot::logging::LoggingChatbotClient`] wrapper.
|
||||
//! Rows are retained for `llm.request_log.retention_days` days (default 14).
|
||||
|
||||
use anyhow::Result;
|
||||
@@ -21,147 +21,138 @@ pub mod stack_mcp_grants;
|
||||
pub mod tool_permission_groups;
|
||||
pub mod users;
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use sqlx::{SqlitePool, sqlite::{SqliteConnectOptions, SqliteJournalMode, SqliteSynchronous}};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::str::FromStr;
|
||||
use std::time::Duration;
|
||||
|
||||
/// System database: instance-wide state, shared by every user.
|
||||
///
|
||||
/// Fixed path — not configurable. Sibling `database/{userid}.db` files hold
|
||||
/// per-user content; keeping them in one directory makes backup, export and
|
||||
/// per-user erasure a matter of files rather than tables.
|
||||
use anyhow::{Context, Result};
|
||||
use sqlx::{SqlitePool, sqlite::{SqliteConnectOptions, SqliteJournalMode, SqliteSynchronous}};
|
||||
|
||||
use crate::crypto::Dek;
|
||||
|
||||
/// Every database file lives here, so backup, export and per-user erasure are a
|
||||
/// matter of files rather than tables.
|
||||
pub const DATABASE_DIR: &str = "database";
|
||||
|
||||
/// System database: instance-wide state, shared by every user. Fixed path — not
|
||||
/// configurable.
|
||||
pub const SYSTEM_DB_PATH: &str = "database/system.db";
|
||||
|
||||
pub async fn init_pool(path: &str) -> Result<SqlitePool> {
|
||||
/// `{dir}/{userid}.db`. Keyed by the opaque user id and never the username, so a
|
||||
/// rename never has to touch the file. `dir` is a parameter rather than the
|
||||
/// [`DATABASE_DIR`] constant so nothing depends on the process's working
|
||||
/// directory — tests in particular.
|
||||
pub fn user_db_path(dir: &Path, user_id: &str) -> PathBuf {
|
||||
dir.join(format!("{user_id}.db"))
|
||||
}
|
||||
|
||||
/// The `-wal` and `-shm` sidecars SQLite keeps beside a database in WAL mode.
|
||||
/// Erasing a user means erasing these too.
|
||||
pub fn user_db_sidecars(path: &Path) -> [PathBuf; 2] {
|
||||
let ext = |suffix: &str| {
|
||||
let mut p = path.to_path_buf().into_os_string();
|
||||
p.push(suffix);
|
||||
PathBuf::from(p)
|
||||
};
|
||||
[ext("-wal"), ext("-shm")]
|
||||
}
|
||||
|
||||
fn ensure_parent(path: &Path) -> Result<()> {
|
||||
// `create_if_missing` creates the file, never its parent directory.
|
||||
if let Some(parent) = std::path::Path::new(path).parent()
|
||||
if let Some(parent) = path.parent()
|
||||
&& !parent.as_os_str().is_empty()
|
||||
{
|
||||
std::fs::create_dir_all(parent)
|
||||
.with_context(|| format!("failed to create database directory {}", parent.display()))?;
|
||||
}
|
||||
let opts = SqliteConnectOptions::from_str(path)?
|
||||
.create_if_missing(true)
|
||||
// WAL lets readers run alongside a single writer, and `busy_timeout`
|
||||
// makes a writer *wait* for the lock instead of failing immediately with
|
||||
// SQLITE_BUSY ("database is locked"). Without these, concurrent writers —
|
||||
// e.g. the mobile-connector persisting its E2E `send_counter` while the
|
||||
// chat loop / cron write history — abort mid-operation, which silently
|
||||
// drops outbound mobile messages (inbox_update never reaches the device).
|
||||
.journal_mode(SqliteJournalMode::Wal)
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn tuned(opts: SqliteConnectOptions) -> SqliteConnectOptions {
|
||||
// WAL lets readers run alongside a single writer, and `busy_timeout` makes a
|
||||
// writer *wait* for the lock instead of failing immediately with SQLITE_BUSY
|
||||
// ("database is locked"). Without these, concurrent writers — e.g. the
|
||||
// mobile-connector persisting its E2E `send_counter` while the chat loop /
|
||||
// cron write history — abort mid-operation, which silently drops outbound
|
||||
// mobile messages (inbox_update never reaches the device).
|
||||
opts.journal_mode(SqliteJournalMode::Wal)
|
||||
.synchronous(SqliteSynchronous::Normal)
|
||||
.busy_timeout(Duration::from_secs(5));
|
||||
.busy_timeout(Duration::from_secs(5))
|
||||
}
|
||||
|
||||
/// Connect options for one user's database.
|
||||
///
|
||||
/// When `key` is present the pool is a SQLCipher pool: sqlx puts `PRAGMA key`
|
||||
/// ahead of every other pragma, so the page cipher is armed before `journal_mode`
|
||||
/// touches the file. Without a key the same code opens an ordinary SQLite file —
|
||||
/// which is exactly what a cleartext user gets.
|
||||
///
|
||||
/// The returned value carries the raw key. **Never** `{:?}` it: `SqliteConnectOptions`
|
||||
/// prints its pragmas, so a debug format anywhere near this would write the DEK,
|
||||
/// in hex, into the logs.
|
||||
fn user_options(path: &Path, key: Option<&Dek>, create: bool) -> SqliteConnectOptions {
|
||||
let opts = SqliteConnectOptions::new().filename(path).create_if_missing(create);
|
||||
let opts = match key {
|
||||
Some(dek) => opts.pragma("key", dek.to_pragma()),
|
||||
None => opts,
|
||||
};
|
||||
tuned(opts)
|
||||
}
|
||||
|
||||
/// Forces a page read so a wrong key fails here, at open time, rather than at the
|
||||
/// first unrelated query. SQLCipher answers "file is not a database" when the key
|
||||
/// does not decrypt the header.
|
||||
async fn probe(pool: &SqlitePool) -> Result<()> {
|
||||
sqlx::query_scalar::<_, i64>("SELECT count(*) FROM sqlite_master")
|
||||
.fetch_one(pool)
|
||||
.await
|
||||
.context("database did not open — wrong key, or not a database")?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The system database: registry tables, plus — for now — the owner tables.
|
||||
///
|
||||
/// Owner tables live here **transitionally**. Nothing has been migrated to
|
||||
/// per-user pools yet, so sessions, history and jobs still land in `system.db`.
|
||||
/// When the call sites move to `UserManager::pool_of`, this second call goes
|
||||
/// away and the owner-without-a-user gets a file of its own.
|
||||
pub async fn init_system_pool(path: &str) -> Result<SqlitePool> {
|
||||
ensure_parent(Path::new(path))?;
|
||||
let opts = tuned(SqliteConnectOptions::from_str(path)?.create_if_missing(true));
|
||||
let pool = SqlitePool::connect_with(opts).await?;
|
||||
create_tables(&pool).await?;
|
||||
create_registry_tables(&pool).await?;
|
||||
create_owner_tables(&pool).await?;
|
||||
crate::boot::section("Database initialised".to_string());
|
||||
Ok(pool)
|
||||
}
|
||||
|
||||
async fn create_tables(pool: &SqlitePool) -> Result<()> {
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS chat_sessions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
title TEXT,
|
||||
source TEXT NOT NULL DEFAULT 'web',
|
||||
agent_id TEXT NOT NULL DEFAULT 'main',
|
||||
is_interactive INTEGER NOT NULL DEFAULT 1,
|
||||
is_ephemeral INTEGER NOT NULL DEFAULT 0,
|
||||
run_context TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
/// Provisions `database/{userid}.db` and lays down the owner schema.
|
||||
///
|
||||
/// Only this function may create a user's database. Login goes through
|
||||
/// [`open_user_pool`], which refuses to create anything: a missing file there is
|
||||
/// data loss, and creating a fresh empty one under the right password would hide
|
||||
/// it instead of reporting it.
|
||||
pub async fn create_user_pool(path: &Path, key: Option<&Dek>) -> Result<SqlitePool> {
|
||||
ensure_parent(path)?;
|
||||
let pool = SqlitePool::connect_with(user_options(path, key, true)).await?;
|
||||
probe(&pool).await?;
|
||||
create_owner_tables(&pool).await?;
|
||||
Ok(pool)
|
||||
}
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS chat_sessions_stack (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_id INTEGER NOT NULL REFERENCES chat_sessions(id),
|
||||
agent_id TEXT NOT NULL DEFAULT 'main',
|
||||
agent_prompt TEXT,
|
||||
depth INTEGER NOT NULL DEFAULT 0,
|
||||
parent_tool_call_id INTEGER,
|
||||
terminated_at TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
/// Opens an existing user database. Never creates one — see [`create_user_pool`].
|
||||
pub async fn open_user_pool(path: &Path, key: Option<&Dek>) -> Result<SqlitePool> {
|
||||
let pool = SqlitePool::connect_with(user_options(path, key, false)).await?;
|
||||
probe(&pool).await?;
|
||||
Ok(pool)
|
||||
}
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS chat_history (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_stack_id INTEGER NOT NULL REFERENCES chat_sessions_stack(id),
|
||||
role TEXT NOT NULL CHECK(role IN ('user', 'assistant', 'agent')),
|
||||
content TEXT NOT NULL DEFAULT '',
|
||||
status TEXT NOT NULL DEFAULT 'ok' CHECK(status IN ('ok', 'failed')),
|
||||
input_tokens INTEGER,
|
||||
output_tokens INTEGER,
|
||||
duration_ms INTEGER,
|
||||
model_db_id INTEGER REFERENCES llm_models(id),
|
||||
is_synthetic INTEGER NOT NULL DEFAULT 0,
|
||||
reasoning_content TEXT,
|
||||
cost REAL,
|
||||
metadata TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS chat_llm_tools (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
message_id INTEGER NOT NULL REFERENCES chat_history(id),
|
||||
name TEXT NOT NULL,
|
||||
arguments TEXT,
|
||||
result TEXT,
|
||||
status TEXT NOT NULL DEFAULT 'running' CHECK(status IN ('running', 'pending', 'done', 'failed', 'cancelled', 'rejected')),
|
||||
result_type TEXT NOT NULL DEFAULT 'string' CHECK(result_type IN ('string', 'json')),
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_stack_session ON chat_sessions_stack(session_id)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_history_stack ON chat_history(session_stack_id)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_tools_message ON chat_llm_tools(message_id)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS mcp_servers (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
name TEXT NOT NULL UNIQUE,
|
||||
transport TEXT NOT NULL DEFAULT 'stdio',
|
||||
command TEXT,
|
||||
args_json TEXT,
|
||||
env_json TEXT,
|
||||
url TEXT,
|
||||
api_key TEXT,
|
||||
description TEXT,
|
||||
friendly_name TEXT,
|
||||
enabled INTEGER NOT NULL DEFAULT 1,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
// ── Registry tables ───────────────────────────────────────────────────────────
|
||||
//
|
||||
// Instance-wide, readable without any user key: the directory you must open
|
||||
// before you know who exists. Nothing here is scoped to one user.
|
||||
|
||||
async fn create_registry_tables(pool: &SqlitePool) -> Result<()> {
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS llm_providers (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
@@ -205,82 +196,6 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS scheduled_jobs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
title TEXT NOT NULL,
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
cron TEXT NOT NULL,
|
||||
prompt TEXT NOT NULL,
|
||||
agent_id TEXT NOT NULL DEFAULT 'main',
|
||||
session_id INTEGER REFERENCES chat_sessions(id),
|
||||
enabled INTEGER NOT NULL DEFAULT 1,
|
||||
last_run_at TEXT,
|
||||
next_run_at TEXT,
|
||||
single_run INTEGER NOT NULL DEFAULT 0,
|
||||
running_session_id INTEGER,
|
||||
kind TEXT NOT NULL DEFAULT 'cron',
|
||||
parent_session_id INTEGER REFERENCES chat_sessions(id),
|
||||
run_context TEXT,
|
||||
running_since TEXT,
|
||||
origin_ref TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS plugins (
|
||||
id TEXT PRIMARY KEY,
|
||||
enabled INTEGER NOT NULL DEFAULT 0,
|
||||
config TEXT NOT NULL DEFAULT '{}',
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS session_scratchpad (
|
||||
session_id INTEGER NOT NULL REFERENCES chat_sessions(id),
|
||||
key TEXT NOT NULL,
|
||||
value TEXT NOT NULL,
|
||||
PRIMARY KEY (session_id, key)
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS tool_permission_groups (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
description TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS approval_rules (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
agent_id TEXT,
|
||||
source TEXT,
|
||||
tool_pattern TEXT NOT NULL,
|
||||
action TEXT NOT NULL DEFAULT 'require'
|
||||
CHECK(action IN ('require', 'allow', 'deny')),
|
||||
note TEXT,
|
||||
priority INTEGER NOT NULL DEFAULT 100,
|
||||
path_pattern TEXT,
|
||||
group_id TEXT REFERENCES tool_permission_groups(id),
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS transcribe_models (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
@@ -332,10 +247,40 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS sources (
|
||||
id TEXT PRIMARY KEY,
|
||||
active_session_id INTEGER REFERENCES chat_sessions(id),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
"CREATE TABLE IF NOT EXISTS plugins (
|
||||
id TEXT PRIMARY KEY,
|
||||
enabled INTEGER NOT NULL DEFAULT 0,
|
||||
config TEXT NOT NULL DEFAULT '{}',
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS tool_permission_groups (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
description TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS approval_rules (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
agent_id TEXT,
|
||||
source TEXT,
|
||||
tool_pattern TEXT NOT NULL,
|
||||
action TEXT NOT NULL DEFAULT 'require'
|
||||
CHECK(action IN ('require', 'allow', 'deny')),
|
||||
note TEXT,
|
||||
priority INTEGER NOT NULL DEFAULT 100,
|
||||
path_pattern TEXT,
|
||||
group_id TEXT REFERENCES tool_permission_groups(id),
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
@@ -351,63 +296,26 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
// Every tool ever offered to the LLM, recorded by `ToolDiscovery` at
|
||||
// injection time. Lets the approval / Security-groups UI list and gate tools
|
||||
// that are injected dynamically outside the `ToolRegistry` (interface tools,
|
||||
// plugin tools, provider tools).
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS secrets (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS mcp_events (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
source TEXT NOT NULL,
|
||||
method TEXT NOT NULL,
|
||||
payload TEXT NOT NULL,
|
||||
processed INTEGER NOT NULL DEFAULT 0,
|
||||
processed_at TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_mcp_events_pending
|
||||
ON mcp_events (processed, created_at)
|
||||
WHERE processed = 0",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS session_mcp_grants (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_id INTEGER NOT NULL,
|
||||
mcp_name TEXT NOT NULL,
|
||||
granted_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
UNIQUE(session_id, mcp_name)
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS stack_mcp_grants (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
stack_id INTEGER NOT NULL,
|
||||
mcp_name TEXT NOT NULL,
|
||||
granted_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
UNIQUE(stack_id, mcp_name)
|
||||
"CREATE TABLE IF NOT EXISTS known_tools (
|
||||
name TEXT PRIMARY KEY,
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
schema TEXT,
|
||||
first_seen INTEGER NOT NULL DEFAULT (strftime('%s','now')),
|
||||
last_seen INTEGER NOT NULL DEFAULT (strftime('%s','now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
// Spend/telemetry metadata. Stays in the registry so cost charts run without
|
||||
// decrypting anything: the admin sees how much, when and which model — never
|
||||
// what was said. `session_id` / `stack_id` are bare integers, not foreign
|
||||
// keys, precisely because the rows they point at live in another file.
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS llm_requests (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
@@ -437,6 +345,140 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
// User directory + auth material. Read before every login, so it lives in
|
||||
// the registry — which means it must never hold anything that derives a
|
||||
// user's key: `database_password` is the DEK sealed under a key derived
|
||||
// from the password, useless without it.
|
||||
//
|
||||
// `role_id` has no `REFERENCES roles(id)` yet: sqlx turns on
|
||||
// `PRAGMA foreign_keys`, so pointing at a table that does not exist would
|
||||
// make every INSERT fail. The constraint lands with the `roles` table.
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS users (
|
||||
id TEXT PRIMARY KEY,
|
||||
username TEXT NOT NULL UNIQUE,
|
||||
display_name TEXT,
|
||||
role_id TEXT NOT NULL,
|
||||
encrypted INTEGER NOT NULL,
|
||||
kdf_params TEXT,
|
||||
kdf_salt BLOB,
|
||||
database_password BLOB,
|
||||
password_hash BLOB,
|
||||
active INTEGER NOT NULL DEFAULT 1,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
CHECK (
|
||||
(encrypted = 1 AND database_password IS NOT NULL AND password_hash IS NULL)
|
||||
OR (encrypted = 0 AND database_password IS NULL)
|
||||
)
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// ── Owner tables ──────────────────────────────────────────────────────────────
|
||||
//
|
||||
// One owner's content. The schema is identical in every file that has it — only
|
||||
// the file says who owns the rows — so this runs verbatim against `system.db`
|
||||
// and against each `database/{userid}.db`.
|
||||
//
|
||||
// **No foreign key here may point at a registry table.** SQLite cannot enforce a
|
||||
// key across files, not even through `ATTACH`, and sqlx turns on
|
||||
// `PRAGMA foreign_keys`: the `CREATE TABLE` would succeed and every `INSERT`
|
||||
// would fail. `create_owner_tables_stand_alone` in the tests guards this by
|
||||
// running the schema against a file that has nothing else in it.
|
||||
|
||||
pub async fn create_owner_tables(pool: &SqlitePool) -> Result<()> {
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS chat_sessions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
title TEXT,
|
||||
source TEXT NOT NULL DEFAULT 'web',
|
||||
agent_id TEXT NOT NULL DEFAULT 'main',
|
||||
is_interactive INTEGER NOT NULL DEFAULT 1,
|
||||
is_ephemeral INTEGER NOT NULL DEFAULT 0,
|
||||
run_context TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS chat_sessions_stack (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_id INTEGER NOT NULL REFERENCES chat_sessions(id),
|
||||
agent_id TEXT NOT NULL DEFAULT 'main',
|
||||
agent_prompt TEXT,
|
||||
depth INTEGER NOT NULL DEFAULT 0,
|
||||
parent_tool_call_id INTEGER,
|
||||
terminated_at TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
// `model_db_id` used to point at `llm_models(id)`. It was write-only — never
|
||||
// selected, never joined — and it was the one key that crossed into the
|
||||
// registry. Which model answered is already recorded in
|
||||
// `llm_requests.model_name`.
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS chat_history (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_stack_id INTEGER NOT NULL REFERENCES chat_sessions_stack(id),
|
||||
role TEXT NOT NULL CHECK(role IN ('user', 'assistant', 'agent')),
|
||||
content TEXT NOT NULL DEFAULT '',
|
||||
status TEXT NOT NULL DEFAULT 'ok' CHECK(status IN ('ok', 'failed')),
|
||||
input_tokens INTEGER,
|
||||
output_tokens INTEGER,
|
||||
duration_ms INTEGER,
|
||||
is_synthetic INTEGER NOT NULL DEFAULT 0,
|
||||
reasoning_content TEXT,
|
||||
cost REAL,
|
||||
metadata TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS chat_llm_tools (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
message_id INTEGER NOT NULL REFERENCES chat_history(id),
|
||||
name TEXT NOT NULL,
|
||||
arguments TEXT,
|
||||
result TEXT,
|
||||
status TEXT NOT NULL DEFAULT 'running' CHECK(status IN ('running', 'pending', 'done', 'failed', 'cancelled', 'rejected')),
|
||||
result_type TEXT NOT NULL DEFAULT 'string' CHECK(result_type IN ('string', 'json')),
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_stack_session ON chat_sessions_stack(session_id)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_history_stack ON chat_history(session_stack_id)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_tools_message ON chat_llm_tools(message_id)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS chat_summaries (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
@@ -456,6 +498,66 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS session_scratchpad (
|
||||
session_id INTEGER NOT NULL REFERENCES chat_sessions(id),
|
||||
key TEXT NOT NULL,
|
||||
value TEXT NOT NULL,
|
||||
PRIMARY KEY (session_id, key)
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS session_mcp_grants (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_id INTEGER NOT NULL,
|
||||
mcp_name TEXT NOT NULL,
|
||||
granted_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
UNIQUE(session_id, mcp_name)
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS stack_mcp_grants (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
stack_id INTEGER NOT NULL,
|
||||
mcp_name TEXT NOT NULL,
|
||||
granted_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
UNIQUE(stack_id, mcp_name)
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS scheduled_jobs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
title TEXT NOT NULL,
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
cron TEXT NOT NULL,
|
||||
prompt TEXT NOT NULL,
|
||||
agent_id TEXT NOT NULL DEFAULT 'main',
|
||||
session_id INTEGER REFERENCES chat_sessions(id),
|
||||
enabled INTEGER NOT NULL DEFAULT 1,
|
||||
last_run_at TEXT,
|
||||
next_run_at TEXT,
|
||||
single_run INTEGER NOT NULL DEFAULT 0,
|
||||
running_session_id INTEGER,
|
||||
kind TEXT NOT NULL DEFAULT 'cron',
|
||||
parent_session_id INTEGER REFERENCES chat_sessions(id),
|
||||
run_context TEXT,
|
||||
running_since TEXT,
|
||||
origin_ref TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS job_runs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
@@ -481,6 +583,68 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS mcp_servers (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
name TEXT NOT NULL UNIQUE,
|
||||
transport TEXT NOT NULL DEFAULT 'stdio',
|
||||
command TEXT,
|
||||
args_json TEXT,
|
||||
env_json TEXT,
|
||||
url TEXT,
|
||||
api_key TEXT,
|
||||
description TEXT,
|
||||
friendly_name TEXT,
|
||||
enabled INTEGER NOT NULL DEFAULT 1,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS mcp_events (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
source TEXT NOT NULL,
|
||||
method TEXT NOT NULL,
|
||||
payload TEXT NOT NULL,
|
||||
processed INTEGER NOT NULL DEFAULT 0,
|
||||
processed_at TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_mcp_events_pending
|
||||
ON mcp_events (processed, created_at)
|
||||
WHERE processed = 0",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS sources (
|
||||
id TEXT PRIMARY KEY,
|
||||
active_session_id INTEGER REFERENCES chat_sessions(id),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS secrets (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS projects (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
@@ -516,52 +680,111 @@ async fn create_tables(pool: &SqlitePool) -> Result<()> {
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
// Every tool ever offered to the LLM, recorded by `ToolDiscovery` at
|
||||
// injection time. Lets the approval / Security-groups UI list and gate tools
|
||||
// that are injected dynamically outside the `ToolRegistry` (interface tools,
|
||||
// plugin tools, provider tools). See docs/approval + docs/tools.
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS known_tools (
|
||||
name TEXT PRIMARY KEY,
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
schema TEXT,
|
||||
first_seen INTEGER NOT NULL DEFAULT (strftime('%s','now')),
|
||||
last_seen INTEGER NOT NULL DEFAULT (strftime('%s','now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
// User directory + auth material. Read before every login, so it lives in
|
||||
// the system DB — which means it must never hold anything that derives a
|
||||
// user's key: `database_password` is the DEK sealed under a key derived
|
||||
// from the password, useless without it.
|
||||
//
|
||||
// `role_id` has no `REFERENCES roles(id)` yet: sqlx turns on
|
||||
// `PRAGMA foreign_keys`, so pointing at a table that does not exist would
|
||||
// make every INSERT fail. The constraint lands with the `roles` table.
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS users (
|
||||
id TEXT PRIMARY KEY,
|
||||
username TEXT NOT NULL UNIQUE,
|
||||
display_name TEXT,
|
||||
role_id TEXT NOT NULL,
|
||||
encrypted INTEGER NOT NULL,
|
||||
kdf_params TEXT,
|
||||
kdf_salt BLOB,
|
||||
database_password BLOB,
|
||||
password_hash BLOB,
|
||||
active INTEGER NOT NULL DEFAULT 1,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
CHECK (
|
||||
(encrypted = 1 AND database_password IS NOT NULL AND password_hash IS NULL)
|
||||
OR (encrypted = 0 AND database_password IS NULL)
|
||||
)
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn temp_dir(tag: &str) -> PathBuf {
|
||||
let nanos = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos();
|
||||
let mut p = std::env::temp_dir();
|
||||
p.push(format!("skald-db-{tag}-{}-{nanos}", std::process::id()));
|
||||
p
|
||||
}
|
||||
|
||||
/// The guardrail the file split would have given for free.
|
||||
///
|
||||
/// `create_owner_tables` runs here against a database that holds *nothing
|
||||
/// else* — no `llm_models`, no `users`. Every table then takes a row. Any
|
||||
/// foreign key that reaches into a registry table passes `CREATE TABLE` and
|
||||
/// dies on the `INSERT`, right here, instead of lying dormant until a real
|
||||
/// user logs in and writes to their own file.
|
||||
#[tokio::test]
|
||||
async fn owner_tables_stand_alone_with_foreign_keys_on() {
|
||||
let dir = temp_dir("owner-standalone");
|
||||
let path = dir.join("owner.db");
|
||||
let pool = create_user_pool(&path, None).await.unwrap();
|
||||
|
||||
let (fk,): (i64,) = sqlx::query_as("PRAGMA foreign_keys")
|
||||
.fetch_one(&pool).await.unwrap();
|
||||
assert_eq!(fk, 1, "the guardrail is meaningless without FK enforcement");
|
||||
|
||||
let one = |q: &'static str| sqlx::query(q).execute(&pool);
|
||||
|
||||
one("INSERT INTO chat_sessions (id, title) VALUES (1, 't')").await.unwrap();
|
||||
one("INSERT INTO chat_sessions_stack (id, session_id) VALUES (1, 1)").await.unwrap();
|
||||
one("INSERT INTO chat_history (id, session_stack_id, role, content) VALUES (1, 1, 'user', 'hi')")
|
||||
.await.unwrap();
|
||||
one("INSERT INTO chat_llm_tools (message_id, name) VALUES (1, 'exec')").await.unwrap();
|
||||
one("INSERT INTO chat_summaries (stack_id, content, covers_up_to_message_id) VALUES (1, 's', 1)")
|
||||
.await.unwrap();
|
||||
one("INSERT INTO session_scratchpad (session_id, key, value) VALUES (1, 'k', 'v')").await.unwrap();
|
||||
one("INSERT INTO session_mcp_grants (session_id, mcp_name) VALUES (1, 'm')").await.unwrap();
|
||||
one("INSERT INTO stack_mcp_grants (stack_id, mcp_name) VALUES (1, 'm')").await.unwrap();
|
||||
one("INSERT INTO scheduled_jobs (id, title, cron, prompt, session_id) VALUES (1, 't', '* * * * *', 'p', 1)")
|
||||
.await.unwrap();
|
||||
one("INSERT INTO job_runs (job_id, started_at, status) VALUES (1, 'now', 'completed')").await.unwrap();
|
||||
one("INSERT INTO mcp_servers (name) VALUES ('srv')").await.unwrap();
|
||||
one("INSERT INTO mcp_events (source, method, payload) VALUES ('s', 'm', '{}')").await.unwrap();
|
||||
one("INSERT INTO sources (id, active_session_id) VALUES ('web', 1)").await.unwrap();
|
||||
one("INSERT INTO secrets (key, value) VALUES ('k', 'v')").await.unwrap();
|
||||
one("INSERT INTO projects (id, name, path) VALUES (1, 'p', '/tmp')").await.unwrap();
|
||||
one("INSERT INTO project_tickets (project_id, title, job_id) VALUES (1, 't', 1)").await.unwrap();
|
||||
|
||||
pool.close().await;
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
/// A cleartext user's database is an ordinary SQLite file; an encrypted one
|
||||
/// is unreadable without the key, and does not even carry SQLite's header.
|
||||
#[tokio::test]
|
||||
async fn an_encrypted_database_is_opaque_without_its_key() {
|
||||
let dir = temp_dir("opaque");
|
||||
let clear_path = dir.join("clear.db");
|
||||
let enc_path = dir.join("enc.db");
|
||||
|
||||
let clear = create_user_pool(&clear_path, None).await.unwrap();
|
||||
clear.close().await;
|
||||
|
||||
let dek = Dek::random();
|
||||
let enc = create_user_pool(&enc_path, Some(&dek)).await.unwrap();
|
||||
sqlx::query("INSERT INTO chat_sessions (title) VALUES ('secret')")
|
||||
.execute(&enc).await.unwrap();
|
||||
enc.close().await;
|
||||
|
||||
let magic = b"SQLite format 3\0";
|
||||
assert_eq!(&std::fs::read(&clear_path).unwrap()[..16], magic);
|
||||
assert_ne!(&std::fs::read(&enc_path).unwrap()[..16], magic,
|
||||
"an encrypted file must not advertise itself as SQLite");
|
||||
|
||||
assert!(open_user_pool(&enc_path, None).await.is_err(), "no key must not open it");
|
||||
assert!(open_user_pool(&enc_path, Some(&Dek::random())).await.is_err(), "wrong key must not open it");
|
||||
|
||||
// ...and the right key still does, with the row intact.
|
||||
let reopened = open_user_pool(&enc_path, Some(&dek)).await.unwrap();
|
||||
let (title,): (String,) = sqlx::query_as("SELECT title FROM chat_sessions")
|
||||
.fetch_one(&reopened).await.unwrap();
|
||||
assert_eq!(title, "secret");
|
||||
reopened.close().await;
|
||||
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
/// Login must never conjure a database. A missing file is data loss and has
|
||||
/// to be reported, not papered over with a fresh empty one.
|
||||
#[tokio::test]
|
||||
async fn open_never_creates_a_database() {
|
||||
let dir = temp_dir("no-create");
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let path = dir.join("ghost.db");
|
||||
|
||||
assert!(open_user_pool(&path, None).await.is_err());
|
||||
assert!(open_user_pool(&path, Some(&Dek::random())).await.is_err());
|
||||
assert!(!path.exists(), "open_user_pool must not have created the file");
|
||||
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
}
|
||||
@@ -358,7 +358,7 @@ pub async fn delete(pool: &SqlitePool, id: &str) -> Result<()> {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// Nested on purpose: also covers `init_pool` creating the parent directory.
|
||||
/// Nested on purpose: also covers `init_system_pool` creating the parent directory.
|
||||
fn temp_db_path(tag: &str) -> String {
|
||||
let mut p = std::env::temp_dir();
|
||||
let nanos = std::time::SystemTime::now()
|
||||
@@ -392,11 +392,11 @@ mod tests {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn init_pool_creates_the_database_directory() {
|
||||
async fn init_system_pool_creates_the_database_directory() {
|
||||
let path = temp_db_path("users-mkdir");
|
||||
assert!(!std::path::Path::new(&path).parent().unwrap().exists());
|
||||
|
||||
let pool = crate::core::db::init_pool(&path).await.unwrap();
|
||||
let pool = crate::db::init_system_pool(&path).await.unwrap();
|
||||
assert!(std::path::Path::new(&path).exists(), "system.db must exist under a fresh database/");
|
||||
|
||||
pool.close().await;
|
||||
@@ -406,7 +406,7 @@ mod tests {
|
||||
#[tokio::test]
|
||||
async fn encrypted_user_round_trips_and_keeps_the_wrapped_dek() {
|
||||
let path = temp_db_path("users-enc");
|
||||
let pool = crate::core::db::init_pool(&path).await.unwrap();
|
||||
let pool = crate::db::init_system_pool(&path).await.unwrap();
|
||||
|
||||
insert(&pool, "u-1", "ada", Some("Ada"), "admin", &encrypted()).await.unwrap();
|
||||
|
||||
@@ -429,7 +429,7 @@ mod tests {
|
||||
#[tokio::test]
|
||||
async fn cleartext_user_round_trips_with_and_without_a_verifier() {
|
||||
let path = temp_db_path("users-clear");
|
||||
let pool = crate::core::db::init_pool(&path).await.unwrap();
|
||||
let pool = crate::db::init_system_pool(&path).await.unwrap();
|
||||
|
||||
insert(&pool, "u-1", "kid", None, "children", &cleartext()).await.unwrap();
|
||||
insert(&pool, "u-2", "kiosk", None, "children", &Credentials::Cleartext(None)).await.unwrap();
|
||||
@@ -455,7 +455,7 @@ mod tests {
|
||||
#[tokio::test]
|
||||
async fn set_credentials_rewraps_and_migrates() {
|
||||
let path = temp_db_path("users-rewrap");
|
||||
let pool = crate::core::db::init_pool(&path).await.unwrap();
|
||||
let pool = crate::db::init_system_pool(&path).await.unwrap();
|
||||
|
||||
insert(&pool, "u-1", "ada", None, "admin", &encrypted()).await.unwrap();
|
||||
|
||||
@@ -485,7 +485,7 @@ mod tests {
|
||||
#[tokio::test]
|
||||
async fn check_constraint_rejects_impossible_rows() {
|
||||
let path = temp_db_path("users-check");
|
||||
let pool = crate::core::db::init_pool(&path).await.unwrap();
|
||||
let pool = crate::db::init_system_pool(&path).await.unwrap();
|
||||
|
||||
// encrypted without a wrapped DEK
|
||||
let err = sqlx::query(
|
||||
@@ -7,7 +7,7 @@
|
||||
//! secret it carries) flows straight back to the server's stdin — it is **never**
|
||||
//! logged, broadcast in an event, or written to the DB.
|
||||
//!
|
||||
//! Mirrors [`crate::core::clarification`], but with the `accept`/`decline`/`cancel`
|
||||
//! Mirrors [`crate::clarification`], but with the `accept`/`decline`/`cancel`
|
||||
//! outcome and a `sensitive` flag that elicitation needs and clarification lacks.
|
||||
|
||||
use std::sync::Arc;
|
||||
@@ -23,8 +23,8 @@ use tracing::{debug, info};
|
||||
|
||||
use mcp_client::{ElicitationAction, ElicitationHandler, ElicitationReply, ElicitationRequest};
|
||||
|
||||
use crate::core::events::{GlobalEvent, ServerEvent};
|
||||
use crate::core::pending_registry::PendingRegistry;
|
||||
use crate::events::{GlobalEvent, ServerEvent};
|
||||
use crate::pending_registry::PendingRegistry;
|
||||
|
||||
/// How long the user has to answer an elicitation before we reply `cancel`.
|
||||
/// Independent of any secret-cache TTL the MCP server keeps in its own RAM.
|
||||
+6
-6
@@ -21,10 +21,10 @@ use tracing::{info, warn};
|
||||
|
||||
use core_api::image_generate::ImageGenerateRegistry;
|
||||
|
||||
use crate::core::llm::LlmProviderRecord;
|
||||
use crate::core::llm::db as llm_db;
|
||||
use crate::core::provider::ProviderRegistry;
|
||||
use crate::core::tools::Tool;
|
||||
use crate::llm::LlmProviderRecord;
|
||||
use crate::llm::db as llm_db;
|
||||
use crate::provider::ProviderRegistry;
|
||||
use crate::tools::Tool;
|
||||
|
||||
use super::{ImageGenerate, ImageGenerateInfo, ImageGenerateModelInfo, ImageGenerateModelRecord};
|
||||
use super::db as image_db;
|
||||
@@ -231,8 +231,8 @@ impl ImageGeneratorManager {
|
||||
}
|
||||
drop(state);
|
||||
vec![
|
||||
Arc::new(crate::core::tools::image_generate::ImageGenerateProvidersList { mgr: Arc::clone(&self) }) as Arc<dyn Tool>,
|
||||
Arc::new(crate::core::tools::image_generate::ImageGenerateTool { mgr: Arc::clone(&self) }) as Arc<dyn Tool>,
|
||||
Arc::new(crate::tools::image_generate::ImageGenerateProvidersList { mgr: Arc::clone(&self) }) as Arc<dyn Tool>,
|
||||
Arc::new(crate::tools::image_generate::ImageGenerateTool { mgr: Arc::clone(&self) }) as Arc<dyn Tool>,
|
||||
]
|
||||
}
|
||||
|
||||
@@ -10,10 +10,10 @@ use core_api::inbox::{
|
||||
};
|
||||
use core_api::tool::ToolDescriptionLength;
|
||||
|
||||
use crate::core::approval::{ApprovalManager, PendingApprovalInfo};
|
||||
use crate::core::clarification::{ClarificationManager, PendingClarificationInfo};
|
||||
use crate::core::elicitation::{ElicitationManager, ElicitationOutcome, PendingElicitationInfo};
|
||||
use crate::core::tools::ToolRegistry;
|
||||
use crate::approval::{ApprovalManager, PendingApprovalInfo};
|
||||
use crate::clarification::{ClarificationManager, PendingClarificationInfo};
|
||||
use crate::elicitation::{ElicitationManager, ElicitationOutcome, PendingElicitationInfo};
|
||||
use crate::tools::ToolRegistry;
|
||||
|
||||
#[derive(Serialize)]
|
||||
pub struct InboxItems {
|
||||
@@ -597,29 +597,43 @@ OUTPUT hello.log
|
||||
}
|
||||
}
|
||||
|
||||
/// Two real files to hash. These used to be `file!()` and `Cargo.toml`,
|
||||
/// which quietly stopped resolving when this module moved into its own
|
||||
/// crate: `file!()` is relative to the workspace root while a test's working
|
||||
/// directory is the package root. Owning the fixtures keeps the test about
|
||||
/// hashing.
|
||||
fn hash_fixture(tag: &str) -> (PathBuf, PathBuf, PathBuf) {
|
||||
let nanos = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos();
|
||||
let dir = std::env::temp_dir().join(format!("skald-hash-{tag}-{}-{nanos}", std::process::id()));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let (a, b) = (dir.join("a.tex"), dir.join("b.tex"));
|
||||
std::fs::write(&a, b"\\documentclass{article}").unwrap();
|
||||
std::fs::write(&b, b"\\usepackage{amsmath}").unwrap();
|
||||
(dir, a, b)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn composite_hash_is_deterministic_for_same_contents() {
|
||||
// Use the test file itself as a stand-in dependency: it exists on disk
|
||||
// and its contents are stable for the duration of the test.
|
||||
let me = Path::new(file!());
|
||||
let deps_a = vec![me.to_path_buf()];
|
||||
let deps_b = vec![me.to_path_buf()];
|
||||
let (dir, a, _b) = hash_fixture("deterministic");
|
||||
let deps = vec![a];
|
||||
assert_eq!(
|
||||
composite_hash_of(&deps_a).await.unwrap(),
|
||||
composite_hash_of(&deps_b).await.unwrap()
|
||||
composite_hash_of(&deps).await.unwrap(),
|
||||
composite_hash_of(&deps).await.unwrap()
|
||||
);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn composite_hash_is_order_independent() {
|
||||
let me = Path::new(file!());
|
||||
let cargo = Path::new("Cargo.toml");
|
||||
let a = vec![me.to_path_buf(), cargo.to_path_buf()];
|
||||
let b = vec![cargo.to_path_buf(), me.to_path_buf()];
|
||||
let (dir, a, b) = hash_fixture("order");
|
||||
let forwards = vec![a.clone(), b.clone()];
|
||||
let backwards = vec![b, a];
|
||||
assert_eq!(
|
||||
composite_hash_of(&a).await.unwrap(),
|
||||
composite_hash_of(&b).await.unwrap()
|
||||
composite_hash_of(&forwards).await.unwrap(),
|
||||
composite_hash_of(&backwards).await.unwrap()
|
||||
);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -1,3 +1,11 @@
|
||||
//! The headless Skald core: storage, identity, LLM stack, tools, sessions.
|
||||
//!
|
||||
//! Nothing here knows what runs it. The process shell — HTTP server, desktop
|
||||
//! webview, setup wizard — lives in the crates that depend on this one. Concrete
|
||||
//! plugins are never named: `plugin::PluginManager` only ever sees
|
||||
//! `Arc<dyn Plugin>`, constructed by the consumer and handed to `Skald::new`.
|
||||
|
||||
pub mod boot;
|
||||
pub mod config;
|
||||
pub mod config_store;
|
||||
pub mod skald;
|
||||
@@ -9,6 +17,7 @@ pub mod chatbot;
|
||||
pub mod clarification;
|
||||
pub mod command;
|
||||
pub mod compactor;
|
||||
pub mod crypto;
|
||||
pub mod elicitation;
|
||||
pub mod cron;
|
||||
pub mod db;
|
||||
@@ -35,3 +44,4 @@ pub mod tool_discovery;
|
||||
pub mod tools;
|
||||
pub mod transcribe;
|
||||
pub mod tts;
|
||||
pub mod users;
|
||||
@@ -1,7 +1,7 @@
|
||||
use anyhow::{Context, Result};
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
use crate::config::LlmStrength;
|
||||
use core_api::provider::LlmStrength;
|
||||
use super::{LlmModelRecord, LlmProviderRecord};
|
||||
|
||||
// ── Provider rows ─────────────────────────────────────────────────────────────
|
||||
@@ -124,11 +124,13 @@ pub async fn insert_model(pool: &SqlitePool, r: &LlmModelRecord) -> Result<i64>
|
||||
let extra_params = r.extra_params.as_ref().map(|v| v.to_string());
|
||||
let capabilities = serde_json::to_string(&r.capabilities)?;
|
||||
let reasoning = r.reasoning.as_ref().map(|v| v.to_string());
|
||||
// A model row is never hard-deleted (llm_requests.model_db_id references it),
|
||||
// only soft-deleted via `removed_at`. The unique identity is `name` (also the
|
||||
// resolution key), so re-adding a previously removed model with the same alias
|
||||
// would collide with the lingering soft-deleted row. Upsert on `name` so that
|
||||
// existing row is revived (removed_at cleared) and every field overwritten.
|
||||
// A model row is never hard-deleted, only soft-deleted via `removed_at`:
|
||||
// history and telemetry name it, and `llm_requests` keeps only the model's
|
||||
// name, so the row is what maps that name back to a provider. The unique
|
||||
// identity is `name` (also the resolution key), so re-adding a previously
|
||||
// removed model with the same alias would collide with the lingering
|
||||
// soft-deleted row. Upsert on `name` so that existing row is revived
|
||||
// (removed_at cleared) and every field overwritten.
|
||||
let id = sqlx::query_scalar::<_, i64>(
|
||||
"INSERT INTO llm_models (provider_id, model_id, name, strength, scope, is_default, priority, extra_params,
|
||||
context_length, max_output_tokens, knowledge_cutoff, capabilities, reasoning)
|
||||
@@ -8,10 +8,10 @@ use sqlx::SqlitePool;
|
||||
use tokio::sync::RwLock;
|
||||
use tracing::{info, warn};
|
||||
|
||||
use crate::core::chatbot::ChatbotClient;
|
||||
use crate::core::chatbot::logging::{LoggingChatbotClient, LogSaveFlags};
|
||||
use crate::config::LlmStrength;
|
||||
use crate::core::provider::{ApiProvider, ProviderRegistry, ReasoningMode};
|
||||
use crate::chatbot::ChatbotClient;
|
||||
use crate::chatbot::logging::{LoggingChatbotClient, LogSaveFlags};
|
||||
use core_api::provider::LlmStrength;
|
||||
use crate::provider::{ApiProvider, ProviderRegistry, ReasoningMode};
|
||||
|
||||
use super::providers::RemoteLlmModelInfo;
|
||||
use super::{ClientStatus, LlmEntry, LlmModelInfo, LlmModelRecord, LlmProviderInfo, LlmProviderRecord};
|
||||
@@ -4,8 +4,8 @@ pub mod providers;
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use crate::core::chatbot::ChatbotClient;
|
||||
use crate::core::provider::ServiceType;
|
||||
use crate::chatbot::ChatbotClient;
|
||||
use crate::provider::ServiceType;
|
||||
|
||||
pub use core_api::provider::{LlmProviderRecord, LlmModelRecord, LlmStrength, ReasoningMode};
|
||||
pub use manager::{LlmManager, sort_models_for_agent};
|
||||
+4
-4
@@ -2,10 +2,10 @@ use std::sync::Arc;
|
||||
|
||||
use anyhow::{Context, Result, anyhow};
|
||||
|
||||
use crate::core::chatbot::anthropic::AnthropicClient;
|
||||
use crate::core::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::core::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
|
||||
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
|
||||
use crate::chatbot::anthropic::AnthropicClient;
|
||||
use crate::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
|
||||
use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
|
||||
|
||||
pub struct AnthropicProvider {
|
||||
http: reqwest::Client,
|
||||
+4
-4
@@ -2,10 +2,10 @@ use std::sync::Arc;
|
||||
|
||||
use anyhow::{Context, Result, anyhow};
|
||||
|
||||
use crate::core::chatbot::openai::OpenAiClient;
|
||||
use crate::core::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::core::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
|
||||
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
|
||||
use crate::chatbot::openai::OpenAiClient;
|
||||
use crate::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
|
||||
use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
|
||||
|
||||
pub struct DeepSeekProvider {
|
||||
http: reqwest::Client,
|
||||
+4
-4
@@ -2,10 +2,10 @@ use std::sync::Arc;
|
||||
|
||||
use anyhow::{Result, anyhow};
|
||||
|
||||
use crate::core::chatbot::lm_studio::LmStudioClient;
|
||||
use crate::core::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::core::llm::providers::RemoteLlmModelInfo;
|
||||
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ServiceType};
|
||||
use crate::chatbot::lm_studio::LmStudioClient;
|
||||
use crate::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::llm::providers::RemoteLlmModelInfo;
|
||||
use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ServiceType};
|
||||
|
||||
pub struct LmStudioProvider {
|
||||
http: reqwest::Client,
|
||||
@@ -7,7 +7,7 @@ pub mod openrouter;
|
||||
pub mod zai;
|
||||
|
||||
// Re-export so existing code that uses `providers::ServiceType` / `providers::RemoteLlmModelInfo` keeps working.
|
||||
pub use crate::core::provider::ServiceType;
|
||||
pub use crate::provider::ServiceType;
|
||||
pub use core_api::provider::RemoteLlmModelInfo;
|
||||
|
||||
use core_api::provider::{ApiProvider, LlmModelRecord};
|
||||
@@ -2,10 +2,10 @@ use std::sync::Arc;
|
||||
|
||||
use anyhow::{Result, anyhow};
|
||||
|
||||
use crate::core::chatbot::ollama::OllamaClient;
|
||||
use crate::core::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::core::llm::providers::RemoteLlmModelInfo;
|
||||
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ServiceType};
|
||||
use crate::chatbot::ollama::OllamaClient;
|
||||
use crate::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::llm::providers::RemoteLlmModelInfo;
|
||||
use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ServiceType};
|
||||
|
||||
pub struct OllamaProvider {
|
||||
http: reqwest::Client,
|
||||
@@ -2,14 +2,14 @@ use std::sync::Arc;
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
|
||||
use crate::core::chatbot::openai::OpenAiClient;
|
||||
use crate::core::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::core::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
|
||||
use crate::core::transcribe::TranscribeModelRecord;
|
||||
use crate::core::transcribe::openai_audio::OpenAiAudioTranscriber;
|
||||
use crate::core::tts::TtsModelRecord;
|
||||
use crate::core::tts::openai_tts::OpenAiTtsSynthesiser;
|
||||
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
|
||||
use crate::chatbot::openai::OpenAiClient;
|
||||
use crate::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
|
||||
use crate::transcribe::TranscribeModelRecord;
|
||||
use crate::transcribe::openai_audio::OpenAiAudioTranscriber;
|
||||
use crate::tts::TtsModelRecord;
|
||||
use crate::tts::openai_tts::OpenAiTtsSynthesiser;
|
||||
use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
|
||||
|
||||
pub struct OpenAiProvider;
|
||||
|
||||
@@ -58,7 +58,7 @@ impl ApiProvider for OpenAiProvider {
|
||||
})())
|
||||
}
|
||||
|
||||
fn build_tts(&self, record: &LlmProviderRecord, model: &TtsModelRecord) -> Option<Result<Arc<dyn crate::core::tts::TextToSpeech>>> {
|
||||
fn build_tts(&self, record: &LlmProviderRecord, model: &TtsModelRecord) -> Option<Result<Arc<dyn crate::tts::TextToSpeech>>> {
|
||||
Some((|| {
|
||||
let base_url = record.base_url.clone()
|
||||
.unwrap_or_else(|| "https://api.openai.com/v1".to_string());
|
||||
@@ -67,11 +67,11 @@ impl ApiProvider for OpenAiProvider {
|
||||
Ok(Arc::new(OpenAiTtsSynthesiser::new(
|
||||
&model.name, base_url, api_key, &model.model_id,
|
||||
model.voice_id.clone(), model.instructions.clone(), model.response_format.clone(),
|
||||
)) as Arc<dyn crate::core::tts::TextToSpeech>)
|
||||
)) as Arc<dyn crate::tts::TextToSpeech>)
|
||||
})())
|
||||
}
|
||||
|
||||
fn build_transcriber(&self, record: &LlmProviderRecord, model: &TranscribeModelRecord) -> Option<Result<Arc<dyn crate::core::transcribe::Transcribe>>> {
|
||||
fn build_transcriber(&self, record: &LlmProviderRecord, model: &TranscribeModelRecord) -> Option<Result<Arc<dyn crate::transcribe::Transcribe>>> {
|
||||
Some((|| {
|
||||
let base_url = record.base_url.clone()
|
||||
.unwrap_or_else(|| "https://api.openai.com/v1".to_string());
|
||||
@@ -79,7 +79,7 @@ impl ApiProvider for OpenAiProvider {
|
||||
.with_context(|| format!("provider '{}': api_key required for open_ai", record.name))?;
|
||||
Ok(Arc::new(OpenAiAudioTranscriber::new(
|
||||
&model.name, base_url, api_key, &model.model_id, model.language.clone(),
|
||||
)) as Arc<dyn crate::core::transcribe::Transcribe>)
|
||||
)) as Arc<dyn crate::transcribe::Transcribe>)
|
||||
})())
|
||||
}
|
||||
|
||||
+16
-16
@@ -2,16 +2,16 @@ use std::sync::Arc;
|
||||
|
||||
use anyhow::{Context, Result, anyhow};
|
||||
|
||||
use crate::core::chatbot::openai::OpenAiClient;
|
||||
use crate::core::image_generate::ImageGenerateModelRecord;
|
||||
use crate::core::image_generate::openrouter_image::OpenRouterImageGenerator;
|
||||
use crate::core::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::core::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
|
||||
use crate::core::transcribe::TranscribeModelRecord;
|
||||
use crate::core::transcribe::openai_audio::OpenAiAudioTranscriber;
|
||||
use crate::core::tts::TtsModelRecord;
|
||||
use crate::core::tts::openai_tts::OpenAiTtsSynthesiser;
|
||||
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
|
||||
use crate::chatbot::openai::OpenAiClient;
|
||||
use crate::image_generate::ImageGenerateModelRecord;
|
||||
use crate::image_generate::openrouter_image::OpenRouterImageGenerator;
|
||||
use crate::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
|
||||
use crate::transcribe::TranscribeModelRecord;
|
||||
use crate::transcribe::openai_audio::OpenAiAudioTranscriber;
|
||||
use crate::tts::TtsModelRecord;
|
||||
use crate::tts::openai_tts::OpenAiTtsSynthesiser;
|
||||
use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
|
||||
|
||||
pub struct OpenRouterProvider {
|
||||
http: reqwest::Client,
|
||||
@@ -163,7 +163,7 @@ impl ApiProvider for OpenRouterProvider {
|
||||
})())
|
||||
}
|
||||
|
||||
fn build_tts(&self, record: &LlmProviderRecord, model: &TtsModelRecord) -> Option<Result<Arc<dyn crate::core::tts::TextToSpeech>>> {
|
||||
fn build_tts(&self, record: &LlmProviderRecord, model: &TtsModelRecord) -> Option<Result<Arc<dyn crate::tts::TextToSpeech>>> {
|
||||
Some((|| {
|
||||
let base_url = record.base_url.clone()
|
||||
.unwrap_or_else(|| "https://openrouter.ai/api/v1".to_string());
|
||||
@@ -172,11 +172,11 @@ impl ApiProvider for OpenRouterProvider {
|
||||
Ok(Arc::new(OpenAiTtsSynthesiser::new(
|
||||
&model.name, base_url, api_key, &model.model_id,
|
||||
model.voice_id.clone(), model.instructions.clone(), model.response_format.clone(),
|
||||
)) as Arc<dyn crate::core::tts::TextToSpeech>)
|
||||
)) as Arc<dyn crate::tts::TextToSpeech>)
|
||||
})())
|
||||
}
|
||||
|
||||
fn build_transcriber(&self, record: &LlmProviderRecord, model: &TranscribeModelRecord) -> Option<Result<Arc<dyn crate::core::transcribe::Transcribe>>> {
|
||||
fn build_transcriber(&self, record: &LlmProviderRecord, model: &TranscribeModelRecord) -> Option<Result<Arc<dyn crate::transcribe::Transcribe>>> {
|
||||
Some((|| {
|
||||
let base_url = record.base_url.clone()
|
||||
.unwrap_or_else(|| "https://openrouter.ai/api/v1".to_string());
|
||||
@@ -184,11 +184,11 @@ impl ApiProvider for OpenRouterProvider {
|
||||
.with_context(|| format!("provider '{}': api_key required for openrouter", record.name))?;
|
||||
Ok(Arc::new(OpenAiAudioTranscriber::new(
|
||||
&model.name, base_url, api_key, &model.model_id, model.language.clone(),
|
||||
)) as Arc<dyn crate::core::transcribe::Transcribe>)
|
||||
)) as Arc<dyn crate::transcribe::Transcribe>)
|
||||
})())
|
||||
}
|
||||
|
||||
fn build_image_generator(&self, record: &LlmProviderRecord, model: &ImageGenerateModelRecord) -> Option<Result<Arc<dyn crate::core::image_generate::ImageGenerate>>> {
|
||||
fn build_image_generator(&self, record: &LlmProviderRecord, model: &ImageGenerateModelRecord) -> Option<Result<Arc<dyn crate::image_generate::ImageGenerate>>> {
|
||||
Some((|| {
|
||||
let base_url = record.base_url.clone()
|
||||
.unwrap_or_else(|| "https://openrouter.ai/api/v1".to_string());
|
||||
@@ -196,7 +196,7 @@ impl ApiProvider for OpenRouterProvider {
|
||||
.with_context(|| format!("provider '{}': api_key required for openrouter", record.name))?;
|
||||
Ok(Arc::new(OpenRouterImageGenerator::new(
|
||||
&model.name, base_url, api_key, &model.model_id,
|
||||
)) as Arc<dyn crate::core::image_generate::ImageGenerate>)
|
||||
)) as Arc<dyn crate::image_generate::ImageGenerate>)
|
||||
})())
|
||||
}
|
||||
|
||||
@@ -2,10 +2,10 @@ use std::sync::Arc;
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
|
||||
use crate::core::chatbot::openai::OpenAiClient;
|
||||
use crate::core::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::core::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
|
||||
use crate::core::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
|
||||
use crate::chatbot::openai::OpenAiClient;
|
||||
use crate::llm::{LlmModelRecord, LlmProviderRecord};
|
||||
use crate::llm::providers::{RemoteLlmModelInfo, extra_with_reasoning};
|
||||
use crate::provider::{ApiProvider, BuiltLlmClient, ProviderField, ProviderUiMeta, ReasoningMode, ServiceType};
|
||||
|
||||
/// Z.AI (Zhipu AI) — OpenAI-compatible GLM API.
|
||||
///
|
||||
@@ -11,7 +11,7 @@ use tokio::sync::mpsc;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
use tracing::{info, warn};
|
||||
|
||||
use crate::core::tools::ToolResult;
|
||||
use crate::tools::ToolResult;
|
||||
|
||||
pub use mcp_client::{
|
||||
ElicitationHandler,
|
||||
@@ -105,7 +105,7 @@ impl McpManager {
|
||||
Some((source, payload)) => {
|
||||
let method = payload["method"].as_str().unwrap_or("unknown").to_string();
|
||||
let params = serde_json::to_string(&payload["params"]).unwrap_or_else(|_| "{}".to_string());
|
||||
match crate::core::db::mcp_events::insert(&pool, &source, &method, ¶ms).await {
|
||||
match crate::db::mcp_events::insert(&pool, &source, &method, ¶ms).await {
|
||||
Ok(id) => info!("mcp_event stored: id={id} source={source} method={method}"),
|
||||
Err(e) => warn!("mcp_events insert failed (source={source} method={method}): {e}"),
|
||||
}
|
||||
@@ -116,7 +116,7 @@ impl McpManager {
|
||||
}
|
||||
}
|
||||
|
||||
fn cfg_from_row(row: &crate::core::db::mcp_servers::McpServerRow) -> McpServerConfig {
|
||||
fn cfg_from_row(row: &crate::db::mcp_servers::McpServerRow) -> McpServerConfig {
|
||||
McpServerConfig {
|
||||
name: row.name.clone(),
|
||||
transport: match row.transport.as_str() {
|
||||
@@ -155,7 +155,7 @@ impl McpManager {
|
||||
}
|
||||
|
||||
pub async fn initialize(&self) {
|
||||
let rows = match crate::core::db::mcp_servers::all_enabled(&self.pool).await {
|
||||
let rows = match crate::db::mcp_servers::all_enabled(&self.pool).await {
|
||||
Ok(r) => r,
|
||||
Err(e) => { warn!("McpManager::initialize: failed to read DB: {e}"); return; }
|
||||
};
|
||||
@@ -218,12 +218,12 @@ impl McpManager {
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn register(&self, p: crate::core::db::mcp_servers::UpsertParams<'_>) -> Result<Vec<String>> {
|
||||
pub async fn register(&self, p: crate::db::mcp_servers::UpsertParams<'_>) -> Result<Vec<String>> {
|
||||
let name = p.name.to_string();
|
||||
|
||||
crate::core::db::mcp_servers::upsert(&self.pool, p).await?;
|
||||
crate::db::mcp_servers::upsert(&self.pool, p).await?;
|
||||
|
||||
let rows = crate::core::db::mcp_servers::all_enabled(&self.pool).await?;
|
||||
let rows = crate::db::mcp_servers::all_enabled(&self.pool).await?;
|
||||
let row = rows.into_iter().find(|r| r.name == name)
|
||||
.ok_or_else(|| anyhow::anyhow!("register: server '{}' not found after upsert", name))?;
|
||||
let cfg = Self::cfg_from_row(&row);
|
||||
@@ -251,7 +251,7 @@ impl McpManager {
|
||||
}
|
||||
|
||||
pub async fn unregister(&self, name: &str) -> Result<()> {
|
||||
crate::core::db::mcp_servers::delete(&self.pool, name).await?;
|
||||
crate::db::mcp_servers::delete(&self.pool, name).await?;
|
||||
self.servers.write().unwrap().remove(name);
|
||||
self.errors.write().unwrap().remove(name);
|
||||
self.descriptions.write().unwrap().remove(name);
|
||||
@@ -259,11 +259,11 @@ impl McpManager {
|
||||
}
|
||||
|
||||
pub async fn set_enabled(&self, name: &str, enabled: bool) -> Result<()> {
|
||||
crate::core::db::mcp_servers::set_enabled(&self.pool, name, enabled).await
|
||||
crate::db::mcp_servers::set_enabled(&self.pool, name, enabled).await
|
||||
}
|
||||
|
||||
pub async fn list(&self) -> Result<Vec<McpServerInfo>> {
|
||||
let rows = crate::core::db::mcp_servers::all(&self.pool).await?;
|
||||
let rows = crate::db::mcp_servers::all(&self.pool).await?;
|
||||
let servers = self.servers.read().unwrap();
|
||||
let errors = self.errors.read().unwrap();
|
||||
|
||||
@@ -24,7 +24,7 @@ use tracing::{error, info};
|
||||
|
||||
pub use core_api::memory::Memory;
|
||||
|
||||
use crate::core::tools::Tool;
|
||||
use crate::tools::Tool;
|
||||
|
||||
// ── MemoryManager ─────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
// The manager only ever handles `Arc<dyn Plugin>`. Naming a concrete plugin crate
|
||||
// here would make the core depend on every plugin — and, through
|
||||
// `plugin-transcribe-whisper-local`, on a C build — for no gain: the consumer
|
||||
// constructs the plugin list and passes it to `Skald::new`.
|
||||
pub use core_api::plugin::{Plugin, PluginContext, RouterFactory};
|
||||
pub use plugin_comfyui::ComfyUIPlugin;
|
||||
#[cfg(feature = "whisper-local")]
|
||||
pub use plugin_transcribe_whisper_local::WhisperLocalPlugin;
|
||||
pub use plugin_telegram_bot::TelegramPlugin;
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::{Arc, OnceLock};
|
||||
@@ -19,8 +19,8 @@ use tokio::sync::Mutex;
|
||||
use tokio::time::timeout;
|
||||
use tracing::{error, info, warn};
|
||||
|
||||
use crate::core::db::plugins as db;
|
||||
use crate::core::skald::Skald;
|
||||
use crate::db::plugins as db;
|
||||
use crate::skald::Skald;
|
||||
|
||||
// ── Public plugin info (returned by list_items tool and REST API) ─────────────
|
||||
|
||||
@@ -238,7 +238,7 @@ impl PluginManager {
|
||||
/// Toggle only the enabled flag, keeping existing config.
|
||||
pub async fn toggle(&self, id: &str, enabled: bool) -> Result<()> {
|
||||
let row = db::get(&self.db, id).await?
|
||||
.unwrap_or_else(|| crate::core::db::plugins::PluginRow {
|
||||
.unwrap_or_else(|| crate::db::plugins::PluginRow {
|
||||
id: id.to_string(),
|
||||
enabled,
|
||||
config: "{}".to_string(),
|
||||
@@ -329,6 +329,13 @@ impl PluginManager {
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Every registered plugin, enabled or not. Lets the core ask each one for
|
||||
/// its contributions (`Plugin::tools`, `Plugin::http_router`) without ever
|
||||
/// naming a concrete plugin type.
|
||||
pub fn all(&self) -> &[Arc<dyn Plugin>] {
|
||||
&self.plugins
|
||||
}
|
||||
|
||||
pub fn get_plugin_typed<T: Plugin + 'static>(&self, id: &str) -> Option<Arc<T>> {
|
||||
self.plugins.iter()
|
||||
.find(|p| p.id() == id)
|
||||
@@ -5,8 +5,8 @@ use std::sync::Arc;
|
||||
use anyhow::Result;
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
use crate::core::db::projects::{self, Project};
|
||||
use crate::core::run_context::RunContext;
|
||||
use crate::db::projects::{self, Project};
|
||||
use crate::run_context::RunContext;
|
||||
|
||||
pub struct ProjectManager {
|
||||
db: Arc<SqlitePool>,
|
||||
@@ -7,9 +7,9 @@ use tracing::warn;
|
||||
|
||||
use core_api::system_bus::{SystemEvent, SystemEventBus};
|
||||
|
||||
use crate::core::cron::TaskManager;
|
||||
use crate::core::db::{project_tickets, project_tickets::ProjectTicket, projects};
|
||||
use crate::core::run_context::RunContext;
|
||||
use crate::cron::TaskManager;
|
||||
use crate::db::{project_tickets, project_tickets::ProjectTicket, projects};
|
||||
use crate::run_context::RunContext;
|
||||
|
||||
pub struct ProjectTicketManager {
|
||||
db: Arc<SqlitePool>,
|
||||
@@ -6,9 +6,9 @@ use serde::{Deserialize, Serialize};
|
||||
use sqlx::SqlitePool;
|
||||
use tracing::info;
|
||||
|
||||
pub use crate::core::db::tool_permission_groups::ToolPermissionGroup;
|
||||
use crate::core::approval::{ApprovalManager, RuleAction};
|
||||
use crate::core::tools::fs::{canonicalize_for_policy, path_under};
|
||||
pub use crate::db::tool_permission_groups::ToolPermissionGroup;
|
||||
use crate::approval::{ApprovalManager, RuleAction};
|
||||
use crate::tools::fs::{canonicalize_for_policy, path_under};
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
|
||||
pub struct RunContext {
|
||||
@@ -113,7 +113,7 @@ impl RunContextManager {
|
||||
/// Seeds the built-in "default" permission group and migrates legacy rules.
|
||||
/// Safe to call at every startup (idempotent).
|
||||
pub async fn seed_defaults(&self) -> Result<()> {
|
||||
crate::core::db::tool_permission_groups::insert_or_ignore(
|
||||
crate::db::tool_permission_groups::insert_or_ignore(
|
||||
&self.db, "default", "Default", Some("Built-in default permission group"),
|
||||
).await?;
|
||||
|
||||
@@ -133,11 +133,11 @@ impl RunContextManager {
|
||||
// ── ToolPermissionGroup CRUD ───────────────────────────────────────────────
|
||||
|
||||
pub async fn list_groups(&self) -> Result<Vec<ToolPermissionGroup>> {
|
||||
crate::core::db::tool_permission_groups::list(&self.db).await
|
||||
crate::db::tool_permission_groups::list(&self.db).await
|
||||
}
|
||||
|
||||
pub async fn get_group(&self, id: &str) -> Result<Option<ToolPermissionGroup>> {
|
||||
crate::core::db::tool_permission_groups::get(&self.db, id).await
|
||||
crate::db::tool_permission_groups::get(&self.db, id).await
|
||||
}
|
||||
|
||||
pub async fn create_group(
|
||||
@@ -149,7 +149,7 @@ impl RunContextManager {
|
||||
if id == "default" {
|
||||
bail!("cannot create a permission group with reserved id 'default'");
|
||||
}
|
||||
crate::core::db::tool_permission_groups::insert(&self.db, id, name, description).await
|
||||
crate::db::tool_permission_groups::insert(&self.db, id, name, description).await
|
||||
}
|
||||
|
||||
pub async fn update_group(
|
||||
@@ -158,14 +158,14 @@ impl RunContextManager {
|
||||
name: &str,
|
||||
description: Option<&str>,
|
||||
) -> Result<bool> {
|
||||
crate::core::db::tool_permission_groups::update(&self.db, id, name, description).await
|
||||
crate::db::tool_permission_groups::update(&self.db, id, name, description).await
|
||||
}
|
||||
|
||||
pub async fn delete_group(&self, id: &str) -> Result<bool> {
|
||||
if id == "default" {
|
||||
bail!("cannot delete the built-in 'default' permission group");
|
||||
}
|
||||
crate::core::db::tool_permission_groups::delete(&self.db, id).await
|
||||
crate::db::tool_permission_groups::delete(&self.db, id).await
|
||||
}
|
||||
|
||||
/// Duplicates a permission group and all its rules atomically.
|
||||
@@ -178,7 +178,7 @@ impl RunContextManager {
|
||||
if new_id == "default" {
|
||||
bail!("cannot create a permission group with reserved id 'default'");
|
||||
}
|
||||
let source = crate::core::db::tool_permission_groups::get(&self.db, source_id).await?
|
||||
let source = crate::db::tool_permission_groups::get(&self.db, source_id).await?
|
||||
.ok_or_else(|| anyhow::anyhow!("source group '{source_id}' not found"))?;
|
||||
|
||||
let mut tx = self.db.begin().await?;
|
||||
@@ -1,4 +1,4 @@
|
||||
use crate::core::provider::ServiceType;
|
||||
use crate::provider::ServiceType;
|
||||
|
||||
/// Light umbrella trait shared by all model managers.
|
||||
/// Enables grouping managers generically (e.g. models-hub routing, diagnostics)
|
||||
+6
-6
@@ -6,8 +6,8 @@ use tokio::sync::mpsc;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
use tracing::info;
|
||||
|
||||
use crate::core::db::{chat_history, chat_llm_tools, chat_sessions_stack, scratchpad, stack_mcp_grants};
|
||||
use crate::core::events::ServerEvent;
|
||||
use crate::db::{chat_history, chat_llm_tools, chat_sessions_stack, scratchpad, stack_mcp_grants};
|
||||
use crate::events::ServerEvent;
|
||||
|
||||
use super::{ChatSessionHandler, MAX_AGENT_DEPTH, TurnOutcome};
|
||||
use super::emitter::TurnEmitter;
|
||||
@@ -41,7 +41,7 @@ impl ChatSessionHandler {
|
||||
// Only `task` agents are dispatchable: this rejects `chat` (e.g. `main`,
|
||||
// `project-coordinator`) and `system` (e.g. `tic`) agents, and surfaces a
|
||||
// not-found error for unknown ids — all in one gate.
|
||||
let target_meta = crate::core::agents::load_task_meta(target_id)
|
||||
let target_meta = crate::agents::load_task_meta(target_id)
|
||||
.map_err(|e| anyhow::anyhow!("dispatch_sub_agent: {e}"))?;
|
||||
|
||||
let parent_frame = chat_sessions_stack::find_by_id(pool, parent_stack_id).await?
|
||||
@@ -91,7 +91,7 @@ impl ChatSessionHandler {
|
||||
{
|
||||
let group_id = self.tool_group_id().await;
|
||||
let gid = group_id.as_deref().unwrap_or("default");
|
||||
let group_rules = crate::core::db::approval_rules::list_for_group(
|
||||
let group_rules = crate::db::approval_rules::list_for_group(
|
||||
pool, Some(gid),
|
||||
).await.unwrap_or_default();
|
||||
child_config.base_tool_defs.retain(|def| {
|
||||
@@ -107,7 +107,7 @@ impl ChatSessionHandler {
|
||||
let mcp_clone = Arc::clone(&self.mcp);
|
||||
let grants_clone = Arc::clone(&active_mcp_grants);
|
||||
|
||||
let activate_tool = crate::core::tools::activate_tools::ActivateTools {
|
||||
let activate_tool = crate::tools::activate_tools::ActivateTools {
|
||||
pool: pool_clone,
|
||||
session_id,
|
||||
stack_id: Some(stack_id),
|
||||
@@ -118,7 +118,7 @@ impl ChatSessionHandler {
|
||||
child_config.interface_tools.push(InterfaceTool {
|
||||
definition: activate_tools_tool_def(),
|
||||
handler: Arc::new(move |args| -> ToolFuture {
|
||||
use crate::core::tools::Tool as _;
|
||||
use crate::tools::Tool as _;
|
||||
let tool = Arc::clone(&activate_tool);
|
||||
Box::pin(async move {
|
||||
tokio::task::spawn_blocking(move || tool.execute(args))
|
||||
+2
-2
@@ -3,7 +3,7 @@ use tracing::debug;
|
||||
|
||||
use super::ChatSessionHandler;
|
||||
use super::emitter::TurnEmitter;
|
||||
use crate::core::tools::{is_file_write_tool, tool_names as tn};
|
||||
use crate::tools::{is_file_write_tool, tool_names as tn};
|
||||
|
||||
impl ChatSessionHandler {
|
||||
/// Emits the appropriate frontend approval event for the given tool call.
|
||||
@@ -55,7 +55,7 @@ impl ChatSessionHandler {
|
||||
|
||||
/// Reads the current content of a file from disk (for diff generation in PendingWrite events).
|
||||
pub(super) async fn read_current_content(&self, path: &str) -> Option<String> {
|
||||
let abs = crate::core::tools::fs::resolve(path).ok()?;
|
||||
let abs = crate::tools::fs::resolve(path).ok()?;
|
||||
tokio::fs::read_to_string(&abs).await.ok()
|
||||
}
|
||||
|
||||
+7
-7
@@ -3,7 +3,7 @@ use std::sync::{Arc, RwLock};
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::core::tools::tool_names as tn;
|
||||
use crate::tools::tool_names as tn;
|
||||
use super::{ChatSessionHandler, update_scratchpad_tool_def, write_todos_tool_def};
|
||||
use super::interface_tools::{AgentRunConfig, InterfaceTool, ToolFuture};
|
||||
|
||||
@@ -47,7 +47,7 @@ impl ChatSessionHandler {
|
||||
mut interface_tools: Vec<InterfaceTool>,
|
||||
system_substitutions: HashMap<String, String>,
|
||||
) -> anyhow::Result<AgentRunConfig> {
|
||||
let meta = crate::core::agents::load_meta(&self.agent_id).ok();
|
||||
let meta = crate::agents::load_meta(&self.agent_id).ok();
|
||||
let (key, _) = self.llm_manager.resolve(
|
||||
client_name.as_deref(),
|
||||
meta.as_ref().and_then(|m| m.scope.as_deref()),
|
||||
@@ -68,7 +68,7 @@ impl ChatSessionHandler {
|
||||
// Inbox alone.
|
||||
let is_system = meta
|
||||
.as_ref()
|
||||
.map(|m| m.agent_type == crate::core::agents::AgentType::System)
|
||||
.map(|m| m.agent_type == crate::agents::AgentType::System)
|
||||
.unwrap_or(false);
|
||||
if !is_system {
|
||||
base_tool_defs.push(super::ask_user_clarification_tool_def());
|
||||
@@ -114,7 +114,7 @@ impl ChatSessionHandler {
|
||||
{
|
||||
let group_id = self.tool_group_id().await;
|
||||
let gid = group_id.as_deref().unwrap_or("default");
|
||||
let group_rules = crate::core::db::approval_rules::list_for_group(
|
||||
let group_rules = crate::db::approval_rules::list_for_group(
|
||||
&self.db, Some(gid),
|
||||
).await.unwrap_or_default();
|
||||
let visible = |def: &Value| {
|
||||
@@ -130,7 +130,7 @@ impl ChatSessionHandler {
|
||||
// Load persisted session grants from DB (MCP server names and/or the reserved
|
||||
// `config` keyword), then inject `activate_tools` so the LLM can activate
|
||||
// additional groups on demand.
|
||||
let persisted = crate::core::db::session_mcp_grants::list_for_session(
|
||||
let persisted = crate::db::session_mcp_grants::list_for_session(
|
||||
&self.db, self.session_id,
|
||||
).await.unwrap_or_default();
|
||||
|
||||
@@ -143,7 +143,7 @@ impl ChatSessionHandler {
|
||||
let mcp_clone = Arc::clone(&self.mcp);
|
||||
let grants_clone = Arc::clone(&active_mcp_grants);
|
||||
|
||||
let activate_tool = crate::core::tools::activate_tools::ActivateTools {
|
||||
let activate_tool = crate::tools::activate_tools::ActivateTools {
|
||||
pool: pool_clone,
|
||||
session_id,
|
||||
stack_id: None,
|
||||
@@ -155,7 +155,7 @@ impl ChatSessionHandler {
|
||||
interface_tools.push(InterfaceTool {
|
||||
definition: activate_tools_tool_def(),
|
||||
handler: Arc::new(move |args| -> ToolFuture {
|
||||
use crate::core::tools::Tool as _;
|
||||
use crate::tools::Tool as _;
|
||||
let tool = Arc::clone(&activate_tool);
|
||||
Box::pin(async move {
|
||||
tokio::task::spawn_blocking(move || tool.execute(args))
|
||||
+2
-2
@@ -10,8 +10,8 @@ use tokio::sync::mpsc;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
use tracing::warn;
|
||||
|
||||
use crate::core::events::ServerEvent;
|
||||
use crate::core::tools::{drive_execution, tool_names as tn, ExecutionOutcome, ToolResult};
|
||||
use crate::events::ServerEvent;
|
||||
use crate::tools::{drive_execution, tool_names as tn, ExecutionOutcome, ToolResult};
|
||||
|
||||
use super::ChatSessionHandler;
|
||||
use super::interface_tools::AgentRunConfig;
|
||||
+1
-1
@@ -16,7 +16,7 @@ use tokio::sync::mpsc;
|
||||
|
||||
use core_api::message_meta::Attachment;
|
||||
|
||||
use crate::core::events::ServerEvent;
|
||||
use crate::events::ServerEvent;
|
||||
|
||||
/// Borrows the per-turn event sender and emits typed [`ServerEvent`]s.
|
||||
pub(super) struct TurnEmitter<'a> {
|
||||
@@ -11,10 +11,10 @@ use std::sync::atomic::Ordering;
|
||||
use serde_json::Value;
|
||||
use tracing::{info, warn};
|
||||
|
||||
use crate::core::approval::GateResult;
|
||||
use crate::core::db::chat_llm_tools;
|
||||
use crate::core::run_context::RunContext;
|
||||
use crate::core::tools::{is_file_read_tool, is_file_write_tool};
|
||||
use crate::approval::GateResult;
|
||||
use crate::db::chat_llm_tools;
|
||||
use crate::run_context::RunContext;
|
||||
use crate::tools::{is_file_read_tool, is_file_write_tool};
|
||||
|
||||
use super::{ApprovalDecision, ChatSessionHandler};
|
||||
use super::emitter::TurnEmitter;
|
||||
+5
-5
@@ -3,9 +3,9 @@ use std::sync::{Arc, RwLock};
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::core::mcp::McpManager;
|
||||
use crate::core::tools::Tool;
|
||||
use crate::core::tools::tool_names as tn;
|
||||
use crate::mcp::McpManager;
|
||||
use crate::tools::Tool;
|
||||
use crate::tools::tool_names as tn;
|
||||
|
||||
pub use core_api::interface_tool::{InterfaceTool, ToolFuture};
|
||||
|
||||
@@ -89,7 +89,7 @@ impl AgentRunConfig {
|
||||
|
||||
// MCP servers: include tools for the granted server names.
|
||||
let servers: Vec<String> = granted.iter()
|
||||
.filter(|n| n.as_str() != crate::core::tools::tool_names::CONFIG_GROUP)
|
||||
.filter(|n| n.as_str() != crate::tools::tool_names::CONFIG_GROUP)
|
||||
.cloned()
|
||||
.collect();
|
||||
if !servers.is_empty() {
|
||||
@@ -101,7 +101,7 @@ impl AgentRunConfig {
|
||||
}
|
||||
|
||||
// `config` group: include the built-in Config-category tools on demand.
|
||||
if granted.contains(crate::core::tools::tool_names::CONFIG_GROUP) {
|
||||
if granted.contains(crate::tools::tool_names::CONFIG_GROUP) {
|
||||
defs.extend(self.config_tool_defs.iter().cloned());
|
||||
}
|
||||
|
||||
+2
-2
@@ -12,8 +12,8 @@ use serde_json::Value;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
use tracing::{error, warn};
|
||||
|
||||
use crate::core::chatbot::{ChatOptions, LlmTurn};
|
||||
use crate::core::llm::{LlmEntry, LlmStrength};
|
||||
use crate::chatbot::{ChatOptions, LlmTurn};
|
||||
use crate::llm::{LlmEntry, LlmStrength};
|
||||
|
||||
use super::ChatSessionHandler;
|
||||
use super::emitter::TurnEmitter;
|
||||
+8
-10
@@ -3,12 +3,12 @@ use tokio::sync::mpsc;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
use tracing::{debug, info, trace};
|
||||
|
||||
use crate::core::tools::tool_names as tn;
|
||||
use crate::core::chat_event_bus::ToolCallEvent;
|
||||
use crate::core::chatbot::{LlmTurn, ToolCall};
|
||||
use crate::core::db::{chat_history, chat_llm_tools};
|
||||
use crate::core::events::ServerEvent;
|
||||
use crate::core::tools::{
|
||||
use crate::tools::tool_names as tn;
|
||||
use crate::chat_event_bus::ToolCallEvent;
|
||||
use crate::chatbot::{LlmTurn, ToolCall};
|
||||
use crate::db::{chat_history, chat_llm_tools};
|
||||
use crate::events::ServerEvent;
|
||||
use crate::tools::{
|
||||
ExecutionOutcome, SimpleExecution, ToolDescriptionLength, ToolExecution, ToolResult,
|
||||
};
|
||||
use futures::stream::{self, StreamExt};
|
||||
@@ -67,7 +67,7 @@ impl ChatSessionHandler {
|
||||
.ok_or_else(|| anyhow::anyhow!("LLM client '{}' not found", cur_name))?;
|
||||
|
||||
// Scope/strength needed for fallback re-selection.
|
||||
let meta = crate::core::agents::load_meta(&config.agent_id).ok();
|
||||
let meta = crate::agents::load_meta(&config.agent_id).ok();
|
||||
let req_scope = meta.as_ref().and_then(|m| m.scope.as_deref()).map(str::to_string);
|
||||
let req_strength = meta.as_ref().and_then(|m| m.strength);
|
||||
|
||||
@@ -144,7 +144,6 @@ impl ChatSessionHandler {
|
||||
pool, stack_id, &chat_history::Role::Assistant, &resp.content, false,
|
||||
resp.reasoning_content.as_deref(),
|
||||
).await?;
|
||||
chat_history::set_model_db_id(pool, message_id, cur_llm.model_db_id).await?;
|
||||
if let (Some(i), Some(o)) = (resp.input_tokens, resp.output_tokens) {
|
||||
chat_history::set_usage(pool, message_id, i, o, 0, resp.cost).await?;
|
||||
}
|
||||
@@ -163,7 +162,6 @@ impl ChatSessionHandler {
|
||||
pool, stack_id, &chat_history::Role::Assistant, &assistant_text, false,
|
||||
reasoning_content.as_deref(),
|
||||
).await?;
|
||||
chat_history::set_model_db_id(pool, message_id, cur_llm.model_db_id).await?;
|
||||
if let (Some(i), Some(o)) = (input_tokens, output_tokens) {
|
||||
chat_history::set_usage(pool, message_id, i, o, 0, cost).await?;
|
||||
}
|
||||
@@ -418,7 +416,7 @@ impl ChatSessionHandler {
|
||||
return Some(tool.run(args));
|
||||
}
|
||||
// MCP tools (`server::tool`). Clone the Arc so the work future is 'static.
|
||||
if let Some((srv, mcp_tool)) = crate::core::mcp::parse_mcp_tool_name(name) {
|
||||
if let Some((srv, mcp_tool)) = crate::mcp::parse_mcp_tool_name(name) {
|
||||
let mcp = std::sync::Arc::clone(&self.mcp);
|
||||
let srv = srv.to_string();
|
||||
let mcp_tool = mcp_tool.to_string();
|
||||
+9
-9
@@ -4,11 +4,11 @@ use std::sync::Arc;
|
||||
use serde_json::{Value, json};
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
use crate::core::compactor::{ContextCompactor, SUMMARY_PREFIX};
|
||||
use crate::core::config::DatetimeConfig;
|
||||
use crate::core::db::{chat_history, chat_llm_tools, chat_summaries};
|
||||
use crate::core::mcp::McpManager;
|
||||
use crate::core::tools::tool_names as tn;
|
||||
use crate::compactor::{ContextCompactor, SUMMARY_PREFIX};
|
||||
use crate::config::DatetimeConfig;
|
||||
use crate::db::{chat_history, chat_llm_tools, chat_summaries};
|
||||
use crate::mcp::McpManager;
|
||||
use crate::tools::tool_names as tn;
|
||||
|
||||
/// Registry of installed skills, relative to Skald's process cwd. Injected into agents
|
||||
/// that have `inject_skills` enabled (the default).
|
||||
@@ -84,9 +84,9 @@ impl MessageBuilder {
|
||||
let pool = &*self.pool;
|
||||
|
||||
// ── 1. Static system message ──────────────────────────────────────────
|
||||
let mut static_content = crate::core::agents::load_prompt(agent_id)?;
|
||||
let mut static_content = crate::agents::load_prompt(agent_id)?;
|
||||
|
||||
let meta = crate::core::agents::load_meta(agent_id)?;
|
||||
let meta = crate::agents::load_meta(agent_id)?;
|
||||
if !meta.inject_memory.is_empty() {
|
||||
static_content.push_str(
|
||||
"\n\n---\nThe following memory files have been loaded automatically. \
|
||||
@@ -154,7 +154,7 @@ impl MessageBuilder {
|
||||
let mut out = vec![static_msg];
|
||||
|
||||
// ── 2. Scratchpad system message (before conversation) ────────────────
|
||||
let scratch = crate::core::db::scratchpad::for_session(pool, self.session_id).await?;
|
||||
let scratch = crate::db::scratchpad::for_session(pool, self.session_id).await?;
|
||||
if !scratch.is_empty() {
|
||||
let mut s = String::from(
|
||||
"<scratchpad>\n \
|
||||
@@ -409,7 +409,7 @@ impl MessageBuilder {
|
||||
let wd = self.working_directory.clone()
|
||||
.unwrap_or_else(|| std::env::current_dir().unwrap_or_default());
|
||||
let expanded = mem_path.replace("$WD", &wd.display().to_string());
|
||||
let abs = crate::core::tools::fs::resolve(&expanded)
|
||||
let abs = crate::tools::fs::resolve(&expanded)
|
||||
.unwrap_or_else(|_| std::path::PathBuf::from(&expanded));
|
||||
let display = match abs.strip_prefix(&wd) {
|
||||
Ok(rel) => rel.to_string_lossy().into_owned(),
|
||||
@@ -10,22 +10,22 @@ use tokio_util::sync::CancellationToken;
|
||||
|
||||
use tracing::{error, info, trace, warn};
|
||||
|
||||
use crate::core::approval::ApprovalManager;
|
||||
use crate::core::run_context::RunContext;
|
||||
use crate::core::tools::tool_names as tn;
|
||||
use crate::core::chat_event_bus::{ChatEvent, ChatEventBus, ChatEventRole};
|
||||
use crate::core::clarification::ClarificationManager;
|
||||
use crate::core::compactor::ContextCompactor;
|
||||
use crate::core::config::DatetimeConfig;
|
||||
use crate::core::db::{chat_history, chat_sessions_stack};
|
||||
use crate::core::events::ServerEvent;
|
||||
use crate::approval::ApprovalManager;
|
||||
use crate::run_context::RunContext;
|
||||
use crate::tools::tool_names as tn;
|
||||
use crate::chat_event_bus::{ChatEvent, ChatEventBus, ChatEventRole};
|
||||
use crate::clarification::ClarificationManager;
|
||||
use crate::compactor::ContextCompactor;
|
||||
use crate::config::DatetimeConfig;
|
||||
use crate::db::{chat_history, chat_sessions_stack};
|
||||
use crate::events::ServerEvent;
|
||||
use core_api::message_meta::MessageMetadata;
|
||||
use crate::core::llm::LlmManager;
|
||||
use crate::core::mcp::McpManager;
|
||||
use crate::core::image_generate::ImageGeneratorManager;
|
||||
use crate::core::memory::MemoryManager;
|
||||
use crate::core::tool_discovery::ToolDiscovery;
|
||||
use crate::core::tools::ToolRegistry;
|
||||
use crate::llm::LlmManager;
|
||||
use crate::mcp::McpManager;
|
||||
use crate::image_generate::ImageGeneratorManager;
|
||||
use crate::memory::MemoryManager;
|
||||
use crate::tool_discovery::ToolDiscovery;
|
||||
use crate::tools::ToolRegistry;
|
||||
|
||||
mod approval;
|
||||
mod agent_dispatch;
|
||||
@@ -102,7 +102,7 @@ pub(super) enum TurnOutcome {
|
||||
output_tokens: Option<u32>,
|
||||
truncated: bool,
|
||||
/// All tool calls executed during this turn, across all rounds.
|
||||
tool_calls: Vec<crate::core::chat_event_bus::ToolCallEvent>,
|
||||
tool_calls: Vec<crate::chat_event_bus::ToolCallEvent>,
|
||||
},
|
||||
Cancelled,
|
||||
Exhausted,
|
||||
+4
-4
@@ -8,9 +8,9 @@
|
||||
use serde_json::Value;
|
||||
use tracing::{debug, info, warn};
|
||||
|
||||
use crate::core::chat_event_bus::ToolCallEvent;
|
||||
use crate::core::db::chat_llm_tools;
|
||||
use crate::core::tools::{is_file_write_tool, ExecutionOutcome};
|
||||
use crate::chat_event_bus::ToolCallEvent;
|
||||
use crate::db::chat_llm_tools;
|
||||
use crate::tools::{is_file_write_tool, ExecutionOutcome};
|
||||
|
||||
use super::ChatSessionHandler;
|
||||
use super::emitter::TurnEmitter;
|
||||
@@ -52,7 +52,7 @@ impl ChatSessionHandler {
|
||||
if is_file_write_tool(tool_name)
|
||||
&& let Some(p) = args["path"].as_str()
|
||||
{
|
||||
em.file_changed(crate::core::approval::normalize_path(p)).await;
|
||||
em.file_changed(crate::approval::normalize_path(p)).await;
|
||||
}
|
||||
acc.push(ToolCallEvent {
|
||||
name: tool_name.to_string(),
|
||||
+5
-5
@@ -3,9 +3,9 @@ use tokio::sync::mpsc;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
use tracing::{error, info, warn};
|
||||
|
||||
use crate::core::db::{chat_history, chat_llm_tools, chat_sessions_stack};
|
||||
use crate::core::events::ServerEvent;
|
||||
use crate::core::tools::{ToolDescriptionLength, ToolResult, tool_names as tn};
|
||||
use crate::db::{chat_history, chat_llm_tools, chat_sessions_stack};
|
||||
use crate::events::ServerEvent;
|
||||
use crate::tools::{ToolDescriptionLength, ToolResult, tool_names as tn};
|
||||
|
||||
use super::{ChatSessionHandler, TurnOutcome};
|
||||
use super::emitter::TurnEmitter;
|
||||
@@ -18,7 +18,7 @@ impl ChatSessionHandler {
|
||||
/// Used by the REST `resolve` endpoint and by `resume_pending_tools`.
|
||||
/// Does NOT update the DB — caller is responsible for `complete` / `fail`.
|
||||
pub async fn execute_tool(&self, name: &str, args: Value) -> anyhow::Result<ToolResult> {
|
||||
if let Some((srv, mcp_tool)) = crate::core::mcp::parse_mcp_tool_name(name) {
|
||||
if let Some((srv, mcp_tool)) = crate::mcp::parse_mcp_tool_name(name) {
|
||||
return self.mcp.call(srv, mcp_tool, args).await;
|
||||
}
|
||||
self.tools.dispatch(name, args).await.map(ToolResult::Text)
|
||||
@@ -365,7 +365,7 @@ fn shallowest_parallel_depth(active: &[chat_sessions_stack::SessionStack]) -> Op
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::shallowest_parallel_depth;
|
||||
use crate::core::db::chat_sessions_stack::SessionStack;
|
||||
use crate::db::chat_sessions_stack::SessionStack;
|
||||
|
||||
fn frame(id: i64, depth: i64, parent: Option<i64>) -> SessionStack {
|
||||
SessionStack { id, agent_id: "agent".into(), depth, parent_tool_call_id: parent }
|
||||
@@ -4,19 +4,19 @@ use std::sync::Arc;
|
||||
use sqlx::SqlitePool;
|
||||
use tokio::sync::Mutex;
|
||||
|
||||
use crate::core::approval::ApprovalManager;
|
||||
use crate::core::chat_event_bus::ChatEventBus;
|
||||
use crate::core::clarification::ClarificationManager;
|
||||
use crate::core::compactor::ContextCompactor;
|
||||
use crate::core::config::DatetimeConfig;
|
||||
use crate::core::db::{chat_sessions, chat_sessions_stack};
|
||||
use crate::core::llm::LlmManager;
|
||||
use crate::core::mcp::McpManager;
|
||||
use crate::core::image_generate::ImageGeneratorManager;
|
||||
use crate::core::memory::MemoryManager;
|
||||
use crate::core::run_context::{RunContext, RunContextManager};
|
||||
use crate::core::tool_discovery::ToolDiscovery;
|
||||
use crate::core::tools::ToolRegistry;
|
||||
use crate::approval::ApprovalManager;
|
||||
use crate::chat_event_bus::ChatEventBus;
|
||||
use crate::clarification::ClarificationManager;
|
||||
use crate::compactor::ContextCompactor;
|
||||
use crate::config::DatetimeConfig;
|
||||
use crate::db::{chat_sessions, chat_sessions_stack};
|
||||
use crate::llm::LlmManager;
|
||||
use crate::mcp::McpManager;
|
||||
use crate::image_generate::ImageGeneratorManager;
|
||||
use crate::memory::MemoryManager;
|
||||
use crate::run_context::{RunContext, RunContextManager};
|
||||
use crate::tool_discovery::ToolDiscovery;
|
||||
use crate::tools::ToolRegistry;
|
||||
|
||||
use super::handler::ChatSessionHandler;
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
//! The logical API surface of `Skald`: one accessor per manager, named after the
|
||||
//! historical field, delegating into the domain bundle that now owns it. This is
|
||||
//! the intentional surface consumers (frontend handlers, plugin context) use — the
|
||||
//! bundles themselves stay internal. Promote this block to a `SkaldApi` trait if/
|
||||
//! when `src/core/` is lifted into its own crate.
|
||||
//! bundles themselves stay internal.
|
||||
//!
|
||||
//! Now that the core is its own crate, this is a real boundary rather than a
|
||||
//! convention: everything here is `pub` because the `skald` binary lives outside,
|
||||
//! and everything not here is unreachable from it. Promote the block to a
|
||||
//! `SkaldApi` trait if the shells ever need to mock it.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
@@ -13,43 +17,45 @@ use tokio_util::sync::CancellationToken;
|
||||
use core_api::remote::RemoteAccess;
|
||||
use core_api::system_bus::SystemEventBus;
|
||||
|
||||
use crate::core::approval::ApprovalManager;
|
||||
use crate::core::chat_event_bus::ChatEventBus;
|
||||
use crate::core::chat_hub::ChatHub;
|
||||
use crate::core::clarification::ClarificationManager;
|
||||
use crate::core::command::LlmCommandManager;
|
||||
use crate::core::config_store::GlobalConfigManager;
|
||||
use crate::core::cron::TaskManager;
|
||||
use crate::core::elicitation::ElicitationManager;
|
||||
use crate::core::image_generate::ImageGeneratorManager;
|
||||
use crate::core::inbox::Inbox;
|
||||
use crate::core::latex::LatexCompiler;
|
||||
use crate::core::llm::LlmManager;
|
||||
use crate::core::location::LocationManager;
|
||||
use crate::core::mcp::McpManager;
|
||||
use crate::core::memory::MemoryManager;
|
||||
use crate::core::plugin::PluginManager;
|
||||
use crate::core::projects::tickets::ProjectTicketManager;
|
||||
use crate::core::projects::ProjectManager;
|
||||
use crate::core::provider::ProviderRegistry;
|
||||
use crate::core::run_context::RunContextManager;
|
||||
use crate::core::secrets::SecretsStore;
|
||||
use crate::core::session::manager::ChatSessionManager;
|
||||
use crate::core::tic::TicManager;
|
||||
use crate::core::tool_catalog::ToolCatalog;
|
||||
use crate::core::tools::ToolRegistry;
|
||||
use crate::core::transcribe::TranscribeManager;
|
||||
use crate::core::tts::TtsManager;
|
||||
use crate::approval::ApprovalManager;
|
||||
use crate::chat_event_bus::ChatEventBus;
|
||||
use crate::chat_hub::ChatHub;
|
||||
use crate::clarification::ClarificationManager;
|
||||
use crate::command::LlmCommandManager;
|
||||
use crate::config_store::GlobalConfigManager;
|
||||
use crate::cron::TaskManager;
|
||||
use crate::elicitation::ElicitationManager;
|
||||
use crate::image_generate::ImageGeneratorManager;
|
||||
use crate::inbox::Inbox;
|
||||
use crate::latex::LatexCompiler;
|
||||
use crate::llm::LlmManager;
|
||||
use crate::location::LocationManager;
|
||||
use crate::mcp::McpManager;
|
||||
use crate::memory::MemoryManager;
|
||||
use crate::plugin::PluginManager;
|
||||
use crate::projects::tickets::ProjectTicketManager;
|
||||
use crate::projects::ProjectManager;
|
||||
use crate::provider::ProviderRegistry;
|
||||
use crate::run_context::RunContextManager;
|
||||
use crate::secrets::SecretsStore;
|
||||
use crate::session::manager::ChatSessionManager;
|
||||
use crate::tic::TicManager;
|
||||
use crate::tool_catalog::ToolCatalog;
|
||||
use crate::tools::ToolRegistry;
|
||||
use crate::transcribe::TranscribeManager;
|
||||
use crate::tts::TtsManager;
|
||||
use crate::users::UserManager;
|
||||
|
||||
use super::Skald;
|
||||
|
||||
impl Skald {
|
||||
// Runtime / cross-cutting
|
||||
pub(crate) fn db(&self) -> &Arc<SqlitePool> { &self.rt.db }
|
||||
pub fn db(&self) -> &Arc<SqlitePool> { &self.rt.db }
|
||||
pub fn users(&self) -> &Arc<UserManager> { &self.rt.users }
|
||||
pub fn config(&self) -> &Arc<GlobalConfigManager> { &self.rt.config }
|
||||
pub fn config_properties(&self) -> &[core_api::ConfigSet] { &self.rt.config_properties }
|
||||
pub(crate) fn system_bus(&self) -> &Arc<SystemEventBus> { &self.rt.system_bus }
|
||||
pub(crate) fn event_bus(&self) -> &Arc<ChatEventBus> { &self.rt.event_bus }
|
||||
pub fn system_bus(&self) -> &Arc<SystemEventBus> { &self.rt.system_bus }
|
||||
pub fn event_bus(&self) -> &Arc<ChatEventBus> { &self.rt.event_bus }
|
||||
pub fn shutdown_token(&self) -> &CancellationToken { &self.rt.shutdown_token }
|
||||
|
||||
// Models
|
||||
@@ -14,35 +14,35 @@ use tracing::{debug, info, warn};
|
||||
|
||||
use core_api::remote::RemoteAccess;
|
||||
|
||||
use crate::core::approval::ApprovalManager;
|
||||
use crate::core::chat_hub::ChatHub;
|
||||
use crate::core::clarification::ClarificationManager;
|
||||
use crate::core::command::LlmCommandManager;
|
||||
use crate::core::compactor::ContextCompactor;
|
||||
use crate::core::config::{CoreConfig, DatetimeConfig};
|
||||
use crate::core::cron::TaskManager;
|
||||
use crate::core::elicitation::ElicitationManager;
|
||||
use crate::core::image_generate::ImageGeneratorManager;
|
||||
use crate::core::inbox::Inbox;
|
||||
use crate::core::latex::LatexCompiler;
|
||||
use crate::core::llm::LlmManager;
|
||||
use crate::core::location::LocationManager;
|
||||
use crate::core::mcp::McpManager;
|
||||
use crate::core::memory::MemoryManager;
|
||||
use crate::core::plugin::PluginManager;
|
||||
use crate::core::projects::tickets::ProjectTicketManager;
|
||||
use crate::core::projects::ProjectManager;
|
||||
use crate::core::provider::ProviderRegistry;
|
||||
use crate::core::run_context::RunContextManager;
|
||||
use crate::core::secrets::SecretsStore;
|
||||
use crate::core::session::handler::{DEFAULT_MAX_PARALLEL_SUBAGENTS, DEFAULT_MAX_TOOL_ROUNDS};
|
||||
use crate::core::session::manager::ChatSessionManager;
|
||||
use crate::core::tic::TicManager;
|
||||
use crate::core::tool_catalog::ToolCatalog;
|
||||
use crate::core::tool_discovery::ToolDiscovery;
|
||||
use crate::core::tools::ToolRegistry;
|
||||
use crate::core::transcribe::TranscribeManager;
|
||||
use crate::core::tts::TtsManager;
|
||||
use crate::approval::ApprovalManager;
|
||||
use crate::chat_hub::ChatHub;
|
||||
use crate::clarification::ClarificationManager;
|
||||
use crate::command::LlmCommandManager;
|
||||
use crate::compactor::ContextCompactor;
|
||||
use crate::config::{CoreConfig, DatetimeConfig};
|
||||
use crate::cron::TaskManager;
|
||||
use crate::elicitation::ElicitationManager;
|
||||
use crate::image_generate::ImageGeneratorManager;
|
||||
use crate::inbox::Inbox;
|
||||
use crate::latex::LatexCompiler;
|
||||
use crate::llm::LlmManager;
|
||||
use crate::location::LocationManager;
|
||||
use crate::mcp::McpManager;
|
||||
use crate::memory::MemoryManager;
|
||||
use crate::plugin::PluginManager;
|
||||
use crate::projects::tickets::ProjectTicketManager;
|
||||
use crate::projects::ProjectManager;
|
||||
use crate::provider::ProviderRegistry;
|
||||
use crate::run_context::RunContextManager;
|
||||
use crate::secrets::SecretsStore;
|
||||
use crate::session::handler::{DEFAULT_MAX_PARALLEL_SUBAGENTS, DEFAULT_MAX_TOOL_ROUNDS};
|
||||
use crate::session::manager::ChatSessionManager;
|
||||
use crate::tic::TicManager;
|
||||
use crate::tool_catalog::ToolCatalog;
|
||||
use crate::tool_discovery::ToolDiscovery;
|
||||
use crate::tools::ToolRegistry;
|
||||
use crate::transcribe::TranscribeManager;
|
||||
use crate::tts::TtsManager;
|
||||
|
||||
use tokio::sync::RwLock;
|
||||
|
||||
@@ -62,18 +62,18 @@ pub(super) struct Models {
|
||||
impl Models {
|
||||
pub(super) async fn build(rt: &Runtime, config: &CoreConfig) -> Result<Self> {
|
||||
let mut provider_registry = ProviderRegistry::new(Arc::clone(&rt.system_bus));
|
||||
provider_registry.register_builtin(crate::core::llm::providers::openai::OpenAiProvider);
|
||||
provider_registry.register_builtin(crate::core::llm::providers::anthropic::AnthropicProvider::new());
|
||||
provider_registry.register_builtin(crate::core::llm::providers::openrouter::OpenRouterProvider::new());
|
||||
provider_registry.register_builtin(crate::core::llm::providers::ollama::OllamaProvider::new());
|
||||
provider_registry.register_builtin(crate::core::llm::providers::lm_studio::LmStudioProvider::new());
|
||||
provider_registry.register_builtin(crate::core::llm::providers::deepseek::DeepSeekProvider::new());
|
||||
provider_registry.register_builtin(crate::core::llm::providers::zai::ZaiProvider::new());
|
||||
provider_registry.register_builtin(crate::llm::providers::openai::OpenAiProvider);
|
||||
provider_registry.register_builtin(crate::llm::providers::anthropic::AnthropicProvider::new());
|
||||
provider_registry.register_builtin(crate::llm::providers::openrouter::OpenRouterProvider::new());
|
||||
provider_registry.register_builtin(crate::llm::providers::ollama::OllamaProvider::new());
|
||||
provider_registry.register_builtin(crate::llm::providers::lm_studio::LmStudioProvider::new());
|
||||
provider_registry.register_builtin(crate::llm::providers::deepseek::DeepSeekProvider::new());
|
||||
provider_registry.register_builtin(crate::llm::providers::zai::ZaiProvider::new());
|
||||
let provider_registry = Arc::new(provider_registry);
|
||||
info!("provider registry ready ({} built-in providers)", provider_registry.all().len());
|
||||
|
||||
let log_flags = config.llm.requests_log.as_ref().filter(|r| r.enabled).map(|r| {
|
||||
use crate::core::chatbot::logging::LogSaveFlags;
|
||||
use crate::chatbot::logging::LogSaveFlags;
|
||||
LogSaveFlags {
|
||||
request_payload: r.request_payload_save,
|
||||
response_payload: r.response_payload_save,
|
||||
@@ -214,35 +214,38 @@ impl Tools {
|
||||
/// per interactive session by `ChatHub::send_message`.
|
||||
pub(super) fn build(integrations: &Integrations, tasks: &Tasks, models: &Models) -> Self {
|
||||
let mut tool_registry = ToolRegistry::new();
|
||||
crate::core::tools::fs::register_all(&mut tool_registry);
|
||||
tool_registry.register(crate::core::tools::ast_outline::AstOutline::new());
|
||||
tool_registry.register(crate::core::tools::exec::ExecuteCmd);
|
||||
tool_registry.register(crate::core::tools::read_notification::ReadNotification);
|
||||
tool_registry.register(crate::core::tools::restart::Restart);
|
||||
crate::tools::fs::register_all(&mut tool_registry);
|
||||
tool_registry.register(crate::tools::ast_outline::AstOutline::new());
|
||||
tool_registry.register(crate::tools::exec::ExecuteCmd);
|
||||
tool_registry.register(crate::tools::read_notification::ReadNotification);
|
||||
tool_registry.register(crate::tools::restart::Restart);
|
||||
// Unified listing / toggling across mcp, plugins, cron (+ agents for list).
|
||||
tool_registry.register(crate::core::tools::list_items::ListItems::new(
|
||||
tool_registry.register(crate::tools::list_items::ListItems::new(
|
||||
Arc::clone(&integrations.mcp), Arc::clone(&integrations.plugin_manager), Arc::clone(&tasks.cron)));
|
||||
tool_registry.register(crate::core::tools::toggle_item::ToggleItem::new(
|
||||
tool_registry.register(crate::tools::toggle_item::ToggleItem::new(
|
||||
Arc::clone(&integrations.mcp), Arc::clone(&integrations.plugin_manager), Arc::clone(&tasks.cron)));
|
||||
tool_registry.register(crate::core::tools::register_mcp::RegisterMcp::new(Arc::clone(&integrations.mcp)));
|
||||
tool_registry.register(crate::core::tools::register_mcp::DeleteMcp::new(Arc::clone(&integrations.mcp)));
|
||||
tool_registry.register(crate::core::tools::cron_jobs::DeleteCronJob(Arc::clone(&tasks.cron)));
|
||||
tool_registry.register(crate::core::tools::set_secret::SetSecret(Arc::clone(&models.secrets)));
|
||||
tool_registry.register(crate::core::tools::list_secrets::ListSecrets(Arc::clone(&models.secrets)));
|
||||
tool_registry.register(crate::core::tools::configure_plugin::ConfigurePlugin(Arc::clone(&integrations.plugin_manager)));
|
||||
tool_registry.register(crate::tools::register_mcp::RegisterMcp::new(Arc::clone(&integrations.mcp)));
|
||||
tool_registry.register(crate::tools::register_mcp::DeleteMcp::new(Arc::clone(&integrations.mcp)));
|
||||
tool_registry.register(crate::tools::cron_jobs::DeleteCronJob(Arc::clone(&tasks.cron)));
|
||||
tool_registry.register(crate::tools::set_secret::SetSecret(Arc::clone(&models.secrets)));
|
||||
tool_registry.register(crate::tools::list_secrets::ListSecrets(Arc::clone(&models.secrets)));
|
||||
tool_registry.register(crate::tools::configure_plugin::ConfigurePlugin(Arc::clone(&integrations.plugin_manager)));
|
||||
|
||||
// Mobile-connector control tools (plugin.md §11). The plugin Arc itself
|
||||
// implements `RelayAgent`; we look it up and bind the tools to it. The tools
|
||||
// call into the plugin lazily, so registering them before the plugin's
|
||||
// runloop starts is fine (they fail gracefully while stopped).
|
||||
if let Some(mc) = integrations.plugin_manager
|
||||
.get_plugin_typed::<plugin_mobile_connector::MobileConnectorPlugin>("mobile-connector")
|
||||
{
|
||||
let agent: Arc<dyn plugin_mobile_connector::RelayAgent> = mc;
|
||||
for tool in plugin_mobile_connector::mobile_tools(agent) {
|
||||
// Tools contributed by plugins (plugin.md §11), via `Plugin::tools()`.
|
||||
// The core never names a plugin crate: each one hands over whatever tools
|
||||
// it wants, bound to its own handle. They are built before the plugins'
|
||||
// runloops start, so they must tolerate being called while stopped.
|
||||
for plugin in integrations.plugin_manager.all() {
|
||||
let id = plugin.id().to_string();
|
||||
let tools = Arc::clone(plugin).tools();
|
||||
if tools.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let n = tools.len();
|
||||
for tool in tools {
|
||||
tool_registry.register_arc(tool);
|
||||
}
|
||||
info!("mobile-connector tools registered");
|
||||
info!(plugin = %id, count = n, "plugin tools registered");
|
||||
}
|
||||
debug!("tool registry built");
|
||||
|
||||
@@ -92,5 +92,8 @@ impl Skald {
|
||||
self.rt.shutdown_token.cancel();
|
||||
self.rt.supervisor.join_all(tokio::time::Duration::from_secs(10)).await;
|
||||
self.integrations.plugin_manager.stop_all().await;
|
||||
// Last: every user key leaves RAM. A restarted box is opaque again until
|
||||
// each user unlocks their own database (§9).
|
||||
self.rt.users.lock_all().await;
|
||||
}
|
||||
}
|
||||
@@ -17,13 +17,17 @@ use tracing::info;
|
||||
use core_api::events::GlobalEvent;
|
||||
use core_api::system_bus::SystemEventBus;
|
||||
|
||||
use crate::core::chat_event_bus::ChatEventBus;
|
||||
use crate::core::config_store::GlobalConfigManager;
|
||||
use crate::chat_event_bus::ChatEventBus;
|
||||
use crate::config_store::GlobalConfigManager;
|
||||
use crate::users::UserManager;
|
||||
|
||||
use super::supervisor::TaskSupervisor;
|
||||
|
||||
pub(super) struct Runtime {
|
||||
/// The registry pool (`system.db`). Still the only pool anything reads or
|
||||
/// writes: nothing has moved to per-user pools yet. `users` owns those.
|
||||
pub(super) db: Arc<SqlitePool>,
|
||||
pub(super) users: Arc<UserManager>,
|
||||
pub(super) config: Arc<GlobalConfigManager>,
|
||||
pub(super) config_properties: Vec<core_api::ConfigSet>,
|
||||
pub(super) system_bus: Arc<SystemEventBus>,
|
||||
@@ -41,6 +45,8 @@ impl Runtime {
|
||||
pub(super) fn bootstrap(pool: Arc<SqlitePool>) -> Self {
|
||||
let config = Arc::new(GlobalConfigManager::new(Arc::clone(&pool)));
|
||||
|
||||
let users = Arc::new(UserManager::new(Arc::clone(&pool)));
|
||||
|
||||
let system_bus = Arc::new(SystemEventBus::new());
|
||||
info!("system event bus ready");
|
||||
|
||||
@@ -51,8 +57,9 @@ impl Runtime {
|
||||
|
||||
Runtime {
|
||||
db: pool,
|
||||
users,
|
||||
config,
|
||||
config_properties: vec![crate::core::tic::config_set()],
|
||||
config_properties: vec![crate::tic::config_set()],
|
||||
system_bus,
|
||||
event_bus,
|
||||
global_tx,
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user