dguiducci/Skald-Circle
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: dguiducci/Skald-Circle:587958ffe0c9281bfad6ab723884cb4728b99da1
Branches Tags
dguiducci/Skald-Circle:main dguiducci/Skald-Circle:release
..
compare: dguiducci/Skald-Circle:a847dda88f43e266c62db3adbbe1e73f61a70596
Branches Tags
dguiducci/Skald-Circle:main dguiducci/Skald-Circle:release
2 Commits
587958ffe0 .. a847dda88f
Author SHA1 Message Date
dguiducci a847dda88f feat(memory): dual-pool memory namespace, FTS search, and prompt injection 
Add a virtual memory namespace backed by SQLite, surfaced through the
fs-tools, with private (per-user) and shared (system) stores.

Storage
- `memory_docs` owner table + external-content FTS5 index with sync triggers.
- `db/memory_docs.rs` accessor: get / upsert / list / search (bm25+snippet) / delete.

Routing (tools/fs)
- `classify_memory` splits paths on the raw first component; `..` clamps inside
  the store, never escaping to disk.
- read/write/list/edit/insert/replace/search_file route `user-memory/` to the
  owner pool and `shared-memory/` to the system pool (a singleton captured in
  `register_all`); every other path stays on disk. Each tool extracts a pure
  transform shared between its disk and memory paths.
- New `memory_search` tool over the FTS index (scope private/shared/all),
  with a sanitised FTS5 query. grep_files stays disk-only.

Approval
- `user-memory/*` allow (read+write); `shared-memory/*` reads allow,
  writes require approval so the agent can't silently push one person's data
  into shared memory. `memory_search` allowed via a path-less rule.
- migrate away the old `memory/*` and blanket `shared-memory/*` rows.

Prompt injection
- `MessageBuilder::load_inject_memory` reads `user-memory/` (owner pool) and
  `shared-memory/` (system pool) inject entries from SQLite; disk paths
  unchanged. The system pool is threaded ChatSessionManager -> handler ->
  MessageBuilder.
- main and project-coordinator inject `user-memory/index.md` +
  `shared-memory/index.md`; common/memory.md rewritten for the two stores.
 2026-07-11 02:11:00 +01:00
dguiducci 5848829a92 telegram bot user isolation, config store, user context channels 2026-07-11 01:02:37 +01:00
42 changed files with 2428 additions and 673 deletions
Expand all files Collapse all files
CLAUDE.md
+6 -2
View File
@@ -69,7 +69,7 @@ Two rules keep the boundary real, and both are enforced by the compiler:
| `crates/skald-core/src/chat_hub/` | `ChatHub`: broadcast events to all connected WS clients |
| `crates/skald-core/src/chat_event_bus.rs` | Global async bus for cross-session events |
| `crates/skald-core/src/agents.rs` | Discovers agents from `agents/*/`, loads meta + system prompt |
| `crates/skald-core/src/tools/` | Built-in tools: `exec`, `restart`, `list_agents`, `fs/*`, `notify`, `ast_outline`, `image_generate`, MCP tools, plugin tools, cron tools |
| `crates/skald-core/src/tools/` | Built-in tools: `exec`, `restart`, `list_agents`, `fs/*` (also route `user-memory/`/`shared-memory/` to `memory_docs` — see DB tables), `notify`, `ast_outline`, `image_generate`, MCP tools, plugin tools, cron tools |
| `crates/skald-core/src/tool_catalog.rs` | `ToolCatalog`: unified tool listing façade (wraps ToolRegistry + McpManager) |
| `crates/skald-core/src/events.rs` | `ServerEvent` enum streamed over WebSocket to the frontend |
| `crates/skald-core/src/db/` | sqlx SQLite — see below |
@@ -100,10 +100,14 @@ Two rules keep the boundary real, and both are enforced by the compiler:
The schema is split into two buckets (§5.1), and the split is the point:
- **`create_registry_tables`** — instance-wide, readable without any user key: `users`, `llm_providers`, `llm_models`, `transcribe_models`, `tts_models`, `image_generate_models`, `plugins`, `approval_rules`, `tool_permission_groups`, `config`, `known_tools`, `llm_requests`.
- **`create_owner_tables`** — one owner's content, **identical schema in every file that has it**: `chat_sessions`, `chat_sessions_stack`, `chat_history`, `chat_llm_tools`, `chat_summaries`, `session_scratchpad`, `session_mcp_grants`, `stack_mcp_grants`, `scheduled_jobs`, `job_runs`, `mcp_servers`, `mcp_events`, `sources`, `secrets`, `projects`, `project_tickets`.
- **`create_owner_tables`** — one owner's content, **identical schema in every file that has it**: `chat_sessions`, `chat_sessions_stack`, `chat_history`, `chat_llm_tools`, `chat_summaries`, `session_scratchpad`, `session_mcp_grants`, `stack_mcp_grants`, `scheduled_jobs`, `job_runs`, `mcp_servers`, `mcp_events`, `sources`, `secrets`, `projects`, `project_tickets`, `llm_request_payloads`, `memory_docs` (+ FTS5 `memory_docs_fts`). Because `memory_docs` is an owner table, one definition backs **private** memory in each `{userid}.db` and **shared** memory in `system.db` (the household owner) — see the memory namespace note below.
**No foreign key in the owner bucket may point at a registry table.** SQLite cannot enforce a key across files, not even through `ATTACH`, and sqlx turns on `PRAGMA foreign_keys`: the `CREATE TABLE` succeeds and every `INSERT` fails. `db::tests::owner_tables_stand_alone_with_foreign_keys_on` enforces this by running the owner schema against a database holding nothing else, then inserting a row into each table. Two keys crossed and were fixed: `chat_history.model_db_id` (dropped — write-only, and `llm_requests.model_name` already records the model) and `project_tickets.job_id` (fixed by moving `projects`/`project_tickets` into the owner bucket).
**Memory namespace (blueprint §5).** `memory_docs` (accessor `db/memory_docs.rs` — `get`/`upsert`/`list`/`search`(FTS)/`delete`) backs a virtual note store surfaced through the fs-tools, **not** the disk. Two sibling roots (not the blueprint's nested `memory/{userid}` + `memory/shared`): `user-memory/…` routes to the caller's own pool (`ToolContext::pool`), `shared-memory/…` to the system pool (a singleton captured in `fs::register_all`). `tools/fs/classify_memory()` decides on the raw first path component (a `..` in the tail clamps inside the store, never escapes to disk); `read_file`/`write_file`/`list_files`/`edit_file`/`insert_at_line`/`replace_lines`/`search_file` override `run_with` to route memory paths (each extracting a pure transform shared with its on-disk `execute`) and leave every other path on disk. Approval (seeded in `seed_fs_path_rules`): `user-memory/*` is `@fs_any allow` (private, frictionless); `shared-memory/*` is `@fs_read allow` + `@fs_write require` — reads free, **writes need approval** so the agent can't silently push one person's data into shared memory. `grep_files` stays disk-only (regex-across-tree ≠ FTS); ranked full-text recall over notes is a separate tool, `memory_search` (`tools/fs/memory_search.rs`), over the `memory_docs` FTS index — allowed by a path-less rule (it takes `query`, not `path`).
**Memory injection into the prompt**: `MessageBuilder::load_inject_memory` routes each `meta.inject_memory` entry — `user-memory/…` → owner pool, `shared-memory/…` → the shared (`system.db`) pool, both via `memory_docs::get`; anything else (`data/…`, `$WD/…`) is a disk read. The shared pool is threaded `ChatSessionManager` → handler → `MessageBuilder`. `main` and `project-coordinator` inject `user-memory/index.md` + `shared-memory/index.md`.
`system.db` currently gets **both** bucket functions, because nothing has migrated to per-user pools yet. That is transitional.
`users` (`crates/skald-core/src/db/users.rs`) holds the directory plus auth material. It lives in the system DB, which the box owner can read, so it must never store anything that derives a user's key. `Credentials` is an enum mirroring the table's `CHECK`: an encrypted user carries a **wrapped DEK** (whose AEAD tag *is* the password verifier — hence no `password_hash`); a cleartext user carries an ordinary verifier, or none. `User` is deliberately not `Serialize` and its `Debug` redacts key material — use `User::summary()` for anything leaving the process. `role_id` has no foreign key yet: sqlx enables `PRAGMA foreign_keys`, so referencing the not-yet-existing `roles` table would fail every insert.
Cargo.lock
Generated
+417 -14
View File
@@ -88,6 +88,29 @@ version = "1.0.102"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
[[package]]
name = "aquamarine"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0f50776554130342de4836ba542aa85a4ddb361690d7e8df13774d7284c3d5c2"
dependencies = [
"include_dir",
"itertools 0.10.5",
"proc-macro-error2",
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "ar_archive_writer"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4087686b4b0a3427190bae57a1d9a478dbb2d40c5dc1bd6e2b6d797913bdd348"
dependencies = [
"object",
]
[[package]]
name = "argon2"
version = "0.5.3"
@@ -639,6 +662,15 @@ version = "0.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a"
[[package]]
name = "colored"
version = "3.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "faf9468729b8cbcea668e36183cb69d317348c2e08e994829fb56ebfdfbaac34"
dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "combine"
version = "4.6.7"
@@ -1109,13 +1141,34 @@ dependencies = [
"serde_core",
]
[[package]]
name = "derive_more"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4a9b99b9cbbe49445b21764dc0625032a89b145a2642e67603e1c936f5458d05"
dependencies = [
"derive_more-impl 1.0.0",
]
[[package]]
name = "derive_more"
version = "2.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d751e9e49156b02b44f9c1815bcb94b984cdcc4396ecc32521c739452808b134"
dependencies = [
"derive_more-impl",
"derive_more-impl 2.1.1",
]
[[package]]
name = "derive_more-impl"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb7330aeadfbe296029522e6c40f315320aba36fc43a5b3632f3795348f3bd22"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
"unicode-xid",
]
[[package]]
@@ -1276,6 +1329,16 @@ dependencies = [
"serde",
]
[[package]]
name = "dptree"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db96968fcf52fe063a98c75df1d1f2b1fba304e7ae29b72fdc81c1165b7e2fd0"
dependencies = [
"colored",
"futures",
]
[[package]]
name = "dtoa"
version = "1.0.11"
@@ -1406,6 +1469,16 @@ version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "erasable"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "437cfb75878119ed8265685c41a115724eae43fb7cc5a0bf0e4ecc3b803af1c4"
dependencies = [
"autocfg",
"scopeguard",
]
[[package]]
name = "erased-serde"
version = "0.4.10"
@@ -1859,11 +1932,13 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555"
dependencies = [
"cfg-if",
"js-sys",
"libc",
"r-efi 6.0.0",
"rand_core 0.10.1",
"wasip2",
"wasip3",
"wasm-bindgen",
]
[[package]]
@@ -2188,7 +2263,7 @@ checksum = "ec9d92d097f4749b64e8cc33d924d9f40a2d4eb91402b458014b781f5733d60f"
name = "honcho-client"
version = "0.1.0"
dependencies = [
"reqwest",
"reqwest 0.13.4",
"serde",
"serde_json",
"tracing",
@@ -2311,6 +2386,7 @@ dependencies = [
"tokio",
"tokio-rustls",
"tower-service",
"webpki-roots 1.0.7",
]
[[package]]
@@ -2487,6 +2563,25 @@ dependencies = [
"icu_properties",
]
[[package]]
name = "include_dir"
version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "923d117408f1e49d914f1a379a309cffe4f18c05cf4e3d12e613a15fc81bd0dd"
dependencies = [
"include_dir_macros",
]
[[package]]
name = "include_dir_macros"
version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7cab85a7ed0bd5f0e76d93846e0147172bed2e2d3f859bcc33a8d9699cad1a75"
dependencies = [
"proc-macro2",
"quote",
]
[[package]]
name = "indexmap"
version = "1.9.3"
@@ -2919,7 +3014,7 @@ dependencies = [
"anyhow",
"async-trait",
"core-api",
"reqwest",
"reqwest 0.13.4",
"serde",
"serde_json",
"tracing",
@@ -2940,6 +3035,12 @@ version = "0.4.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897"
[[package]]
name = "lru-slab"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154"
[[package]]
name = "managed"
version = "0.8.0"
@@ -2979,7 +3080,7 @@ dependencies = [
"anyhow",
"async-trait",
"base64 0.22.1",
"reqwest",
"reqwest 0.13.4",
"serde",
"serde_json",
"tokio",
@@ -3478,6 +3579,15 @@ dependencies = [
"objc2-foundation",
]
[[package]]
name = "object"
version = "0.37.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff76201f031d8863c38aa7f905eca4f53abbfa15f609db4277d44cd8938f33fe"
dependencies = [
"memchr",
]
[[package]]
name = "once_cell"
version = "1.21.4"
@@ -3793,6 +3903,26 @@ dependencies = [
"siphasher",
]
[[package]]
name = "pin-project"
version = "1.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2466b2336ed02bcdca6b294417127b90ec92038d1d5c4fbeac971a922e0e0924"
dependencies = [
"pin-project-internal",
]
[[package]]
name = "pin-project-internal"
version = "1.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c96395f0a926bc13b1c17622aaddda1ecb55d49c8f1bf9777e4d877800a43f8b"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "pin-project-lite"
version = "0.2.17"
@@ -3835,7 +3965,7 @@ dependencies = [
"anyhow",
"async-trait",
"core-api",
"reqwest",
"reqwest 0.13.4",
"serde",
"serde_json",
"tokio",
@@ -3850,7 +3980,7 @@ dependencies = [
"async-trait",
"core-api",
"parking_lot",
"reqwest",
"reqwest 0.13.4",
"serde_json",
"tokio",
"tracing",
@@ -3871,6 +4001,24 @@ dependencies = [
"tracing",
]
[[package]]
name = "plugin-telegram-bot"
version = "0.1.0"
dependencies = [
"anyhow",
"async-trait",
"chrono",
"core-api",
"rand 0.10.1",
"regex",
"serde",
"serde_json",
"teloxide",
"tokio",
"tokio-util",
"tracing",
]
[[package]]
name = "plugin-transcribe-whisper-local"
version = "0.1.0"
@@ -3892,7 +4040,7 @@ dependencies = [
"anyhow",
"async-trait",
"core-api",
"reqwest",
"reqwest 0.13.4",
"serde_json",
"tokio",
"tracing",
@@ -3905,7 +4053,7 @@ dependencies = [
"anyhow",
"async-trait",
"core-api",
"reqwest",
"reqwest 0.13.4",
"serde_json",
"tokio",
"tracing",
@@ -4234,6 +4382,16 @@ version = "3.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "95067976aca6421a523e491fce939a3e65249bac4b977adee0ee9771568e8aa3"
[[package]]
name = "psm"
version = "0.1.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "645dbe486e346d9b5de3ef16ede18c26e6c70ad97418f4874b8b1889d6e761ea"
dependencies = [
"ar_archive_writer",
"cc",
]
[[package]]
name = "quick-xml"
version = "0.41.0"
@@ -4243,6 +4401,62 @@ dependencies = [
"memchr",
]
[[package]]
name = "quinn"
version = "0.11.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c1a41e437b6bbd489372cd4971de128e85c855f56c57f283d20ff016cf7c0a8"
dependencies = [
"bytes",
"cfg_aliases",
"pin-project-lite",
"quinn-proto",
"quinn-udp",
"rustc-hash",
"rustls",
"socket2",
"thiserror 2.0.18",
"tokio",
"tracing",
"web-time",
]
[[package]]
name = "quinn-proto"
version = "0.11.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2f4bfc015262b9df63c8845072ce59068853ff5872180c2ce2f13038b970e560"
dependencies = [
"bytes",
"getrandom 0.4.2",
"lru-slab",
"rand 0.10.1",
"rand_pcg",
"ring",
"rustc-hash",
"rustls",
"rustls-pki-types",
"slab",
"thiserror 2.0.18",
"tinyvec",
"tracing",
"web-time",
]
[[package]]
name = "quinn-udp"
version = "0.5.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "35a133f956daabe89a61a685c2649f13d82d5aa4bd5d12d1277e1072a21c0694"
dependencies = [
"cfg_aliases",
"libc",
"once_cell",
"socket2",
"tracing",
"windows-sys 0.61.2",
]
[[package]]
name = "quote"
version = "1.0.45"
@@ -4350,12 +4564,30 @@ version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69"
[[package]]
name = "rand_pcg"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "caa0f4137e1c0a72f4c651489402276c8e8e1cf081f3b0ba156d2cbeef09e86a"
dependencies = [
"rand_core 0.10.1",
]
[[package]]
name = "raw-window-handle"
version = "0.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20675572f6f24e9e76ef639bc5552774ed45f1c30e2951e1e99c59888861c539"
[[package]]
name = "rc-box"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "897fecc9fac6febd4408f9e935e86df739b0023b625e610e0357535b9c8adad0"
dependencies = [
"erasable",
]
[[package]]
name = "redox_syscall"
version = "0.5.18"
@@ -4442,6 +4674,48 @@ version = "0.8.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a"
[[package]]
name = "reqwest"
version = "0.12.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eddd3ca559203180a307f12d114c268abf583f59b03cb906fd0b3ff8646c1147"
dependencies = [
"base64 0.22.1",
"bytes",
"futures-core",
"futures-util",
"http",
"http-body",
"http-body-util",
"hyper",
"hyper-rustls",
"hyper-util",
"js-sys",
"log",
"mime_guess",
"percent-encoding",
"pin-project-lite",
"quinn",
"rustls",
"rustls-pki-types",
"serde",
"serde_json",
"serde_urlencoded",
"sync_wrapper",
"tokio",
"tokio-rustls",
"tokio-util",
"tower",
"tower-http 0.6.11",
"tower-service",
"url",
"wasm-bindgen",
"wasm-bindgen-futures",
"wasm-streams 0.4.2",
"web-sys",
"webpki-roots 1.0.7",
]
[[package]]
name = "reqwest"
version = "0.13.4"
@@ -4481,10 +4755,19 @@ dependencies = [
"url",
"wasm-bindgen",
"wasm-bindgen-futures",
"wasm-streams",
"wasm-streams 0.5.0",
"web-sys",
]
[[package]]
name = "rgb"
version = "0.8.53"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "47b34b781b31e5d73e9fbc8689c70551fd1ade9a19e3e28cfec8580a79290cc4"
dependencies = [
"bytemuck",
]
[[package]]
name = "ring"
version = "0.17.14"
@@ -4601,6 +4884,7 @@ version = "1.14.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "30a7197ae7eb376e574fe940d068c30fe0462554a3ddbe4eca7838e049c937a9"
dependencies = [
"web-time",
"zeroize",
]
@@ -4770,7 +5054,7 @@ checksum = "c5d9c0c92a92d33f08817311cf3f2c29a3538a8240e94a6a3c622ce652d7e00c"
dependencies = [
"bitflags 2.11.1",
"cssparser",
"derive_more",
"derive_more 2.1.1",
"log",
"new_debug_unreachable",
"phf 0.13.1",
@@ -5124,10 +5408,11 @@ dependencies = [
"plugin-comfyui",
"plugin-elevenlabs",
"plugin-tailscale-remote",
"plugin-telegram-bot",
"plugin-transcribe-whisper-local",
"plugin-tts-kokoro",
"plugin-tts-orpheus-3b",
"reqwest",
"reqwest 0.13.4",
"rustls",
"serde",
"serde_json",
@@ -5174,7 +5459,7 @@ dependencies = [
"quote",
"rand 0.10.1",
"regex",
"reqwest",
"reqwest 0.13.4",
"serde",
"serde_json",
"sha2 0.10.9",
@@ -5268,7 +5553,7 @@ dependencies = [
"jsonwebtoken",
"prost",
"rand 0.8.6",
"reqwest",
"reqwest 0.13.4",
"serde",
"serde_json",
"sha2 0.10.9",
@@ -5576,6 +5861,19 @@ version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
[[package]]
name = "stacker"
version = "0.1.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "640c8cdd92b6b12f5bcb1803ca3bbf5ab96e5e6b6b96b9ab77dabe9e880b3190"
dependencies = [
"cc",
"cfg-if",
"libc",
"psm",
"windows-sys 0.61.2",
]
[[package]]
name = "static_assertions"
version = "1.1.0"
@@ -5771,6 +6069,18 @@ dependencies = [
"url",
]
[[package]]
name = "take_mut"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f764005d11ee5f36500a149ace24e00e3da98b0158b3e2d53a7495660d3f4d60"
[[package]]
name = "takecell"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20f34339676cdcab560c9a82300c4c2581f68b9369aedf0fae86f2ff9565ff3e"
[[package]]
name = "tao"
version = "0.35.3"
@@ -5858,7 +6168,7 @@ dependencies = [
"percent-encoding",
"plist",
"raw-window-handle",
"reqwest",
"reqwest 0.13.4",
"serde",
"serde_json",
"serde_repr",
@@ -6041,6 +6351,76 @@ dependencies = [
"toml 1.1.2+spec-1.1.0",
]
[[package]]
name = "teloxide"
version = "0.17.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "84992abeed3ae42e8401b25d266d12bcba1def0abe59d22f6b9781167545f71e"
dependencies = [
"aquamarine",
"bytes",
"derive_more 1.0.0",
"dptree",
"either",
"futures",
"log",
"mime",
"pin-project",
"serde",
"serde_json",
"teloxide-core",
"teloxide-macros",
"thiserror 2.0.18",
"tokio",
"tokio-stream",
"tokio-util",
"url",
]
[[package]]
name = "teloxide-core"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f7a34ca8e971fa892e633858c07547fe138ef4a02e4a4eaa1d35e517d6e0bc4"
dependencies = [
"bitflags 2.11.1",
"bytes",
"chrono",
"derive_more 1.0.0",
"either",
"futures",
"log",
"mime",
"once_cell",
"pin-project",
"rc-box",
"reqwest 0.12.28",
"rgb",
"serde",
"serde_json",
"serde_with",
"stacker",
"take_mut",
"takecell",
"thiserror 2.0.18",
"tokio",
"tokio-util",
"url",
"uuid",
]
[[package]]
name = "teloxide-macros"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "300fadcaf0c182f19b5ca10bf23a45dc9a48925f00c704405fd90ee2c03942f9"
dependencies = [
"heck 0.5.0",
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "tempfile"
version = "3.27.0"
@@ -7554,6 +7934,19 @@ dependencies = [
"wasmparser",
]
[[package]]
name = "wasm-streams"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "15053d8d85c7eccdbefef60f06769760a563c7f0a9d6902a13d35c7800b0ad65"
dependencies = [
"futures-util",
"js-sys",
"wasm-bindgen",
"wasm-bindgen-futures",
"web-sys",
]
[[package]]
name = "wasm-streams"
version = "0.5.0"
@@ -7589,6 +7982,16 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "web-time"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a6580f308b1fad9207618087a65c04e7a10bc77e02c8e84e9b00dd4b12fa0bb"
dependencies = [
"js-sys",
"wasm-bindgen",
]
[[package]]
name = "web_atoms"
version = "0.2.5"
Cargo.toml
+2
View File
@@ -8,6 +8,7 @@ members = [
"crates/core-api",
"crates/mcp-client",
"crates/plugin-tailscale-remote",
"crates/plugin-telegram-bot",
"crates/plugin-transcribe-whisper-local",
"crates/plugin-comfyui",
"crates/plugin-tts-orpheus-3b",
@@ -81,6 +82,7 @@ llm-client = { path = "crates/llm-client" }
core-api = { path = "crates/core-api" }
mcp-client = { path = "crates/mcp-client" }
plugin-tailscale-remote = { path = "crates/plugin-tailscale-remote" }
plugin-telegram-bot = { path = "crates/plugin-telegram-bot" }
plugin-transcribe-whisper-local = { path = "crates/plugin-transcribe-whisper-local", optional = true }
plugin-comfyui = { path = "crates/plugin-comfyui" }
plugin-tts-orpheus-3b = { path = "crates/plugin-tts-orpheus-3b" }
agents/common/memory.md
+30 -10
View File
@@ -1,20 +1,41 @@
# Persistent memory
All memory lives in `data/memory/`. Entry point: `data/memory/index.md` — one line per file with a brief summary.
You have two persistent note stores, kept as Markdown and searchable. **Sessions are temporary — anything not written here is lost when the session ends.** Save proactively.
- **`user-memory/`** — your **private** memory for this user. Nobody else can read it. Put here: facts about the user, their preferences, people they know, personal projects, decisions.
- **`shared-memory/`** — memory **shared with the whole group**. Every member can read it. Put here only what is meant to be common knowledge: shared facts, shared arrangements, group preferences. **Never** put one person's private information here. Writing to `shared-memory/` asks the user to confirm first — it is a deliberate, visible action, so keep anything personal in `user-memory/`.
When unsure where something belongs, prefer `user-memory/`.
## The indexes
Each store has an `index.md` — one line per note with a brief summary — and **both are injected into your context automatically** at the start of each session (look for them below):
- `user-memory/index.md` — your private notes.
- `shared-memory/index.md` — the group's shared notes.
Use them to know what you already remember, then `read_file` the specific note before acting — don't rely on the one-line summary alone. **Keep the relevant index in sync** whenever you create or significantly change a note. Updating `shared-memory/index.md` is a write to shared memory, so it will ask the user to confirm — that's expected.
## When to save
Save **immediately** (do not postpone) when:
- The user shares a new fact about themselves, a project, a person, or a preference
- A decision is made that may be relevant in future sessions
- You notice an inconsistency with what was previously saved → correct it
- A decision is made that may matter in a future session
- You notice that something you saved before is now wrong → correct it
## When to read
At the start of each session, read `data/memory/index.md` silently. Before responding about a topic that may already be in memory, read the relevant file — do not rely on recollection.
Before responding about a topic that may already be in memory, look it up — do not rely on recollection:
## File format
- The injected `user-memory/index.md` tells you what exists; `read_file` the note it points to.
- `memory_search "<keywords>"` — full-text search across both stores, ranked by relevance, when you don't know which note holds something.
## Organising notes
Use clear, topic-based paths — e.g. `user-memory/people/alice.md`, `user-memory/projects/website.md`, `shared-memory/wifi.md`. Keep one topic per note.
## Note format
```md
# Title
@@ -28,8 +49,7 @@ _Updated: YYYY-MM-DD_
## How to update
1. `read_file` to get the exact current content
2. `edit_file` to modify — always keep the `_Updated: YYYY-MM-DD_` date in sync
3. Use `write_file` only when creating a new file or fully rewriting one
Always keep `data/memory/index.md` in sync when you create or significantly update a file.
1. `read_file` the note to get its exact current content.
2. `edit_file` to change part of it — keep the `_Updated:_` date in sync.
3. Use `write_file` only to create a new note or fully rewrite one.
4. Keep `user-memory/index.md` in sync when you add or significantly change a note.
agents/main/AGENT.md
+2 -4
View File
@@ -2,11 +2,9 @@
You are an extremely powerful general-purpose personal assistant. You help the user with any task — research, writing, planning, analysis, coding, or anything else they bring to you.
Your personality and tone are defined in `data/memory/SOUL.md`. If the file exists, it is automatically injected into your system context — look for it at the end of this prompt.
Think outside the box: you can use tools, write and execute Python scripts on the fly, or even modify your own source code.
The `data/` directory (inside your working directory) is your own space — write there freely; you have permission to create and modify anything under it. **Default to `data/` for everything you produce**: generated files, notes, one-shot scripts, downloads, and persistent memory (e.g. `data/memory/`, `data/notifications.md`). When a path is relative, prefix it with `data/` — a bare filename lands in the project root, which is not where your working files belong. Write **outside** `data/` (the project root, `src/`, `web/`, `agents/`, config, …) only when a specific, well-defined goal genuinely requires it and cannot be accomplished within `data/`.
The `data/` directory (inside your working directory) is your own space — write there freely; you have permission to create and modify anything under it. **Default to `data/` for everything you produce**: generated files, notes, one-shot scripts, downloads. When a path is relative, prefix it with `data/` — a bare filename lands in the project root, which is not where your working files belong. (Persistent **memory** is separate: durable facts go to `user-memory/` or `shared-memory/`, not under `data/` — see the Memory section.) Write **outside** `data/` (the project root, `src/`, `web/`, `agents/`, config, …) only when a specific, well-defined goal genuinely requires it and cannot be accomplished within `data/`.
You have access to tools, persistent memory system and sub agents. Use both proactively. Sub agents also help to keep your context windows small and concise.
@@ -113,7 +111,7 @@ Configuration tools are hidden by default to keep context small. Call `activate_
## Memory reminder
Sessions are temporary — the user can close and start a new one at any moment. **Context alone is not enough.** If something is worth remembering, write it to a file in `data/memory/` immediately. If it stays only in context, it is gone forever when the session ends.
Sessions are temporary — the user can close and start a new one at any moment. **Context alone is not enough.** If something is worth remembering, save it to `user-memory/` immediately (or `shared-memory/` if it's meant for the whole group). If it stays only in context, it is gone forever when the session ends.
---
agents/main/meta.json
+2 -2
View File
@@ -1,9 +1,9 @@
{
"name": "Main Assistant",
"description": "General-purpose assistant: helps the user with any task using tools, and persists all relevant information in data/memory",
"description": "General-purpose assistant: helps the user with any task using tools, and persists all relevant information in memory",
"friendly_description": "Your general-purpose assistant — helps with any task and remembers what matters in memory.",
"type": "chat",
"inject_memory": ["data/memory/index.md", "data/memory/SOUL.md"],
"inject_memory": ["user-memory/index.md", "shared-memory/index.md"],
"icon": "icon.png",
"strength": "average"
}
agents/project-coordinator/AGENT.md
+1 -1
View File
@@ -29,7 +29,7 @@ Delegate work to these task specialists via `execute_task` / `execute_subtask`:
Your system prompt already contains, without you asking:
- The project's **name**, **description**, and **working directory** (the project root — all relative file paths resolve there). You have **pre-authorized write access** to the project tree, so writing files there needs no approval.
- **`data/memory/index.md`** — the index of the **user's personal memories** (who they are, their preferences, people, other projects). It is injected automatically. Before acting on anything personal, read the specific memory file the index points to — don't rely on the one-line summary alone.
- **`user-memory/index.md`** and **`shared-memory/index.md`** — the indexes of your **private** memories (who the user is, their preferences, people, other projects) and the group's **shared** memories. Both are injected automatically. Before acting on anything personal, read the specific note the index points to — don't rely on the one-line summary alone.
- **`SKALD.md`** at the project root — this project's **living diary** (see below). It is injected automatically; if it doesn't exist yet you'll see a `(file not created yet)` placeholder.
Treat all of this as ground truth. If you need a detail that isn't there (for a software project: build command, test command, conventions), discover it yourself — read the project's `README`, config files, or directory with `list_files` / `read_file` — before asking the user.
agents/project-coordinator/meta.json
+1 -1
View File
@@ -5,6 +5,6 @@
"type": "chat",
"scope": "reasoning",
"strength": "average",
"inject_memory": ["data/memory/index.md", "$WD/SKALD.md"],
"inject_memory": ["user-memory/index.md", "shared-memory/index.md", "$WD/SKALD.md"],
"icon": "icon.png"
}
crates/core-api/src/config_api.rs
+14
View File
@@ -0,0 +1,14 @@
use anyhow::Result;
use async_trait::async_trait;
/// Read/write access to the instance-wide key/value config store
/// (`config` table in `system.db`).
///
/// [`ConfigApi::set`] emits `ConfigKeyUpdated` on the system bus when the
/// value changes, so subscribers (e.g. the Telegram plugin reloading its
/// bindings) are notified without polling.
#[async_trait]
pub trait ConfigApi: Send + Sync {
async fn get(&self, key: &str) -> Result<Option<String>>;
async fn set(&self, key: &str, value: &str) -> Result<()>;
}
crates/core-api/src/lib.rs
+2
View File
@@ -3,6 +3,7 @@ pub const APP_NAME: &str = "Skald";
pub mod approval;
pub mod bus;
pub mod config_api;
pub mod system_bus;
pub mod chatbot;
pub mod chat_hub;
@@ -18,6 +19,7 @@ pub mod plugin;
pub mod provider;
pub mod remote;
pub mod tool;
pub mod user_channel;
pub mod secrets;
pub mod transcribe;
pub mod tts;
crates/core-api/src/plugin.rs
+9 -3
View File
@@ -5,12 +5,10 @@ use async_trait::async_trait;
use serde_json::Value;
use tokio::sync::RwLock;
use crate::approval::ApprovalApi;
use crate::command::CommandApi;
use crate::config_api::ConfigApi;
use crate::system_bus::SystemEventBus;
use crate::chat_hub::ChatHubApi;
use crate::image_generate::ImageGenerateRegistry;
use crate::inbox::InboxApi;
use crate::location::LocationUpdater;
use crate::memory::Memory;
use crate::provider::ApiProviderRegistry;
@@ -18,6 +16,7 @@ use crate::remote::RemoteAccess;
use crate::secrets::SecretsApi;
use crate::transcribe::{TranscribeProvider, TranscribeRegistry};
use crate::tts::{TtsProvider, TtsRegistry};
use crate::user_channel::UserChannelApi;
/// Closure that builds a fresh Axum router (e.g. for the mesh-facing server).
pub type RouterFactory = Arc<dyn Fn() -> axum::Router + Send + Sync>;
@@ -33,6 +32,9 @@ pub struct PluginContext {
/// Custom file-based slash commands (`commands/<name>/`). Read-only from the
/// plugin side — lets the Telegram bot resolve `/command` expansions.
pub command: Arc<dyn CommandApi>,
/// Key/value config store (`config` table in `system.db`). `set` emits
/// `ConfigKeyUpdated` on the system bus.
pub config: Arc<dyn ConfigApi>,
/// Skald's shared SQLite pool — lets plugins create/use their own tables
/// (e.g. `relay_*`) in the main DB. See plugin.md §12.1.
pub db: Arc<sqlx::SqlitePool>,
@@ -45,6 +47,10 @@ pub struct PluginContext {
pub api_provider_registry: Arc<dyn ApiProviderRegistry>,
pub location: Arc<dyn LocationUpdater>,
pub system_bus: Arc<SystemEventBus>,
/// Channel-to-session resolver (blueprint §13). Lets channel plugins
/// (Telegram, mobile, …) look up an unlocked user's chat hub, approval
/// manager and event stream by user id.
pub user_channel: Arc<dyn UserChannelApi>,
pub web_port: u16,
pub remote_slot: Arc<RwLock<Option<Arc<dyn RemoteAccess>>>>,
pub router_factory: RouterFactory,
crates/core-api/src/user_channel.rs
+57
View File
@@ -0,0 +1,57 @@
//! Channel-to-session contract (blueprint §13).
//!
//! In the multi-user architecture, chat hubs, approval managers and event
//! streams are per-user (inside [`UserContext`]). External channels (Telegram,
//! mobile, …) need a way to resolve a user's owner-bound runtime at runtime,
//! without depending on the concrete `Skald` / `UserContext` types.
//!
//! [`UserChannelApi`] is the lookup seam: given a `user_id`, returns a
//! [`UserChannelHandle`] when the user's database is unlocked (§9), or `None`
//! when it is still locked. The handle exposes the per-user [`ChatHubApi`],
//! [`ApprovalApi`] and event stream — everything a channel adapter needs to
//! route a message and receive the response events.
//!
//! This is the "one contract" of §13: each channel (Telegram, mobile, …) is a
//! thin adapter over it, so N channel rewrites become 1 contract + N adapters.
use std::sync::Arc;
use async_trait::async_trait;
use tokio::sync::broadcast;
use crate::approval::ApprovalApi;
use crate::chat_hub::ChatHubApi;
use crate::events::GlobalEvent;
/// Resolves an unlocked user's channel handle.
///
/// Implemented by the application core (`Skald`) and injected into
/// [`crate::plugin::PluginContext`] as `user_channel`.
#[async_trait]
pub trait UserChannelApi: Send + Sync {
/// Returns the user's handle if their database is unlocked in this boot
/// (§9: from first login until restart). `None` = locked — the caller
/// should prompt the user to log in.
async fn resolve_user(&self, user_id: &str) -> Option<Arc<dyn UserChannelHandle>>;
}
/// Handle to one unlocked user's owner-bound runtime.
///
/// Lifetime = the user's pool lifetime (§9). Cloning the returned `Arc`s is
/// cheap (they share the underlying state). The event receiver obtained from
/// [`UserChannelHandle::subscribe`] is independent per call — each subscriber
/// gets every future event.
pub trait UserChannelHandle: Send + Sync {
/// The opaque user id this handle belongs to.
fn user_id(&self) -> &str;
/// The user's chat hub — send messages, manage sessions, query context.
fn chat_hub(&self) -> Arc<dyn ChatHubApi>;
/// The user's approval manager — resolve pending tool-call approvals.
fn approval(&self) -> Arc<dyn ApprovalApi>;
/// Subscribe to the user's server→client event stream.
/// Events are scoped to this user; no cross-user leakage.
fn subscribe(&self) -> broadcast::Receiver<GlobalEvent>;
}
crates/plugin-telegram-bot/src/auth.rs
+87 -95
View File
@@ -1,79 +1,91 @@
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::SystemTime;
use anyhow::Result;
use chrono::{DateTime, Local, Utc};
use rand::RngExt;
use serde::{Deserialize, Serialize};
use teloxide::prelude::*;
use teloxide::types::ParseMode;
use tokio::time::Duration;
use tokio::sync::broadcast;
use tokio_util::sync::CancellationToken;
use tracing::{error, info};
use tracing::{error, info, warn};
use core_api::config_api::ConfigApi;
use core_api::system_bus::SystemEvent;
use super::TgShared;
// ── Whitelist file schema ─────────────────────────────────────────────────────
//
// Written to secrets/telegram_whitelist.json.
// The main agent edits this file directly to authorise users.
/// Config-table key under which all Telegram bindings are stored as JSON.
pub(crate) const CONFIG_KEY: &str = "telegram";
#[derive(Debug, Serialize, Deserialize, Default)]
pub struct WhitelistFile {
// ── Bindings schema (stored as JSON in the `config` table) ────────────────────
#[derive(Debug, Serialize, Deserialize, Default, Clone)]
pub struct TelegramConfig {
#[serde(default)]
pub whitelist: Vec<i64>,
pub bindings: Vec<Binding>,
#[serde(default)]
pub pending_pairings: Vec<PairingEntry>,
}
#[derive(Debug, Serialize, Deserialize)]
pub struct PairingEntry {
pub code: String,
pub chat_id: i64,
pub issued_at: String,
#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct Binding {
pub chat_id: i64,
pub user_id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub display: Option<String>,
}
pub(crate) async fn load_wl(secrets_dir: &Path) -> WhitelistFile {
let path = secrets_dir.join("telegram_whitelist.json");
match tokio::fs::read_to_string(&path).await {
Ok(s) => serde_json::from_str(&s).unwrap_or_default(),
Err(_) => WhitelistFile::default(),
#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct PairingEntry {
pub code: String,
pub chat_id: i64,
pub issued_at: String,
}
// ── Config-table read/write ────────────────────────────────────────────────────
/// Reads the Telegram config from the `config` table. Returns `Default` when
/// the key is absent or unparseable (never fails the caller).
pub(crate) async fn load_config(config: &dyn ConfigApi) -> anyhow::Result<TelegramConfig> {
match config.get(CONFIG_KEY).await? {
Some(json) => Ok(serde_json::from_str(&json).unwrap_or_default()),
None => Ok(TelegramConfig::default()),
}
}
pub(crate) async fn save_wl(secrets_dir: &Path, wl: &WhitelistFile) -> Result<()> {
tokio::fs::create_dir_all(secrets_dir).await?;
let path = secrets_dir.join("telegram_whitelist.json");
tokio::fs::write(&path, serde_json::to_string_pretty(wl)?).await?;
Ok(())
/// Writes the Telegram config to the `config` table. `ConfigApi::set` emits a
/// `ConfigKeyUpdated` event when the value changes, so the in-memory cache and
/// any forwarders are updated automatically.
pub(crate) async fn save_config(
config: &dyn ConfigApi,
cfg: &TelegramConfig,
) -> anyhow::Result<()> {
config.set(CONFIG_KEY, &serde_json::to_string_pretty(cfg)?).await
}
// ── Pairing ───────────────────────────────────────────────────────────────────
/// Pairing codes older than this are considered abandoned and pruned, so the
/// whitelist file does not accumulate stale `pending_pairings` entries.
/// Pairing codes older than this are considered abandoned and pruned.
const PAIRING_TTL_HOURS: i64 = 24;
/// Called when an unbound `chat_id` sends a message. Generates (or reuses) a
/// pairing code, persists it to the config table, and replies with instructions.
pub(crate) async fn handle_pairing(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>) {
let mut wl = load_wl(&shared.secrets_dir).await;
let mut cfg = shared.bindings.read().await.clone();
// Drop pairing codes past their TTL. Entries with an unparseable timestamp
// are kept (don't silently lose data on a format change).
// Prune expired codes.
let cutoff = Utc::now() - chrono::Duration::hours(PAIRING_TTL_HOURS);
let before = wl.pending_pairings.len();
wl.pending_pairings.retain(|e| match DateTime::parse_from_rfc3339(&e.issued_at) {
cfg.pending_pairings.retain(|e| match DateTime::parse_from_rfc3339(&e.issued_at) {
Ok(ts) => ts.with_timezone(&Utc) > cutoff,
Err(_) => true,
});
let pruned = wl.pending_pairings.len() != before;
// Re-use an existing (non-expired) code if one is already pending for this chat.
let (code, added) = if let Some(entry) = wl.pending_pairings.iter().find(|e| e.chat_id == chat_id.0) {
// Reuse an existing code for this chat, or generate a new one.
let (code, added) = if let Some(entry) = cfg.pending_pairings.iter().find(|e| e.chat_id == chat_id.0) {
(entry.code.clone(), false)
} else {
let code = generate_code();
wl.pending_pairings.push(PairingEntry {
cfg.pending_pairings.push(PairingEntry {
code: code.clone(),
chat_id: chat_id.0,
issued_at: Local::now().format("%Y-%m-%dT%H:%M:%S%:z").to_string(),
@@ -81,14 +93,16 @@ pub(crate) async fn handle_pairing(bot: &Bot, chat_id: ChatId, shared: &Arc<TgSh
(code, true)
};
// Persist if we added a new code or pruned expired ones.
if added || pruned {
if let Err(e) = save_wl(&shared.secrets_dir, &wl).await {
error!(error = %e, "telegram: failed to write whitelist file");
}
}
if added {
info!(chat_id = chat_id.0, code = %code, "TELEGRAM PAIRING: code written to telegram_whitelist.json");
if let Err(e) = save_config(&*shared.config, &cfg).await {
error!(error = %e, "telegram: failed to write pairing to config table");
} else {
// Update the in-memory cache immediately (the config_listener will
// also fire, but this avoids a race if the user sends another
// message before the event arrives).
*shared.bindings.write().await = cfg.clone();
}
info!(chat_id = chat_id.0, code = %code, "TELEGRAM PAIRING: code written to config table");
}
bot.send_message(
@@ -96,7 +110,7 @@ pub(crate) async fn handle_pairing(bot: &Bot, chat_id: ChatId, shared: &Arc<TgSh
format!(
"🔐 <b>Pairing required.</b>\n\n\
Code: <code>{code}</code>\n\n\
Provide this code to the web agent to authorize access.",
Ask the admin to authorize this chat using the telegram_pairing tool.",
),
)
.parse_mode(ParseMode::Html)
@@ -110,60 +124,38 @@ pub(crate) fn generate_code() -> String {
(0..6).map(|_| CHARS[rng.random_range(0..CHARS.len())] as char).collect()
}
// ── Whitelist watchdog ────────────────────────────────────────────────────────
//
// Polls telegram_whitelist.json every 10 s for mtime changes.
// When a new chat_id appears in `whitelist` (agent moved it from pending),
// sends a welcome message so the user knows they are authorized.
pub(crate) async fn whitelist_watchdog(bot: Bot, secrets_dir: PathBuf, cancel: CancellationToken) {
let path = secrets_dir.join("telegram_whitelist.json");
let mut last_mtime: Option<SystemTime> = tokio::fs::metadata(&path).await.ok()
.and_then(|m| m.modified().ok());
let mut known_wl = load_wl(&secrets_dir).await.whitelist;
let mut interval = tokio::time::interval(Duration::from_secs(10));
interval.tick().await; // skip the immediate first tick
// ── Config listener ────────────────────────────────────────────────────────────
/// Subscribes to the system bus and reloads the in-memory bindings whenever the
/// `"telegram"` config key changes. Replaces the old file-polling watchdog.
pub(crate) async fn config_listener(
shared: Arc<TgShared>,
mut rx: broadcast::Receiver<SystemEvent>,
cancel: CancellationToken,
) {
info!("telegram: config listener started");
loop {
tokio::select! {
_ = cancel.cancelled() => break,
_ = interval.tick() => {
let new_mtime = tokio::fs::metadata(&path).await.ok()
.and_then(|m| m.modified().ok());
if new_mtime.is_none() || new_mtime == last_mtime {
continue;
}
last_mtime = new_mtime;
let wl = load_wl(&secrets_dir).await;
let newly_authorized: Vec<i64> = wl.whitelist.iter()
.filter(|id| !known_wl.contains(id))
.cloned()
.collect();
if !newly_authorized.is_empty() {
info!(users = ?newly_authorized, "telegram: new users authorized — sending welcome");
for &chat_id in &newly_authorized {
bot.send_message(
ChatId(chat_id),
"✅ <b>Access granted!</b>\n\
You can now talk to your agent.\n\n\
/help for available commands.",
)
.parse_mode(ParseMode::Html)
.await
.ok();
_ = cancel.cancelled() => {
info!("telegram: config listener stopped");
return;
}
result = rx.recv() => match result {
Ok(SystemEvent::ConfigKeyUpdated { key, new_value, .. }) if key == CONFIG_KEY => {
match serde_json::from_str::<TelegramConfig>(&new_value) {
Ok(cfg) => {
let n = cfg.bindings.len();
*shared.bindings.write().await = cfg;
info!(bindings = n, "telegram: bindings reloaded from config event");
}
Err(e) => warn!(error = %e, "telegram: failed to parse config from event"),
}
}
known_wl = wl.whitelist;
info!(
whitelist = known_wl.len(),
pending = wl.pending_pairings.len(),
"telegram: whitelist file reloaded"
);
Ok(_) => {}
Err(broadcast::error::RecvError::Lagged(n)) => {
warn!(skipped = n, "telegram: config listener lagged");
}
Err(broadcast::error::RecvError::Closed) => return,
}
}
}
crates/plugin-telegram-bot/src/events.rs
+129 -106
View File
@@ -1,4 +1,5 @@
use std::sync::Arc;
use teloxide::prelude::*;
use teloxide::types::{InlineKeyboardButton, InlineKeyboardMarkup, ParseMode};
use tokio::sync::broadcast;
@@ -6,17 +7,17 @@ use tokio_util::sync::CancellationToken;
use tracing::{error, info, warn};
use core_api::events::{GlobalEvent, ServerEvent};
use core_api::user_channel::UserChannelHandle;
use super::TgShared;
use super::auth::load_wl;
use super::helpers::{escape_html, label_to_html, send_long};
/// Sends an inline keyboard for an approval request and records the request_id.
/// Sends an inline keyboard for an approval request and records the request.
async fn send_approval_keyboard(
bot: &Bot,
chat_id: ChatId,
text: String,
user_id: String,
request_id: i64,
shared: &Arc<TgShared>,
) {
@@ -37,101 +38,122 @@ async fn send_approval_keyboard(
.reply_markup(keyboard)
.await
{
Ok(m) => { shared.pending_approvals.lock().await.insert(m.id, request_id); }
Ok(m) => {
shared.pending_approvals.lock().await.insert(
m.id,
super::PendingApproval { user_id, request_id },
);
}
Err(e) => error!(error = %e, "telegram: failed to send approval message"),
}
}
// ── Persistent background forwarder ──────────────────────────────────────────
// ── Per-user forwarder ────────────────────────────────────────────────────────
/// Spawned once when the plugin starts.
/// Stays subscribed to the "telegram" broadcast channel forever, forwarding
/// events to the home chat_id. This is the only subscriber — per-message
/// subscriptions are not used — so it also catches background notifications
/// that arrive without a user message triggering them.
///
/// Re-subscribes immediately after each `Done`/`Error` so no events from the
/// next turn are missed. Safe because Tokio's cooperative scheduler guarantees
/// no other task runs between the re-subscription point and the next `await`,
/// and the processing mutex in `ChatSessionHandler` serialises turns.
pub(crate) async fn persistent_forwarder(
bot: Bot,
shared: Arc<TgShared>,
cancel: CancellationToken,
/// Spawns forwarders for all bound users whose contexts are already unlocked.
/// Called at plugin start. Users who log in later get their forwarder spawned
/// lazily on first incoming message.
pub(crate) async fn spawn_forwarders_for_bound_users(
bot: &Bot,
shared: &Arc<TgShared>,
cancel: &CancellationToken,
) {
info!("telegram: persistent forwarder started");
let bindings = shared.bindings.read().await.clone();
for b in &bindings.bindings {
if let Some(handle) = shared.user_channel.resolve_user(&b.user_id).await {
ensure_forwarder(bot.clone(), Arc::clone(shared), &b.user_id, b.chat_id, handle, cancel.clone()).await;
}
}
}
let mut rx = shared.chat_hub.events("telegram");
/// Spawns a per-user forwarder if one is not already running for `user_id`.
/// The forwarder subscribes to the user's event stream and routes `ServerEvent`s
/// to the bound Telegram `chat_id`.
pub(crate) async fn ensure_forwarder(
bot: Bot,
shared: Arc<TgShared>,
user_id: &str,
chat_id: i64,
handle: Arc<dyn UserChannelHandle>,
cancel: CancellationToken,
) {
let mut forwarders = shared.forwarders.lock().await;
if forwarders.contains(user_id) {
return;
}
forwarders.insert(user_id.to_string());
let uid = user_id.to_string();
info!(user_id = %uid, chat_id, "telegram: spawning per-user forwarder");
let shared_c = Arc::clone(&shared);
let cancel_c = cancel.clone();
tokio::spawn(user_forwarder(bot, shared_c, uid, chat_id, handle, cancel_c));
}
/// One forwarder per unlocked user. Subscribes to the user's `global_tx` and
/// routes events to Telegram. Exits when the broadcast channel closes (user
/// context dropped at restart / lock) or the plugin is cancelled.
async fn user_forwarder(
bot: Bot,
shared: Arc<TgShared>,
user_id: String,
chat_id: i64,
handle: Arc<dyn UserChannelHandle>,
cancel: CancellationToken,
) {
let mut rx = handle.subscribe();
let tg_chat = ChatId(chat_id);
// Single loop: rx is updated in-place on Done/Error so we never miss events
// from the next turn (re-subscription happens before the async send).
loop {
let ge: GlobalEvent = tokio::select! {
_ = cancel.cancelled() => {
info!("telegram: persistent forwarder stopped");
return;
info!(user_id = %user_id, "telegram: forwarder cancelled");
break;
}
result = rx.recv() => match result {
Ok(e) => e,
Err(broadcast::error::RecvError::Lagged(n)) => {
warn!(skipped = n, "telegram: persistent forwarder lagged");
warn!(user_id = %user_id, skipped = n, "telegram: forwarder lagged");
continue;
}
Err(broadcast::error::RecvError::Closed) => return,
Err(broadcast::error::RecvError::Closed) => {
info!(user_id = %user_id, "telegram: forwarder — user context closed, exiting");
break;
}
},
};
// ApprovalResolved is handled regardless of source so Telegram removes its
// keyboard even when the approval was resolved via web or REST.
if let ServerEvent::ApprovalResolved { request_id, approved, .. } = ge.event {
let label = if approved { "✅ Approved" } else { "❌ Rejected" };
// ApprovalResolved is handled regardless of source so Telegram removes
// its keyboard even when the approval was resolved via web or REST.
if let ServerEvent::ApprovalResolved { request_id, .. } = ge.event {
let mut pending = shared.pending_approvals.lock().await;
if let Some((&msg_id, _)) = pending.iter().find(|(_, rid)| **rid == request_id) {
if let Some((&msg_id, _)) = pending.iter().find(|(_, pa)| pa.request_id == request_id) {
let msg_id = msg_id;
pending.remove(&msg_id);
drop(pending);
if let Some(cid) = resolve_chat_id(&shared).await {
bot.delete_message(cid, msg_id).await.ok();
}
bot.delete_message(tg_chat, msg_id).await.ok();
}
continue;
}
// All other events: only process if they belong to the "telegram" source.
// Only process events from the "telegram" source.
if ge.source.as_deref() != Some("telegram") {
tracing::debug!(event_type = ge.event.type_name(), source = ?ge.source, "persistent_forwarder: skipping non-telegram event");
continue;
}
let event = ge.event;
tracing::debug!(event_type = event.type_name(), "persistent_forwarder: processing telegram event");
// Resolve the destination chat_id (last known user, or first in whitelist).
// For terminal events (Done/Error) with no known chat, still re-subscribe.
let chat_id = match resolve_chat_id(&shared).await {
Some(id) => id,
None => {
warn!(event_type = %event.type_name(), "telegram: persistent_forwarder — no chat_id resolved, dropping event");
if matches!(event, ServerEvent::Done { .. } | ServerEvent::Error { .. }) {
rx = shared.chat_hub.events("telegram");
}
continue;
}
};
match event {
ServerEvent::Done { content, .. } => {
// Re-subscribe BEFORE any await so we don't miss the next turn.
rx = shared.chat_hub.events("telegram");
if !content.trim().is_empty() {
send_long(&bot, chat_id, &content, Some(ParseMode::Html)).await;
send_long(&bot, tg_chat, &content, Some(ParseMode::Html)).await;
}
}
ServerEvent::Error { message } => {
rx = shared.chat_hub.events("telegram");
bot.send_message(
chat_id,
tg_chat,
format!("⚠️ <b>Error:</b> {}", escape_html(&message)),
)
.parse_mode(ParseMode::Html)
@@ -140,7 +162,7 @@ pub(crate) async fn persistent_forwarder(
}
ServerEvent::ToolStart { label_short, .. } => {
bot.send_message(chat_id, format!("🔧 <i>{}</i>…", label_to_html(&label_short)))
bot.send_message(tg_chat, format!("🔧 <i>{}</i>…", label_to_html(&label_short)))
.parse_mode(ParseMode::Html)
.await
.ok();
@@ -148,7 +170,7 @@ pub(crate) async fn persistent_forwarder(
ServerEvent::Thinking { content, .. } => {
if !content.trim().is_empty() {
send_long(&bot, chat_id, &content, Some(ParseMode::Html)).await;
send_long(&bot, tg_chat, &content, Some(ParseMode::Html)).await;
}
}
@@ -156,7 +178,7 @@ pub(crate) async fn persistent_forwarder(
let preview = prompt_preview.chars().take(300).collect::<String>();
let ellipsis = if prompt_preview.len() > 300 { "…" } else { "" };
bot.send_message(
chat_id,
tg_chat,
format!(
"🤖 <b>{}</b> → <b>{}</b>\n<blockquote>{}{ellipsis}</blockquote>",
escape_html(&parent_agent_id),
@@ -173,7 +195,7 @@ pub(crate) async fn persistent_forwarder(
let preview = result_preview.chars().take(300).collect::<String>();
let ellipsis = if result_preview.len() > 300 { "…" } else { "" };
bot.send_message(
chat_id,
tg_chat,
format!(
"✅ <b>{}</b> finished → <b>{}</b>\n<blockquote>{}{ellipsis}</blockquote>",
escape_html(&agent_id),
@@ -195,7 +217,7 @@ pub(crate) async fn persistent_forwarder(
escape_html(&path),
escape_html(&preview),
);
send_approval_keyboard(&bot, chat_id, text, request_id, &shared).await;
send_approval_keyboard(&bot, tg_chat, text, user_id.clone(), request_id, &shared).await;
}
ServerEvent::ApprovalRequired { request_id, tool_name, arguments, .. } => {
@@ -209,16 +231,15 @@ pub(crate) async fn persistent_forwarder(
escape_html(&tool_name),
escape_html(&args_preview),
);
send_approval_keyboard(&bot, chat_id, text, request_id, &shared).await;
send_approval_keyboard(&bot, tg_chat, text, user_id.clone(), request_id, &shared).await;
}
ServerEvent::AgentQuestion { request_id, tool_call_id, title, question, suggested_answers, .. } => {
info!(request_id, tool_call_id, %question, "telegram: persistent_forwarder received AgentQuestion");
ServerEvent::AgentQuestion { request_id, title, question, suggested_answers, .. } => {
info!(request_id, %question, "telegram: forwarder received AgentQuestion");
// If a previous question is still pending, disable its (now-dead)
// buttons so tapping them doesn't silently no-op.
if let Some(prev) = shared.pending_question.lock().await.take() {
bot.edit_message_reply_markup(chat_id, prev.message_id)
// Disable any previously-pending question for this chat.
if let Some(prev) = shared.pending_questions.lock().await.remove(&chat_id) {
bot.edit_message_reply_markup(tg_chat, prev.message_id)
.reply_markup(InlineKeyboardMarkup::new(vec![vec![
InlineKeyboardButton::callback("⏭ Superseded by a newer question", "noop"),
]]))
@@ -240,27 +261,27 @@ pub(crate) async fn persistent_forwarder(
.collect();
Some(InlineKeyboardMarkup::new(buttons))
};
let mut req = bot.send_message(chat_id, header).parse_mode(ParseMode::Html);
let mut req = bot.send_message(tg_chat, header).parse_mode(ParseMode::Html);
if let Some(kb) = keyboard {
req = req.reply_markup(kb);
}
match req.await {
Ok(m) => {
info!(request_id, msg_id = m.id.0, "telegram: AgentQuestion sent to user, pending_question set");
*shared.pending_question.lock().await = Some(super::PendingQuestion {
shared.pending_questions.lock().await.insert(chat_id, super::PendingQuestion {
user_id: user_id.clone(),
request_id,
message_id: m.id,
suggested_answers,
});
}
Err(e) => error!(error = %e, request_id, "telegram: failed to send AgentQuestion to user"),
Err(e) => error!(error = %e, request_id, "telegram: failed to send AgentQuestion"),
}
}
ServerEvent::LlmFailed { tried, last_error } => {
let models = tried.join(", ");
bot.send_message(
chat_id,
tg_chat,
format!(
"⚠️ <b>LLM unavailable</b>\nTried: <code>{}</code>\n{}",
escape_html(&models),
@@ -277,23 +298,14 @@ pub(crate) async fn persistent_forwarder(
_ => {}
}
}
}
/// Resolves the Telegram chat_id to use for outbound messages.
/// Prefers the last chat_id that sent a message; falls back to the first
/// whitelisted user.
async fn resolve_chat_id(shared: &TgShared) -> Option<ChatId> {
if let Some(id) = *shared.home_chat_id.lock().await {
return Some(id);
}
let wl = load_wl(&shared.secrets_dir).await;
wl.whitelist.first().map(|&id| ChatId(id))
// Clean up: remove this user from the active forwarders set.
shared.forwarders.lock().await.remove(&user_id);
info!(user_id = %user_id, "telegram: forwarder exited");
}
// ── Helpers ───────────────────────────────────────────────────────────────────
/// Truncates `s` to at most `max_chars` Unicode scalar values.
/// Appends `…` if truncated. Never panics on multibyte UTF-8 content.
fn truncate_chars(s: &str, max_chars: usize) -> String {
let mut chars = s.chars();
let truncated: String = chars.by_ref().take(max_chars).collect();
@@ -333,14 +345,20 @@ pub(crate) async fn callback_handler(
let req_id = parts.next().and_then(|s| s.parse::<i64>().ok());
let idx_str = parts.next().and_then(|s| s.parse::<usize>().ok());
if let (Some(request_id), Some(idx)) = (req_id, idx_str) {
let mut pq = shared.pending_question.lock().await;
if let Some(pq_inner) = pq.as_ref() {
if pq_inner.request_id == request_id {
let answer = pq_inner.suggested_answers.get(idx).cloned().unwrap_or_default();
drop(pq);
*shared.pending_question.lock().await = None;
shared.chat_hub.resolve_question("telegram", request_id, answer.clone()).await;
info!(request_id, %answer, "telegram: clarification answered via button");
let pq_map = shared.pending_questions.lock().await;
if let Some(pq) = pq_map.get(&msg_chat_id.0) {
if pq.request_id == request_id {
let user_id = pq.user_id.clone();
let answer = pq.suggested_answers.get(idx).cloned().unwrap_or_default();
drop(pq_map);
shared.pending_questions.lock().await.remove(&msg_chat_id.0);
if let Some(handle) = shared.user_channel.resolve_user(&user_id).await {
handle.chat_hub().resolve_question("telegram", request_id, answer.clone()).await;
info!(request_id, %answer, "telegram: clarification answered via button");
} else {
warn!(user_id = %user_id, "telegram: user locked, cannot resolve clarification");
}
bot.edit_message_reply_markup(msg_chat_id, msg_id)
.reply_markup(InlineKeyboardMarkup::new(vec![vec![
InlineKeyboardButton::callback(format!("✅ {answer}"), "noop"),
@@ -380,20 +398,25 @@ pub(crate) async fn callback_handler(
if let Some((request_id, action, label)) = parsed {
let stored = shared.pending_approvals.lock().await.remove(&msg_id);
if let Some(stored_id) = stored {
if stored_id == request_id {
match action {
ApprovalAction::Approve =>
shared.approval.approve(request_id).await,
ApprovalAction::Reject =>
shared.approval.reject(request_id, String::new()).await,
ApprovalAction::BypassTime(secs) =>
shared.approval.approve_with_bypass(request_id, Some(secs)).await,
ApprovalAction::BypassSession =>
shared.approval.approve_with_bypass(request_id, None).await,
if let Some(pa) = stored {
if pa.request_id == request_id {
if let Some(handle) = shared.user_channel.resolve_user(&pa.user_id).await {
let approval = handle.approval();
match action {
ApprovalAction::Approve =>
approval.approve(request_id).await,
ApprovalAction::Reject =>
approval.reject(request_id, String::new()).await,
ApprovalAction::BypassTime(secs) =>
approval.approve_with_bypass(request_id, Some(secs)).await,
ApprovalAction::BypassSession =>
approval.approve_with_bypass(request_id, None).await,
}
info!(request_id, label, "telegram: approval resolved");
bot.delete_message(msg_chat_id, msg_id).await.ok();
} else {
warn!(user_id = %pa.user_id, "telegram: user locked, cannot resolve approval");
}
info!(request_id, label, "telegram: approval resolved");
bot.delete_message(msg_chat_id, msg_id).await.ok();
}
} else {
warn!(request_id, "telegram: approval not found (already resolved?)");
crates/plugin-telegram-bot/src/handlers.rs
+89 -103
View File
@@ -8,11 +8,13 @@ use core_api::chat_hub::{ModelCommandOutcome, SendMessageOptions};
use core_api::command::expand_template;
use core_api::location::GpsCoord;
use core_api::message_meta::{CommandRef, MessageMetadata};
use core_api::user_channel::UserChannelHandle;
use super::TELEGRAM_FORMAT_CONTEXT;
use super::TgShared;
use super::attachments::TelegramAttachment;
use super::auth::{handle_pairing, load_wl};
use super::auth::handle_pairing;
use super::events::ensure_forwarder;
// ── Available commands help text (shared by /help and unknown-command replies) ──
const HELP_TEXT: &str = "<b>Available commands</b>\n\n\
@@ -28,9 +30,6 @@ const HELP_TEXT: &str = "<b>Available commands</b>\n\n\
/sethome — receive agent notifications here\n\
/help — this message";
/// Builds the `/help` text: the static system-command list plus a dynamically
/// discovered "Custom commands" section (`commands/<name>/`). Descriptions are
/// HTML-escaped since the message is sent with `ParseMode::Html`.
fn help_text(command: &dyn core_api::command::CommandApi) -> String {
let mut out = String::from(HELP_TEXT);
let cmds = command.list_enabled();
@@ -48,9 +47,6 @@ fn help_text(command: &dyn core_api::command::CommandApi) -> String {
}
// ── Incoming message classification ───────────────────────────────────────────
//
// To add a new media type: add a variant to IncomingEvent, handle it in
// classify_message, then dispatch it in message_handler.
pub(crate) enum IncomingEvent {
Text(String),
@@ -93,13 +89,6 @@ pub(crate) fn classify_message(msg: &Message) -> Option<IncomingEvent> {
let text = msg.text()?;
// A command is any message that *starts* with '/'. We deliberately do NOT
// rely on teloxide's BotCommand entities: those are emitted for every
// "/token" anywhere in the text, so a normal sentence containing a "/path"
// (e.g. "stop /usr/bin/foo") would be misclassified as a command. A leading
// slash is the only signal. Arguments are parsed from the message `text`
// (not `entity.text()`, which spans only "/model" and would drop the arg).
// An unknown command is handled by the dispatcher, which replies with help.
if text.starts_with('/') {
let full = text.trim_start_matches('/');
let mut parts = full.splitn(2, ' ');
@@ -126,31 +115,58 @@ pub(crate) async fn message_handler(
) -> ResponseResult<()> {
let chat_id = msg.chat.id;
// Whitelist check — re-read the file on every message so agent edits are
// picked up without a plugin restart.
let wl = load_wl(&shared.secrets_dir).await;
if !wl.whitelist.contains(&chat_id.0) {
handle_pairing(&bot, chat_id, &shared).await;
return Ok(());
}
// Resolve chat_id → user_id from bindings.
let user_id = match shared.user_for_chat(chat_id.0).await {
Some(uid) => uid,
None => {
handle_pairing(&bot, chat_id, &shared).await;
return Ok(());
}
};
// Track the last active chat_id so the persistent forwarder knows
// where to send background notifications.
*shared.home_chat_id.lock().await = Some(chat_id);
// Resolve the user's per-user context (must be unlocked, §9).
let handle = match shared.user_channel.resolve_user(&user_id).await {
Some(h) => h,
None => {
bot.send_message(
chat_id,
"🔒 Your account is locked. Please log in via the web app first, \
then send another message.",
)
.await
.ok();
return Ok(());
}
};
// Ensure a per-user forwarder is running so the response events reach
// this Telegram chat. The forwarder will exit on broadcast-close (user
// locked) or plugin stop; a fresh CancellationToken is fine here since
// the global plugin cancel drops the dispatcher (and thus the shared Arc).
ensure_forwarder(
bot.clone(),
Arc::clone(&shared),
&user_id,
chat_id.0,
Arc::clone(&handle),
tokio_util::sync::CancellationToken::new(),
).await;
let Some(incoming) = classify_message(&msg) else {
bot.send_message(chat_id, "Unsupported message format.").await.ok();
return Ok(());
};
let hub = handle.chat_hub();
match incoming {
IncomingEvent::Command { ref name, .. } if name == "clear" || name == "new" => {
handle_clear(&bot, chat_id, &shared).await;
handle_clear(&bot, chat_id, &hub).await;
}
IncomingEvent::Command { ref name, .. } if name == "sethome" => {
match shared.chat_hub.set_home("telegram").await {
match hub.set_home("telegram").await {
Ok(_) => {
info!("telegram: set as home source");
info!("telegram: set as home source for user {}", user_id);
bot.send_message(chat_id, "🏠 Telegram set as <b>home</b>. Agent notifications will be delivered here.")
.parse_mode(ParseMode::Html)
.await
@@ -168,30 +184,28 @@ pub(crate) async fn message_handler(
.ok();
}
IncomingEvent::Command { ref name, .. } if name == "stop" => {
handle_stop(&bot, chat_id, &shared).await;
hub.cancel("telegram").await;
info!("telegram: agent cancelled via /stop");
bot.send_message(chat_id, "⏹ Agent stopped.").await.ok();
}
IncomingEvent::Command { ref name, .. } if name == "context" => {
handle_context(&bot, chat_id, &shared).await;
handle_context(&bot, chat_id, &hub).await;
}
IncomingEvent::Command { ref name, .. } if name == "cost" => {
handle_cost(&bot, chat_id, &shared).await;
handle_cost(&bot, chat_id, &hub).await;
}
IncomingEvent::Command { ref name, .. } if name == "compact" => {
handle_compact(&bot, chat_id, &shared).await;
handle_compact(&bot, chat_id, &hub).await;
}
IncomingEvent::Command { ref name, .. } if name == "resettools" => {
handle_reset_mcp(&bot, chat_id, &shared).await;
handle_reset_mcp(&bot, chat_id, &hub).await;
}
IncomingEvent::Command { ref name, .. } if name == "models" => {
handle_list_models(&bot, chat_id, &shared).await;
handle_list_models(&bot, chat_id, &hub).await;
}
IncomingEvent::Command { ref name, ref args, .. } if name == "model" => {
handle_set_model(&bot, chat_id, args, &shared).await;
handle_set_model(&bot, chat_id, args, &hub).await;
}
// A recognised custom `/command` expands its `COMMAND.md` template into a
// normal user message (fully interactive: the model can then ask questions,
// iterate, dispatch sub-agents). Any other `/...` is an unknown command and
// is never forwarded to the LLM — reply with a not-found notice + help.
IncomingEvent::Command { ref name, ref args, .. } => {
if let Some(resolved) = shared.command.resolve(name) {
let args_str = args.join(" ");
@@ -208,7 +222,7 @@ pub(crate) async fn message_handler(
}),
..Default::default()
};
handle_llm_message(bot, chat_id, content, Some(metadata), shared).await;
handle_llm_message(bot, chat_id, content, Some(metadata), shared, &handle).await;
} else {
bot.send_message(
chat_id,
@@ -220,10 +234,10 @@ pub(crate) async fn message_handler(
}
}
IncomingEvent::Voice { file_id } => {
handle_voice(&bot, chat_id, file_id, &shared).await;
handle_voice(&bot, chat_id, file_id, &shared, &handle).await;
}
IncomingEvent::Attachment(attachment) => {
handle_attachment(bot, chat_id, attachment, shared).await;
handle_attachment(bot, chat_id, attachment, shared, &handle).await;
}
_ => {
let text = match &incoming {
@@ -233,14 +247,15 @@ pub(crate) async fn message_handler(
| IncomingEvent::Attachment(_) => unreachable!(),
};
// If a clarification question is pending, treat any text as the answer.
// If a clarification question is pending for this chat, treat any
// text as the answer.
{
let mut pq = shared.pending_question.lock().await;
if let Some(pq_inner) = pq.take() {
let request_id = pq_inner.request_id;
let question_msg_id = pq_inner.message_id;
drop(pq);
shared.chat_hub.resolve_question("telegram", request_id, text.clone()).await;
let mut pq_map = shared.pending_questions.lock().await;
if let Some(pq) = pq_map.remove(&chat_id.0) {
let request_id = pq.request_id;
let question_msg_id = pq.message_id;
drop(pq_map);
hub.resolve_question("telegram", request_id, text.clone()).await;
tracing::info!(request_id, %text, "telegram: clarification answered via text");
bot.edit_message_reply_markup(chat_id, question_msg_id)
.reply_markup(teloxide::types::InlineKeyboardMarkup::new(vec![vec![
@@ -255,17 +270,17 @@ pub(crate) async fn message_handler(
}
}
handle_llm_message(bot, chat_id, text, None, shared).await;
handle_llm_message(bot, chat_id, text, None, shared, &handle).await;
}
}
Ok(())
}
// ── /clear command ────────────────────────────────────────────────────────────
// ── Command handlers ──────────────────────────────────────────────────────────
async fn handle_clear(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>) {
match shared.chat_hub.clear("telegram").await {
async fn handle_clear(bot: &Bot, chat_id: ChatId, hub: &Arc<dyn core_api::chat_hub::ChatHubApi>) {
match hub.clear("telegram").await {
Ok(_) => {
info!("telegram: session cleared via /clear");
bot.send_message(chat_id, "🆕 New conversation started.").await.ok();
@@ -277,10 +292,8 @@ async fn handle_clear(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>) {
}
}
// ── /context command ──────────────────────────────────────────────────────────
async fn handle_context(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>) {
match shared.chat_hub.context_info("telegram").await {
async fn handle_context(bot: &Bot, chat_id: ChatId, hub: &Arc<dyn core_api::chat_hub::ChatHubApi>) {
match hub.context_info("telegram").await {
Ok((input, output)) => {
let input_str = input.map_or("?".to_string(), |t| t.to_string());
let output_str = output.map_or("?".to_string(), |t| t.to_string());
@@ -298,10 +311,8 @@ async fn handle_context(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>) {
}
}
// ── /cost command ─────────────────────────────────────────────────────────────
async fn handle_cost(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>) {
match shared.chat_hub.cost_info("telegram").await {
async fn handle_cost(bot: &Bot, chat_id: ChatId, hub: &Arc<dyn core_api::chat_hub::ChatHubApi>) {
match hub.cost_info("telegram").await {
Ok(Some(c)) => {
bot.send_message(chat_id, format!("💰 Session cost: ${c:.4}")).await.ok();
}
@@ -314,10 +325,8 @@ async fn handle_cost(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>) {
}
}
// ── /compact command ──────────────────────────────────────────────────────────
async fn handle_compact(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>) {
match shared.chat_hub.force_compact("telegram").await {
async fn handle_compact(bot: &Bot, chat_id: ChatId, hub: &Arc<dyn core_api::chat_hub::ChatHubApi>) {
match hub.force_compact("telegram").await {
Ok(true) => {
info!("telegram: manual compaction succeeded");
bot.send_message(chat_id, "✅ Context compacted.").await.ok();
@@ -332,10 +341,8 @@ async fn handle_compact(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>) {
}
}
// ── /resettools command ───────────────────────────────────────────────────────
async fn handle_reset_mcp(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>) {
match shared.chat_hub.reset_mcp("telegram").await {
async fn handle_reset_mcp(bot: &Bot, chat_id: ChatId, hub: &Arc<dyn core_api::chat_hub::ChatHubApi>) {
match hub.reset_mcp("telegram").await {
Ok(()) => {
info!("telegram: tool-group grants reset via /resettools");
bot.send_message(chat_id, "✅ Activated tool groups removed from the session.").await.ok();
@@ -347,23 +354,8 @@ async fn handle_reset_mcp(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>) {
}
}
// ── /stop command ────────────────────────────────────────────────────────────
async fn handle_stop(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>) {
shared.chat_hub.cancel("telegram").await;
info!("telegram: agent cancelled via /stop");
bot.send_message(chat_id, "⏹ Agent stopped.").await.ok();
}
// ── /models and /model commands ──────────────────────────────────────────────
//
// Business logic (resolve arg, mutate pin, broadcast) lives in
// `ChatHub::apply_model_command` / `ChatHub::list_clients_marked`. Here we only
// format for Telegram (HTML) and send via the bot — same pattern the web WS
// handler uses with Markdown.
async fn handle_list_models(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>) {
let items = shared.chat_hub.list_clients_marked("telegram").await;
async fn handle_list_models(bot: &Bot, chat_id: ChatId, hub: &Arc<dyn core_api::chat_hub::ChatHubApi>) {
let items = hub.list_clients_marked("telegram").await;
let mut text = String::from("<b>Available models</b>\n\n");
for (i, name, is_current) in &items {
let marker = if *is_current { "●" } else { "○" };
@@ -381,9 +373,9 @@ async fn handle_list_models(bot: &Bot, chat_id: ChatId, shared: &Arc<TgShared>)
.ok();
}
async fn handle_set_model(bot: &Bot, chat_id: ChatId, args: &[String], shared: &Arc<TgShared>) {
async fn handle_set_model(bot: &Bot, chat_id: ChatId, args: &[String], hub: &Arc<dyn core_api::chat_hub::ChatHubApi>) {
let arg = args.first().cloned().unwrap_or_default();
let outcome = shared.chat_hub.apply_model_command("telegram", &arg).await;
let outcome = hub.apply_model_command("telegram", &arg).await;
let text = match outcome {
ModelCommandOutcome::Set(name) => format!("✅ Model set: <b>{}</b>", super::helpers::escape_html(&name)),
ModelCommandOutcome::Cleared => "✅ Model reset to <b>auto</b>.".to_string(),
@@ -403,25 +395,22 @@ async fn handle_llm_message(
text: String,
metadata: Option<MessageMetadata>,
shared: Arc<TgShared>,
handle: &Arc<dyn UserChannelHandle>,
) {
bot.send_chat_action(chat_id, ChatAction::Typing).await.ok();
// The persistent_forwarder (spawned once in start()) is always subscribed
// to the "telegram" broadcast channel and will pick up all events for this
// turn — including Done → send to Telegram. No per-message subscription needed.
let client_name = shared.chat_hub.get_selected_client("telegram").await;
let hub = handle.chat_hub();
let client_name = hub.get_selected_client("telegram").await;
let opts = SendMessageOptions {
client_name,
extra_system_context: Some(TELEGRAM_FORMAT_CONTEXT.to_string()),
tail_reminder: Some(super::TELEGRAM_FORMAT_REMINDER.to_string()),
interface_tools: super::tools::interface_tools(bot, chat_id, &*shared.tts).await,
interface_tools: super::tools::interface_tools(bot.clone(), chat_id, &*shared.tts).await,
metadata,
..Default::default()
};
// send_message only enqueues — the turn runs on ChatHub's per-source consumer —
// so awaiting inline keeps this message handler responsive.
if let Err(e) = shared.chat_hub.send_message("telegram", &text, opts).await {
if let Err(e) = hub.send_message("telegram", &text, opts).await {
error!(error = %e, "telegram: enqueue error");
}
}
@@ -433,6 +422,7 @@ async fn handle_voice(
chat_id: ChatId,
file_id: String,
shared: &Arc<TgShared>,
handle: &Arc<dyn UserChannelHandle>,
) {
use teloxide::net::Download;
@@ -477,7 +467,7 @@ async fn handle_voice(
The user sent a voice message. The following is the audio transcript:\n\n\
{text}"
);
handle_llm_message(bot.clone(), chat_id, message, None, Arc::clone(shared)).await;
handle_llm_message(bot.clone(), chat_id, message, None, Arc::clone(shared), handle).await;
}
// ── Edited message (live location updates) ────────────────────────────────────
@@ -500,8 +490,8 @@ async fn handle_attachment(
chat_id: ChatId,
attachment: TelegramAttachment,
shared: Arc<TgShared>,
handle: &Arc<dyn UserChannelHandle>,
) {
// Update LocationManager immediately, before any LLM dispatch.
if let TelegramAttachment::Location { latitude, longitude, accuracy, is_live } = &attachment {
let coord = GpsCoord { latitude: *latitude, longitude: *longitude };
shared.location.update("telegram", coord, *accuracy, *is_live);
@@ -519,9 +509,6 @@ async fn handle_attachment(
};
match saved {
// Document / Photo: carry the file as structured metadata (rendered as a
// chip in the copilot UI; the LLM gets the shared [SYSTEM INFO] block).
// The caption, if any, becomes the user's text for this turn.
Some(att) => {
info!(chat_id = chat_id.0, path = %att.path, "telegram: attachment saved, forwarding to LLM");
let caption = match &attachment {
@@ -530,12 +517,11 @@ async fn handle_attachment(
TelegramAttachment::Location { .. } => None,
}.unwrap_or_default();
let metadata = MessageMetadata { attachments: vec![att], ..Default::default() };
handle_llm_message(bot, chat_id, caption, Some(metadata), shared).await;
handle_llm_message(bot, chat_id, caption, Some(metadata), shared, handle).await;
}
// Location (no file): keep the textual system-info block.
None => {
let message = attachment.system_info_message(None);
handle_llm_message(bot, chat_id, message, None, shared).await;
handle_llm_message(bot, chat_id, message, None, shared, handle).await;
}
}
}
crates/plugin-telegram-bot/src/lib.rs
+123 -79
View File
@@ -1,22 +1,29 @@
/// Telegram plugin — connects the Skald LLM to a private Telegram bot.
///
/// # Multi-user architecture (blueprint §13)
///
/// One bot serves many Telegram chats, each bound to a Skald user via the
/// `chat_id ↔ user_id` pairing stored in the config table (key `"telegram"`).
/// Incoming messages resolve the user's per-user context via
/// [`UserChannelApi`], then dispatch through that user's `ChatHub`. A per-user
/// forwarder subscribes to the user's event stream and routes `ServerEvent`s
/// back to the bound Telegram chat.
///
/// # Pairing
/// Unknown users receive a pairing code in chat. The code is also written to
/// `secrets/telegram_whitelist.json` under `pending_pairings`. The main agent
/// (via `read_file` / `write_file`) can inspect that file and move the
/// `chat_id` into the `whitelist` array to complete the authorisation — no
/// code changes required, just a file edit.
///
/// Unknown chats receive a pairing code. The admin's agent calls the
/// `telegram_pairing` tool (category `Config`) to bind the `chat_id` to a
/// `user_id`. The binding is written to the config table; the resulting
/// `ConfigKeyUpdated` event reloads the in-memory cache instantly.
///
/// # Human-in-the-loop approvals
/// Tool calls requiring approval emit a `PendingWrite` event; the plugin
/// forwards it to Telegram as an inline-keyboard message with
/// [✅ Approve] [❌ Reject] / [⏱ 15 min] [🔄 Session] buttons.
///
/// # Adding new message types
/// 1. Add a variant to `IncomingEvent` in `handlers.rs`.
/// 2. Handle it in `classify_message` (same file).
/// 3. Dispatch it in `message_handler` (same file).
use std::collections::HashMap;
/// Tool calls requiring approval emit a `PendingWrite` / `ApprovalRequired`
/// event; the per-user forwarder sends it to Telegram as an inline-keyboard
/// message. Button presses resolve the approval through that user's
/// `ApprovalApi`.
use std::collections::{HashMap, HashSet};
use std::path::PathBuf;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
@@ -26,18 +33,18 @@ use async_trait::async_trait;
use serde_json::{Value, json};
use teloxide::prelude::*;
use teloxide::types::MessageId;
use tokio::sync::Mutex;
use tokio::sync::{Mutex, RwLock};
use tokio::task::JoinHandle;
use tokio_util::sync::CancellationToken;
use tracing::{info, warn};
use core_api::approval::ApprovalApi;
use core_api::chat_hub::ChatHubApi;
use core_api::command::CommandApi;
use core_api::config_api::ConfigApi;
use core_api::location::LocationUpdater;
use core_api::plugin::{Plugin, PluginContext};
use core_api::transcribe::{Transcribe, TranscribeProvider};
use core_api::transcribe::TranscribeProvider;
use core_api::tts::TtsProvider;
use core_api::user_channel::UserChannelApi;
mod attachments;
mod auth;
@@ -56,7 +63,6 @@ FORBIDDEN (will appear as raw symbols): ** * _ ` # | and Markdown tables.\n\
• Headers → <b>text</b>\n\
• Structured data → bullet lists with •, never | tables\n\
• Escape & < > as &amp; &lt; &gt;";
/// Short reminder injected near the end of the message list to counter
/// instruction drift in long conversations.
pub(crate) const TELEGRAM_FORMAT_REMINDER: &str = "\
@@ -67,61 +73,90 @@ No Markdown: no ** * _ ` # |. No tables — use bullet lists.";
/// A pending `ask_user_clarification` question waiting for the user's reply.
pub(crate) struct PendingQuestion {
pub(crate) request_id: i64,
pub(crate) message_id: MessageId,
pub(crate) user_id: String,
pub(crate) request_id: i64,
pub(crate) message_id: MessageId,
/// Suggested answers (used to resolve the selection when the user taps a button).
pub(crate) suggested_answers: Vec<String>,
}
/// A pending tool-call approval shown as an inline keyboard.
pub(crate) struct PendingApproval {
pub(crate) user_id: String,
pub(crate) request_id: i64,
}
/// Global state shared across all Telegram handlers and the per-user forwarders.
///
/// Per-user state (ChatHub, ApprovalApi, event stream) is resolved at runtime
/// via [`UserChannelApi`] — it is NOT held here. Only global capabilities and
/// pairing/multiplexing state live in `TgShared`.
pub(crate) struct TgShared {
pub(crate) chat_hub: Arc<dyn ChatHubApi>,
/// Custom slash-command resolver (`commands/<name>/`). Read-only: lets the
/// bot expand a recognised `/command` into a template before forwarding it to
/// the LLM, mirroring the WS handler.
pub(crate) command: Arc<dyn CommandApi>,
pub(crate) approval: Arc<dyn ApprovalApi>,
pub(crate) transcribe: Arc<dyn TranscribeProvider>,
pub(crate) tts: Arc<dyn TtsProvider>,
pub(crate) location: Arc<dyn LocationUpdater>,
/// MessageId of the approval message → request_id.
pub(crate) pending_approvals: Mutex<HashMap<MessageId, i64>>,
/// Currently active clarification question (at most one at a time per session).
pub(crate) pending_question: Mutex<Option<PendingQuestion>>,
pub(crate) secrets_dir: PathBuf,
/// Base directory for file attachments: `<data_root>/uploads/telegram/`.
pub(crate) uploads_dir: PathBuf,
/// Last chat_id that sent a message — used as the target for background notifications.
/// Set on every incoming message; read by the persistent event forwarder.
pub(crate) home_chat_id: Mutex<Option<ChatId>>,
// ── Global capabilities ──
pub(crate) user_channel: Arc<dyn UserChannelApi>,
pub(crate) command: Arc<dyn CommandApi>,
pub(crate) config: Arc<dyn ConfigApi>,
pub(crate) transcribe: Arc<dyn TranscribeProvider>,
pub(crate) tts: Arc<dyn TtsProvider>,
pub(crate) location: Arc<dyn LocationUpdater>,
pub(crate) uploads_dir: PathBuf,
// ── Pairing / bindings (config-table-backed, cached in memory) ──
pub(crate) bindings: RwLock<auth::TelegramConfig>,
// ── Per-chat pending state ──
/// Approval message_id → pending approval (carries user_id for routing).
pub(crate) pending_approvals: Mutex<HashMap<MessageId, PendingApproval>>,
/// chat_id → pending clarification question (at most one per chat).
pub(crate) pending_questions: Mutex<HashMap<i64, PendingQuestion>>,
// ── Forwarder tracking ──
/// user_ids with an active per-user forwarder task.
pub(crate) forwarders: Mutex<HashSet<String>>,
}
impl TgShared {
pub(crate) async fn transcriber(&self) -> Option<Arc<dyn Transcribe>> {
pub(crate) async fn transcriber(&self) -> Option<Arc<dyn core_api::transcribe::Transcribe>> {
self.transcribe.get().await
}
/// Looks up the `user_id` bound to a Telegram `chat_id`, if any.
pub(crate) async fn user_for_chat(&self, chat_id: i64) -> Option<String> {
self.bindings.read().await
.bindings.iter()
.find(|b| b.chat_id == chat_id)
.map(|b| b.user_id.clone())
}
}
// ── Plugin struct ─────────────────────────────────────────────────────────────
pub struct TelegramPlugin {
secrets_dir: PathBuf,
/// Bot token — set by reload() before start() is called.
token: Mutex<String>,
running: Arc<AtomicBool>,
cancel: Mutex<Option<CancellationToken>>,
handle: Mutex<Option<JoinHandle<()>>>,
/// Runtime shared state, populated by `start()`. Accessible to the pairing
/// tool so it can write bindings before/after the dispatcher is running.
shared: std::sync::OnceLock<Arc<TgShared>>,
}
impl TelegramPlugin {
pub fn new(secrets_dir: impl Into<PathBuf>) -> Self {
pub fn new() -> Self {
Self {
secrets_dir: secrets_dir.into(),
token: Mutex::new(String::new()),
running: Arc::new(AtomicBool::new(false)),
cancel: Mutex::new(None),
handle: Mutex::new(None),
token: Mutex::new(String::new()),
running: Arc::new(AtomicBool::new(false)),
cancel: Mutex::new(None),
handle: Mutex::new(None),
shared: std::sync::OnceLock::new(),
}
}
/// Returns the shared runtime state if the plugin is running.
pub(crate) fn shared(&self) -> Option<&Arc<TgShared>> {
self.shared.get()
}
}
#[async_trait]
@@ -149,7 +184,6 @@ impl Plugin for TelegramPlugin {
}
fn as_any(&self) -> &dyn std::any::Any { self }
fn as_arc_any(self: Arc<Self>) -> Arc<dyn std::any::Any + Send + Sync> { self }
async fn reload(&self, enabled: bool, config: Value, ctx: PluginContext) -> Result<()> {
@@ -189,49 +223,59 @@ impl Plugin for TelegramPlugin {
anyhow::bail!("telegram: token is empty — set it via the plugins API");
}
let uploads_dir = self.secrets_dir
.parent()
.unwrap_or(std::path::Path::new("."))
let uploads_dir = std::env::current_dir()
.unwrap_or_default()
.join("uploads")
.join("telegram");
// Register "telegram" source with ChatHub (idempotent).
// ChatHub restores the active session from the sources table automatically.
ctx.chat_hub.register("telegram").await;
info!("telegram: registered with ChatHub");
// Load bindings from the config table (or default if absent).
let telegram_config = auth::load_config(&*ctx.config).await
.unwrap_or_default();
info!(
bindings = telegram_config.bindings.len(),
pending = telegram_config.pending_pairings.len(),
"telegram: config loaded",
);
let shared = Arc::new(TgShared {
chat_hub: Arc::clone(&ctx.chat_hub),
user_channel: Arc::clone(&ctx.user_channel),
command: Arc::clone(&ctx.command),
approval: Arc::clone(&ctx.approval),
config: Arc::clone(&ctx.config),
transcribe: Arc::clone(&ctx.transcribe),
tts: Arc::clone(&ctx.tts_provider),
location: Arc::clone(&ctx.location),
pending_approvals: Mutex::new(HashMap::new()),
pending_question: Mutex::new(None),
secrets_dir: self.secrets_dir.clone(),
uploads_dir,
home_chat_id: Mutex::new(None),
bindings: RwLock::new(telegram_config),
pending_approvals: Mutex::new(HashMap::new()),
pending_questions: Mutex::new(HashMap::new()),
forwarders: Mutex::new(HashSet::new()),
});
let _ = self.shared.set(Arc::clone(&shared));
let bot = Bot::new(&token);
let cancel = CancellationToken::new();
tokio::spawn(events::persistent_forwarder(
bot.clone(),
Arc::clone(&shared),
cancel.clone(),
));
// Config listener: reloads bindings when the "telegram" config key
// changes (e.g. the pairing tool writes a new binding).
{
let shared_c = Arc::clone(&shared);
let cancel_c = cancel.clone();
let bus_rx = ctx.system_bus.subscribe();
tokio::spawn(auth::config_listener(shared_c, bus_rx, cancel_c));
}
let hub_clone = Arc::clone(&ctx.chat_hub);
tokio::spawn(async move {
if let Err(e) = hub_clone.resume("telegram").await {
tracing::warn!(error = %e, "telegram: startup resume failed");
}
});
// Spawn forwarders for already-unlocked paired users.
{
let shared_c = Arc::clone(&shared);
let bot_c = bot.clone();
let cancel_c = cancel.clone();
tokio::spawn(async move {
events::spawn_forwarders_for_bound_users(&bot_c, &shared_c, &cancel_c).await;
});
}
let cancel_clone = cancel.clone();
let cancel_wdg = cancel.clone();
let running_clone = Arc::clone(&self.running);
self.running.store(true, Ordering::Relaxed);
@@ -240,9 +284,6 @@ impl Plugin for TelegramPlugin {
.branch(Update::filter_edited_message().endpoint(handlers::edited_message_handler))
.branch(Update::filter_callback_query().endpoint(events::callback_handler));
let secrets_dir_wdg = self.secrets_dir.clone();
let bot_wdg = bot.clone();
let task = tokio::spawn(async move {
let mut dispatcher = Dispatcher::builder(bot, handler)
.dependencies(dptree::deps![shared])
@@ -250,9 +291,8 @@ impl Plugin for TelegramPlugin {
info!("telegram plugin: dispatcher starting");
tokio::select! {
_ = cancel_clone.cancelled() => info!("telegram plugin: cancellation received"),
_ = dispatcher.dispatch() => warn!("telegram plugin: dispatcher exited unexpectedly"),
_ = auth::whitelist_watchdog(bot_wdg, secrets_dir_wdg, cancel_wdg) => {}
_ = cancel_clone.cancelled() => info!("telegram plugin: cancellation received"),
_ = dispatcher.dispatch() => warn!("telegram plugin: dispatcher exited unexpectedly"),
}
running_clone.store(false, Ordering::Relaxed);
info!("telegram plugin: stopped");
@@ -273,4 +313,8 @@ impl Plugin for TelegramPlugin {
self.running.store(false, Ordering::Relaxed);
Ok(())
}
fn tools(self: Arc<Self>) -> Vec<Arc<dyn core_api::tool::Tool>> {
vec![Arc::new(tools::TelegramPairingTool::new(self))]
}
}
crates/plugin-telegram-bot/src/tools.rs
+158 -1
View File
@@ -1,12 +1,17 @@
use std::sync::Arc;
use serde_json::json;
use anyhow::Result;
use serde_json::{json, Value};
use teloxide::prelude::*;
use teloxide::types::InputFile;
use core_api::interface_tool::InterfaceTool;
use core_api::tool::{Tool, ToolCategory, ToolDescriptionLength};
use core_api::tts::{TextToSpeech, TtsProvider};
use super::auth::{Binding, load_config, save_config};
use super::TelegramPlugin;
/// Returns all LLM-callable tools available in a Telegram session.
///
/// Each tool captures `bot` and `chat_id` so its handler can send content
@@ -225,3 +230,155 @@ async fn to_ogg_opus(audio: Vec<u8>, format: &str) -> anyhow::Result<Vec<u8>> {
}
Ok(out.stdout)
}
// ── telegram_pairing (registry tool, category Config) ─────────────────────────
/// Tool that binds a Telegram `chat_id` to a Skald `user_id`.
///
/// Category `Config` — excluded from the default tool list, activated
/// explicitly by the admin's agent. The admin calls this after a user reports
/// their pairing code from Telegram.
///
/// The binding is written to the config table (key `"telegram"`); the
/// resulting `ConfigKeyUpdated` event reloads the plugin's in-memory cache
/// instantly.
pub struct TelegramPairingTool {
plugin: Arc<TelegramPlugin>,
}
impl TelegramPairingTool {
pub fn new(plugin: Arc<TelegramPlugin>) -> Self {
Self { plugin }
}
}
impl Tool for TelegramPairingTool {
fn name(&self) -> &str { "telegram_pairing" }
fn category(&self) -> ToolCategory { ToolCategory::Config }
fn description(&self) -> &str {
"Bind a Telegram chat to a Skald user so they can chat with the agent via Telegram. \
Use `action: \"bind\"` with either a `code` (from the pairing message the user received) \
or a `chat_id` + `user_id`. Use `action: \"list\"` to see current bindings. \
Use `action: \"unbind\"` with a `chat_id` to remove a binding."
}
fn parameters_schema(&self) -> Value {
json!({
"type": "object",
"properties": {
"action": {
"type": "string",
"enum": ["bind", "unbind", "list"],
"description": "bind: create a chat_id→user_id binding. unbind: remove it. list: show all bindings.",
"default": "bind"
},
"code": {
"type": "string",
"description": "Pairing code shown to the Telegram user (alternative to chat_id+user_id)."
},
"chat_id": {
"type": "integer",
"description": "Telegram chat id (use when not resolving via code)."
},
"user_id": {
"type": "string",
"description": "Skald user id to bind to (required for bind when not using code)."
}
}
})
}
fn describe(&self, _args: &Value, _length: ToolDescriptionLength) -> String {
"telegram_pairing".to_string()
}
fn execute(&self, args: Value) -> Result<String> {
let shared = self.plugin.shared()
.ok_or_else(|| anyhow::anyhow!("telegram: plugin is not running"))?
.clone();
let action = args.get("action")
.and_then(Value::as_str)
.unwrap_or("bind");
// Block on since Tool::execute is sync.
let rt = tokio::runtime::Handle::try_current()
.map_err(|e| anyhow::anyhow!("telegram_pairing: no tokio runtime: {e}"))?;
rt.block_on(async {
let cfg_api = &*shared.config;
match action {
"list" => {
let cfg = load_config(cfg_api).await.unwrap_or_default();
if cfg.bindings.is_empty() {
return Ok("No Telegram bindings.".to_string());
}
let lines: Vec<String> = cfg.bindings.iter()
.map(|b| format!(" chat_id={} → user_id={}{}", b.chat_id, b.user_id,
b.display.as_ref().map(|d| format!(" ({d})")).unwrap_or_default()))
.collect();
Ok(format!("Telegram bindings:\n{}", lines.join("\n")))
}
"unbind" => {
let chat_id = args.get("chat_id")
.and_then(Value::as_i64)
.ok_or_else(|| anyhow::anyhow!("telegram_pairing: `chat_id` required for unbind"))?;
let mut cfg = load_config(cfg_api).await.unwrap_or_default();
let before = cfg.bindings.len();
cfg.bindings.retain(|b| b.chat_id != chat_id);
if cfg.bindings.len() == before {
return Ok(format!("chat_id {chat_id} is not bound."));
}
save_config(cfg_api, &cfg).await?;
Ok(format!("Unbound chat_id {chat_id}."))
}
"bind" => {
let mut cfg = load_config(cfg_api).await.unwrap_or_default();
// Resolve chat_id + user_id either from a pairing code or
// from explicit arguments.
let (chat_id, user_id) = if let Some(code) = args.get("code").and_then(Value::as_str) {
let entry = cfg.pending_pairings.iter()
.find(|e| e.code == code)
.ok_or_else(|| anyhow::anyhow!("telegram_pairing: code '{code}' not found (it may have expired or already been used)"))?;
let chat_id = entry.chat_id;
let user_id = args.get("user_id")
.and_then(Value::as_str)
.ok_or_else(|| anyhow::anyhow!("telegram_pairing: `user_id` required (the code only identifies the chat)"))?
.to_string();
// Remove the used pairing entry.
cfg.pending_pairings.retain(|e| e.code != code);
(chat_id, user_id)
} else {
let chat_id = args.get("chat_id")
.and_then(Value::as_i64)
.ok_or_else(|| anyhow::anyhow!("telegram_pairing: either `code` or `chat_id`+`user_id` required"))?;
let user_id = args.get("user_id")
.and_then(Value::as_str)
.ok_or_else(|| anyhow::anyhow!("telegram_pairing: `user_id` required"))?
.to_string();
(chat_id, user_id)
};
// Replace any existing binding for this chat_id.
cfg.bindings.retain(|b| b.chat_id != chat_id);
cfg.bindings.push(Binding {
chat_id,
user_id: user_id.clone(),
display: None,
});
save_config(cfg_api, &cfg).await?;
Ok(format!("Bound Telegram chat_id {chat_id} to user_id {user_id}."))
}
other => Err(anyhow::anyhow!("telegram_pairing: unknown action '{other}'")),
}
})
}
}
crates/skald-core/src/approval/mod.rs
+77 -29
View File
@@ -29,11 +29,12 @@
//! determines the action; if none matches the tool requires approval
//! (default-closed).
//!
//! ## Hardcoded exception
//! ## Memory namespace
//!
//! File-write tools targeting `memory/` paths bypass the rule engine and are
//! always allowed (this mirrors the original behaviour and can be replaced by
//! an explicit `allow` rule later).
//! The virtual memory roots `user-memory/` and `shared-memory/` (blueprint §5) are
//! allowed by **seeded rules**, not a hardcoded bypass: `seed_fs_path_rules` stamps
//! `@fs_any allow user-memory/*` and `@fs_any allow shared-memory/*`, editable in the
//! File System panel like any other path rule.
use std::collections::HashMap;
use std::sync::Arc;
@@ -327,31 +328,52 @@ impl ApprovalManager {
/// Priority `5` places these path-scoped rules *before* the `*` catch-all (999999),
/// so an unmatched path still falls through to the default `require`.
///
/// - `memory/*` → **allow** (the LLM manages its own memory; replaces the former
/// hardcoded `is_memory_path` bypass).
/// - `user-memory/*` → **allow** (the caller's private memory namespace, blueprint §5;
/// the LLM manages its own memory). Reads *and* writes: `@fs_any`.
/// - `shared-memory/*` → reads **allow** (`@fs_read`), writes **require** (`@fs_write`):
/// shared memory is visible to everyone, so a write is a deliberate, human-confirmed
/// act — the agent must not silently push one person's information into it.
/// - `data/*` → **allow** (scratch/data workspace).
/// - `secrets/*` → **deny** (`@fs_any` denies reads *and* writes; a read would leak
/// the secret into the LLM context / history / WS stream, and `Deny` is
/// non-bypassable). The `/*` pattern also matches the `secrets` dir node itself, so
/// recursive `list_files`/`grep_files` rooted at it are covered.
/// - `memory_search` → **allow**, path-less: it searches note *content* (arg `query`,
/// not `path`), so it needs a tool-scoped rule rather than a path pattern.
pub async fn seed_fs_path_rules(&self) -> Result<()> {
// (tool_pattern, path_pattern, action, note)
let rules: &[(&str, &str, &str, &str)] = &[
("@fs_any", "memory/*", "allow", "auto-allow memory/"),
("@fs_any", "data/*", "allow", "auto-allow data/"),
("@fs_any", "secrets/*", "deny", "deny secrets/ access"),
// (tool_pattern, path_pattern, action, note). `path_pattern = None` is a
// tool-scoped rule that matches regardless of args.
let rules: &[(&str, Option<&str>, &str, &str)] = &[
("@fs_any", Some("user-memory/*"), "allow", "auto-allow user-memory/"),
("@fs_read", Some("shared-memory/*"), "allow", "auto-allow read shared-memory/"),
("@fs_write", Some("shared-memory/*"), "require", "require write shared-memory/"),
("@fs_any", Some("data/*"), "allow", "auto-allow data/"),
("@fs_any", Some("secrets/*"), "deny", "deny secrets/ access"),
("memory_search", None, "allow", "allow memory_search"),
];
let mut seeded = 0;
for (tool_pattern, path_pattern, action, note) in rules {
let exists: i64 = sqlx::query_scalar(
"SELECT COUNT(*) FROM approval_rules
WHERE tool_pattern = ? AND path_pattern = ? AND group_id = 'default'",
)
.bind(tool_pattern)
.bind(path_pattern)
.fetch_one(self.db.as_ref())
.await?;
for &(tool_pattern, path_pattern, action, note) in rules {
// A NULL path can't be matched with `=` (NULL comparisons are never true),
// so the existence check branches on it — otherwise the row would re-insert
// on every boot.
let exists: i64 = match path_pattern {
Some(p) => sqlx::query_scalar(
"SELECT COUNT(*) FROM approval_rules
WHERE tool_pattern = ? AND path_pattern = ? AND group_id = 'default'",
)
.bind(tool_pattern)
.bind(p)
.fetch_one(self.db.as_ref())
.await?,
None => sqlx::query_scalar(
"SELECT COUNT(*) FROM approval_rules
WHERE tool_pattern = ? AND path_pattern IS NULL AND group_id = 'default'",
)
.bind(tool_pattern)
.fetch_one(self.db.as_ref())
.await?,
};
if exists > 0 {
continue;
}
@@ -360,7 +382,7 @@ impl ApprovalManager {
VALUES (?, ?, ?, ?, 5, 'default')",
)
.bind(tool_pattern)
.bind(path_pattern)
.bind(path_pattern) // Option<&str> → NULL when None
.bind(action)
.bind(note)
.execute(self.db.as_ref())
@@ -383,7 +405,10 @@ impl ApprovalManager {
/// - the per-tool write `require` defaults (`note = 'default rule'`, no path) — fs
/// gating now lives in the File System panel + the `*` catch-all;
/// - the old `data/*` allow rows (`note = 'auto-allow data/ writes'`);
/// - the old `secrets` deny rows (`note = 'deny reading secrets/'`).
/// - the old `secrets` deny rows (`note = 'deny reading secrets/'`);
/// - the old single `memory/*` allow row (`note = 'auto-allow memory/'`) — the memory
/// namespace split into `user-memory/` + `shared-memory/`, so the `memory/*` pattern
/// no longer routes and would otherwise linger as a stale allow on a disk `./memory/`.
///
/// Idempotent: a no-op once the legacy rows are gone. Run before `seed_fs_path_rules`.
pub async fn migrate_legacy_fs_rules(&self) -> Result<()> {
@@ -406,7 +431,20 @@ impl ApprovalManager {
.await?
.rows_affected();
let total = n1 + n2 + n3;
let n4 = sqlx::query("DELETE FROM approval_rules WHERE note = 'auto-allow memory/'")
.execute(self.db.as_ref())
.await?
.rows_affected();
// An earlier build seeded `@fs_any allow shared-memory/*`; shared writes now
// require approval, so that blanket allow must go (the new @fs_read/@fs_write
// rows are seeded fresh by `seed_fs_path_rules`).
let n5 = sqlx::query("DELETE FROM approval_rules WHERE note = 'auto-allow shared-memory/'")
.execute(self.db.as_ref())
.await?
.rows_affected();
let total = n1 + n2 + n3 + n4 + n5;
if total > 0 {
info!("approval_rules: migrated {total} legacy filesystem rules to @fs_* File System panel");
}
@@ -419,8 +457,8 @@ impl ApprovalManager {
///
/// Evaluation order:
/// 1. Rules for `group_id` first, then "default" group as fallback, sorted by priority ASC.
/// First match wins. (`memory/` auto-allow is a seeded `@fs_any allow memory/*` rule,
/// not a hardcoded exception — see `seed_fs_path_rules`.)
/// First match wins. (The `user-memory/` and `shared-memory/` auto-allows are seeded
/// `@fs_any allow …/*` rules, not a hardcoded exception — see `seed_fs_path_rules`.)
/// 2. Session bypass: if a matching bypass is active, `Require` → `Allow`.
/// `Deny` is never bypassed.
/// 3. No match → `Require` (default-closed policy).
@@ -1119,21 +1157,31 @@ mod tests {
.unwrap();
assert_eq!(legacy, 0, "legacy fs rules should be removed by migration");
// …and replaced by exactly the three @fs_* token rows.
// …and replaced by exactly the five @fs_* token rows (shared-memory has two:
// read-allow and write-require).
let fs_rows: i64 = sqlx::query_scalar(
"SELECT COUNT(*) FROM approval_rules WHERE tool_pattern LIKE '@fs%'",
)
.fetch_one(db.as_ref())
.await
.unwrap();
assert_eq!(fs_rows, 3, "memory/data/secrets @fs_* rules should be seeded");
assert_eq!(fs_rows, 5, "user-memory + shared-memory(r/w) + data + secrets @fs_* rules should be seeded");
// Gate decisions through the real check() path.
async fn decide(mgr: &ApprovalManager, tool: &str, path: &str) -> GateResult {
mgr.check(1, None, "main", "web", tool, &json!({ "path": path }), Some("default")).await
}
assert!(matches!(decide(&mgr, "write_file", "memory/notes.md").await, GateResult::Allow));
assert!(matches!(decide(&mgr, "read_file", "data/x.txt").await, GateResult::Allow));
// user-memory auto-allows reads and writes; the old `memory/*` no longer matches.
assert!(matches!(decide(&mgr, "write_file", "user-memory/notes.md").await, GateResult::Allow));
assert!(matches!(decide(&mgr, "list_files", "user-memory").await, GateResult::Allow));
assert!(matches!(decide(&mgr, "write_file", "memory/notes.md").await, GateResult::Require));
// shared-memory: reads allowed, writes require approval.
assert!(matches!(decide(&mgr, "read_file", "shared-memory/casa.md").await, GateResult::Allow));
assert!(matches!(decide(&mgr, "write_file", "shared-memory/casa.md").await, GateResult::Require));
assert!(matches!(decide(&mgr, "edit_file", "shared-memory/casa.md").await, GateResult::Require));
assert!(matches!(decide(&mgr, "read_file", "data/x.txt").await, GateResult::Allow));
// memory_search is allowed by a path-less tool rule (it has `query`, not `path`).
assert!(matches!(decide(&mgr, "memory_search", "ignored").await, GateResult::Allow));
// Improvement over legacy: secrets *writes* are now denied too, not just reads.
assert!(matches!(decide(&mgr, "write_file", "secrets/key").await, GateResult::Deny));
assert!(matches!(decide(&mgr, "read_file", "secrets/key").await, GateResult::Deny));
crates/skald-core/src/config_store.rs
+32 -3
View File
@@ -2,13 +2,16 @@ use std::sync::Arc;
use sqlx::SqlitePool;
use core_api::system_bus::{SystemEvent, SystemEventBus};
pub struct GlobalConfigManager {
pool: Arc<SqlitePool>,
pool: Arc<SqlitePool>,
system_bus: Arc<SystemEventBus>,
}
impl GlobalConfigManager {
pub fn new(pool: Arc<SqlitePool>) -> Self {
Self { pool }
pub fn new(pool: Arc<SqlitePool>, system_bus: Arc<SystemEventBus>) -> Self {
Self { pool, system_bus }
}
pub async fn get(&self, key: &str) -> anyhow::Result<Option<String>> {
@@ -19,7 +22,16 @@ impl GlobalConfigManager {
Ok(row.map(|(v,)| v))
}
/// Sets a config key and emits [`SystemEvent::ConfigKeyUpdated`] on the
/// system bus when the value actually changes. No-op (no write, no event)
/// when the new value equals the current one.
pub async fn set(&self, key: &str, value: &str) -> anyhow::Result<()> {
let old_value = self.get(key).await?;
if old_value.as_deref() == Some(value) {
return Ok(());
}
sqlx::query(
"INSERT INTO config (key, value, updated_at) VALUES (?, ?, datetime('now'))
ON CONFLICT(key) DO UPDATE SET
@@ -30,6 +42,13 @@ impl GlobalConfigManager {
.bind(value)
.execute(&*self.pool)
.await?;
self.system_bus.send(SystemEvent::ConfigKeyUpdated {
key: key.to_string(),
old_value,
new_value: value.to_string(),
});
Ok(())
}
@@ -41,3 +60,13 @@ impl GlobalConfigManager {
Ok(())
}
}
#[async_trait::async_trait]
impl core_api::config_api::ConfigApi for GlobalConfigManager {
async fn get(&self, key: &str) -> anyhow::Result<Option<String>> {
GlobalConfigManager::get(self, key).await
}
async fn set(&self, key: &str, value: &str) -> anyhow::Result<()> {
GlobalConfigManager::set(self, key, value).await
}
}
crates/skald-core/src/db/memory_docs.rs
+203
View File
@@ -0,0 +1,203 @@
//! Accessor for `memory_docs` — the backing store of the virtual `memory/`
//! namespace (blueprint §5).
//!
//! The **pool is the namespace**: a user pool holds that user's private notes
//! (`memory/{userid}`), the system pool holds shared notes (`memory/shared`).
//! Callers pass a `path` already stripped of the `memory/…` prefix — the file
//! it lands in decides the namespace, the row keeps only the tail. `path` is
//! UNIQUE, so [`upsert`] is the single write path for both create and edit, and
//! the `memory_docs_fts` triggers keep the full-text index in step underneath.
use anyhow::Result;
use sqlx::SqlitePool;
#[derive(Debug, Clone, sqlx::FromRow)]
pub struct MemoryDoc {
pub id: i64,
pub path: String,
pub content: String,
pub created_at: String,
pub updated_at: String,
}
/// One row of a directory-style listing: metadata only, no `content` body.
#[derive(Debug, Clone, sqlx::FromRow)]
pub struct MemoryEntry {
pub path: String,
pub updated_at: String,
}
/// One full-text hit: the matching note's path and a highlighted excerpt.
#[derive(Debug, Clone, sqlx::FromRow)]
pub struct MemoryHit {
pub path: String,
pub snippet: String,
}
const SELECT: &str = "SELECT id, path, content, created_at, updated_at FROM memory_docs";
/// Fetch one note by its exact path.
pub async fn get(pool: &SqlitePool, path: &str) -> Result<Option<MemoryDoc>> {
let row = sqlx::query_as::<_, MemoryDoc>(sqlx::AssertSqlSafe(format!("{SELECT} WHERE path = ?")))
.bind(path)
.fetch_optional(pool)
.await?;
Ok(row)
}
/// Create the note at `path`, or overwrite it if it already exists. `created_at`
/// survives an overwrite; `updated_at` is bumped. Returns the stored row.
pub async fn upsert(pool: &SqlitePool, path: &str, content: &str) -> Result<MemoryDoc> {
sqlx::query(
"INSERT INTO memory_docs (path, content)
VALUES (?, ?)
ON CONFLICT(path) DO UPDATE SET
content = excluded.content,
updated_at = datetime('now')",
)
.bind(path)
.bind(content)
.execute(pool)
.await?;
let row = sqlx::query_as::<_, MemoryDoc>(sqlx::AssertSqlSafe(format!("{SELECT} WHERE path = ?")))
.bind(path)
.fetch_one(pool)
.await?;
Ok(row)
}
/// List notes whose path starts with `prefix` (pass `""` for all), most recently
/// edited first. Metadata only — the `content` body is not loaded.
pub async fn list(pool: &SqlitePool, prefix: &str) -> Result<Vec<MemoryEntry>> {
// Escaped LIKE prefix: a literal `%`/`_` in the caller's path must match as
// itself, not as a wildcard. `\` is the escape character.
let pattern = format!("{}%", escape_like(prefix));
let rows = sqlx::query_as::<_, MemoryEntry>(
"SELECT path, updated_at FROM memory_docs
WHERE path LIKE ? ESCAPE '\\'
ORDER BY updated_at DESC",
)
.bind(pattern)
.fetch_all(pool)
.await?;
Ok(rows)
}
/// Full-text search over note bodies and paths, best match first. `query` is
/// FTS5 MATCH syntax; `snippet` is a short excerpt of the body with the matched
/// terms wrapped in `[` … `]`.
pub async fn search(pool: &SqlitePool, query: &str, limit: i64) -> Result<Vec<MemoryHit>> {
let rows = sqlx::query_as::<_, MemoryHit>(
// `memory_docs_fts` is external-content, so its rowid is `memory_docs.id`;
// join back for the path, and read the excerpt from content column 1.
"SELECT d.path AS path,
snippet(memory_docs_fts, 1, '[', ']', '…', 12) AS snippet
FROM memory_docs_fts
JOIN memory_docs d ON d.id = memory_docs_fts.rowid
WHERE memory_docs_fts MATCH ?
ORDER BY bm25(memory_docs_fts)
LIMIT ?",
)
.bind(query)
.bind(limit)
.fetch_all(pool)
.await?;
Ok(rows)
}
/// Delete the note at `path`. Returns whether a row was removed.
pub async fn delete(pool: &SqlitePool, path: &str) -> Result<bool> {
let n = sqlx::query("DELETE FROM memory_docs WHERE path = ?")
.bind(path)
.execute(pool)
.await?
.rows_affected();
Ok(n > 0)
}
/// Escapes `%`, `_` and `\` so a caller-supplied string is a literal LIKE prefix.
fn escape_like(s: &str) -> String {
let mut out = String::with_capacity(s.len());
for c in s.chars() {
if matches!(c, '%' | '_' | '\\') {
out.push('\\');
}
out.push(c);
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::PathBuf;
/// A standalone owner-schema database in a throwaway temp dir. `tag` plus an
/// atomic counter keep parallel tests from colliding on the same file.
/// Returns the pool and the dir so the caller can wipe it (SQLite leaves
/// `-wal`/`-shm` sidecars beside the file).
async fn owner_pool(tag: &str) -> (SqlitePool, PathBuf) {
use std::sync::atomic::{AtomicU64, Ordering};
static SEQ: AtomicU64 = AtomicU64::new(0);
let n = SEQ.fetch_add(1, Ordering::Relaxed);
let dir = std::env::temp_dir()
.join(format!("skald-memdocs-{}-{tag}-{n}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let pool = crate::db::create_user_pool(&dir.join("owner.db"), None).await.unwrap();
(pool, dir)
}
#[tokio::test]
async fn upsert_is_create_then_overwrite_and_fts_follows() {
let (pool, dir) = owner_pool("upsert").await;
// create
let doc = upsert(&pool, "notes/spesa.md", "latte e pane").await.unwrap();
assert_eq!(doc.path, "notes/spesa.md");
assert_eq!(doc.content, "latte e pane");
let first_id = doc.id;
// get by exact path
assert_eq!(get(&pool, "notes/spesa.md").await.unwrap().unwrap().content, "latte e pane");
assert!(get(&pool, "notes/altro.md").await.unwrap().is_none());
// overwrite: same row, new content, created_at preserved
let doc2 = upsert(&pool, "notes/spesa.md", "latte, pane, uova").await.unwrap();
assert_eq!(doc2.id, first_id, "upsert must update in place, not insert a new row");
assert_eq!(doc2.content, "latte, pane, uova");
assert_eq!(doc2.created_at, doc.created_at, "created_at survives an overwrite");
assert_eq!(list(&pool, "").await.unwrap().len(), 1, "still one row for that path");
// FTS follows the update: the removed token is gone, the new one is found
assert!(search(&pool, "uova", 10).await.unwrap().iter().any(|h| h.path == "notes/spesa.md"));
assert!(search(&pool, "latte", 10).await.unwrap().iter().any(|h| h.path == "notes/spesa.md"));
pool.close().await;
let _ = std::fs::remove_dir_all(&dir);
}
#[tokio::test]
async fn list_by_prefix_and_delete_deindexes() {
let (pool, dir) = owner_pool("list").await;
upsert(&pool, "notes/spesa.md", "latte pane uova").await.unwrap();
upsert(&pool, "notes/idee.md", "un'idea brillante").await.unwrap();
upsert(&pool, "diary/2026.md", "oggi e' successo").await.unwrap();
let notes = list(&pool, "notes/").await.unwrap();
assert_eq!(notes.len(), 2, "prefix listing is scoped to the subtree");
assert!(notes.iter().all(|e| e.path.starts_with("notes/")));
assert_eq!(list(&pool, "").await.unwrap().len(), 3, "empty prefix lists everything");
// delete removes the row and de-indexes it from FTS
assert!(delete(&pool, "notes/spesa.md").await.unwrap());
assert!(get(&pool, "notes/spesa.md").await.unwrap().is_none());
assert!(search(&pool, "uova", 10).await.unwrap().is_empty(), "delete must de-index");
assert!(!delete(&pool, "notes/spesa.md").await.unwrap(), "second delete is a no-op");
pool.close().await;
let _ = std::fs::remove_dir_all(&dir);
}
}
crates/skald-core/src/db/mod.rs
+64
View File
@@ -13,6 +13,7 @@ pub mod llm_requests;
pub mod llm_request_payloads;
pub mod mcp_events;
pub mod mcp_servers;
pub mod memory_docs;
pub mod plugins;
pub mod roles;
pub mod scheduled_jobs;
@@ -710,6 +711,60 @@ pub async fn create_owner_tables(pool: &SqlitePool) -> Result<()> {
.execute(pool)
.await?;
// Backing store for the virtual `memory/` namespace (blueprint §5). MD-only
// notes keyed by a path *relative to the namespace root* — the file is the
// namespace, so no `memory/{userid}` / `memory/shared` prefix is stored:
// routing picks the pool, the row keeps only the tail. Because this is an
// owner table, the same schema backs private memory in each `{userid}.db`
// (behind SQLCipher) and shared memory in `system.db` (cleartext, the
// household owner) — §5.1. `path` is UNIQUE, so a write is an upsert.
sqlx::query(
"CREATE TABLE IF NOT EXISTS memory_docs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
path TEXT NOT NULL UNIQUE,
content TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
// Full-text index over memory notes: the payoff of a SQLite backing over
// opaque file blobs (§5) — a decrypted session can search / RAG its own
// memory. External-content FTS5 keeps no second copy of `content`; the
// triggers below mirror every change from `memory_docs`. FTS5 is compiled
// into the bundled SQLCipher build, so this works inside encrypted user
// files too.
sqlx::query(
"CREATE VIRTUAL TABLE IF NOT EXISTS memory_docs_fts USING fts5(
path, content,
content='memory_docs',
content_rowid='id'
)",
)
.execute(pool)
.await?;
for trigger in [
"CREATE TRIGGER IF NOT EXISTS memory_docs_ai AFTER INSERT ON memory_docs BEGIN
INSERT INTO memory_docs_fts(rowid, path, content)
VALUES (new.id, new.path, new.content);
END",
"CREATE TRIGGER IF NOT EXISTS memory_docs_ad AFTER DELETE ON memory_docs BEGIN
INSERT INTO memory_docs_fts(memory_docs_fts, rowid, path, content)
VALUES ('delete', old.id, old.path, old.content);
END",
"CREATE TRIGGER IF NOT EXISTS memory_docs_au AFTER UPDATE ON memory_docs BEGIN
INSERT INTO memory_docs_fts(memory_docs_fts, rowid, path, content)
VALUES ('delete', old.id, old.path, old.content);
INSERT INTO memory_docs_fts(rowid, path, content)
VALUES (new.id, new.path, new.content);
END",
] {
sqlx::query(trigger).execute(pool).await?;
}
Ok(())
}
@@ -764,6 +819,15 @@ mod tests {
one("INSERT INTO projects (id, name, path) VALUES (1, 'p', '/tmp')").await.unwrap();
one("INSERT INTO project_tickets (project_id, title, job_id) VALUES (1, 't', 1)").await.unwrap();
one("INSERT INTO llm_request_payloads (request_id, request_json) VALUES ('r1', '{}')").await.unwrap();
// Fires the AFTER INSERT trigger into the external-content FTS5 table.
one("INSERT INTO memory_docs (path, content) VALUES ('notes/x.md', 'hello world')").await.unwrap();
// ...and the FTS index actually answers a MATCH.
let (hits,): (i64,) = sqlx::query_as(
"SELECT count(*) FROM memory_docs_fts WHERE memory_docs_fts MATCH 'world'",
)
.fetch_one(&pool).await.unwrap();
assert_eq!(hits, 1, "memory_docs_fts must index inserted notes");
pool.close().await;
let _ = std::fs::remove_dir_all(&dir);
crates/skald-core/src/plugin/mod.rs
+2
View File
@@ -97,6 +97,7 @@ impl PluginManager {
Ok(PluginContext {
command: Arc::clone(skald.command_manager()) as _,
config: Arc::clone(skald.config()) as Arc<dyn core_api::config_api::ConfigApi>,
db: Arc::clone(skald.db()),
secrets: Arc::clone(skald.secrets()) as _,
transcribe: Arc::clone(skald.transcribe_manager()) as _,
@@ -107,6 +108,7 @@ impl PluginManager {
api_provider_registry: Arc::clone(skald.provider_registry()) as _,
location: Arc::clone(skald.location_manager()) as _,
system_bus: Arc::clone(skald.system_bus()),
user_channel: self.skald()? as Arc<dyn core_api::user_channel::UserChannelApi>,
web_port,
remote_slot: Arc::clone(skald.remote()),
router_factory,
crates/skald-core/src/session/handler/message_builder.rs
+25 -3
View File
@@ -34,6 +34,9 @@ fn system_timezone() -> Option<&'static str> {
/// without needing the full handler and all its dependencies.
pub struct MessageBuilder {
pub pool: Arc<SqlitePool>,
/// The shared (`system.db`) pool, for injecting `shared-memory/` notes. The
/// owner `pool` above backs `user-memory/`.
pub shared_pool: Arc<SqlitePool>,
pub session_id: i64,
pub mcp: Arc<McpManager>,
pub datetime_config: DatetimeConfig,
@@ -93,9 +96,7 @@ impl MessageBuilder {
You can edit them with `edit_file` or `write_file` using the path shown.\n"
);
for mem_path in &meta.inject_memory {
// Resolve the entry to (absolute path to read, path to show the agent).
let (abs, display) = self.resolve_memory_path(mem_path);
let content = tokio::fs::read_to_string(&abs).await.ok();
let (content, display) = self.load_inject_memory(mem_path).await;
match content {
Some(c) => static_content.push_str(&format!(
"\n<memory_file path=\"{display}\">\n{c}\n</memory_file>\n"
@@ -405,6 +406,27 @@ impl MessageBuilder {
/// when the file lives under it, absolute otherwise** — so when the agent references
/// it back via `edit_file`/`write_file`, the loop's working-directory injection
/// (which rewrites relative paths against the WD) resolves to the very same file.
/// Loads an `inject_memory` entry, returning `(content, display_path)`.
///
/// Virtual memory paths are read from SQLite: `user-memory/…` from the owner
/// `pool`, `shared-memory/…` from the `shared_pool` (`system.db`). Everything
/// else (`data/…`, `$WD/…`) is an ordinary disk read. A missing note / file
/// yields `None`, rendered as "(file not created yet)".
async fn load_inject_memory(&self, mem_path: &str) -> (Option<String>, String) {
use crate::tools::fs::{classify_memory, MemScope};
if let Some(m) = classify_memory(mem_path) {
let pool = match m.scope {
MemScope::User => &self.pool,
MemScope::Shared => &self.shared_pool,
};
let content = crate::db::memory_docs::get(pool, &m.rel)
.await.ok().flatten().map(|d| d.content);
return (content, mem_path.to_string());
}
let (abs, display) = self.resolve_memory_path(mem_path);
(tokio::fs::read_to_string(&abs).await.ok(), display)
}
fn resolve_memory_path(&self, mem_path: &str) -> (std::path::PathBuf, String) {
let wd = self.working_directory.clone()
.unwrap_or_else(|| std::env::current_dir().unwrap_or_default());
crates/skald-core/src/session/handler/messages.rs
+1
View File
@@ -28,6 +28,7 @@ impl ChatSessionHandler {
.map(|rc| rc.effective_working_dir());
let builder = MessageBuilder {
pool: Arc::clone(&self.db),
shared_pool: Arc::clone(&self.shared_pool),
session_id: self.scratchpad_sid(),
mcp: Arc::clone(&self.mcp),
datetime_config: self.datetime_config.clone(),
crates/skald-core/src/session/handler/mod.rs
+5
View File
@@ -262,6 +262,9 @@ impl ApprovalDecision {
pub struct ChatSessionHandler {
pub session_id: i64,
pub(super) db: Arc<SqlitePool>,
/// The shared (`system.db`) pool. Owner-bound work uses `db`; this is only for
/// cross-owner reads, e.g. injecting `shared-memory/` notes into the prompt.
pub(super) shared_pool: Arc<SqlitePool>,
/// The authenticated user who owns this session. Threaded into `ChatOptions`
/// so the telemetry metadata row in `system.db` carries `user_id`.
pub(super) user_id: String,
@@ -332,6 +335,7 @@ impl ChatSessionHandler {
pub fn new(
session_id: i64,
db: Arc<SqlitePool>,
shared_pool: Arc<SqlitePool>,
user_id: String,
llm_manager: Arc<LlmManager>,
max_history_messages: usize,
@@ -357,6 +361,7 @@ impl ChatSessionHandler {
Self {
session_id,
db,
shared_pool,
user_id,
llm_manager,
max_history_messages,
crates/skald-core/src/session/manager.rs
+6
View File
@@ -22,6 +22,9 @@ use super::handler::ChatSessionHandler;
pub struct ChatSessionManager {
db: Arc<SqlitePool>,
/// The shared (`system.db`) pool, threaded to each handler for cross-owner
/// reads such as injecting `shared-memory/` notes.
shared_pool: Arc<SqlitePool>,
user_id: String,
llm_manager: Arc<LlmManager>,
max_history_messages: usize,
@@ -48,6 +51,7 @@ pub struct ChatSessionManager {
impl ChatSessionManager {
pub fn new(
db: Arc<SqlitePool>,
shared_pool: Arc<SqlitePool>,
user_id: String,
llm_manager: Arc<LlmManager>,
max_history_messages: usize,
@@ -68,6 +72,7 @@ impl ChatSessionManager {
) -> Self {
Self {
db,
shared_pool,
user_id,
llm_manager,
max_history_messages,
@@ -155,6 +160,7 @@ impl ChatSessionManager {
let handler = Arc::new(ChatSessionHandler::new(
session_id,
self.db.clone(),
self.shared_pool.clone(),
self.user_id.clone(),
Arc::clone(&self.llm_manager),
self.max_history_messages,
crates/skald-core/src/skald/accessors.rs
+13
View File
@@ -16,6 +16,7 @@ use tokio_util::sync::CancellationToken;
use core_api::remote::RemoteAccess;
use core_api::system_bus::SystemEventBus;
use core_api::user_channel::UserChannelApi;
use crate::approval::ApprovalManager;
use crate::chat_event_bus::ChatEventBus;
@@ -112,3 +113,15 @@ impl Skald {
pub fn location_manager(&self) -> &Arc<LocationManager> { &self.infra.location_manager }
pub fn remote(&self) -> &Arc<RwLock<Option<Arc<dyn RemoteAccess>>>> { &self.infra.remote }
}
// ── UserChannelApi ────────────────────────────────────────────────────────────
use super::user_context::UserContextHandle;
#[async_trait::async_trait]
impl UserChannelApi for Skald {
async fn resolve_user(&self, user_id: &str) -> Option<std::sync::Arc<dyn core_api::user_channel::UserChannelHandle>> {
let ctx = self.user_context(user_id).await?;
Some(std::sync::Arc::new(UserContextHandle::new(ctx)))
}
}
crates/skald-core/src/skald/bundles.rs
+3 -2
View File
@@ -204,9 +204,9 @@ impl Tools {
/// Captures sibling managers (mcp, plugins, cron, secrets) into the tool
/// registry. `execute_task` is deliberately NOT registered here — it is injected
/// per interactive session by `ChatHub::send_message`.
pub(super) fn build(integrations: &Integrations, tasks: &Tasks, models: &Models) -> Self {
pub(super) fn build(rt: &Runtime, integrations: &Integrations, tasks: &Tasks, models: &Models) -> Self {
let mut tool_registry = ToolRegistry::new();
crate::tools::fs::register_all(&mut tool_registry);
crate::tools::fs::register_all(&mut tool_registry, Arc::clone(&rt.db));
tool_registry.register(crate::tools::ast_outline::AstOutline::new());
tool_registry.register(crate::tools::exec::ExecuteCmd);
tool_registry.register(crate::tools::read_notification::ReadNotification);
@@ -338,6 +338,7 @@ impl Conversation {
let manager = Arc::new(ChatSessionManager::new(
Arc::clone(&rt.db),
Arc::clone(&rt.db), // shared pool == system.db (this is the ownerless manager)
String::new(),
Arc::clone(&models.llm_manager),
config.llm.max_history_messages,
crates/skald-core/src/skald/mod.rs
+1 -1
View File
@@ -66,7 +66,7 @@ impl Skald {
let media = Media::build(&rt, &models).await?;
let integrations = Integrations::build(&rt, plugins);
let tasks = Tasks::build(&rt, config);
let tools = Tools::build(&integrations, &tasks, &models);
let tools = Tools::build(&rt, &integrations, &tasks, &models);
let interaction = Interaction::build(&rt, &tools).await?;
let conversation = Conversation::build(&rt, &models, &media, &tools, &integrations, &interaction, config).await?;
let infra = Infra::build();
crates/skald-core/src/skald/runtime.rs
+4 -4
View File
@@ -45,14 +45,14 @@ pub(super) struct Runtime {
impl Runtime {
/// Wires the cross-cutting primitives. Infallible.
pub(super) fn bootstrap(pool: Arc<SqlitePool>) -> Self {
let config = Arc::new(GlobalConfigManager::new(Arc::clone(&pool)));
let system_bus = Arc::new(SystemEventBus::new());
info!("system event bus ready");
let config = Arc::new(GlobalConfigManager::new(Arc::clone(&pool), Arc::clone(&system_bus)));
let users = Arc::new(UserManager::new(Arc::clone(&pool)));
let sessions = Arc::new(SessionStore::new(Arc::clone(&users)));
let system_bus = Arc::new(SystemEventBus::new());
info!("system event bus ready");
let event_bus = Arc::new(ChatEventBus::new());
info!("chat event bus ready");
crates/skald-core/src/skald/user_context.rs
+39
View File
@@ -29,8 +29,11 @@ use sqlx::SqlitePool;
use tokio::sync::{broadcast, Mutex};
use tokio_util::sync::CancellationToken;
use core_api::approval::ApprovalApi;
use core_api::chat_hub::ChatHubApi;
use core_api::events::GlobalEvent;
use core_api::system_bus::SystemEventBus;
use core_api::user_channel::UserChannelHandle;
use crate::approval::ApprovalManager;
use crate::chat_event_bus::ChatEventBus;
@@ -156,6 +159,7 @@ impl UserContextFactory {
let manager = Arc::new(ChatSessionManager::new(
Arc::clone(&pool),
Arc::clone(&self.registry_pool), // shared pool = system.db, for shared-memory injection
user_id.to_string(),
Arc::clone(&self.llm_manager),
self.max_history_messages,
@@ -257,3 +261,38 @@ impl UserContextRegistry {
Ok(ctx)
}
}
// ── UserChannelHandle impl ────────────────────────────────────────────────────
/// Concrete [`UserChannelHandle`] wrapping a live [`UserContext`].
///
/// Constructed by [`Skald`](super::Skald) when resolving a user for a channel
/// plugin. The concrete type stays private — callers receive
/// `Arc<dyn UserChannelHandle>`.
pub(super) struct UserContextHandle {
ctx: Arc<UserContext>,
}
impl UserContextHandle {
pub(super) fn new(ctx: Arc<UserContext>) -> Self {
Self { ctx }
}
}
impl UserChannelHandle for UserContextHandle {
fn user_id(&self) -> &str {
&self.ctx.user_id
}
fn chat_hub(&self) -> Arc<dyn ChatHubApi> {
Arc::clone(&self.ctx.chat_hub) as Arc<dyn ChatHubApi>
}
fn approval(&self) -> Arc<dyn ApprovalApi> {
Arc::clone(&self.ctx.approval) as Arc<dyn ApprovalApi>
}
fn subscribe(&self) -> broadcast::Receiver<GlobalEvent> {
self.ctx.global_tx.subscribe()
}
}
crates/skald-core/src/tools/fs/edit_file.rs
+76 -40
View File
@@ -1,8 +1,55 @@
use std::sync::Arc;
use anyhow::Result;
use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT};
use super::{read_to_string, write_string};
use crate::tools::{
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT,
};
use super::{classify_memory, read_to_string, write_string, MemScope};
/// Applies the substring edit to `content`, returning the new content. Shared by
/// the on-disk [`EditFile::execute`] and the `memory/` routing in `run_with`;
/// `display` is the path used in error messages.
fn apply_edit(content: &str, args: &Value, display: &str) -> Result<String> {
let old = args["old"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: old"))?;
let new = args["new"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: new"))?;
let replace_all = args["replace_all"].as_bool().unwrap_or(false);
let updated = if replace_all {
if !content.contains(old) {
anyhow::bail!(
"Text not found in {display}. \
Call read_file first and copy the text exactly as shown after the '| ' prefix."
);
}
content.replace(old, new)
} else {
let exact_count = content.matches(old).count();
if exact_count > 1 {
anyhow::bail!(
"Text found {exact_count} times in {display}. \
Include more surrounding context in `old` to make it unique, or set replace_all=true."
);
}
if exact_count == 1 {
content.replacen(old, new, 1)
} else {
let normalized_old = normalize_ws(old);
let (start, end) = find_normalized(content, &normalized_old)
.ok_or_else(|| anyhow::anyhow!(
"Text not found in {display}. \
Call read_file first and copy the text exactly as shown after the '| ' prefix."
))?;
format!("{}{}{}", &content[..start], new, &content[end..])
}
};
Ok(updated)
}
fn normalize_ws(s: &str) -> String {
s.lines()
@@ -56,10 +103,13 @@ fn find_normalized(haystack: &str, normalized_needle: &str) -> Option<(usize, us
None
}
pub struct EditFile;
pub struct EditFile {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl EditFile {
pub fn new() -> Self { Self }
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
}
impl Tool for EditFile {
@@ -102,46 +152,32 @@ impl Tool for EditFile {
truncate_label(&format!("edit_file `{path}`"), MAX_LABEL_SHORT)
}
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute).
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = super::path_arg(&args).unwrap_or_default();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
Box::new(SimpleExecution::new(Box::pin(async move {
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
anyhow::bail!("No note at {path}");
};
let updated = apply_edit(&doc.content, &args, &path)?;
crate::db::memory_docs::upsert(&pool, &rel, &updated).await?;
Ok(ToolResult::Text(format!("Edited {path}.")))
})))
}
fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
let old = args["old"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: old"))?;
let new = args["new"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: new"))?;
let replace_all = args["replace_all"].as_bool().unwrap_or(false);
let content = read_to_string(user_path)?;
let updated = if replace_all {
if !content.contains(old) {
anyhow::bail!(
"Text not found in {user_path}. \
Call read_file first and copy the text exactly as shown after the '| ' prefix."
);
}
content.replace(old, new)
} else {
let exact_count = content.matches(old).count();
if exact_count > 1 {
anyhow::bail!(
"Text found {exact_count} times in {user_path}. \
Include more surrounding context in `old` to make it unique, or set replace_all=true."
);
}
if exact_count == 1 {
content.replacen(old, new, 1)
} else {
let normalized_old = normalize_ws(old);
let (start, end) = find_normalized(&content, &normalized_old)
.ok_or_else(|| anyhow::anyhow!(
"Text not found in {user_path}. \
Call read_file first and copy the text exactly as shown after the '| ' prefix."
))?;
format!("{}{}{}", &content[..start], new, &content[end..])
}
};
let updated = apply_edit(&content, &args, user_path)?;
write_string(user_path, &updated)?;
Ok(format!("Edited {user_path}."))
}
crates/skald-core/src/tools/fs/insert_at_line.rs
+63 -26
View File
@@ -1,13 +1,49 @@
use std::sync::Arc;
use anyhow::Result;
use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL};
use super::{read_to_string, write_string};
use crate::tools::{
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL,
};
use super::{classify_memory, read_to_string, write_string, MemScope};
pub struct InsertAtLine;
pub struct InsertAtLine {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl InsertAtLine {
pub fn new() -> Self { Self }
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
}
/// Inserts `content` before/after `line` in `text`, returning the new text and a
/// result message. Shared by the on-disk `execute` and the `memory/` routing;
/// `display` is the path used in the message.
fn apply_insert(text: &str, args: &Value, display: &str) -> Result<(String, String)> {
let line_num = args["line"].as_u64()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: line"))? as usize;
let new_text = args["content"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: content"))?;
let placement = args["placement"].as_str().unwrap_or("after");
anyhow::ensure!(line_num >= 1, "line must be >= 1");
let mut lines: Vec<&str> = text.split('\n').collect();
let idx = (line_num - 1).min(lines.len().saturating_sub(1));
let insert_idx = if placement == "before" { idx } else { idx + 1 };
let new_lines: Vec<&str> = new_text.split('\n').collect();
for (i, l) in new_lines.iter().enumerate() {
lines.insert(insert_idx + i, l);
}
let updated = lines.join("\n");
let msg = format!(
"Inserted {} line(s) {} line {} in {display}.",
new_lines.len(), placement, line_num
);
Ok((updated, msg))
}
impl Tool for InsertAtLine {
@@ -53,32 +89,33 @@ impl Tool for InsertAtLine {
}
}
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute).
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = super::path_arg(&args).unwrap_or_default();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
Box::new(SimpleExecution::new(Box::pin(async move {
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
anyhow::bail!("No note at {path}");
};
let (updated, msg) = apply_insert(&doc.content, &args, &path)?;
crate::db::memory_docs::upsert(&pool, &rel, &updated).await?;
Ok(ToolResult::Text(msg))
})))
}
fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
let line_num = args["line"].as_u64()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: line"))? as usize;
let new_text = args["content"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: content"))?;
let placement = args["placement"].as_str().unwrap_or("after");
anyhow::ensure!(line_num >= 1, "line must be >= 1");
let text = read_to_string(user_path)?;
let mut lines: Vec<&str> = text.split('\n').collect();
let idx = (line_num - 1).min(lines.len().saturating_sub(1));
let insert_idx = if placement == "before" { idx } else { idx + 1 };
let new_lines: Vec<&str> = new_text.split('\n').collect();
for (i, l) in new_lines.iter().enumerate() {
lines.insert(insert_idx + i, l);
}
let updated = lines.join("\n");
let (updated, msg) = apply_insert(&text, &args, user_path)?;
write_string(user_path, &updated)?;
Ok(format!(
"Inserted {} line(s) {} line {} in {user_path}.",
new_lines.len(), placement, line_num
))
Ok(msg)
}
}
crates/skald-core/src/tools/fs/list_files.rs
+40 -5
View File
@@ -1,20 +1,28 @@
use std::path::Path;
use std::sync::Arc;
use anyhow::Result;
use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT};
use super::resolve;
use crate::tools::{
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT,
};
use super::{classify_memory, resolve, MemScope};
/// Directories to skip unconditionally when walking.
/// `secrets` is skipped so a recursive listing rooted at a parent (e.g. the auto-read
/// working directory) never reveals the contents of the secrets store.
const SKIP_DIRS: &[&str] = &[".git", "target", "node_modules", ".cache", "secrets"];
pub struct ListFiles;
pub struct ListFiles {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl ListFiles {
pub fn new() -> Self { Self }
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
}
impl Tool for ListFiles {
@@ -27,7 +35,8 @@ impl Tool for ListFiles {
Relative paths are resolved from the project root; absolute paths (starting with /) are used as-is. \
Skips .git, target, node_modules, .cache. \
Returns a JSON array of paths relative to the requested directory. \
Use depth=1 for immediate contents only, depth=2-3 for moderate exploration."
Use depth=1 for immediate contents only, depth=2-3 for moderate exploration. \
Listing under user-memory/ (private) or shared-memory/ (shared) lists your memory notes instead of disk."
}
fn parameters_schema(&self) -> Value {
@@ -56,6 +65,32 @@ impl Tool for ListFiles {
truncate_label(&format!("list_files `{path}`"), MAX_LABEL_SHORT)
}
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute). Memory is a
/// flat key space, so `depth`/`dirs_only` don't apply — the whole subtree
/// under the prefix is returned, keyed relative to the requested directory.
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = args["path"].as_str().unwrap_or("").to_string();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
Box::new(SimpleExecution::new(Box::pin(async move {
// Treat `rel` as a directory prefix: match `rel/…` (or everything at
// the root), then strip it so results are relative to what was asked.
let prefix = if rel.is_empty() || rel.ends_with('/') { rel } else { format!("{rel}/") };
let entries = crate::db::memory_docs::list(&pool, &prefix).await?;
let mut paths: Vec<String> = entries.into_iter()
.map(|e| e.path.strip_prefix(&prefix).unwrap_or(&e.path).to_string())
.collect();
paths.sort();
Ok(ToolResult::Text(serde_json::to_string(&paths)?))
})))
}
fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str().unwrap_or(".");
let max_depth = args["depth"].as_u64().unwrap_or(3) as usize;
crates/skald-core/src/tools/fs/memory_search.rs
+125
View File
@@ -0,0 +1,125 @@
//! `memory_search` — full-text search over the virtual memory namespace (§5).
//!
//! Unlike the fs-tools, this does **not** route a path: it searches note *content*
//! through the `memory_docs` FTS5 index (`memory_docs::search`, bm25-ranked with a
//! highlighted snippet). `user-memory` is the caller's own pool (`ToolContext::pool`);
//! `shared-memory` is the system pool captured at registration. Kept a distinct tool
//! rather than folding FTS into `grep_files`: grep is regex-per-line over a tree,
//! this is ranked keyword recall — different semantics, so different names.
use std::sync::Arc;
use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::db::memory_docs::{self, MemoryHit};
use crate::tools::{
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT,
};
pub struct MemorySearch {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl MemorySearch {
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
}
/// Turns free text into a robust FTS5 MATCH query: each whitespace token becomes a
/// quoted term (AND-combined), so arbitrary input — colons, dashes, punctuation —
/// can't trip an FTS5 syntax error. Returns `None` when there are no tokens.
fn fts_query(input: &str) -> Option<String> {
let terms: Vec<String> = input
.split_whitespace()
.map(|t| format!("\"{}\"", t.replace('"', "\"\"")))
.collect();
(!terms.is_empty()).then(|| terms.join(" "))
}
fn render_hits(store: &str, hits: &[MemoryHit], out: &mut String) {
for h in hits {
out.push_str(&format!("[{store}] {} — {}\n", h.path, h.snippet));
}
}
impl Tool for MemorySearch {
fn name(&self) -> &str { "memory_search" }
fn category(&self) -> crate::tools::ToolCategory { crate::tools::ToolCategory::Introspection }
fn description(&self) -> &str {
"Full-text search across your memory notes by keyword, ranked by relevance. \
Searches user-memory/ (private) and shared-memory/ (shared); set scope to narrow it. \
Returns matching note paths with a short highlighted snippet — open one with read_file. \
Use this to recall where you wrote something instead of listing and reading notes one by one."
}
fn parameters_schema(&self) -> Value {
json!({
"type": "object",
"properties": {
"query": { "type": "string", "description": "Keywords to search for. Plain words; all must appear." },
"scope": {
"type": "string",
"enum": ["all", "private", "shared"],
"description": "Which store to search: 'private' (user-memory), 'shared' (shared-memory), or 'all' (default)."
},
"limit": { "type": "integer", "description": "Max results per store (default 10, max 50).", "default": 10 }
},
"required": ["query"]
})
}
fn describe(&self, args: &Value, _length: ToolDescriptionLength) -> String {
let q = args["query"].as_str().unwrap_or("?");
truncate_label(&format!("memory_search \"{q}\""), MAX_LABEL_SHORT)
}
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let user_pool = Arc::clone(&ctx.pool);
let shared_pool = Arc::clone(&self.shared_pool);
Box::new(SimpleExecution::new(Box::pin(async move {
let raw = args["query"].as_str().unwrap_or("");
let Some(q) = fts_query(raw) else {
anyhow::bail!("memory_search needs a non-empty query");
};
let scope = args["scope"].as_str().unwrap_or("all");
let limit = args["limit"].as_u64().unwrap_or(10).clamp(1, 50) as i64;
let mut out = String::new();
let mut total = 0usize;
if scope == "all" || scope == "private" {
let hits = memory_docs::search(&user_pool, &q, limit).await?;
total += hits.len();
render_hits("user-memory", &hits, &mut out);
}
if scope == "all" || scope == "shared" {
let hits = memory_docs::search(&shared_pool, &q, limit).await?;
total += hits.len();
render_hits("shared-memory", &hits, &mut out);
}
if total == 0 {
return Ok(ToolResult::Text(format!("No memory notes match {raw:?}.")));
}
Ok(ToolResult::Text(format!("{total} result(s):\n{out}")))
})))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn fts_query_quotes_tokens_and_survives_punctuation() {
assert_eq!(fts_query("spesa settimana").unwrap(), "\"spesa\" \"settimana\"");
// colons / dashes would be FTS5 operators unquoted; quoting makes them literal
assert_eq!(fts_query("budget: 2026-07").unwrap(), "\"budget:\" \"2026-07\"");
// an embedded quote is escaped by doubling
assert_eq!(fts_query("say \"hi\"").unwrap(), "\"say\" \"\"\"hi\"\"\"");
assert!(fts_query(" ").is_none());
}
}
crates/skald-core/src/tools/fs/mod.rs
+262 -9
View File
@@ -2,15 +2,18 @@ mod edit_file;
mod grep_files;
mod insert_at_line;
mod list_files;
mod memory_search;
mod read_file;
mod replace_lines;
mod search_file;
mod write_file;
use std::path::{Component, Path, PathBuf};
use std::sync::Arc;
use anyhow::{Context, Result};
use serde_json::Value;
use sqlx::SqlitePool;
use crate::tools::ToolRegistry;
@@ -25,11 +28,65 @@ pub use edit_file::EditFile;
pub use grep_files::GrepFiles;
pub use insert_at_line::InsertAtLine;
pub use list_files::ListFiles;
pub use memory_search::MemorySearch;
pub use read_file::ReadFile;
pub use replace_lines::ReplaceLines;
pub use search_file::SearchFile;
pub use write_file::WriteFile;
// ── Virtual memory namespace (blueprint §5) ───────────────────────────────────
//
// Two sibling top-level roots, each backed by the `memory_docs` table in SQLite
// rather than the disk. The fs-tools intercept these prefixes in `run_with` and
// route reads/writes to the `memory_docs` accessor on the right pool, so the LLM
// uses ordinary read/write/list against what looks like two folders.
/// The current user's **private** memory — routed to `ctx.pool` (`{userid}.db`,
/// behind SQLCipher).
pub const USER_MEMORY_ROOT: &str = "user-memory";
/// The instance-wide **shared** memory — routed to the system pool (`system.db`,
/// cleartext, readable by every member).
pub const SHARED_MEMORY_ROOT: &str = "shared-memory";
/// Which memory store a path resolves to.
pub(crate) enum MemScope {
/// `user-memory/…` → the caller's own pool (`ToolContext::pool`).
User,
/// `shared-memory/…` → the shared system pool.
Shared,
}
/// A path that falls inside the virtual memory namespace: the store it belongs to
/// and the note key **relative to that store's root** (the root prefix stripped).
pub(crate) struct MemRef {
pub scope: MemScope,
pub rel: String,
}
/// Classifies a user-supplied path. Returns `Some` when it lands under one of the
/// virtual memory roots — to be routed to SQLite — and `None` for an ordinary
/// disk path.
///
/// The **first** component decides the store, taken raw *before* normalization, so
/// a `..` in the tail can never drop the memory root and silently fall back to a
/// disk path. The tail is then normalized (resolving `.`/`..`) and clamped at the
/// store root, so a memory path stays within its store and an absolute path is
/// always disk.
pub(crate) fn classify_memory(user_path: &str) -> Option<MemRef> {
let mut parts = user_path.trim_start_matches("./").splitn(2, ['/', '\\']);
let scope = match parts.next()? {
USER_MEMORY_ROOT => MemScope::User,
SHARED_MEMORY_ROOT => MemScope::Shared,
_ => return None,
};
// Normalize the tail within the store (empty = the root itself). `..` clamps
// at the root rather than escaping upward.
let tail = parts.next().unwrap_or("");
let rel = lexical_normalize(Path::new(tail)).to_string_lossy().replace('\\', "/");
Some(MemRef { scope, rel })
}
/// Resolve a user-supplied path:
/// - starts with `/` → absolute path, used as-is
/// - otherwise → relative to the process working directory (project root)
@@ -127,13 +184,209 @@ pub(super) fn write_string(user_path: &str, content: &str) -> Result<()> {
.with_context(|| format!("Failed to write: {}", abs.display()))
}
pub fn register_all(registry: &mut ToolRegistry) {
registry.register(EditFile::new());
registry.register(GrepFiles::new());
registry.register(InsertAtLine::new());
registry.register(ListFiles::new());
registry.register(ReadFile::new());
registry.register(ReplaceLines::new());
registry.register(SearchFile::new());
registry.register(WriteFile::new());
/// Registers the filesystem tools. `shared_pool` is the system (`shared-memory`)
/// pool captured once here — a global singleton — and handed to the memory-aware
/// tools; each still resolves the per-user (`user-memory`) pool per call from the
/// `ToolContext`.
pub fn register_all(registry: &mut ToolRegistry, shared_pool: Arc<SqlitePool>) {
registry.register(EditFile::new(Arc::clone(&shared_pool)));
registry.register(GrepFiles::new()); // not memory-aware yet — see blueprint Prossimi passi
registry.register(InsertAtLine::new(Arc::clone(&shared_pool)));
registry.register(ListFiles::new(Arc::clone(&shared_pool)));
registry.register(ReadFile::new(Arc::clone(&shared_pool)));
registry.register(ReplaceLines::new(Arc::clone(&shared_pool)));
registry.register(SearchFile::new(Arc::clone(&shared_pool)));
registry.register(MemorySearch::new(Arc::clone(&shared_pool)));
registry.register(WriteFile::new(shared_pool));
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::PathBuf;
use serde_json::json;
use crate::tools::{ExecutionOutcome, Tool, ToolContext};
#[test]
fn classify_memory_splits_root_from_key() {
let u = classify_memory("user-memory/notes/x.md").unwrap();
assert!(matches!(u.scope, MemScope::User));
assert_eq!(u.rel, "notes/x.md");
let s = classify_memory("./shared-memory/casa.md").unwrap();
assert!(matches!(s.scope, MemScope::Shared));
assert_eq!(s.rel, "casa.md");
// bare roots (with/without trailing slash) resolve to the empty key
assert_eq!(classify_memory("user-memory").unwrap().rel, "");
assert_eq!(classify_memory("shared-memory/").unwrap().rel, "");
// `..` clamps inside the store instead of falling back to a disk path
assert_eq!(classify_memory("user-memory/../secret.md").unwrap().rel, "secret.md");
// ordinary, absolute, and look-alike paths are disk (None)
assert!(classify_memory("src/main.rs").is_none());
assert!(classify_memory("/etc/hosts").is_none());
assert!(classify_memory("user-memoryish/x").is_none());
}
/// A throwaway owner-schema pool (as `Arc`, ready for a `ToolContext`), plus its
/// dir for cleanup. `tag` + a counter keep parallel tests off the same file.
async fn store(tag: &str) -> (Arc<SqlitePool>, PathBuf) {
use std::sync::atomic::{AtomicU64, Ordering};
static SEQ: AtomicU64 = AtomicU64::new(0);
let n = SEQ.fetch_add(1, Ordering::Relaxed);
let dir = std::env::temp_dir().join(format!("skald-fsmem-{}-{tag}-{n}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let pool = crate::db::create_user_pool(&dir.join("owner.db"), None).await.unwrap();
(Arc::new(pool), dir)
}
/// Drives a tool through the context-aware path and returns its text result.
async fn drive(tool: &dyn Tool, ctx: &ToolContext, args: Value) -> Result<String, String> {
let exec = tool.run_with(ctx, args);
match exec.wait().await {
ExecutionOutcome::Completed(r) => Ok(r.to_wire()),
ExecutionOutcome::Failed(e) => Err(e),
ExecutionOutcome::Cancelled => Err("cancelled".into()),
}
}
#[tokio::test]
async fn memory_tools_route_and_isolate_user_vs_shared() {
let (user, udir) = store("user").await;
let (shared, sdir) = store("shared").await;
// The shared pool is captured by the tools; the user pool arrives per call.
let write = WriteFile::new(Arc::clone(&shared));
let read = ReadFile::new(Arc::clone(&shared));
let list = ListFiles::new(Arc::clone(&shared));
let ctx = ToolContext { session_id: 1, pool: Arc::clone(&user) };
// Private write lands in the user pool — and never in the shared one.
let out = drive(&write, &ctx, json!({"path":"user-memory/spesa.md","content":"latte\npane"}))
.await.unwrap();
assert!(out.starts_with("Created user-memory/spesa.md"), "{out}");
assert!(crate::db::memory_docs::get(&user, "spesa.md").await.unwrap().is_some());
assert!(crate::db::memory_docs::get(&shared, "spesa.md").await.unwrap().is_none(),
"a user-memory write must not touch the shared store");
// Shared write lands in the shared pool — and never in the user one.
drive(&write, &ctx, json!({"path":"shared-memory/casa.md","content":"wifi 1234"}))
.await.unwrap();
assert!(crate::db::memory_docs::get(&shared, "casa.md").await.unwrap().is_some());
assert!(crate::db::memory_docs::get(&user, "casa.md").await.unwrap().is_none());
// Read back with 1-based line numbers; a missing note errors.
let r = drive(&read, &ctx, json!({"path":"user-memory/spesa.md"})).await.unwrap();
assert!(r.contains("| latte") && r.contains("| pane"), "{r}");
assert!(drive(&read, &ctx, json!({"path":"user-memory/nope.md"})).await.is_err());
// A second write to the same key overwrites (and says so).
let out = drive(&write, &ctx, json!({"path":"user-memory/spesa.md","content":"latte"}))
.await.unwrap();
assert!(out.starts_with("Overwrote user-memory/spesa.md"), "{out}");
// Listing returns keys relative to the requested directory.
drive(&write, &ctx, json!({"path":"user-memory/notes/idee.md","content":"x"}))
.await.unwrap();
let l = drive(&list, &ctx, json!({"path":"user-memory"})).await.unwrap();
assert_eq!(serde_json::from_str::<Vec<String>>(&l).unwrap(),
vec!["notes/idee.md".to_string(), "spesa.md".to_string()]);
let l = drive(&list, &ctx, json!({"path":"user-memory/notes"})).await.unwrap();
assert_eq!(serde_json::from_str::<Vec<String>>(&l).unwrap(),
vec!["idee.md".to_string()]);
let _ = std::fs::remove_dir_all(&udir);
let _ = std::fs::remove_dir_all(&sdir);
}
#[tokio::test]
async fn memory_edit_insert_replace_search_route_to_the_note() {
let (user, udir) = store("edit-user").await;
let (shared, sdir) = store("edit-shared").await;
let write = WriteFile::new(Arc::clone(&shared));
let edit = EditFile::new(Arc::clone(&shared));
let insert = InsertAtLine::new(Arc::clone(&shared));
let replace = ReplaceLines::new(Arc::clone(&shared));
let search = SearchFile::new(Arc::clone(&shared));
let ctx = ToolContext { session_id: 1, pool: Arc::clone(&user) };
async fn note(pool: &SqlitePool, path: &str) -> String {
crate::db::memory_docs::get(pool, path).await.unwrap().unwrap().content
}
drive(&write, &ctx, json!({"path":"user-memory/todo.md","content":"latte\npane\nuvoa"}))
.await.unwrap();
// edit_file: fix the typo, in place
let out = drive(&edit, &ctx, json!({"path":"user-memory/todo.md","old":"uvoa","new":"uova"}))
.await.unwrap();
assert_eq!(out, "Edited user-memory/todo.md.");
assert_eq!(note(&user, "todo.md").await, "latte\npane\nuova");
// insert_at_line: add a line after line 1
drive(&insert, &ctx, json!({"path":"user-memory/todo.md","line":1,"content":"burro","placement":"after"}))
.await.unwrap();
assert_eq!(note(&user, "todo.md").await, "latte\nburro\npane\nuova");
// replace_lines: collapse lines 2–3 into one
drive(&replace, &ctx, json!({"path":"user-memory/todo.md","from_line":2,"to_line":3,"new":"olio"}))
.await.unwrap();
assert_eq!(note(&user, "todo.md").await, "latte\nolio\nuova");
// search_file: find a line inside the note
let s = drive(&search, &ctx, json!({"path":"user-memory/todo.md","query":"olio"})).await.unwrap();
assert!(s.contains("match(es) in user-memory/todo.md"), "{s}");
assert!(s.contains("| olio"), "{s}");
// editing a note that doesn't exist errors, not creates
assert!(drive(&edit, &ctx, json!({"path":"user-memory/ghost.md","old":"a","new":"b"}))
.await.is_err());
let _ = std::fs::remove_dir_all(&udir);
let _ = std::fs::remove_dir_all(&sdir);
}
#[tokio::test]
async fn memory_search_scopes_to_user_shared_or_all() {
let (user, udir) = store("search-user").await;
let (shared, sdir) = store("search-shared").await;
let write = WriteFile::new(Arc::clone(&shared));
let search = MemorySearch::new(Arc::clone(&shared));
let ctx = ToolContext { session_id: 1, pool: Arc::clone(&user) };
// one note in each store, both mentioning "wifi"
drive(&write, &ctx, json!({"path":"user-memory/rete.md","content":"la mia wifi privata"}))
.await.unwrap();
drive(&write, &ctx, json!({"path":"shared-memory/casa.md","content":"wifi di casa 1234"}))
.await.unwrap();
// scope=private → only the user store
let r = drive(&search, &ctx, json!({"query":"wifi","scope":"private"})).await.unwrap();
assert!(r.contains("[user-memory] rete.md"), "{r}");
assert!(!r.contains("shared-memory"), "{r}");
// scope=shared → only the shared store
let r = drive(&search, &ctx, json!({"query":"wifi","scope":"shared"})).await.unwrap();
assert!(r.contains("[shared-memory] casa.md"), "{r}");
assert!(!r.contains("[user-memory]"), "{r}");
// scope=all (default) → both, and the snippet highlights the term
let r = drive(&search, &ctx, json!({"query":"wifi"})).await.unwrap();
assert!(r.contains("[user-memory] rete.md") && r.contains("[shared-memory] casa.md"), "{r}");
assert!(r.contains("[wifi]"), "snippet should highlight the match: {r}");
// no match → a friendly message, not an error
let r = drive(&search, &ctx, json!({"query":"inesistente"})).await.unwrap();
assert!(r.starts_with("No memory notes match"), "{r}");
let _ = std::fs::remove_dir_all(&udir);
let _ = std::fs::remove_dir_all(&sdir);
}
}
crates/skald-core/src/tools/fs/read_file.rs
+68 -32
View File
@@ -1,13 +1,50 @@
use std::sync::Arc;
use anyhow::Result;
use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL};
use super::read_to_string;
use crate::tools::{
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL,
};
use super::{classify_memory, read_to_string, MemScope};
pub struct ReadFile;
pub struct ReadFile {
/// The `shared-memory` (system) pool. `user-memory` resolves per call from the
/// `ToolContext`; only the shared store is a global singleton captured here.
shared_pool: Arc<SqlitePool>,
}
impl ReadFile {
pub fn new() -> Self { Self }
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
}
/// Render `content` with 1-based line numbers, honouring the same
/// `start`/`end_line`/`limit` windowing as the disk path. Shared by the on-disk
/// [`ReadFile::execute`] and the `memory/` routing in [`ReadFile::run_with`].
fn number_lines(content: &str, start: usize, end_line: Option<usize>, limit: Option<usize>) -> String {
let lines: Vec<&str> = content.lines().collect();
let total = lines.len();
let end = match (end_line, limit) {
(Some(e), _) => e.min(total),
(None, Some(l)) => (start + l).min(total),
(None, None) => total,
};
if start >= total && total > 0 {
return format!("(file has only {total} lines; start_line {} is out of range)", start + 1);
}
let end = end.max(start);
let width = total.to_string().len().max(3);
lines[start..end]
.iter()
.enumerate()
.map(|(i, line)| format!("{:>width$} | {line}", start + i + 1))
.collect::<Vec<_>>()
.join("\n")
}
impl Tool for ReadFile {
@@ -19,7 +56,8 @@ impl Tool for ReadFile {
Use instead of cat/head/tail in the terminal. \
Returns text prefixed as ' N | line'. When calling edit_file, copy the text after '| ' exactly. \
For large files use start_line/end_line to read in chunks — files over ~2000 lines should never be read whole. \
Use limit to cap output when end_line is unknown."
Use limit to cap output when end_line is unknown. \
Paths under user-memory/ (private) or shared-memory/ (shared) read a note from your memory instead of disk."
}
fn parameters_schema(&self) -> Value {
@@ -70,37 +108,35 @@ impl Tool for ReadFile {
}
}
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute).
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = super::path_arg(&args).unwrap_or_default();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
let start = args["start_line"].as_u64().map(|n| (n as usize).saturating_sub(1)).unwrap_or(0);
let end_line = args["end_line"].as_u64().map(|n| n as usize);
let limit = args["limit"].as_u64().map(|n| n.min(2000) as usize);
Box::new(SimpleExecution::new(Box::pin(async move {
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
anyhow::bail!("No note at {path}");
};
Ok(ToolResult::Text(number_lines(&doc.content, start, end_line, limit)))
})))
}
fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
let content = read_to_string(user_path)?;
let lines: Vec<&str> = content.lines().collect();
let total = lines.len();
let start = args["start_line"].as_u64().map(|n| (n as usize).saturating_sub(1)).unwrap_or(0);
let end_line = args["end_line"].as_u64().map(|n| n as usize);
let limit = args["limit"].as_u64().map(|n| n.min(2000) as usize);
let start = args["start_line"].as_u64()
.map(|n| (n as usize).saturating_sub(1))
.unwrap_or(0);
let end = match (args["end_line"].as_u64(), limit) {
(Some(e), _) => (e as usize).min(total),
(None, Some(l)) => (start + l).min(total),
(None, None) => total,
};
if start >= total && total > 0 {
return Ok(format!("(file has only {total} lines; start_line {start_line} is out of range)",
start_line = start + 1));
}
let end = end.max(start);
let width = total.to_string().len().max(3);
let numbered = lines[start..end]
.iter()
.enumerate()
.map(|(i, line)| format!("{:>width$} | {line}", start + i + 1))
.collect::<Vec<_>>()
.join("\n");
Ok(numbered)
Ok(number_lines(&content, start, end_line, limit))
}
}
crates/skald-core/src/tools/fs/replace_lines.rs
+70 -32
View File
@@ -1,13 +1,56 @@
use std::sync::Arc;
use anyhow::Result;
use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL};
use super::{read_to_string, write_string};
use crate::tools::{
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL,
};
use super::{classify_memory, read_to_string, write_string, MemScope};
pub struct ReplaceLines;
pub struct ReplaceLines {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl ReplaceLines {
pub fn new() -> Self { Self }
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
}
/// Replaces the inclusive 1-based line range with `new`, returning the new content
/// and a result message. Shared by the on-disk `execute` and the `memory/` routing;
/// `display` is the path used in the message.
fn apply_replace(content: &str, args: &Value, display: &str) -> Result<(String, String)> {
let from_line = args["from_line"].as_u64()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: from_line"))? as usize;
let to_line = args["to_line"].as_u64()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: to_line"))? as usize;
let new = args["new"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: new"))?;
if from_line == 0 { anyhow::bail!("from_line must be >= 1"); }
if to_line < from_line { anyhow::bail!("to_line must be >= from_line"); }
let mut lines: Vec<&str> = content.lines().collect();
let total = lines.len();
if from_line > total {
anyhow::bail!("from_line {from_line} exceeds file length ({total} lines)");
}
let to_clamped = to_line.min(total);
let new_lines: Vec<&str> = new.lines().collect();
lines.splice((from_line - 1)..to_clamped, new_lines);
let has_trailing = content.ends_with('\n');
let mut updated = lines.join("\n");
if has_trailing { updated.push('\n'); }
let msg = format!(
"Replaced lines {from_line}–{to_clamped} in {display} with {} new lines.",
new.lines().count()
);
Ok((updated, msg))
}
impl Tool for ReplaceLines {
@@ -51,38 +94,33 @@ impl Tool for ReplaceLines {
}
}
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute).
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = super::path_arg(&args).unwrap_or_default();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
Box::new(SimpleExecution::new(Box::pin(async move {
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
anyhow::bail!("No note at {path}");
};
let (updated, msg) = apply_replace(&doc.content, &args, &path)?;
crate::db::memory_docs::upsert(&pool, &rel, &updated).await?;
Ok(ToolResult::Text(msg))
})))
}
fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
let from_line = args["from_line"].as_u64()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: from_line"))? as usize;
let to_line = args["to_line"].as_u64()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: to_line"))? as usize;
let new = args["new"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: new"))?;
if from_line == 0 { anyhow::bail!("from_line must be >= 1"); }
if to_line < from_line { anyhow::bail!("to_line must be >= from_line"); }
let content = read_to_string(user_path)?;
let mut lines: Vec<&str> = content.lines().collect();
let total = lines.len();
if from_line > total {
anyhow::bail!("from_line {from_line} exceeds file length ({total} lines)");
}
let to_clamped = to_line.min(total);
let new_lines: Vec<&str> = new.lines().collect();
lines.splice((from_line - 1)..to_clamped, new_lines);
let has_trailing = content.ends_with('\n');
let mut updated = lines.join("\n");
if has_trailing { updated.push('\n'); }
let (updated, msg) = apply_replace(&content, &args, user_path)?;
write_string(user_path, &updated)?;
Ok(format!(
"Replaced lines {from_line}–{to_clamped} in {user_path} with {} new lines.",
new.lines().count()
))
Ok(msg)
}
}
crates/skald-core/src/tools/fs/search_file.rs
+78 -43
View File
@@ -1,13 +1,67 @@
use std::sync::Arc;
use anyhow::Result;
use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL};
use super::read_to_string;
use crate::tools::{
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL,
};
use super::{classify_memory, read_to_string, MemScope};
pub struct SearchFile;
pub struct SearchFile {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl SearchFile {
pub fn new() -> Self { Self }
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
}
/// Renders the case-insensitive substring search over `text` with context lines.
/// Shared by the on-disk `execute` and the `memory/` routing; `display` is the
/// path shown in the output.
fn render_search(text: &str, args: &Value, display: &str) -> Result<String> {
let query = args["query"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: query"))?;
let context = args["context_lines"].as_u64().unwrap_or(3).min(10) as usize;
let lines: Vec<&str> = text.lines().collect();
let lower_query = query.to_lowercase();
let width = lines.len().to_string().len().max(3);
let matches: Vec<usize> = lines.iter().enumerate()
.filter(|(_, l)| l.to_lowercase().contains(&lower_query))
.map(|(i, _)| i)
.collect();
if matches.is_empty() {
return Ok(format!("No matches found for {:?} in {display}.", query));
}
let mut chunks: Vec<(usize, usize)> = Vec::new();
for &m in &matches {
let start = m.saturating_sub(context);
let end = (m + context).min(lines.len() - 1);
if let Some(last) = chunks.last_mut() {
if start <= last.1 + 1 { last.1 = last.1.max(end); continue; }
}
chunks.push((start, end));
}
let match_set: std::collections::HashSet<usize> = matches.into_iter().collect();
let mut out = format!("{} match(es) in {display}:\n", match_set.len());
for (ci, (start, end)) in chunks.iter().enumerate() {
if ci > 0 { out.push_str(" ···\n"); }
for idx in *start..=*end {
let marker = if match_set.contains(&idx) { ">" } else { " " };
out.push_str(&format!("{marker}{:>width$} | {}\n", idx + 1, lines[idx]));
}
}
Ok(out)
}
impl Tool for SearchFile {
@@ -53,48 +107,29 @@ impl Tool for SearchFile {
}
}
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute).
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = super::path_arg(&args).unwrap_or_default();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
Box::new(SimpleExecution::new(Box::pin(async move {
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
anyhow::bail!("No note at {path}");
};
Ok(ToolResult::Text(render_search(&doc.content, &args, &path)?))
})))
}
fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
let query = args["query"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: query"))?;
let context = args["context_lines"].as_u64().unwrap_or(3).min(10) as usize;
let text = read_to_string(user_path)?;
let lines: Vec<&str> = text.lines().collect();
let lower_query = query.to_lowercase();
let width = lines.len().to_string().len().max(3);
let matches: Vec<usize> = lines.iter().enumerate()
.filter(|(_, l)| l.to_lowercase().contains(&lower_query))
.map(|(i, _)| i)
.collect();
if matches.is_empty() {
return Ok(format!("No matches found for {:?} in {user_path}.", query));
}
let mut chunks: Vec<(usize, usize)> = Vec::new();
for &m in &matches {
let start = m.saturating_sub(context);
let end = (m + context).min(lines.len() - 1);
if let Some(last) = chunks.last_mut() {
if start <= last.1 + 1 { last.1 = last.1.max(end); continue; }
}
chunks.push((start, end));
}
let match_set: std::collections::HashSet<usize> = matches.into_iter().collect();
let mut out = format!("{} match(es) in {user_path}:\n", match_set.len());
for (ci, (start, end)) in chunks.iter().enumerate() {
if ci > 0 { out.push_str(" ···\n"); }
for idx in *start..=*end {
let marker = if match_set.contains(&idx) { ">" } else { " " };
out.push_str(&format!("{marker}{:>width$} | {}\n", idx + 1, lines[idx]));
}
}
Ok(out)
render_search(&text, &args, user_path)
}
}
crates/skald-core/src/tools/fs/write_file.rs
+39 -5
View File
@@ -1,13 +1,22 @@
use std::sync::Arc;
use anyhow::Result;
use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT};
use super::{resolve, write_string};
use crate::tools::{
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT,
};
use super::{classify_memory, resolve, write_string, MemScope};
pub struct WriteFile;
pub struct WriteFile {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl WriteFile {
pub fn new() -> Self { Self }
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
}
impl Tool for WriteFile {
@@ -18,7 +27,8 @@ impl Tool for WriteFile {
"Create a new file or fully overwrite an existing one. \
Use instead of echo/cat heredoc in the terminal. \
Relative paths are resolved from the project root; absolute paths (starting with /) are used as-is. \
OVERWRITES the entire file — for targeted edits to an existing file use edit_file instead."
OVERWRITES the entire file — for targeted edits to an existing file use edit_file instead. \
Write Markdown under user-memory/ (private to you) or shared-memory/ (shared with everyone) to save a durable note in your memory instead of on disk."
}
fn parameters_schema(&self) -> Value {
@@ -48,6 +58,30 @@ impl Tool for WriteFile {
truncate_label(&format!("write_file `{path}`"), MAX_LABEL_SHORT)
}
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute).
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = super::path_arg(&args).unwrap_or_default();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
let content = args["content"].as_str().map(str::to_string);
Box::new(SimpleExecution::new(Box::pin(async move {
let content = content.ok_or_else(|| anyhow::anyhow!("Missing required argument: content"))?;
if rel.is_empty() {
anyhow::bail!("{path} is a memory root, not a note — write to a path like {path}/notes.md");
}
let existed = crate::db::memory_docs::get(&pool, &rel).await?.is_some();
crate::db::memory_docs::upsert(&pool, &rel, &content).await?;
let verb = if existed { "Overwrote" } else { "Created" };
Ok(ToolResult::Text(format!("{verb} {path} ({} bytes).", content.len())))
})))
}
fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
src/frontend/api/config.rs
+2 -14
View File
@@ -9,7 +9,6 @@ use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use core_api::PropertyType;
use core_api::system_bus::SystemEvent;
use skald_core::skald::Skald;
use super::ApiError;
@@ -108,20 +107,9 @@ pub async fn set_property(
return Err(ApiError::not_found("unknown config key"));
}
let old_value = skald.config().get(&p.key).await?;
// No-op if value didn't change.
if old_value.as_deref() == Some(body.value.as_str()) {
return Ok(StatusCode::OK);
}
// GlobalConfigManager::set handles the no-op check and emits
// ConfigKeyUpdated on the system bus when the value actually changes.
skald.config().set(&p.key, &body.value).await?;
skald.system_bus().send(SystemEvent::ConfigKeyUpdated {
key: p.key.clone(),
old_value,
new_value: body.value,
});
Ok(StatusCode::OK)
}
src/main.rs
+1 -4
View File
@@ -164,12 +164,9 @@ pub async fn run_backend() -> Result<Backend> {
/// Build the plugin list. Extracted so both entry points share the same set.
fn build_plugins() -> Vec<Arc<dyn Plugin>> {
// NOTE (multi-user slice): telegram-bot, mobile-connector and honcho are
// single-user / global-ChatEventBus-subscriber plugins. They assume one user
// and cannot work under per-user isolation yet, so they are dropped from the
// build until they become multi-user-aware. See blueprint §17 and the plan.
let mut plugins: Vec<Arc<dyn Plugin>> = vec![
Arc::new(plugin_tailscale_remote::RemotePlugin::new()),
Arc::new(plugin_telegram_bot::TelegramPlugin::new()),
Arc::new(plugin_comfyui::ComfyUIPlugin::new()),
Arc::new(plugin_tts_orpheus_3b::OrpheusTtsPlugin::new()),
Arc::new(plugin_tts_kokoro::KokoroTtsPlugin::new()),