feat(memory): dual-pool memory namespace, FTS search, and prompt injection

Add a virtual memory namespace backed by SQLite, surfaced through the
fs-tools, with private (per-user) and shared (system) stores.

Storage
- `memory_docs` owner table + external-content FTS5 index with sync triggers.
- `db/memory_docs.rs` accessor: get / upsert / list / search (bm25+snippet) / delete.

Routing (tools/fs)
- `classify_memory` splits paths on the raw first component; `..` clamps inside
  the store, never escaping to disk.
- read/write/list/edit/insert/replace/search_file route `user-memory/` to the
  owner pool and `shared-memory/` to the system pool (a singleton captured in
  `register_all`); every other path stays on disk. Each tool extracts a pure
  transform shared between its disk and memory paths.
- New `memory_search` tool over the FTS index (scope private/shared/all),
  with a sanitised FTS5 query. grep_files stays disk-only.

Approval
- `user-memory/*` allow (read+write); `shared-memory/*` reads allow,
  writes require approval so the agent can't silently push one person's data
  into shared memory. `memory_search` allowed via a path-less rule.
- migrate away the old `memory/*` and blanket `shared-memory/*` rows.

Prompt injection
- `MessageBuilder::load_inject_memory` reads `user-memory/` (owner pool) and
  `shared-memory/` (system pool) inject entries from SQLite; disk paths
  unchanged. The system pool is threaded ChatSessionManager -> handler ->
  MessageBuilder.
- main and project-coordinator inject `user-memory/index.md` +
  `shared-memory/index.md`; common/memory.md rewritten for the two stores.
This commit is contained in:
2026-07-11 02:08:13 +01:00
parent 5848829a92
commit a847dda88f
25 changed files with 1249 additions and 247 deletions
+6 -2
View File
@@ -69,7 +69,7 @@ Two rules keep the boundary real, and both are enforced by the compiler:
| `crates/skald-core/src/chat_hub/` | `ChatHub`: broadcast events to all connected WS clients | | `crates/skald-core/src/chat_hub/` | `ChatHub`: broadcast events to all connected WS clients |
| `crates/skald-core/src/chat_event_bus.rs` | Global async bus for cross-session events | | `crates/skald-core/src/chat_event_bus.rs` | Global async bus for cross-session events |
| `crates/skald-core/src/agents.rs` | Discovers agents from `agents/*/`, loads meta + system prompt | | `crates/skald-core/src/agents.rs` | Discovers agents from `agents/*/`, loads meta + system prompt |
| `crates/skald-core/src/tools/` | Built-in tools: `exec`, `restart`, `list_agents`, `fs/*`, `notify`, `ast_outline`, `image_generate`, MCP tools, plugin tools, cron tools | | `crates/skald-core/src/tools/` | Built-in tools: `exec`, `restart`, `list_agents`, `fs/*` (also route `user-memory/`/`shared-memory/` to `memory_docs` — see DB tables), `notify`, `ast_outline`, `image_generate`, MCP tools, plugin tools, cron tools |
| `crates/skald-core/src/tool_catalog.rs` | `ToolCatalog`: unified tool listing façade (wraps ToolRegistry + McpManager) | | `crates/skald-core/src/tool_catalog.rs` | `ToolCatalog`: unified tool listing façade (wraps ToolRegistry + McpManager) |
| `crates/skald-core/src/events.rs` | `ServerEvent` enum streamed over WebSocket to the frontend | | `crates/skald-core/src/events.rs` | `ServerEvent` enum streamed over WebSocket to the frontend |
| `crates/skald-core/src/db/` | sqlx SQLite — see below | | `crates/skald-core/src/db/` | sqlx SQLite — see below |
@@ -100,10 +100,14 @@ Two rules keep the boundary real, and both are enforced by the compiler:
The schema is split into two buckets (§5.1), and the split is the point: The schema is split into two buckets (§5.1), and the split is the point:
- **`create_registry_tables`** — instance-wide, readable without any user key: `users`, `llm_providers`, `llm_models`, `transcribe_models`, `tts_models`, `image_generate_models`, `plugins`, `approval_rules`, `tool_permission_groups`, `config`, `known_tools`, `llm_requests`. - **`create_registry_tables`** — instance-wide, readable without any user key: `users`, `llm_providers`, `llm_models`, `transcribe_models`, `tts_models`, `image_generate_models`, `plugins`, `approval_rules`, `tool_permission_groups`, `config`, `known_tools`, `llm_requests`.
- **`create_owner_tables`** — one owner's content, **identical schema in every file that has it**: `chat_sessions`, `chat_sessions_stack`, `chat_history`, `chat_llm_tools`, `chat_summaries`, `session_scratchpad`, `session_mcp_grants`, `stack_mcp_grants`, `scheduled_jobs`, `job_runs`, `mcp_servers`, `mcp_events`, `sources`, `secrets`, `projects`, `project_tickets`. - **`create_owner_tables`** — one owner's content, **identical schema in every file that has it**: `chat_sessions`, `chat_sessions_stack`, `chat_history`, `chat_llm_tools`, `chat_summaries`, `session_scratchpad`, `session_mcp_grants`, `stack_mcp_grants`, `scheduled_jobs`, `job_runs`, `mcp_servers`, `mcp_events`, `sources`, `secrets`, `projects`, `project_tickets`, `llm_request_payloads`, `memory_docs` (+ FTS5 `memory_docs_fts`). Because `memory_docs` is an owner table, one definition backs **private** memory in each `{userid}.db` and **shared** memory in `system.db` (the household owner) — see the memory namespace note below.
**No foreign key in the owner bucket may point at a registry table.** SQLite cannot enforce a key across files, not even through `ATTACH`, and sqlx turns on `PRAGMA foreign_keys`: the `CREATE TABLE` succeeds and every `INSERT` fails. `db::tests::owner_tables_stand_alone_with_foreign_keys_on` enforces this by running the owner schema against a database holding nothing else, then inserting a row into each table. Two keys crossed and were fixed: `chat_history.model_db_id` (dropped — write-only, and `llm_requests.model_name` already records the model) and `project_tickets.job_id` (fixed by moving `projects`/`project_tickets` into the owner bucket). **No foreign key in the owner bucket may point at a registry table.** SQLite cannot enforce a key across files, not even through `ATTACH`, and sqlx turns on `PRAGMA foreign_keys`: the `CREATE TABLE` succeeds and every `INSERT` fails. `db::tests::owner_tables_stand_alone_with_foreign_keys_on` enforces this by running the owner schema against a database holding nothing else, then inserting a row into each table. Two keys crossed and were fixed: `chat_history.model_db_id` (dropped — write-only, and `llm_requests.model_name` already records the model) and `project_tickets.job_id` (fixed by moving `projects`/`project_tickets` into the owner bucket).
**Memory namespace (blueprint §5).** `memory_docs` (accessor `db/memory_docs.rs``get`/`upsert`/`list`/`search`(FTS)/`delete`) backs a virtual note store surfaced through the fs-tools, **not** the disk. Two sibling roots (not the blueprint's nested `memory/{userid}` + `memory/shared`): `user-memory/…` routes to the caller's own pool (`ToolContext::pool`), `shared-memory/…` to the system pool (a singleton captured in `fs::register_all`). `tools/fs/classify_memory()` decides on the raw first path component (a `..` in the tail clamps inside the store, never escapes to disk); `read_file`/`write_file`/`list_files`/`edit_file`/`insert_at_line`/`replace_lines`/`search_file` override `run_with` to route memory paths (each extracting a pure transform shared with its on-disk `execute`) and leave every other path on disk. Approval (seeded in `seed_fs_path_rules`): `user-memory/*` is `@fs_any allow` (private, frictionless); `shared-memory/*` is `@fs_read allow` + `@fs_write require` — reads free, **writes need approval** so the agent can't silently push one person's data into shared memory. `grep_files` stays disk-only (regex-across-tree ≠ FTS); ranked full-text recall over notes is a separate tool, `memory_search` (`tools/fs/memory_search.rs`), over the `memory_docs` FTS index — allowed by a path-less rule (it takes `query`, not `path`).
**Memory injection into the prompt**: `MessageBuilder::load_inject_memory` routes each `meta.inject_memory` entry — `user-memory/…` → owner pool, `shared-memory/…` → the shared (`system.db`) pool, both via `memory_docs::get`; anything else (`data/…`, `$WD/…`) is a disk read. The shared pool is threaded `ChatSessionManager` → handler → `MessageBuilder`. `main` and `project-coordinator` inject `user-memory/index.md` + `shared-memory/index.md`.
`system.db` currently gets **both** bucket functions, because nothing has migrated to per-user pools yet. That is transitional. `system.db` currently gets **both** bucket functions, because nothing has migrated to per-user pools yet. That is transitional.
`users` (`crates/skald-core/src/db/users.rs`) holds the directory plus auth material. It lives in the system DB, which the box owner can read, so it must never store anything that derives a user's key. `Credentials` is an enum mirroring the table's `CHECK`: an encrypted user carries a **wrapped DEK** (whose AEAD tag *is* the password verifier — hence no `password_hash`); a cleartext user carries an ordinary verifier, or none. `User` is deliberately not `Serialize` and its `Debug` redacts key material — use `User::summary()` for anything leaving the process. `role_id` has no foreign key yet: sqlx enables `PRAGMA foreign_keys`, so referencing the not-yet-existing `roles` table would fail every insert. `users` (`crates/skald-core/src/db/users.rs`) holds the directory plus auth material. It lives in the system DB, which the box owner can read, so it must never store anything that derives a user's key. `Credentials` is an enum mirroring the table's `CHECK`: an encrypted user carries a **wrapped DEK** (whose AEAD tag *is* the password verifier — hence no `password_hash`); a cleartext user carries an ordinary verifier, or none. `User` is deliberately not `Serialize` and its `Debug` redacts key material — use `User::summary()` for anything leaving the process. `role_id` has no foreign key yet: sqlx enables `PRAGMA foreign_keys`, so referencing the not-yet-existing `roles` table would fail every insert.
+30 -10
View File
@@ -1,20 +1,41 @@
# Persistent memory # Persistent memory
All memory lives in `data/memory/`. Entry point: `data/memory/index.md` — one line per file with a brief summary. You have two persistent note stores, kept as Markdown and searchable. **Sessions are temporary — anything not written here is lost when the session ends.** Save proactively.
- **`user-memory/`** — your **private** memory for this user. Nobody else can read it. Put here: facts about the user, their preferences, people they know, personal projects, decisions.
- **`shared-memory/`** — memory **shared with the whole group**. Every member can read it. Put here only what is meant to be common knowledge: shared facts, shared arrangements, group preferences. **Never** put one person's private information here. Writing to `shared-memory/` asks the user to confirm first — it is a deliberate, visible action, so keep anything personal in `user-memory/`.
When unsure where something belongs, prefer `user-memory/`.
## The indexes
Each store has an `index.md` — one line per note with a brief summary — and **both are injected into your context automatically** at the start of each session (look for them below):
- `user-memory/index.md` — your private notes.
- `shared-memory/index.md` — the group's shared notes.
Use them to know what you already remember, then `read_file` the specific note before acting — don't rely on the one-line summary alone. **Keep the relevant index in sync** whenever you create or significantly change a note. Updating `shared-memory/index.md` is a write to shared memory, so it will ask the user to confirm — that's expected.
## When to save ## When to save
Save **immediately** (do not postpone) when: Save **immediately** (do not postpone) when:
- The user shares a new fact about themselves, a project, a person, or a preference - The user shares a new fact about themselves, a project, a person, or a preference
- A decision is made that may be relevant in future sessions - A decision is made that may matter in a future session
- You notice an inconsistency with what was previously saved → correct it - You notice that something you saved before is now wrong → correct it
## When to read ## When to read
At the start of each session, read `data/memory/index.md` silently. Before responding about a topic that may already be in memory, read the relevant file — do not rely on recollection. Before responding about a topic that may already be in memory, look it up — do not rely on recollection:
## File format - The injected `user-memory/index.md` tells you what exists; `read_file` the note it points to.
- `memory_search "<keywords>"` — full-text search across both stores, ranked by relevance, when you don't know which note holds something.
## Organising notes
Use clear, topic-based paths — e.g. `user-memory/people/alice.md`, `user-memory/projects/website.md`, `shared-memory/wifi.md`. Keep one topic per note.
## Note format
```md ```md
# Title # Title
@@ -28,8 +49,7 @@ _Updated: YYYY-MM-DD_
## How to update ## How to update
1. `read_file` to get the exact current content 1. `read_file` the note to get its exact current content.
2. `edit_file` to modify — always keep the `_Updated: YYYY-MM-DD_` date in sync 2. `edit_file` to change part of it — keep the `_Updated:_` date in sync.
3. Use `write_file` only when creating a new file or fully rewriting one 3. Use `write_file` only to create a new note or fully rewrite one.
4. Keep `user-memory/index.md` in sync when you add or significantly change a note.
Always keep `data/memory/index.md` in sync when you create or significantly update a file.
+2 -4
View File
@@ -2,11 +2,9 @@
You are an extremely powerful general-purpose personal assistant. You help the user with any task — research, writing, planning, analysis, coding, or anything else they bring to you. You are an extremely powerful general-purpose personal assistant. You help the user with any task — research, writing, planning, analysis, coding, or anything else they bring to you.
Your personality and tone are defined in `data/memory/SOUL.md`. If the file exists, it is automatically injected into your system context — look for it at the end of this prompt.
Think outside the box: you can use tools, write and execute Python scripts on the fly, or even modify your own source code. Think outside the box: you can use tools, write and execute Python scripts on the fly, or even modify your own source code.
The `data/` directory (inside your working directory) is your own space — write there freely; you have permission to create and modify anything under it. **Default to `data/` for everything you produce**: generated files, notes, one-shot scripts, downloads, and persistent memory (e.g. `data/memory/`, `data/notifications.md`). When a path is relative, prefix it with `data/` — a bare filename lands in the project root, which is not where your working files belong. Write **outside** `data/` (the project root, `src/`, `web/`, `agents/`, config, …) only when a specific, well-defined goal genuinely requires it and cannot be accomplished within `data/`. The `data/` directory (inside your working directory) is your own space — write there freely; you have permission to create and modify anything under it. **Default to `data/` for everything you produce**: generated files, notes, one-shot scripts, downloads. When a path is relative, prefix it with `data/` — a bare filename lands in the project root, which is not where your working files belong. (Persistent **memory** is separate: durable facts go to `user-memory/` or `shared-memory/`, not under `data/` — see the Memory section.) Write **outside** `data/` (the project root, `src/`, `web/`, `agents/`, config, …) only when a specific, well-defined goal genuinely requires it and cannot be accomplished within `data/`.
You have access to tools, persistent memory system and sub agents. Use both proactively. Sub agents also help to keep your context windows small and concise. You have access to tools, persistent memory system and sub agents. Use both proactively. Sub agents also help to keep your context windows small and concise.
@@ -113,7 +111,7 @@ Configuration tools are hidden by default to keep context small. Call `activate_
## Memory reminder ## Memory reminder
Sessions are temporary — the user can close and start a new one at any moment. **Context alone is not enough.** If something is worth remembering, write it to a file in `data/memory/` immediately. If it stays only in context, it is gone forever when the session ends. Sessions are temporary — the user can close and start a new one at any moment. **Context alone is not enough.** If something is worth remembering, save it to `user-memory/` immediately (or `shared-memory/` if it's meant for the whole group). If it stays only in context, it is gone forever when the session ends.
--- ---
+2 -2
View File
@@ -1,9 +1,9 @@
{ {
"name": "Main Assistant", "name": "Main Assistant",
"description": "General-purpose assistant: helps the user with any task using tools, and persists all relevant information in data/memory", "description": "General-purpose assistant: helps the user with any task using tools, and persists all relevant information in memory",
"friendly_description": "Your general-purpose assistant — helps with any task and remembers what matters in memory.", "friendly_description": "Your general-purpose assistant — helps with any task and remembers what matters in memory.",
"type": "chat", "type": "chat",
"inject_memory": ["data/memory/index.md", "data/memory/SOUL.md"], "inject_memory": ["user-memory/index.md", "shared-memory/index.md"],
"icon": "icon.png", "icon": "icon.png",
"strength": "average" "strength": "average"
} }
+1 -1
View File
@@ -29,7 +29,7 @@ Delegate work to these task specialists via `execute_task` / `execute_subtask`:
Your system prompt already contains, without you asking: Your system prompt already contains, without you asking:
- The project's **name**, **description**, and **working directory** (the project root — all relative file paths resolve there). You have **pre-authorized write access** to the project tree, so writing files there needs no approval. - The project's **name**, **description**, and **working directory** (the project root — all relative file paths resolve there). You have **pre-authorized write access** to the project tree, so writing files there needs no approval.
- **`data/memory/index.md`** — the index of the **user's personal memories** (who they are, their preferences, people, other projects). It is injected automatically. Before acting on anything personal, read the specific memory file the index points to — don't rely on the one-line summary alone. - **`user-memory/index.md`** and **`shared-memory/index.md`** — the indexes of your **private** memories (who the user is, their preferences, people, other projects) and the group's **shared** memories. Both are injected automatically. Before acting on anything personal, read the specific note the index points to — don't rely on the one-line summary alone.
- **`SKALD.md`** at the project root — this project's **living diary** (see below). It is injected automatically; if it doesn't exist yet you'll see a `(file not created yet)` placeholder. - **`SKALD.md`** at the project root — this project's **living diary** (see below). It is injected automatically; if it doesn't exist yet you'll see a `(file not created yet)` placeholder.
Treat all of this as ground truth. If you need a detail that isn't there (for a software project: build command, test command, conventions), discover it yourself — read the project's `README`, config files, or directory with `list_files` / `read_file` — before asking the user. Treat all of this as ground truth. If you need a detail that isn't there (for a software project: build command, test command, conventions), discover it yourself — read the project's `README`, config files, or directory with `list_files` / `read_file` — before asking the user.
+1 -1
View File
@@ -5,6 +5,6 @@
"type": "chat", "type": "chat",
"scope": "reasoning", "scope": "reasoning",
"strength": "average", "strength": "average",
"inject_memory": ["data/memory/index.md", "$WD/SKALD.md"], "inject_memory": ["user-memory/index.md", "shared-memory/index.md", "$WD/SKALD.md"],
"icon": "icon.png" "icon": "icon.png"
} }
+71 -23
View File
@@ -29,11 +29,12 @@
//! determines the action; if none matches the tool requires approval //! determines the action; if none matches the tool requires approval
//! (default-closed). //! (default-closed).
//! //!
//! ## Hardcoded exception //! ## Memory namespace
//! //!
//! File-write tools targeting `memory/` paths bypass the rule engine and are //! The virtual memory roots `user-memory/` and `shared-memory/` (blueprint §5) are
//! always allowed (this mirrors the original behaviour and can be replaced by //! allowed by **seeded rules**, not a hardcoded bypass: `seed_fs_path_rules` stamps
//! an explicit `allow` rule later). //! `@fs_any allow user-memory/*` and `@fs_any allow shared-memory/*`, editable in the
//! File System panel like any other path rule.
use std::collections::HashMap; use std::collections::HashMap;
use std::sync::Arc; use std::sync::Arc;
@@ -327,31 +328,52 @@ impl ApprovalManager {
/// Priority `5` places these path-scoped rules *before* the `*` catch-all (999999), /// Priority `5` places these path-scoped rules *before* the `*` catch-all (999999),
/// so an unmatched path still falls through to the default `require`. /// so an unmatched path still falls through to the default `require`.
/// ///
/// - `memory/*` → **allow** (the LLM manages its own memory; replaces the former /// - `user-memory/*` → **allow** (the caller's private memory namespace, blueprint §5;
/// hardcoded `is_memory_path` bypass). /// the LLM manages its own memory). Reads *and* writes: `@fs_any`.
/// - `shared-memory/*` → reads **allow** (`@fs_read`), writes **require** (`@fs_write`):
/// shared memory is visible to everyone, so a write is a deliberate, human-confirmed
/// act — the agent must not silently push one person's information into it.
/// - `data/*` → **allow** (scratch/data workspace). /// - `data/*` → **allow** (scratch/data workspace).
/// - `secrets/*` → **deny** (`@fs_any` denies reads *and* writes; a read would leak /// - `secrets/*` → **deny** (`@fs_any` denies reads *and* writes; a read would leak
/// the secret into the LLM context / history / WS stream, and `Deny` is /// the secret into the LLM context / history / WS stream, and `Deny` is
/// non-bypassable). The `/*` pattern also matches the `secrets` dir node itself, so /// non-bypassable). The `/*` pattern also matches the `secrets` dir node itself, so
/// recursive `list_files`/`grep_files` rooted at it are covered. /// recursive `list_files`/`grep_files` rooted at it are covered.
/// - `memory_search` → **allow**, path-less: it searches note *content* (arg `query`,
/// not `path`), so it needs a tool-scoped rule rather than a path pattern.
pub async fn seed_fs_path_rules(&self) -> Result<()> { pub async fn seed_fs_path_rules(&self) -> Result<()> {
// (tool_pattern, path_pattern, action, note) // (tool_pattern, path_pattern, action, note). `path_pattern = None` is a
let rules: &[(&str, &str, &str, &str)] = &[ // tool-scoped rule that matches regardless of args.
("@fs_any", "memory/*", "allow", "auto-allow memory/"), let rules: &[(&str, Option<&str>, &str, &str)] = &[
("@fs_any", "data/*", "allow", "auto-allow data/"), ("@fs_any", Some("user-memory/*"), "allow", "auto-allow user-memory/"),
("@fs_any", "secrets/*", "deny", "deny secrets/ access"), ("@fs_read", Some("shared-memory/*"), "allow", "auto-allow read shared-memory/"),
("@fs_write", Some("shared-memory/*"), "require", "require write shared-memory/"),
("@fs_any", Some("data/*"), "allow", "auto-allow data/"),
("@fs_any", Some("secrets/*"), "deny", "deny secrets/ access"),
("memory_search", None, "allow", "allow memory_search"),
]; ];
let mut seeded = 0; let mut seeded = 0;
for (tool_pattern, path_pattern, action, note) in rules { for &(tool_pattern, path_pattern, action, note) in rules {
let exists: i64 = sqlx::query_scalar( // A NULL path can't be matched with `=` (NULL comparisons are never true),
// so the existence check branches on it — otherwise the row would re-insert
// on every boot.
let exists: i64 = match path_pattern {
Some(p) => sqlx::query_scalar(
"SELECT COUNT(*) FROM approval_rules "SELECT COUNT(*) FROM approval_rules
WHERE tool_pattern = ? AND path_pattern = ? AND group_id = 'default'", WHERE tool_pattern = ? AND path_pattern = ? AND group_id = 'default'",
) )
.bind(tool_pattern) .bind(tool_pattern)
.bind(path_pattern) .bind(p)
.fetch_one(self.db.as_ref()) .fetch_one(self.db.as_ref())
.await?; .await?,
None => sqlx::query_scalar(
"SELECT COUNT(*) FROM approval_rules
WHERE tool_pattern = ? AND path_pattern IS NULL AND group_id = 'default'",
)
.bind(tool_pattern)
.fetch_one(self.db.as_ref())
.await?,
};
if exists > 0 { if exists > 0 {
continue; continue;
} }
@@ -360,7 +382,7 @@ impl ApprovalManager {
VALUES (?, ?, ?, ?, 5, 'default')", VALUES (?, ?, ?, ?, 5, 'default')",
) )
.bind(tool_pattern) .bind(tool_pattern)
.bind(path_pattern) .bind(path_pattern) // Option<&str> → NULL when None
.bind(action) .bind(action)
.bind(note) .bind(note)
.execute(self.db.as_ref()) .execute(self.db.as_ref())
@@ -383,7 +405,10 @@ impl ApprovalManager {
/// - the per-tool write `require` defaults (`note = 'default rule'`, no path) — fs /// - the per-tool write `require` defaults (`note = 'default rule'`, no path) — fs
/// gating now lives in the File System panel + the `*` catch-all; /// gating now lives in the File System panel + the `*` catch-all;
/// - the old `data/*` allow rows (`note = 'auto-allow data/ writes'`); /// - the old `data/*` allow rows (`note = 'auto-allow data/ writes'`);
/// - the old `secrets` deny rows (`note = 'deny reading secrets/'`). /// - the old `secrets` deny rows (`note = 'deny reading secrets/'`);
/// - the old single `memory/*` allow row (`note = 'auto-allow memory/'`) — the memory
/// namespace split into `user-memory/` + `shared-memory/`, so the `memory/*` pattern
/// no longer routes and would otherwise linger as a stale allow on a disk `./memory/`.
/// ///
/// Idempotent: a no-op once the legacy rows are gone. Run before `seed_fs_path_rules`. /// Idempotent: a no-op once the legacy rows are gone. Run before `seed_fs_path_rules`.
pub async fn migrate_legacy_fs_rules(&self) -> Result<()> { pub async fn migrate_legacy_fs_rules(&self) -> Result<()> {
@@ -406,7 +431,20 @@ impl ApprovalManager {
.await? .await?
.rows_affected(); .rows_affected();
let total = n1 + n2 + n3; let n4 = sqlx::query("DELETE FROM approval_rules WHERE note = 'auto-allow memory/'")
.execute(self.db.as_ref())
.await?
.rows_affected();
// An earlier build seeded `@fs_any allow shared-memory/*`; shared writes now
// require approval, so that blanket allow must go (the new @fs_read/@fs_write
// rows are seeded fresh by `seed_fs_path_rules`).
let n5 = sqlx::query("DELETE FROM approval_rules WHERE note = 'auto-allow shared-memory/'")
.execute(self.db.as_ref())
.await?
.rows_affected();
let total = n1 + n2 + n3 + n4 + n5;
if total > 0 { if total > 0 {
info!("approval_rules: migrated {total} legacy filesystem rules to @fs_* File System panel"); info!("approval_rules: migrated {total} legacy filesystem rules to @fs_* File System panel");
} }
@@ -419,8 +457,8 @@ impl ApprovalManager {
/// ///
/// Evaluation order: /// Evaluation order:
/// 1. Rules for `group_id` first, then "default" group as fallback, sorted by priority ASC. /// 1. Rules for `group_id` first, then "default" group as fallback, sorted by priority ASC.
/// First match wins. (`memory/` auto-allow is a seeded `@fs_any allow memory/*` rule, /// First match wins. (The `user-memory/` and `shared-memory/` auto-allows are seeded
/// not a hardcoded exception — see `seed_fs_path_rules`.) /// `@fs_any allow …/*` rules, not a hardcoded exception — see `seed_fs_path_rules`.)
/// 2. Session bypass: if a matching bypass is active, `Require` → `Allow`. /// 2. Session bypass: if a matching bypass is active, `Require` → `Allow`.
/// `Deny` is never bypassed. /// `Deny` is never bypassed.
/// 3. No match → `Require` (default-closed policy). /// 3. No match → `Require` (default-closed policy).
@@ -1119,21 +1157,31 @@ mod tests {
.unwrap(); .unwrap();
assert_eq!(legacy, 0, "legacy fs rules should be removed by migration"); assert_eq!(legacy, 0, "legacy fs rules should be removed by migration");
// …and replaced by exactly the three @fs_* token rows. // …and replaced by exactly the five @fs_* token rows (shared-memory has two:
// read-allow and write-require).
let fs_rows: i64 = sqlx::query_scalar( let fs_rows: i64 = sqlx::query_scalar(
"SELECT COUNT(*) FROM approval_rules WHERE tool_pattern LIKE '@fs%'", "SELECT COUNT(*) FROM approval_rules WHERE tool_pattern LIKE '@fs%'",
) )
.fetch_one(db.as_ref()) .fetch_one(db.as_ref())
.await .await
.unwrap(); .unwrap();
assert_eq!(fs_rows, 3, "memory/data/secrets @fs_* rules should be seeded"); assert_eq!(fs_rows, 5, "user-memory + shared-memory(r/w) + data + secrets @fs_* rules should be seeded");
// Gate decisions through the real check() path. // Gate decisions through the real check() path.
async fn decide(mgr: &ApprovalManager, tool: &str, path: &str) -> GateResult { async fn decide(mgr: &ApprovalManager, tool: &str, path: &str) -> GateResult {
mgr.check(1, None, "main", "web", tool, &json!({ "path": path }), Some("default")).await mgr.check(1, None, "main", "web", tool, &json!({ "path": path }), Some("default")).await
} }
assert!(matches!(decide(&mgr, "write_file", "memory/notes.md").await, GateResult::Allow)); // user-memory auto-allows reads and writes; the old `memory/*` no longer matches.
assert!(matches!(decide(&mgr, "write_file", "user-memory/notes.md").await, GateResult::Allow));
assert!(matches!(decide(&mgr, "list_files", "user-memory").await, GateResult::Allow));
assert!(matches!(decide(&mgr, "write_file", "memory/notes.md").await, GateResult::Require));
// shared-memory: reads allowed, writes require approval.
assert!(matches!(decide(&mgr, "read_file", "shared-memory/casa.md").await, GateResult::Allow));
assert!(matches!(decide(&mgr, "write_file", "shared-memory/casa.md").await, GateResult::Require));
assert!(matches!(decide(&mgr, "edit_file", "shared-memory/casa.md").await, GateResult::Require));
assert!(matches!(decide(&mgr, "read_file", "data/x.txt").await, GateResult::Allow)); assert!(matches!(decide(&mgr, "read_file", "data/x.txt").await, GateResult::Allow));
// memory_search is allowed by a path-less tool rule (it has `query`, not `path`).
assert!(matches!(decide(&mgr, "memory_search", "ignored").await, GateResult::Allow));
// Improvement over legacy: secrets *writes* are now denied too, not just reads. // Improvement over legacy: secrets *writes* are now denied too, not just reads.
assert!(matches!(decide(&mgr, "write_file", "secrets/key").await, GateResult::Deny)); assert!(matches!(decide(&mgr, "write_file", "secrets/key").await, GateResult::Deny));
assert!(matches!(decide(&mgr, "read_file", "secrets/key").await, GateResult::Deny)); assert!(matches!(decide(&mgr, "read_file", "secrets/key").await, GateResult::Deny));
+203
View File
@@ -0,0 +1,203 @@
//! Accessor for `memory_docs` — the backing store of the virtual `memory/`
//! namespace (blueprint §5).
//!
//! The **pool is the namespace**: a user pool holds that user's private notes
//! (`memory/{userid}`), the system pool holds shared notes (`memory/shared`).
//! Callers pass a `path` already stripped of the `memory/…` prefix — the file
//! it lands in decides the namespace, the row keeps only the tail. `path` is
//! UNIQUE, so [`upsert`] is the single write path for both create and edit, and
//! the `memory_docs_fts` triggers keep the full-text index in step underneath.
use anyhow::Result;
use sqlx::SqlitePool;
#[derive(Debug, Clone, sqlx::FromRow)]
pub struct MemoryDoc {
pub id: i64,
pub path: String,
pub content: String,
pub created_at: String,
pub updated_at: String,
}
/// One row of a directory-style listing: metadata only, no `content` body.
#[derive(Debug, Clone, sqlx::FromRow)]
pub struct MemoryEntry {
pub path: String,
pub updated_at: String,
}
/// One full-text hit: the matching note's path and a highlighted excerpt.
#[derive(Debug, Clone, sqlx::FromRow)]
pub struct MemoryHit {
pub path: String,
pub snippet: String,
}
const SELECT: &str = "SELECT id, path, content, created_at, updated_at FROM memory_docs";
/// Fetch one note by its exact path.
pub async fn get(pool: &SqlitePool, path: &str) -> Result<Option<MemoryDoc>> {
let row = sqlx::query_as::<_, MemoryDoc>(sqlx::AssertSqlSafe(format!("{SELECT} WHERE path = ?")))
.bind(path)
.fetch_optional(pool)
.await?;
Ok(row)
}
/// Create the note at `path`, or overwrite it if it already exists. `created_at`
/// survives an overwrite; `updated_at` is bumped. Returns the stored row.
pub async fn upsert(pool: &SqlitePool, path: &str, content: &str) -> Result<MemoryDoc> {
sqlx::query(
"INSERT INTO memory_docs (path, content)
VALUES (?, ?)
ON CONFLICT(path) DO UPDATE SET
content = excluded.content,
updated_at = datetime('now')",
)
.bind(path)
.bind(content)
.execute(pool)
.await?;
let row = sqlx::query_as::<_, MemoryDoc>(sqlx::AssertSqlSafe(format!("{SELECT} WHERE path = ?")))
.bind(path)
.fetch_one(pool)
.await?;
Ok(row)
}
/// List notes whose path starts with `prefix` (pass `""` for all), most recently
/// edited first. Metadata only — the `content` body is not loaded.
pub async fn list(pool: &SqlitePool, prefix: &str) -> Result<Vec<MemoryEntry>> {
// Escaped LIKE prefix: a literal `%`/`_` in the caller's path must match as
// itself, not as a wildcard. `\` is the escape character.
let pattern = format!("{}%", escape_like(prefix));
let rows = sqlx::query_as::<_, MemoryEntry>(
"SELECT path, updated_at FROM memory_docs
WHERE path LIKE ? ESCAPE '\\'
ORDER BY updated_at DESC",
)
.bind(pattern)
.fetch_all(pool)
.await?;
Ok(rows)
}
/// Full-text search over note bodies and paths, best match first. `query` is
/// FTS5 MATCH syntax; `snippet` is a short excerpt of the body with the matched
/// terms wrapped in `[` … `]`.
pub async fn search(pool: &SqlitePool, query: &str, limit: i64) -> Result<Vec<MemoryHit>> {
let rows = sqlx::query_as::<_, MemoryHit>(
// `memory_docs_fts` is external-content, so its rowid is `memory_docs.id`;
// join back for the path, and read the excerpt from content column 1.
"SELECT d.path AS path,
snippet(memory_docs_fts, 1, '[', ']', '…', 12) AS snippet
FROM memory_docs_fts
JOIN memory_docs d ON d.id = memory_docs_fts.rowid
WHERE memory_docs_fts MATCH ?
ORDER BY bm25(memory_docs_fts)
LIMIT ?",
)
.bind(query)
.bind(limit)
.fetch_all(pool)
.await?;
Ok(rows)
}
/// Delete the note at `path`. Returns whether a row was removed.
pub async fn delete(pool: &SqlitePool, path: &str) -> Result<bool> {
let n = sqlx::query("DELETE FROM memory_docs WHERE path = ?")
.bind(path)
.execute(pool)
.await?
.rows_affected();
Ok(n > 0)
}
/// Escapes `%`, `_` and `\` so a caller-supplied string is a literal LIKE prefix.
fn escape_like(s: &str) -> String {
let mut out = String::with_capacity(s.len());
for c in s.chars() {
if matches!(c, '%' | '_' | '\\') {
out.push('\\');
}
out.push(c);
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::PathBuf;
/// A standalone owner-schema database in a throwaway temp dir. `tag` plus an
/// atomic counter keep parallel tests from colliding on the same file.
/// Returns the pool and the dir so the caller can wipe it (SQLite leaves
/// `-wal`/`-shm` sidecars beside the file).
async fn owner_pool(tag: &str) -> (SqlitePool, PathBuf) {
use std::sync::atomic::{AtomicU64, Ordering};
static SEQ: AtomicU64 = AtomicU64::new(0);
let n = SEQ.fetch_add(1, Ordering::Relaxed);
let dir = std::env::temp_dir()
.join(format!("skald-memdocs-{}-{tag}-{n}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let pool = crate::db::create_user_pool(&dir.join("owner.db"), None).await.unwrap();
(pool, dir)
}
#[tokio::test]
async fn upsert_is_create_then_overwrite_and_fts_follows() {
let (pool, dir) = owner_pool("upsert").await;
// create
let doc = upsert(&pool, "notes/spesa.md", "latte e pane").await.unwrap();
assert_eq!(doc.path, "notes/spesa.md");
assert_eq!(doc.content, "latte e pane");
let first_id = doc.id;
// get by exact path
assert_eq!(get(&pool, "notes/spesa.md").await.unwrap().unwrap().content, "latte e pane");
assert!(get(&pool, "notes/altro.md").await.unwrap().is_none());
// overwrite: same row, new content, created_at preserved
let doc2 = upsert(&pool, "notes/spesa.md", "latte, pane, uova").await.unwrap();
assert_eq!(doc2.id, first_id, "upsert must update in place, not insert a new row");
assert_eq!(doc2.content, "latte, pane, uova");
assert_eq!(doc2.created_at, doc.created_at, "created_at survives an overwrite");
assert_eq!(list(&pool, "").await.unwrap().len(), 1, "still one row for that path");
// FTS follows the update: the removed token is gone, the new one is found
assert!(search(&pool, "uova", 10).await.unwrap().iter().any(|h| h.path == "notes/spesa.md"));
assert!(search(&pool, "latte", 10).await.unwrap().iter().any(|h| h.path == "notes/spesa.md"));
pool.close().await;
let _ = std::fs::remove_dir_all(&dir);
}
#[tokio::test]
async fn list_by_prefix_and_delete_deindexes() {
let (pool, dir) = owner_pool("list").await;
upsert(&pool, "notes/spesa.md", "latte pane uova").await.unwrap();
upsert(&pool, "notes/idee.md", "un'idea brillante").await.unwrap();
upsert(&pool, "diary/2026.md", "oggi e' successo").await.unwrap();
let notes = list(&pool, "notes/").await.unwrap();
assert_eq!(notes.len(), 2, "prefix listing is scoped to the subtree");
assert!(notes.iter().all(|e| e.path.starts_with("notes/")));
assert_eq!(list(&pool, "").await.unwrap().len(), 3, "empty prefix lists everything");
// delete removes the row and de-indexes it from FTS
assert!(delete(&pool, "notes/spesa.md").await.unwrap());
assert!(get(&pool, "notes/spesa.md").await.unwrap().is_none());
assert!(search(&pool, "uova", 10).await.unwrap().is_empty(), "delete must de-index");
assert!(!delete(&pool, "notes/spesa.md").await.unwrap(), "second delete is a no-op");
pool.close().await;
let _ = std::fs::remove_dir_all(&dir);
}
}
+64
View File
@@ -13,6 +13,7 @@ pub mod llm_requests;
pub mod llm_request_payloads; pub mod llm_request_payloads;
pub mod mcp_events; pub mod mcp_events;
pub mod mcp_servers; pub mod mcp_servers;
pub mod memory_docs;
pub mod plugins; pub mod plugins;
pub mod roles; pub mod roles;
pub mod scheduled_jobs; pub mod scheduled_jobs;
@@ -710,6 +711,60 @@ pub async fn create_owner_tables(pool: &SqlitePool) -> Result<()> {
.execute(pool) .execute(pool)
.await?; .await?;
// Backing store for the virtual `memory/` namespace (blueprint §5). MD-only
// notes keyed by a path *relative to the namespace root* — the file is the
// namespace, so no `memory/{userid}` / `memory/shared` prefix is stored:
// routing picks the pool, the row keeps only the tail. Because this is an
// owner table, the same schema backs private memory in each `{userid}.db`
// (behind SQLCipher) and shared memory in `system.db` (cleartext, the
// household owner) — §5.1. `path` is UNIQUE, so a write is an upsert.
sqlx::query(
"CREATE TABLE IF NOT EXISTS memory_docs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
path TEXT NOT NULL UNIQUE,
content TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
)",
)
.execute(pool)
.await?;
// Full-text index over memory notes: the payoff of a SQLite backing over
// opaque file blobs (§5) — a decrypted session can search / RAG its own
// memory. External-content FTS5 keeps no second copy of `content`; the
// triggers below mirror every change from `memory_docs`. FTS5 is compiled
// into the bundled SQLCipher build, so this works inside encrypted user
// files too.
sqlx::query(
"CREATE VIRTUAL TABLE IF NOT EXISTS memory_docs_fts USING fts5(
path, content,
content='memory_docs',
content_rowid='id'
)",
)
.execute(pool)
.await?;
for trigger in [
"CREATE TRIGGER IF NOT EXISTS memory_docs_ai AFTER INSERT ON memory_docs BEGIN
INSERT INTO memory_docs_fts(rowid, path, content)
VALUES (new.id, new.path, new.content);
END",
"CREATE TRIGGER IF NOT EXISTS memory_docs_ad AFTER DELETE ON memory_docs BEGIN
INSERT INTO memory_docs_fts(memory_docs_fts, rowid, path, content)
VALUES ('delete', old.id, old.path, old.content);
END",
"CREATE TRIGGER IF NOT EXISTS memory_docs_au AFTER UPDATE ON memory_docs BEGIN
INSERT INTO memory_docs_fts(memory_docs_fts, rowid, path, content)
VALUES ('delete', old.id, old.path, old.content);
INSERT INTO memory_docs_fts(rowid, path, content)
VALUES (new.id, new.path, new.content);
END",
] {
sqlx::query(trigger).execute(pool).await?;
}
Ok(()) Ok(())
} }
@@ -764,6 +819,15 @@ mod tests {
one("INSERT INTO projects (id, name, path) VALUES (1, 'p', '/tmp')").await.unwrap(); one("INSERT INTO projects (id, name, path) VALUES (1, 'p', '/tmp')").await.unwrap();
one("INSERT INTO project_tickets (project_id, title, job_id) VALUES (1, 't', 1)").await.unwrap(); one("INSERT INTO project_tickets (project_id, title, job_id) VALUES (1, 't', 1)").await.unwrap();
one("INSERT INTO llm_request_payloads (request_id, request_json) VALUES ('r1', '{}')").await.unwrap(); one("INSERT INTO llm_request_payloads (request_id, request_json) VALUES ('r1', '{}')").await.unwrap();
// Fires the AFTER INSERT trigger into the external-content FTS5 table.
one("INSERT INTO memory_docs (path, content) VALUES ('notes/x.md', 'hello world')").await.unwrap();
// ...and the FTS index actually answers a MATCH.
let (hits,): (i64,) = sqlx::query_as(
"SELECT count(*) FROM memory_docs_fts WHERE memory_docs_fts MATCH 'world'",
)
.fetch_one(&pool).await.unwrap();
assert_eq!(hits, 1, "memory_docs_fts must index inserted notes");
pool.close().await; pool.close().await;
let _ = std::fs::remove_dir_all(&dir); let _ = std::fs::remove_dir_all(&dir);
@@ -34,6 +34,9 @@ fn system_timezone() -> Option<&'static str> {
/// without needing the full handler and all its dependencies. /// without needing the full handler and all its dependencies.
pub struct MessageBuilder { pub struct MessageBuilder {
pub pool: Arc<SqlitePool>, pub pool: Arc<SqlitePool>,
/// The shared (`system.db`) pool, for injecting `shared-memory/` notes. The
/// owner `pool` above backs `user-memory/`.
pub shared_pool: Arc<SqlitePool>,
pub session_id: i64, pub session_id: i64,
pub mcp: Arc<McpManager>, pub mcp: Arc<McpManager>,
pub datetime_config: DatetimeConfig, pub datetime_config: DatetimeConfig,
@@ -93,9 +96,7 @@ impl MessageBuilder {
You can edit them with `edit_file` or `write_file` using the path shown.\n" You can edit them with `edit_file` or `write_file` using the path shown.\n"
); );
for mem_path in &meta.inject_memory { for mem_path in &meta.inject_memory {
// Resolve the entry to (absolute path to read, path to show the agent). let (content, display) = self.load_inject_memory(mem_path).await;
let (abs, display) = self.resolve_memory_path(mem_path);
let content = tokio::fs::read_to_string(&abs).await.ok();
match content { match content {
Some(c) => static_content.push_str(&format!( Some(c) => static_content.push_str(&format!(
"\n<memory_file path=\"{display}\">\n{c}\n</memory_file>\n" "\n<memory_file path=\"{display}\">\n{c}\n</memory_file>\n"
@@ -405,6 +406,27 @@ impl MessageBuilder {
/// when the file lives under it, absolute otherwise** — so when the agent references /// when the file lives under it, absolute otherwise** — so when the agent references
/// it back via `edit_file`/`write_file`, the loop's working-directory injection /// it back via `edit_file`/`write_file`, the loop's working-directory injection
/// (which rewrites relative paths against the WD) resolves to the very same file. /// (which rewrites relative paths against the WD) resolves to the very same file.
/// Loads an `inject_memory` entry, returning `(content, display_path)`.
///
/// Virtual memory paths are read from SQLite: `user-memory/…` from the owner
/// `pool`, `shared-memory/…` from the `shared_pool` (`system.db`). Everything
/// else (`data/…`, `$WD/…`) is an ordinary disk read. A missing note / file
/// yields `None`, rendered as "(file not created yet)".
async fn load_inject_memory(&self, mem_path: &str) -> (Option<String>, String) {
use crate::tools::fs::{classify_memory, MemScope};
if let Some(m) = classify_memory(mem_path) {
let pool = match m.scope {
MemScope::User => &self.pool,
MemScope::Shared => &self.shared_pool,
};
let content = crate::db::memory_docs::get(pool, &m.rel)
.await.ok().flatten().map(|d| d.content);
return (content, mem_path.to_string());
}
let (abs, display) = self.resolve_memory_path(mem_path);
(tokio::fs::read_to_string(&abs).await.ok(), display)
}
fn resolve_memory_path(&self, mem_path: &str) -> (std::path::PathBuf, String) { fn resolve_memory_path(&self, mem_path: &str) -> (std::path::PathBuf, String) {
let wd = self.working_directory.clone() let wd = self.working_directory.clone()
.unwrap_or_else(|| std::env::current_dir().unwrap_or_default()); .unwrap_or_else(|| std::env::current_dir().unwrap_or_default());
@@ -28,6 +28,7 @@ impl ChatSessionHandler {
.map(|rc| rc.effective_working_dir()); .map(|rc| rc.effective_working_dir());
let builder = MessageBuilder { let builder = MessageBuilder {
pool: Arc::clone(&self.db), pool: Arc::clone(&self.db),
shared_pool: Arc::clone(&self.shared_pool),
session_id: self.scratchpad_sid(), session_id: self.scratchpad_sid(),
mcp: Arc::clone(&self.mcp), mcp: Arc::clone(&self.mcp),
datetime_config: self.datetime_config.clone(), datetime_config: self.datetime_config.clone(),
@@ -262,6 +262,9 @@ impl ApprovalDecision {
pub struct ChatSessionHandler { pub struct ChatSessionHandler {
pub session_id: i64, pub session_id: i64,
pub(super) db: Arc<SqlitePool>, pub(super) db: Arc<SqlitePool>,
/// The shared (`system.db`) pool. Owner-bound work uses `db`; this is only for
/// cross-owner reads, e.g. injecting `shared-memory/` notes into the prompt.
pub(super) shared_pool: Arc<SqlitePool>,
/// The authenticated user who owns this session. Threaded into `ChatOptions` /// The authenticated user who owns this session. Threaded into `ChatOptions`
/// so the telemetry metadata row in `system.db` carries `user_id`. /// so the telemetry metadata row in `system.db` carries `user_id`.
pub(super) user_id: String, pub(super) user_id: String,
@@ -332,6 +335,7 @@ impl ChatSessionHandler {
pub fn new( pub fn new(
session_id: i64, session_id: i64,
db: Arc<SqlitePool>, db: Arc<SqlitePool>,
shared_pool: Arc<SqlitePool>,
user_id: String, user_id: String,
llm_manager: Arc<LlmManager>, llm_manager: Arc<LlmManager>,
max_history_messages: usize, max_history_messages: usize,
@@ -357,6 +361,7 @@ impl ChatSessionHandler {
Self { Self {
session_id, session_id,
db, db,
shared_pool,
user_id, user_id,
llm_manager, llm_manager,
max_history_messages, max_history_messages,
+6
View File
@@ -22,6 +22,9 @@ use super::handler::ChatSessionHandler;
pub struct ChatSessionManager { pub struct ChatSessionManager {
db: Arc<SqlitePool>, db: Arc<SqlitePool>,
/// The shared (`system.db`) pool, threaded to each handler for cross-owner
/// reads such as injecting `shared-memory/` notes.
shared_pool: Arc<SqlitePool>,
user_id: String, user_id: String,
llm_manager: Arc<LlmManager>, llm_manager: Arc<LlmManager>,
max_history_messages: usize, max_history_messages: usize,
@@ -48,6 +51,7 @@ pub struct ChatSessionManager {
impl ChatSessionManager { impl ChatSessionManager {
pub fn new( pub fn new(
db: Arc<SqlitePool>, db: Arc<SqlitePool>,
shared_pool: Arc<SqlitePool>,
user_id: String, user_id: String,
llm_manager: Arc<LlmManager>, llm_manager: Arc<LlmManager>,
max_history_messages: usize, max_history_messages: usize,
@@ -68,6 +72,7 @@ impl ChatSessionManager {
) -> Self { ) -> Self {
Self { Self {
db, db,
shared_pool,
user_id, user_id,
llm_manager, llm_manager,
max_history_messages, max_history_messages,
@@ -155,6 +160,7 @@ impl ChatSessionManager {
let handler = Arc::new(ChatSessionHandler::new( let handler = Arc::new(ChatSessionHandler::new(
session_id, session_id,
self.db.clone(), self.db.clone(),
self.shared_pool.clone(),
self.user_id.clone(), self.user_id.clone(),
Arc::clone(&self.llm_manager), Arc::clone(&self.llm_manager),
self.max_history_messages, self.max_history_messages,
+3 -2
View File
@@ -204,9 +204,9 @@ impl Tools {
/// Captures sibling managers (mcp, plugins, cron, secrets) into the tool /// Captures sibling managers (mcp, plugins, cron, secrets) into the tool
/// registry. `execute_task` is deliberately NOT registered here — it is injected /// registry. `execute_task` is deliberately NOT registered here — it is injected
/// per interactive session by `ChatHub::send_message`. /// per interactive session by `ChatHub::send_message`.
pub(super) fn build(integrations: &Integrations, tasks: &Tasks, models: &Models) -> Self { pub(super) fn build(rt: &Runtime, integrations: &Integrations, tasks: &Tasks, models: &Models) -> Self {
let mut tool_registry = ToolRegistry::new(); let mut tool_registry = ToolRegistry::new();
crate::tools::fs::register_all(&mut tool_registry); crate::tools::fs::register_all(&mut tool_registry, Arc::clone(&rt.db));
tool_registry.register(crate::tools::ast_outline::AstOutline::new()); tool_registry.register(crate::tools::ast_outline::AstOutline::new());
tool_registry.register(crate::tools::exec::ExecuteCmd); tool_registry.register(crate::tools::exec::ExecuteCmd);
tool_registry.register(crate::tools::read_notification::ReadNotification); tool_registry.register(crate::tools::read_notification::ReadNotification);
@@ -338,6 +338,7 @@ impl Conversation {
let manager = Arc::new(ChatSessionManager::new( let manager = Arc::new(ChatSessionManager::new(
Arc::clone(&rt.db), Arc::clone(&rt.db),
Arc::clone(&rt.db), // shared pool == system.db (this is the ownerless manager)
String::new(), String::new(),
Arc::clone(&models.llm_manager), Arc::clone(&models.llm_manager),
config.llm.max_history_messages, config.llm.max_history_messages,
+1 -1
View File
@@ -66,7 +66,7 @@ impl Skald {
let media = Media::build(&rt, &models).await?; let media = Media::build(&rt, &models).await?;
let integrations = Integrations::build(&rt, plugins); let integrations = Integrations::build(&rt, plugins);
let tasks = Tasks::build(&rt, config); let tasks = Tasks::build(&rt, config);
let tools = Tools::build(&integrations, &tasks, &models); let tools = Tools::build(&rt, &integrations, &tasks, &models);
let interaction = Interaction::build(&rt, &tools).await?; let interaction = Interaction::build(&rt, &tools).await?;
let conversation = Conversation::build(&rt, &models, &media, &tools, &integrations, &interaction, config).await?; let conversation = Conversation::build(&rt, &models, &media, &tools, &integrations, &interaction, config).await?;
let infra = Infra::build(); let infra = Infra::build();
@@ -159,6 +159,7 @@ impl UserContextFactory {
let manager = Arc::new(ChatSessionManager::new( let manager = Arc::new(ChatSessionManager::new(
Arc::clone(&pool), Arc::clone(&pool),
Arc::clone(&self.registry_pool), // shared pool = system.db, for shared-memory injection
user_id.to_string(), user_id.to_string(),
Arc::clone(&self.llm_manager), Arc::clone(&self.llm_manager),
self.max_history_messages, self.max_history_messages,
+76 -40
View File
@@ -1,8 +1,55 @@
use std::sync::Arc;
use anyhow::Result; use anyhow::Result;
use serde_json::{Value, json}; use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT}; use crate::tools::{
use super::{read_to_string, write_string}; SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT,
};
use super::{classify_memory, read_to_string, write_string, MemScope};
/// Applies the substring edit to `content`, returning the new content. Shared by
/// the on-disk [`EditFile::execute`] and the `memory/` routing in `run_with`;
/// `display` is the path used in error messages.
fn apply_edit(content: &str, args: &Value, display: &str) -> Result<String> {
let old = args["old"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: old"))?;
let new = args["new"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: new"))?;
let replace_all = args["replace_all"].as_bool().unwrap_or(false);
let updated = if replace_all {
if !content.contains(old) {
anyhow::bail!(
"Text not found in {display}. \
Call read_file first and copy the text exactly as shown after the '| ' prefix."
);
}
content.replace(old, new)
} else {
let exact_count = content.matches(old).count();
if exact_count > 1 {
anyhow::bail!(
"Text found {exact_count} times in {display}. \
Include more surrounding context in `old` to make it unique, or set replace_all=true."
);
}
if exact_count == 1 {
content.replacen(old, new, 1)
} else {
let normalized_old = normalize_ws(old);
let (start, end) = find_normalized(content, &normalized_old)
.ok_or_else(|| anyhow::anyhow!(
"Text not found in {display}. \
Call read_file first and copy the text exactly as shown after the '| ' prefix."
))?;
format!("{}{}{}", &content[..start], new, &content[end..])
}
};
Ok(updated)
}
fn normalize_ws(s: &str) -> String { fn normalize_ws(s: &str) -> String {
s.lines() s.lines()
@@ -56,10 +103,13 @@ fn find_normalized(haystack: &str, normalized_needle: &str) -> Option<(usize, us
None None
} }
pub struct EditFile; pub struct EditFile {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl EditFile { impl EditFile {
pub fn new() -> Self { Self } pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
} }
impl Tool for EditFile { impl Tool for EditFile {
@@ -102,46 +152,32 @@ impl Tool for EditFile {
truncate_label(&format!("edit_file `{path}`"), MAX_LABEL_SHORT) truncate_label(&format!("edit_file `{path}`"), MAX_LABEL_SHORT)
} }
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute).
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = super::path_arg(&args).unwrap_or_default();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
Box::new(SimpleExecution::new(Box::pin(async move {
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
anyhow::bail!("No note at {path}");
};
let updated = apply_edit(&doc.content, &args, &path)?;
crate::db::memory_docs::upsert(&pool, &rel, &updated).await?;
Ok(ToolResult::Text(format!("Edited {path}.")))
})))
}
fn execute(&self, args: Value) -> Result<String> { fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str() let user_path = args["path"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?; .ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
let old = args["old"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: old"))?;
let new = args["new"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: new"))?;
let replace_all = args["replace_all"].as_bool().unwrap_or(false);
let content = read_to_string(user_path)?; let content = read_to_string(user_path)?;
let updated = apply_edit(&content, &args, user_path)?;
let updated = if replace_all {
if !content.contains(old) {
anyhow::bail!(
"Text not found in {user_path}. \
Call read_file first and copy the text exactly as shown after the '| ' prefix."
);
}
content.replace(old, new)
} else {
let exact_count = content.matches(old).count();
if exact_count > 1 {
anyhow::bail!(
"Text found {exact_count} times in {user_path}. \
Include more surrounding context in `old` to make it unique, or set replace_all=true."
);
}
if exact_count == 1 {
content.replacen(old, new, 1)
} else {
let normalized_old = normalize_ws(old);
let (start, end) = find_normalized(&content, &normalized_old)
.ok_or_else(|| anyhow::anyhow!(
"Text not found in {user_path}. \
Call read_file first and copy the text exactly as shown after the '| ' prefix."
))?;
format!("{}{}{}", &content[..start], new, &content[end..])
}
};
write_string(user_path, &updated)?; write_string(user_path, &updated)?;
Ok(format!("Edited {user_path}.")) Ok(format!("Edited {user_path}."))
} }
@@ -1,13 +1,49 @@
use std::sync::Arc;
use anyhow::Result; use anyhow::Result;
use serde_json::{Value, json}; use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL}; use crate::tools::{
use super::{read_to_string, write_string}; SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL,
};
use super::{classify_memory, read_to_string, write_string, MemScope};
pub struct InsertAtLine; pub struct InsertAtLine {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl InsertAtLine { impl InsertAtLine {
pub fn new() -> Self { Self } pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
}
/// Inserts `content` before/after `line` in `text`, returning the new text and a
/// result message. Shared by the on-disk `execute` and the `memory/` routing;
/// `display` is the path used in the message.
fn apply_insert(text: &str, args: &Value, display: &str) -> Result<(String, String)> {
let line_num = args["line"].as_u64()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: line"))? as usize;
let new_text = args["content"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: content"))?;
let placement = args["placement"].as_str().unwrap_or("after");
anyhow::ensure!(line_num >= 1, "line must be >= 1");
let mut lines: Vec<&str> = text.split('\n').collect();
let idx = (line_num - 1).min(lines.len().saturating_sub(1));
let insert_idx = if placement == "before" { idx } else { idx + 1 };
let new_lines: Vec<&str> = new_text.split('\n').collect();
for (i, l) in new_lines.iter().enumerate() {
lines.insert(insert_idx + i, l);
}
let updated = lines.join("\n");
let msg = format!(
"Inserted {} line(s) {} line {} in {display}.",
new_lines.len(), placement, line_num
);
Ok((updated, msg))
} }
impl Tool for InsertAtLine { impl Tool for InsertAtLine {
@@ -53,32 +89,33 @@ impl Tool for InsertAtLine {
} }
} }
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute).
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = super::path_arg(&args).unwrap_or_default();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
Box::new(SimpleExecution::new(Box::pin(async move {
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
anyhow::bail!("No note at {path}");
};
let (updated, msg) = apply_insert(&doc.content, &args, &path)?;
crate::db::memory_docs::upsert(&pool, &rel, &updated).await?;
Ok(ToolResult::Text(msg))
})))
}
fn execute(&self, args: Value) -> Result<String> { fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str() let user_path = args["path"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?; .ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
let line_num = args["line"].as_u64()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: line"))? as usize;
let new_text = args["content"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: content"))?;
let placement = args["placement"].as_str().unwrap_or("after");
anyhow::ensure!(line_num >= 1, "line must be >= 1");
let text = read_to_string(user_path)?; let text = read_to_string(user_path)?;
let mut lines: Vec<&str> = text.split('\n').collect(); let (updated, msg) = apply_insert(&text, &args, user_path)?;
let idx = (line_num - 1).min(lines.len().saturating_sub(1));
let insert_idx = if placement == "before" { idx } else { idx + 1 };
let new_lines: Vec<&str> = new_text.split('\n').collect();
for (i, l) in new_lines.iter().enumerate() {
lines.insert(insert_idx + i, l);
}
let updated = lines.join("\n");
write_string(user_path, &updated)?; write_string(user_path, &updated)?;
Ok(msg)
Ok(format!(
"Inserted {} line(s) {} line {} in {user_path}.",
new_lines.len(), placement, line_num
))
} }
} }
+40 -5
View File
@@ -1,20 +1,28 @@
use std::path::Path; use std::path::Path;
use std::sync::Arc;
use anyhow::Result; use anyhow::Result;
use serde_json::{Value, json}; use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT}; use crate::tools::{
use super::resolve; SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT,
};
use super::{classify_memory, resolve, MemScope};
/// Directories to skip unconditionally when walking. /// Directories to skip unconditionally when walking.
/// `secrets` is skipped so a recursive listing rooted at a parent (e.g. the auto-read /// `secrets` is skipped so a recursive listing rooted at a parent (e.g. the auto-read
/// working directory) never reveals the contents of the secrets store. /// working directory) never reveals the contents of the secrets store.
const SKIP_DIRS: &[&str] = &[".git", "target", "node_modules", ".cache", "secrets"]; const SKIP_DIRS: &[&str] = &[".git", "target", "node_modules", ".cache", "secrets"];
pub struct ListFiles; pub struct ListFiles {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl ListFiles { impl ListFiles {
pub fn new() -> Self { Self } pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
} }
impl Tool for ListFiles { impl Tool for ListFiles {
@@ -27,7 +35,8 @@ impl Tool for ListFiles {
Relative paths are resolved from the project root; absolute paths (starting with /) are used as-is. \ Relative paths are resolved from the project root; absolute paths (starting with /) are used as-is. \
Skips .git, target, node_modules, .cache. \ Skips .git, target, node_modules, .cache. \
Returns a JSON array of paths relative to the requested directory. \ Returns a JSON array of paths relative to the requested directory. \
Use depth=1 for immediate contents only, depth=2-3 for moderate exploration." Use depth=1 for immediate contents only, depth=2-3 for moderate exploration. \
Listing under user-memory/ (private) or shared-memory/ (shared) lists your memory notes instead of disk."
} }
fn parameters_schema(&self) -> Value { fn parameters_schema(&self) -> Value {
@@ -56,6 +65,32 @@ impl Tool for ListFiles {
truncate_label(&format!("list_files `{path}`"), MAX_LABEL_SHORT) truncate_label(&format!("list_files `{path}`"), MAX_LABEL_SHORT)
} }
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute). Memory is a
/// flat key space, so `depth`/`dirs_only` don't apply — the whole subtree
/// under the prefix is returned, keyed relative to the requested directory.
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = args["path"].as_str().unwrap_or("").to_string();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
Box::new(SimpleExecution::new(Box::pin(async move {
// Treat `rel` as a directory prefix: match `rel/…` (or everything at
// the root), then strip it so results are relative to what was asked.
let prefix = if rel.is_empty() || rel.ends_with('/') { rel } else { format!("{rel}/") };
let entries = crate::db::memory_docs::list(&pool, &prefix).await?;
let mut paths: Vec<String> = entries.into_iter()
.map(|e| e.path.strip_prefix(&prefix).unwrap_or(&e.path).to_string())
.collect();
paths.sort();
Ok(ToolResult::Text(serde_json::to_string(&paths)?))
})))
}
fn execute(&self, args: Value) -> Result<String> { fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str().unwrap_or("."); let user_path = args["path"].as_str().unwrap_or(".");
let max_depth = args["depth"].as_u64().unwrap_or(3) as usize; let max_depth = args["depth"].as_u64().unwrap_or(3) as usize;
@@ -0,0 +1,125 @@
//! `memory_search` — full-text search over the virtual memory namespace (§5).
//!
//! Unlike the fs-tools, this does **not** route a path: it searches note *content*
//! through the `memory_docs` FTS5 index (`memory_docs::search`, bm25-ranked with a
//! highlighted snippet). `user-memory` is the caller's own pool (`ToolContext::pool`);
//! `shared-memory` is the system pool captured at registration. Kept a distinct tool
//! rather than folding FTS into `grep_files`: grep is regex-per-line over a tree,
//! this is ranked keyword recall — different semantics, so different names.
use std::sync::Arc;
use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::db::memory_docs::{self, MemoryHit};
use crate::tools::{
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT,
};
pub struct MemorySearch {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl MemorySearch {
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
}
/// Turns free text into a robust FTS5 MATCH query: each whitespace token becomes a
/// quoted term (AND-combined), so arbitrary input — colons, dashes, punctuation —
/// can't trip an FTS5 syntax error. Returns `None` when there are no tokens.
fn fts_query(input: &str) -> Option<String> {
let terms: Vec<String> = input
.split_whitespace()
.map(|t| format!("\"{}\"", t.replace('"', "\"\"")))
.collect();
(!terms.is_empty()).then(|| terms.join(" "))
}
fn render_hits(store: &str, hits: &[MemoryHit], out: &mut String) {
for h in hits {
out.push_str(&format!("[{store}] {}{}\n", h.path, h.snippet));
}
}
impl Tool for MemorySearch {
fn name(&self) -> &str { "memory_search" }
fn category(&self) -> crate::tools::ToolCategory { crate::tools::ToolCategory::Introspection }
fn description(&self) -> &str {
"Full-text search across your memory notes by keyword, ranked by relevance. \
Searches user-memory/ (private) and shared-memory/ (shared); set scope to narrow it. \
Returns matching note paths with a short highlighted snippet — open one with read_file. \
Use this to recall where you wrote something instead of listing and reading notes one by one."
}
fn parameters_schema(&self) -> Value {
json!({
"type": "object",
"properties": {
"query": { "type": "string", "description": "Keywords to search for. Plain words; all must appear." },
"scope": {
"type": "string",
"enum": ["all", "private", "shared"],
"description": "Which store to search: 'private' (user-memory), 'shared' (shared-memory), or 'all' (default)."
},
"limit": { "type": "integer", "description": "Max results per store (default 10, max 50).", "default": 10 }
},
"required": ["query"]
})
}
fn describe(&self, args: &Value, _length: ToolDescriptionLength) -> String {
let q = args["query"].as_str().unwrap_or("?");
truncate_label(&format!("memory_search \"{q}\""), MAX_LABEL_SHORT)
}
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let user_pool = Arc::clone(&ctx.pool);
let shared_pool = Arc::clone(&self.shared_pool);
Box::new(SimpleExecution::new(Box::pin(async move {
let raw = args["query"].as_str().unwrap_or("");
let Some(q) = fts_query(raw) else {
anyhow::bail!("memory_search needs a non-empty query");
};
let scope = args["scope"].as_str().unwrap_or("all");
let limit = args["limit"].as_u64().unwrap_or(10).clamp(1, 50) as i64;
let mut out = String::new();
let mut total = 0usize;
if scope == "all" || scope == "private" {
let hits = memory_docs::search(&user_pool, &q, limit).await?;
total += hits.len();
render_hits("user-memory", &hits, &mut out);
}
if scope == "all" || scope == "shared" {
let hits = memory_docs::search(&shared_pool, &q, limit).await?;
total += hits.len();
render_hits("shared-memory", &hits, &mut out);
}
if total == 0 {
return Ok(ToolResult::Text(format!("No memory notes match {raw:?}.")));
}
Ok(ToolResult::Text(format!("{total} result(s):\n{out}")))
})))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn fts_query_quotes_tokens_and_survives_punctuation() {
assert_eq!(fts_query("spesa settimana").unwrap(), "\"spesa\" \"settimana\"");
// colons / dashes would be FTS5 operators unquoted; quoting makes them literal
assert_eq!(fts_query("budget: 2026-07").unwrap(), "\"budget:\" \"2026-07\"");
// an embedded quote is escaped by doubling
assert_eq!(fts_query("say \"hi\"").unwrap(), "\"say\" \"\"\"hi\"\"\"");
assert!(fts_query(" ").is_none());
}
}
+262 -9
View File
@@ -2,15 +2,18 @@ mod edit_file;
mod grep_files; mod grep_files;
mod insert_at_line; mod insert_at_line;
mod list_files; mod list_files;
mod memory_search;
mod read_file; mod read_file;
mod replace_lines; mod replace_lines;
mod search_file; mod search_file;
mod write_file; mod write_file;
use std::path::{Component, Path, PathBuf}; use std::path::{Component, Path, PathBuf};
use std::sync::Arc;
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use serde_json::Value; use serde_json::Value;
use sqlx::SqlitePool;
use crate::tools::ToolRegistry; use crate::tools::ToolRegistry;
@@ -25,11 +28,65 @@ pub use edit_file::EditFile;
pub use grep_files::GrepFiles; pub use grep_files::GrepFiles;
pub use insert_at_line::InsertAtLine; pub use insert_at_line::InsertAtLine;
pub use list_files::ListFiles; pub use list_files::ListFiles;
pub use memory_search::MemorySearch;
pub use read_file::ReadFile; pub use read_file::ReadFile;
pub use replace_lines::ReplaceLines; pub use replace_lines::ReplaceLines;
pub use search_file::SearchFile; pub use search_file::SearchFile;
pub use write_file::WriteFile; pub use write_file::WriteFile;
// ── Virtual memory namespace (blueprint §5) ───────────────────────────────────
//
// Two sibling top-level roots, each backed by the `memory_docs` table in SQLite
// rather than the disk. The fs-tools intercept these prefixes in `run_with` and
// route reads/writes to the `memory_docs` accessor on the right pool, so the LLM
// uses ordinary read/write/list against what looks like two folders.
/// The current user's **private** memory — routed to `ctx.pool` (`{userid}.db`,
/// behind SQLCipher).
pub const USER_MEMORY_ROOT: &str = "user-memory";
/// The instance-wide **shared** memory — routed to the system pool (`system.db`,
/// cleartext, readable by every member).
pub const SHARED_MEMORY_ROOT: &str = "shared-memory";
/// Which memory store a path resolves to.
pub(crate) enum MemScope {
/// `user-memory/…` → the caller's own pool (`ToolContext::pool`).
User,
/// `shared-memory/…` → the shared system pool.
Shared,
}
/// A path that falls inside the virtual memory namespace: the store it belongs to
/// and the note key **relative to that store's root** (the root prefix stripped).
pub(crate) struct MemRef {
pub scope: MemScope,
pub rel: String,
}
/// Classifies a user-supplied path. Returns `Some` when it lands under one of the
/// virtual memory roots — to be routed to SQLite — and `None` for an ordinary
/// disk path.
///
/// The **first** component decides the store, taken raw *before* normalization, so
/// a `..` in the tail can never drop the memory root and silently fall back to a
/// disk path. The tail is then normalized (resolving `.`/`..`) and clamped at the
/// store root, so a memory path stays within its store and an absolute path is
/// always disk.
pub(crate) fn classify_memory(user_path: &str) -> Option<MemRef> {
let mut parts = user_path.trim_start_matches("./").splitn(2, ['/', '\\']);
let scope = match parts.next()? {
USER_MEMORY_ROOT => MemScope::User,
SHARED_MEMORY_ROOT => MemScope::Shared,
_ => return None,
};
// Normalize the tail within the store (empty = the root itself). `..` clamps
// at the root rather than escaping upward.
let tail = parts.next().unwrap_or("");
let rel = lexical_normalize(Path::new(tail)).to_string_lossy().replace('\\', "/");
Some(MemRef { scope, rel })
}
/// Resolve a user-supplied path: /// Resolve a user-supplied path:
/// - starts with `/` → absolute path, used as-is /// - starts with `/` → absolute path, used as-is
/// - otherwise → relative to the process working directory (project root) /// - otherwise → relative to the process working directory (project root)
@@ -127,13 +184,209 @@ pub(super) fn write_string(user_path: &str, content: &str) -> Result<()> {
.with_context(|| format!("Failed to write: {}", abs.display())) .with_context(|| format!("Failed to write: {}", abs.display()))
} }
pub fn register_all(registry: &mut ToolRegistry) { /// Registers the filesystem tools. `shared_pool` is the system (`shared-memory`)
registry.register(EditFile::new()); /// pool captured once here — a global singleton — and handed to the memory-aware
registry.register(GrepFiles::new()); /// tools; each still resolves the per-user (`user-memory`) pool per call from the
registry.register(InsertAtLine::new()); /// `ToolContext`.
registry.register(ListFiles::new()); pub fn register_all(registry: &mut ToolRegistry, shared_pool: Arc<SqlitePool>) {
registry.register(ReadFile::new()); registry.register(EditFile::new(Arc::clone(&shared_pool)));
registry.register(ReplaceLines::new()); registry.register(GrepFiles::new()); // not memory-aware yet — see blueprint Prossimi passi
registry.register(SearchFile::new()); registry.register(InsertAtLine::new(Arc::clone(&shared_pool)));
registry.register(WriteFile::new()); registry.register(ListFiles::new(Arc::clone(&shared_pool)));
registry.register(ReadFile::new(Arc::clone(&shared_pool)));
registry.register(ReplaceLines::new(Arc::clone(&shared_pool)));
registry.register(SearchFile::new(Arc::clone(&shared_pool)));
registry.register(MemorySearch::new(Arc::clone(&shared_pool)));
registry.register(WriteFile::new(shared_pool));
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::PathBuf;
use serde_json::json;
use crate::tools::{ExecutionOutcome, Tool, ToolContext};
#[test]
fn classify_memory_splits_root_from_key() {
let u = classify_memory("user-memory/notes/x.md").unwrap();
assert!(matches!(u.scope, MemScope::User));
assert_eq!(u.rel, "notes/x.md");
let s = classify_memory("./shared-memory/casa.md").unwrap();
assert!(matches!(s.scope, MemScope::Shared));
assert_eq!(s.rel, "casa.md");
// bare roots (with/without trailing slash) resolve to the empty key
assert_eq!(classify_memory("user-memory").unwrap().rel, "");
assert_eq!(classify_memory("shared-memory/").unwrap().rel, "");
// `..` clamps inside the store instead of falling back to a disk path
assert_eq!(classify_memory("user-memory/../secret.md").unwrap().rel, "secret.md");
// ordinary, absolute, and look-alike paths are disk (None)
assert!(classify_memory("src/main.rs").is_none());
assert!(classify_memory("/etc/hosts").is_none());
assert!(classify_memory("user-memoryish/x").is_none());
}
/// A throwaway owner-schema pool (as `Arc`, ready for a `ToolContext`), plus its
/// dir for cleanup. `tag` + a counter keep parallel tests off the same file.
async fn store(tag: &str) -> (Arc<SqlitePool>, PathBuf) {
use std::sync::atomic::{AtomicU64, Ordering};
static SEQ: AtomicU64 = AtomicU64::new(0);
let n = SEQ.fetch_add(1, Ordering::Relaxed);
let dir = std::env::temp_dir().join(format!("skald-fsmem-{}-{tag}-{n}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let pool = crate::db::create_user_pool(&dir.join("owner.db"), None).await.unwrap();
(Arc::new(pool), dir)
}
/// Drives a tool through the context-aware path and returns its text result.
async fn drive(tool: &dyn Tool, ctx: &ToolContext, args: Value) -> Result<String, String> {
let exec = tool.run_with(ctx, args);
match exec.wait().await {
ExecutionOutcome::Completed(r) => Ok(r.to_wire()),
ExecutionOutcome::Failed(e) => Err(e),
ExecutionOutcome::Cancelled => Err("cancelled".into()),
}
}
#[tokio::test]
async fn memory_tools_route_and_isolate_user_vs_shared() {
let (user, udir) = store("user").await;
let (shared, sdir) = store("shared").await;
// The shared pool is captured by the tools; the user pool arrives per call.
let write = WriteFile::new(Arc::clone(&shared));
let read = ReadFile::new(Arc::clone(&shared));
let list = ListFiles::new(Arc::clone(&shared));
let ctx = ToolContext { session_id: 1, pool: Arc::clone(&user) };
// Private write lands in the user pool — and never in the shared one.
let out = drive(&write, &ctx, json!({"path":"user-memory/spesa.md","content":"latte\npane"}))
.await.unwrap();
assert!(out.starts_with("Created user-memory/spesa.md"), "{out}");
assert!(crate::db::memory_docs::get(&user, "spesa.md").await.unwrap().is_some());
assert!(crate::db::memory_docs::get(&shared, "spesa.md").await.unwrap().is_none(),
"a user-memory write must not touch the shared store");
// Shared write lands in the shared pool — and never in the user one.
drive(&write, &ctx, json!({"path":"shared-memory/casa.md","content":"wifi 1234"}))
.await.unwrap();
assert!(crate::db::memory_docs::get(&shared, "casa.md").await.unwrap().is_some());
assert!(crate::db::memory_docs::get(&user, "casa.md").await.unwrap().is_none());
// Read back with 1-based line numbers; a missing note errors.
let r = drive(&read, &ctx, json!({"path":"user-memory/spesa.md"})).await.unwrap();
assert!(r.contains("| latte") && r.contains("| pane"), "{r}");
assert!(drive(&read, &ctx, json!({"path":"user-memory/nope.md"})).await.is_err());
// A second write to the same key overwrites (and says so).
let out = drive(&write, &ctx, json!({"path":"user-memory/spesa.md","content":"latte"}))
.await.unwrap();
assert!(out.starts_with("Overwrote user-memory/spesa.md"), "{out}");
// Listing returns keys relative to the requested directory.
drive(&write, &ctx, json!({"path":"user-memory/notes/idee.md","content":"x"}))
.await.unwrap();
let l = drive(&list, &ctx, json!({"path":"user-memory"})).await.unwrap();
assert_eq!(serde_json::from_str::<Vec<String>>(&l).unwrap(),
vec!["notes/idee.md".to_string(), "spesa.md".to_string()]);
let l = drive(&list, &ctx, json!({"path":"user-memory/notes"})).await.unwrap();
assert_eq!(serde_json::from_str::<Vec<String>>(&l).unwrap(),
vec!["idee.md".to_string()]);
let _ = std::fs::remove_dir_all(&udir);
let _ = std::fs::remove_dir_all(&sdir);
}
#[tokio::test]
async fn memory_edit_insert_replace_search_route_to_the_note() {
let (user, udir) = store("edit-user").await;
let (shared, sdir) = store("edit-shared").await;
let write = WriteFile::new(Arc::clone(&shared));
let edit = EditFile::new(Arc::clone(&shared));
let insert = InsertAtLine::new(Arc::clone(&shared));
let replace = ReplaceLines::new(Arc::clone(&shared));
let search = SearchFile::new(Arc::clone(&shared));
let ctx = ToolContext { session_id: 1, pool: Arc::clone(&user) };
async fn note(pool: &SqlitePool, path: &str) -> String {
crate::db::memory_docs::get(pool, path).await.unwrap().unwrap().content
}
drive(&write, &ctx, json!({"path":"user-memory/todo.md","content":"latte\npane\nuvoa"}))
.await.unwrap();
// edit_file: fix the typo, in place
let out = drive(&edit, &ctx, json!({"path":"user-memory/todo.md","old":"uvoa","new":"uova"}))
.await.unwrap();
assert_eq!(out, "Edited user-memory/todo.md.");
assert_eq!(note(&user, "todo.md").await, "latte\npane\nuova");
// insert_at_line: add a line after line 1
drive(&insert, &ctx, json!({"path":"user-memory/todo.md","line":1,"content":"burro","placement":"after"}))
.await.unwrap();
assert_eq!(note(&user, "todo.md").await, "latte\nburro\npane\nuova");
// replace_lines: collapse lines 23 into one
drive(&replace, &ctx, json!({"path":"user-memory/todo.md","from_line":2,"to_line":3,"new":"olio"}))
.await.unwrap();
assert_eq!(note(&user, "todo.md").await, "latte\nolio\nuova");
// search_file: find a line inside the note
let s = drive(&search, &ctx, json!({"path":"user-memory/todo.md","query":"olio"})).await.unwrap();
assert!(s.contains("match(es) in user-memory/todo.md"), "{s}");
assert!(s.contains("| olio"), "{s}");
// editing a note that doesn't exist errors, not creates
assert!(drive(&edit, &ctx, json!({"path":"user-memory/ghost.md","old":"a","new":"b"}))
.await.is_err());
let _ = std::fs::remove_dir_all(&udir);
let _ = std::fs::remove_dir_all(&sdir);
}
#[tokio::test]
async fn memory_search_scopes_to_user_shared_or_all() {
let (user, udir) = store("search-user").await;
let (shared, sdir) = store("search-shared").await;
let write = WriteFile::new(Arc::clone(&shared));
let search = MemorySearch::new(Arc::clone(&shared));
let ctx = ToolContext { session_id: 1, pool: Arc::clone(&user) };
// one note in each store, both mentioning "wifi"
drive(&write, &ctx, json!({"path":"user-memory/rete.md","content":"la mia wifi privata"}))
.await.unwrap();
drive(&write, &ctx, json!({"path":"shared-memory/casa.md","content":"wifi di casa 1234"}))
.await.unwrap();
// scope=private → only the user store
let r = drive(&search, &ctx, json!({"query":"wifi","scope":"private"})).await.unwrap();
assert!(r.contains("[user-memory] rete.md"), "{r}");
assert!(!r.contains("shared-memory"), "{r}");
// scope=shared → only the shared store
let r = drive(&search, &ctx, json!({"query":"wifi","scope":"shared"})).await.unwrap();
assert!(r.contains("[shared-memory] casa.md"), "{r}");
assert!(!r.contains("[user-memory]"), "{r}");
// scope=all (default) → both, and the snippet highlights the term
let r = drive(&search, &ctx, json!({"query":"wifi"})).await.unwrap();
assert!(r.contains("[user-memory] rete.md") && r.contains("[shared-memory] casa.md"), "{r}");
assert!(r.contains("[wifi]"), "snippet should highlight the match: {r}");
// no match → a friendly message, not an error
let r = drive(&search, &ctx, json!({"query":"inesistente"})).await.unwrap();
assert!(r.starts_with("No memory notes match"), "{r}");
let _ = std::fs::remove_dir_all(&udir);
let _ = std::fs::remove_dir_all(&sdir);
}
} }
+68 -32
View File
@@ -1,13 +1,50 @@
use std::sync::Arc;
use anyhow::Result; use anyhow::Result;
use serde_json::{Value, json}; use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL}; use crate::tools::{
use super::read_to_string; SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL,
};
use super::{classify_memory, read_to_string, MemScope};
pub struct ReadFile; pub struct ReadFile {
/// The `shared-memory` (system) pool. `user-memory` resolves per call from the
/// `ToolContext`; only the shared store is a global singleton captured here.
shared_pool: Arc<SqlitePool>,
}
impl ReadFile { impl ReadFile {
pub fn new() -> Self { Self } pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
}
/// Render `content` with 1-based line numbers, honouring the same
/// `start`/`end_line`/`limit` windowing as the disk path. Shared by the on-disk
/// [`ReadFile::execute`] and the `memory/` routing in [`ReadFile::run_with`].
fn number_lines(content: &str, start: usize, end_line: Option<usize>, limit: Option<usize>) -> String {
let lines: Vec<&str> = content.lines().collect();
let total = lines.len();
let end = match (end_line, limit) {
(Some(e), _) => e.min(total),
(None, Some(l)) => (start + l).min(total),
(None, None) => total,
};
if start >= total && total > 0 {
return format!("(file has only {total} lines; start_line {} is out of range)", start + 1);
}
let end = end.max(start);
let width = total.to_string().len().max(3);
lines[start..end]
.iter()
.enumerate()
.map(|(i, line)| format!("{:>width$} | {line}", start + i + 1))
.collect::<Vec<_>>()
.join("\n")
} }
impl Tool for ReadFile { impl Tool for ReadFile {
@@ -19,7 +56,8 @@ impl Tool for ReadFile {
Use instead of cat/head/tail in the terminal. \ Use instead of cat/head/tail in the terminal. \
Returns text prefixed as ' N | line'. When calling edit_file, copy the text after '| ' exactly. \ Returns text prefixed as ' N | line'. When calling edit_file, copy the text after '| ' exactly. \
For large files use start_line/end_line to read in chunks — files over ~2000 lines should never be read whole. \ For large files use start_line/end_line to read in chunks — files over ~2000 lines should never be read whole. \
Use limit to cap output when end_line is unknown." Use limit to cap output when end_line is unknown. \
Paths under user-memory/ (private) or shared-memory/ (shared) read a note from your memory instead of disk."
} }
fn parameters_schema(&self) -> Value { fn parameters_schema(&self) -> Value {
@@ -70,37 +108,35 @@ impl Tool for ReadFile {
} }
} }
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute).
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = super::path_arg(&args).unwrap_or_default();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
let start = args["start_line"].as_u64().map(|n| (n as usize).saturating_sub(1)).unwrap_or(0);
let end_line = args["end_line"].as_u64().map(|n| n as usize);
let limit = args["limit"].as_u64().map(|n| n.min(2000) as usize);
Box::new(SimpleExecution::new(Box::pin(async move {
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
anyhow::bail!("No note at {path}");
};
Ok(ToolResult::Text(number_lines(&doc.content, start, end_line, limit)))
})))
}
fn execute(&self, args: Value) -> Result<String> { fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str() let user_path = args["path"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?; .ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
let content = read_to_string(user_path)?; let content = read_to_string(user_path)?;
let lines: Vec<&str> = content.lines().collect(); let start = args["start_line"].as_u64().map(|n| (n as usize).saturating_sub(1)).unwrap_or(0);
let total = lines.len(); let end_line = args["end_line"].as_u64().map(|n| n as usize);
let limit = args["limit"].as_u64().map(|n| n.min(2000) as usize); let limit = args["limit"].as_u64().map(|n| n.min(2000) as usize);
let start = args["start_line"].as_u64() Ok(number_lines(&content, start, end_line, limit))
.map(|n| (n as usize).saturating_sub(1))
.unwrap_or(0);
let end = match (args["end_line"].as_u64(), limit) {
(Some(e), _) => (e as usize).min(total),
(None, Some(l)) => (start + l).min(total),
(None, None) => total,
};
if start >= total && total > 0 {
return Ok(format!("(file has only {total} lines; start_line {start_line} is out of range)",
start_line = start + 1));
}
let end = end.max(start);
let width = total.to_string().len().max(3);
let numbered = lines[start..end]
.iter()
.enumerate()
.map(|(i, line)| format!("{:>width$} | {line}", start + i + 1))
.collect::<Vec<_>>()
.join("\n");
Ok(numbered)
} }
} }
+70 -32
View File
@@ -1,13 +1,56 @@
use std::sync::Arc;
use anyhow::Result; use anyhow::Result;
use serde_json::{Value, json}; use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL}; use crate::tools::{
use super::{read_to_string, write_string}; SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL,
};
use super::{classify_memory, read_to_string, write_string, MemScope};
pub struct ReplaceLines; pub struct ReplaceLines {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl ReplaceLines { impl ReplaceLines {
pub fn new() -> Self { Self } pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
}
/// Replaces the inclusive 1-based line range with `new`, returning the new content
/// and a result message. Shared by the on-disk `execute` and the `memory/` routing;
/// `display` is the path used in the message.
fn apply_replace(content: &str, args: &Value, display: &str) -> Result<(String, String)> {
let from_line = args["from_line"].as_u64()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: from_line"))? as usize;
let to_line = args["to_line"].as_u64()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: to_line"))? as usize;
let new = args["new"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: new"))?;
if from_line == 0 { anyhow::bail!("from_line must be >= 1"); }
if to_line < from_line { anyhow::bail!("to_line must be >= from_line"); }
let mut lines: Vec<&str> = content.lines().collect();
let total = lines.len();
if from_line > total {
anyhow::bail!("from_line {from_line} exceeds file length ({total} lines)");
}
let to_clamped = to_line.min(total);
let new_lines: Vec<&str> = new.lines().collect();
lines.splice((from_line - 1)..to_clamped, new_lines);
let has_trailing = content.ends_with('\n');
let mut updated = lines.join("\n");
if has_trailing { updated.push('\n'); }
let msg = format!(
"Replaced lines {from_line}{to_clamped} in {display} with {} new lines.",
new.lines().count()
);
Ok((updated, msg))
} }
impl Tool for ReplaceLines { impl Tool for ReplaceLines {
@@ -51,38 +94,33 @@ impl Tool for ReplaceLines {
} }
} }
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute).
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = super::path_arg(&args).unwrap_or_default();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
Box::new(SimpleExecution::new(Box::pin(async move {
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
anyhow::bail!("No note at {path}");
};
let (updated, msg) = apply_replace(&doc.content, &args, &path)?;
crate::db::memory_docs::upsert(&pool, &rel, &updated).await?;
Ok(ToolResult::Text(msg))
})))
}
fn execute(&self, args: Value) -> Result<String> { fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str() let user_path = args["path"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?; .ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
let from_line = args["from_line"].as_u64()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: from_line"))? as usize;
let to_line = args["to_line"].as_u64()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: to_line"))? as usize;
let new = args["new"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: new"))?;
if from_line == 0 { anyhow::bail!("from_line must be >= 1"); }
if to_line < from_line { anyhow::bail!("to_line must be >= from_line"); }
let content = read_to_string(user_path)?; let content = read_to_string(user_path)?;
let mut lines: Vec<&str> = content.lines().collect(); let (updated, msg) = apply_replace(&content, &args, user_path)?;
let total = lines.len();
if from_line > total {
anyhow::bail!("from_line {from_line} exceeds file length ({total} lines)");
}
let to_clamped = to_line.min(total);
let new_lines: Vec<&str> = new.lines().collect();
lines.splice((from_line - 1)..to_clamped, new_lines);
let has_trailing = content.ends_with('\n');
let mut updated = lines.join("\n");
if has_trailing { updated.push('\n'); }
write_string(user_path, &updated)?; write_string(user_path, &updated)?;
Ok(msg)
Ok(format!(
"Replaced lines {from_line}{to_clamped} in {user_path} with {} new lines.",
new.lines().count()
))
} }
} }
+78 -43
View File
@@ -1,13 +1,67 @@
use std::sync::Arc;
use anyhow::Result; use anyhow::Result;
use serde_json::{Value, json}; use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL}; use crate::tools::{
use super::read_to_string; SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL,
};
use super::{classify_memory, read_to_string, MemScope};
pub struct SearchFile; pub struct SearchFile {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl SearchFile { impl SearchFile {
pub fn new() -> Self { Self } pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
}
/// Renders the case-insensitive substring search over `text` with context lines.
/// Shared by the on-disk `execute` and the `memory/` routing; `display` is the
/// path shown in the output.
fn render_search(text: &str, args: &Value, display: &str) -> Result<String> {
let query = args["query"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: query"))?;
let context = args["context_lines"].as_u64().unwrap_or(3).min(10) as usize;
let lines: Vec<&str> = text.lines().collect();
let lower_query = query.to_lowercase();
let width = lines.len().to_string().len().max(3);
let matches: Vec<usize> = lines.iter().enumerate()
.filter(|(_, l)| l.to_lowercase().contains(&lower_query))
.map(|(i, _)| i)
.collect();
if matches.is_empty() {
return Ok(format!("No matches found for {:?} in {display}.", query));
}
let mut chunks: Vec<(usize, usize)> = Vec::new();
for &m in &matches {
let start = m.saturating_sub(context);
let end = (m + context).min(lines.len() - 1);
if let Some(last) = chunks.last_mut() {
if start <= last.1 + 1 { last.1 = last.1.max(end); continue; }
}
chunks.push((start, end));
}
let match_set: std::collections::HashSet<usize> = matches.into_iter().collect();
let mut out = format!("{} match(es) in {display}:\n", match_set.len());
for (ci, (start, end)) in chunks.iter().enumerate() {
if ci > 0 { out.push_str(" ···\n"); }
for idx in *start..=*end {
let marker = if match_set.contains(&idx) { ">" } else { " " };
out.push_str(&format!("{marker}{:>width$} | {}\n", idx + 1, lines[idx]));
}
}
Ok(out)
} }
impl Tool for SearchFile { impl Tool for SearchFile {
@@ -53,48 +107,29 @@ impl Tool for SearchFile {
} }
} }
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute).
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = super::path_arg(&args).unwrap_or_default();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
Box::new(SimpleExecution::new(Box::pin(async move {
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
anyhow::bail!("No note at {path}");
};
Ok(ToolResult::Text(render_search(&doc.content, &args, &path)?))
})))
}
fn execute(&self, args: Value) -> Result<String> { fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str() let user_path = args["path"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?; .ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
let query = args["query"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: query"))?;
let context = args["context_lines"].as_u64().unwrap_or(3).min(10) as usize;
let text = read_to_string(user_path)?; let text = read_to_string(user_path)?;
let lines: Vec<&str> = text.lines().collect(); render_search(&text, &args, user_path)
let lower_query = query.to_lowercase();
let width = lines.len().to_string().len().max(3);
let matches: Vec<usize> = lines.iter().enumerate()
.filter(|(_, l)| l.to_lowercase().contains(&lower_query))
.map(|(i, _)| i)
.collect();
if matches.is_empty() {
return Ok(format!("No matches found for {:?} in {user_path}.", query));
}
let mut chunks: Vec<(usize, usize)> = Vec::new();
for &m in &matches {
let start = m.saturating_sub(context);
let end = (m + context).min(lines.len() - 1);
if let Some(last) = chunks.last_mut() {
if start <= last.1 + 1 { last.1 = last.1.max(end); continue; }
}
chunks.push((start, end));
}
let match_set: std::collections::HashSet<usize> = matches.into_iter().collect();
let mut out = format!("{} match(es) in {user_path}:\n", match_set.len());
for (ci, (start, end)) in chunks.iter().enumerate() {
if ci > 0 { out.push_str(" ···\n"); }
for idx in *start..=*end {
let marker = if match_set.contains(&idx) { ">" } else { " " };
out.push_str(&format!("{marker}{:>width$} | {}\n", idx + 1, lines[idx]));
}
}
Ok(out)
} }
} }
+39 -5
View File
@@ -1,13 +1,22 @@
use std::sync::Arc;
use anyhow::Result; use anyhow::Result;
use serde_json::{Value, json}; use serde_json::{Value, json};
use sqlx::SqlitePool;
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT}; use crate::tools::{
use super::{resolve, write_string}; SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
truncate_label, MAX_LABEL_SHORT,
};
use super::{classify_memory, resolve, write_string, MemScope};
pub struct WriteFile; pub struct WriteFile {
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
shared_pool: Arc<SqlitePool>,
}
impl WriteFile { impl WriteFile {
pub fn new() -> Self { Self } pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
} }
impl Tool for WriteFile { impl Tool for WriteFile {
@@ -18,7 +27,8 @@ impl Tool for WriteFile {
"Create a new file or fully overwrite an existing one. \ "Create a new file or fully overwrite an existing one. \
Use instead of echo/cat heredoc in the terminal. \ Use instead of echo/cat heredoc in the terminal. \
Relative paths are resolved from the project root; absolute paths (starting with /) are used as-is. \ Relative paths are resolved from the project root; absolute paths (starting with /) are used as-is. \
OVERWRITES the entire file — for targeted edits to an existing file use edit_file instead." OVERWRITES the entire file — for targeted edits to an existing file use edit_file instead. \
Write Markdown under user-memory/ (private to you) or shared-memory/ (shared with everyone) to save a durable note in your memory instead of on disk."
} }
fn parameters_schema(&self) -> Value { fn parameters_schema(&self) -> Value {
@@ -48,6 +58,30 @@ impl Tool for WriteFile {
truncate_label(&format!("write_file `{path}`"), MAX_LABEL_SHORT) truncate_label(&format!("write_file `{path}`"), MAX_LABEL_SHORT)
} }
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
/// path falls through to the on-disk [`execute`](Self::execute).
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let path = super::path_arg(&args).unwrap_or_default();
let Some(m) = classify_memory(&path) else { return self.run(args); };
let pool = match m.scope {
MemScope::User => Arc::clone(&ctx.pool),
MemScope::Shared => Arc::clone(&self.shared_pool),
};
let rel = m.rel;
let content = args["content"].as_str().map(str::to_string);
Box::new(SimpleExecution::new(Box::pin(async move {
let content = content.ok_or_else(|| anyhow::anyhow!("Missing required argument: content"))?;
if rel.is_empty() {
anyhow::bail!("{path} is a memory root, not a note — write to a path like {path}/notes.md");
}
let existed = crate::db::memory_docs::get(&pool, &rel).await?.is_some();
crate::db::memory_docs::upsert(&pool, &rel, &content).await?;
let verb = if existed { "Overwrote" } else { "Created" };
Ok(ToolResult::Text(format!("{verb} {path} ({} bytes).", content.len())))
})))
}
fn execute(&self, args: Value) -> Result<String> { fn execute(&self, args: Value) -> Result<String> {
let user_path = args["path"].as_str() let user_path = args["path"].as_str()
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?; .ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;