feat(memory): dual-pool memory namespace, FTS search, and prompt injection
Add a virtual memory namespace backed by SQLite, surfaced through the fs-tools, with private (per-user) and shared (system) stores. Storage - `memory_docs` owner table + external-content FTS5 index with sync triggers. - `db/memory_docs.rs` accessor: get / upsert / list / search (bm25+snippet) / delete. Routing (tools/fs) - `classify_memory` splits paths on the raw first component; `..` clamps inside the store, never escaping to disk. - read/write/list/edit/insert/replace/search_file route `user-memory/` to the owner pool and `shared-memory/` to the system pool (a singleton captured in `register_all`); every other path stays on disk. Each tool extracts a pure transform shared between its disk and memory paths. - New `memory_search` tool over the FTS index (scope private/shared/all), with a sanitised FTS5 query. grep_files stays disk-only. Approval - `user-memory/*` allow (read+write); `shared-memory/*` reads allow, writes require approval so the agent can't silently push one person's data into shared memory. `memory_search` allowed via a path-less rule. - migrate away the old `memory/*` and blanket `shared-memory/*` rows. Prompt injection - `MessageBuilder::load_inject_memory` reads `user-memory/` (owner pool) and `shared-memory/` (system pool) inject entries from SQLite; disk paths unchanged. The system pool is threaded ChatSessionManager -> handler -> MessageBuilder. - main and project-coordinator inject `user-memory/index.md` + `shared-memory/index.md`; common/memory.md rewritten for the two stores.
This commit is contained in:
@@ -69,7 +69,7 @@ Two rules keep the boundary real, and both are enforced by the compiler:
|
||||
| `crates/skald-core/src/chat_hub/` | `ChatHub`: broadcast events to all connected WS clients |
|
||||
| `crates/skald-core/src/chat_event_bus.rs` | Global async bus for cross-session events |
|
||||
| `crates/skald-core/src/agents.rs` | Discovers agents from `agents/*/`, loads meta + system prompt |
|
||||
| `crates/skald-core/src/tools/` | Built-in tools: `exec`, `restart`, `list_agents`, `fs/*`, `notify`, `ast_outline`, `image_generate`, MCP tools, plugin tools, cron tools |
|
||||
| `crates/skald-core/src/tools/` | Built-in tools: `exec`, `restart`, `list_agents`, `fs/*` (also route `user-memory/`/`shared-memory/` to `memory_docs` — see DB tables), `notify`, `ast_outline`, `image_generate`, MCP tools, plugin tools, cron tools |
|
||||
| `crates/skald-core/src/tool_catalog.rs` | `ToolCatalog`: unified tool listing façade (wraps ToolRegistry + McpManager) |
|
||||
| `crates/skald-core/src/events.rs` | `ServerEvent` enum streamed over WebSocket to the frontend |
|
||||
| `crates/skald-core/src/db/` | sqlx SQLite — see below |
|
||||
@@ -100,10 +100,14 @@ Two rules keep the boundary real, and both are enforced by the compiler:
|
||||
The schema is split into two buckets (§5.1), and the split is the point:
|
||||
|
||||
- **`create_registry_tables`** — instance-wide, readable without any user key: `users`, `llm_providers`, `llm_models`, `transcribe_models`, `tts_models`, `image_generate_models`, `plugins`, `approval_rules`, `tool_permission_groups`, `config`, `known_tools`, `llm_requests`.
|
||||
- **`create_owner_tables`** — one owner's content, **identical schema in every file that has it**: `chat_sessions`, `chat_sessions_stack`, `chat_history`, `chat_llm_tools`, `chat_summaries`, `session_scratchpad`, `session_mcp_grants`, `stack_mcp_grants`, `scheduled_jobs`, `job_runs`, `mcp_servers`, `mcp_events`, `sources`, `secrets`, `projects`, `project_tickets`.
|
||||
- **`create_owner_tables`** — one owner's content, **identical schema in every file that has it**: `chat_sessions`, `chat_sessions_stack`, `chat_history`, `chat_llm_tools`, `chat_summaries`, `session_scratchpad`, `session_mcp_grants`, `stack_mcp_grants`, `scheduled_jobs`, `job_runs`, `mcp_servers`, `mcp_events`, `sources`, `secrets`, `projects`, `project_tickets`, `llm_request_payloads`, `memory_docs` (+ FTS5 `memory_docs_fts`). Because `memory_docs` is an owner table, one definition backs **private** memory in each `{userid}.db` and **shared** memory in `system.db` (the household owner) — see the memory namespace note below.
|
||||
|
||||
**No foreign key in the owner bucket may point at a registry table.** SQLite cannot enforce a key across files, not even through `ATTACH`, and sqlx turns on `PRAGMA foreign_keys`: the `CREATE TABLE` succeeds and every `INSERT` fails. `db::tests::owner_tables_stand_alone_with_foreign_keys_on` enforces this by running the owner schema against a database holding nothing else, then inserting a row into each table. Two keys crossed and were fixed: `chat_history.model_db_id` (dropped — write-only, and `llm_requests.model_name` already records the model) and `project_tickets.job_id` (fixed by moving `projects`/`project_tickets` into the owner bucket).
|
||||
|
||||
**Memory namespace (blueprint §5).** `memory_docs` (accessor `db/memory_docs.rs` — `get`/`upsert`/`list`/`search`(FTS)/`delete`) backs a virtual note store surfaced through the fs-tools, **not** the disk. Two sibling roots (not the blueprint's nested `memory/{userid}` + `memory/shared`): `user-memory/…` routes to the caller's own pool (`ToolContext::pool`), `shared-memory/…` to the system pool (a singleton captured in `fs::register_all`). `tools/fs/classify_memory()` decides on the raw first path component (a `..` in the tail clamps inside the store, never escapes to disk); `read_file`/`write_file`/`list_files`/`edit_file`/`insert_at_line`/`replace_lines`/`search_file` override `run_with` to route memory paths (each extracting a pure transform shared with its on-disk `execute`) and leave every other path on disk. Approval (seeded in `seed_fs_path_rules`): `user-memory/*` is `@fs_any allow` (private, frictionless); `shared-memory/*` is `@fs_read allow` + `@fs_write require` — reads free, **writes need approval** so the agent can't silently push one person's data into shared memory. `grep_files` stays disk-only (regex-across-tree ≠ FTS); ranked full-text recall over notes is a separate tool, `memory_search` (`tools/fs/memory_search.rs`), over the `memory_docs` FTS index — allowed by a path-less rule (it takes `query`, not `path`).
|
||||
|
||||
**Memory injection into the prompt**: `MessageBuilder::load_inject_memory` routes each `meta.inject_memory` entry — `user-memory/…` → owner pool, `shared-memory/…` → the shared (`system.db`) pool, both via `memory_docs::get`; anything else (`data/…`, `$WD/…`) is a disk read. The shared pool is threaded `ChatSessionManager` → handler → `MessageBuilder`. `main` and `project-coordinator` inject `user-memory/index.md` + `shared-memory/index.md`.
|
||||
|
||||
`system.db` currently gets **both** bucket functions, because nothing has migrated to per-user pools yet. That is transitional.
|
||||
|
||||
`users` (`crates/skald-core/src/db/users.rs`) holds the directory plus auth material. It lives in the system DB, which the box owner can read, so it must never store anything that derives a user's key. `Credentials` is an enum mirroring the table's `CHECK`: an encrypted user carries a **wrapped DEK** (whose AEAD tag *is* the password verifier — hence no `password_hash`); a cleartext user carries an ordinary verifier, or none. `User` is deliberately not `Serialize` and its `Debug` redacts key material — use `User::summary()` for anything leaving the process. `role_id` has no foreign key yet: sqlx enables `PRAGMA foreign_keys`, so referencing the not-yet-existing `roles` table would fail every insert.
|
||||
|
||||
+30
-10
@@ -1,20 +1,41 @@
|
||||
# Persistent memory
|
||||
|
||||
All memory lives in `data/memory/`. Entry point: `data/memory/index.md` — one line per file with a brief summary.
|
||||
You have two persistent note stores, kept as Markdown and searchable. **Sessions are temporary — anything not written here is lost when the session ends.** Save proactively.
|
||||
|
||||
- **`user-memory/`** — your **private** memory for this user. Nobody else can read it. Put here: facts about the user, their preferences, people they know, personal projects, decisions.
|
||||
- **`shared-memory/`** — memory **shared with the whole group**. Every member can read it. Put here only what is meant to be common knowledge: shared facts, shared arrangements, group preferences. **Never** put one person's private information here. Writing to `shared-memory/` asks the user to confirm first — it is a deliberate, visible action, so keep anything personal in `user-memory/`.
|
||||
|
||||
When unsure where something belongs, prefer `user-memory/`.
|
||||
|
||||
## The indexes
|
||||
|
||||
Each store has an `index.md` — one line per note with a brief summary — and **both are injected into your context automatically** at the start of each session (look for them below):
|
||||
|
||||
- `user-memory/index.md` — your private notes.
|
||||
- `shared-memory/index.md` — the group's shared notes.
|
||||
|
||||
Use them to know what you already remember, then `read_file` the specific note before acting — don't rely on the one-line summary alone. **Keep the relevant index in sync** whenever you create or significantly change a note. Updating `shared-memory/index.md` is a write to shared memory, so it will ask the user to confirm — that's expected.
|
||||
|
||||
## When to save
|
||||
|
||||
Save **immediately** (do not postpone) when:
|
||||
|
||||
- The user shares a new fact about themselves, a project, a person, or a preference
|
||||
- A decision is made that may be relevant in future sessions
|
||||
- You notice an inconsistency with what was previously saved → correct it
|
||||
- A decision is made that may matter in a future session
|
||||
- You notice that something you saved before is now wrong → correct it
|
||||
|
||||
## When to read
|
||||
|
||||
At the start of each session, read `data/memory/index.md` silently. Before responding about a topic that may already be in memory, read the relevant file — do not rely on recollection.
|
||||
Before responding about a topic that may already be in memory, look it up — do not rely on recollection:
|
||||
|
||||
## File format
|
||||
- The injected `user-memory/index.md` tells you what exists; `read_file` the note it points to.
|
||||
- `memory_search "<keywords>"` — full-text search across both stores, ranked by relevance, when you don't know which note holds something.
|
||||
|
||||
## Organising notes
|
||||
|
||||
Use clear, topic-based paths — e.g. `user-memory/people/alice.md`, `user-memory/projects/website.md`, `shared-memory/wifi.md`. Keep one topic per note.
|
||||
|
||||
## Note format
|
||||
|
||||
```md
|
||||
# Title
|
||||
@@ -28,8 +49,7 @@ _Updated: YYYY-MM-DD_
|
||||
|
||||
## How to update
|
||||
|
||||
1. `read_file` to get the exact current content
|
||||
2. `edit_file` to modify — always keep the `_Updated: YYYY-MM-DD_` date in sync
|
||||
3. Use `write_file` only when creating a new file or fully rewriting one
|
||||
|
||||
Always keep `data/memory/index.md` in sync when you create or significantly update a file.
|
||||
1. `read_file` the note to get its exact current content.
|
||||
2. `edit_file` to change part of it — keep the `_Updated:_` date in sync.
|
||||
3. Use `write_file` only to create a new note or fully rewrite one.
|
||||
4. Keep `user-memory/index.md` in sync when you add or significantly change a note.
|
||||
|
||||
@@ -2,11 +2,9 @@
|
||||
|
||||
You are an extremely powerful general-purpose personal assistant. You help the user with any task — research, writing, planning, analysis, coding, or anything else they bring to you.
|
||||
|
||||
Your personality and tone are defined in `data/memory/SOUL.md`. If the file exists, it is automatically injected into your system context — look for it at the end of this prompt.
|
||||
|
||||
Think outside the box: you can use tools, write and execute Python scripts on the fly, or even modify your own source code.
|
||||
|
||||
The `data/` directory (inside your working directory) is your own space — write there freely; you have permission to create and modify anything under it. **Default to `data/` for everything you produce**: generated files, notes, one-shot scripts, downloads, and persistent memory (e.g. `data/memory/`, `data/notifications.md`). When a path is relative, prefix it with `data/` — a bare filename lands in the project root, which is not where your working files belong. Write **outside** `data/` (the project root, `src/`, `web/`, `agents/`, config, …) only when a specific, well-defined goal genuinely requires it and cannot be accomplished within `data/`.
|
||||
The `data/` directory (inside your working directory) is your own space — write there freely; you have permission to create and modify anything under it. **Default to `data/` for everything you produce**: generated files, notes, one-shot scripts, downloads. When a path is relative, prefix it with `data/` — a bare filename lands in the project root, which is not where your working files belong. (Persistent **memory** is separate: durable facts go to `user-memory/` or `shared-memory/`, not under `data/` — see the Memory section.) Write **outside** `data/` (the project root, `src/`, `web/`, `agents/`, config, …) only when a specific, well-defined goal genuinely requires it and cannot be accomplished within `data/`.
|
||||
|
||||
You have access to tools, persistent memory system and sub agents. Use both proactively. Sub agents also help to keep your context windows small and concise.
|
||||
|
||||
@@ -113,7 +111,7 @@ Configuration tools are hidden by default to keep context small. Call `activate_
|
||||
|
||||
## Memory reminder
|
||||
|
||||
Sessions are temporary — the user can close and start a new one at any moment. **Context alone is not enough.** If something is worth remembering, write it to a file in `data/memory/` immediately. If it stays only in context, it is gone forever when the session ends.
|
||||
Sessions are temporary — the user can close and start a new one at any moment. **Context alone is not enough.** If something is worth remembering, save it to `user-memory/` immediately (or `shared-memory/` if it's meant for the whole group). If it stays only in context, it is gone forever when the session ends.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
{
|
||||
"name": "Main Assistant",
|
||||
"description": "General-purpose assistant: helps the user with any task using tools, and persists all relevant information in data/memory",
|
||||
"description": "General-purpose assistant: helps the user with any task using tools, and persists all relevant information in memory",
|
||||
"friendly_description": "Your general-purpose assistant — helps with any task and remembers what matters in memory.",
|
||||
"type": "chat",
|
||||
"inject_memory": ["data/memory/index.md", "data/memory/SOUL.md"],
|
||||
"inject_memory": ["user-memory/index.md", "shared-memory/index.md"],
|
||||
"icon": "icon.png",
|
||||
"strength": "average"
|
||||
}
|
||||
|
||||
@@ -29,7 +29,7 @@ Delegate work to these task specialists via `execute_task` / `execute_subtask`:
|
||||
Your system prompt already contains, without you asking:
|
||||
|
||||
- The project's **name**, **description**, and **working directory** (the project root — all relative file paths resolve there). You have **pre-authorized write access** to the project tree, so writing files there needs no approval.
|
||||
- **`data/memory/index.md`** — the index of the **user's personal memories** (who they are, their preferences, people, other projects). It is injected automatically. Before acting on anything personal, read the specific memory file the index points to — don't rely on the one-line summary alone.
|
||||
- **`user-memory/index.md`** and **`shared-memory/index.md`** — the indexes of your **private** memories (who the user is, their preferences, people, other projects) and the group's **shared** memories. Both are injected automatically. Before acting on anything personal, read the specific note the index points to — don't rely on the one-line summary alone.
|
||||
- **`SKALD.md`** at the project root — this project's **living diary** (see below). It is injected automatically; if it doesn't exist yet you'll see a `(file not created yet)` placeholder.
|
||||
|
||||
Treat all of this as ground truth. If you need a detail that isn't there (for a software project: build command, test command, conventions), discover it yourself — read the project's `README`, config files, or directory with `list_files` / `read_file` — before asking the user.
|
||||
|
||||
@@ -5,6 +5,6 @@
|
||||
"type": "chat",
|
||||
"scope": "reasoning",
|
||||
"strength": "average",
|
||||
"inject_memory": ["data/memory/index.md", "$WD/SKALD.md"],
|
||||
"inject_memory": ["user-memory/index.md", "shared-memory/index.md", "$WD/SKALD.md"],
|
||||
"icon": "icon.png"
|
||||
}
|
||||
|
||||
@@ -29,11 +29,12 @@
|
||||
//! determines the action; if none matches the tool requires approval
|
||||
//! (default-closed).
|
||||
//!
|
||||
//! ## Hardcoded exception
|
||||
//! ## Memory namespace
|
||||
//!
|
||||
//! File-write tools targeting `memory/` paths bypass the rule engine and are
|
||||
//! always allowed (this mirrors the original behaviour and can be replaced by
|
||||
//! an explicit `allow` rule later).
|
||||
//! The virtual memory roots `user-memory/` and `shared-memory/` (blueprint §5) are
|
||||
//! allowed by **seeded rules**, not a hardcoded bypass: `seed_fs_path_rules` stamps
|
||||
//! `@fs_any allow user-memory/*` and `@fs_any allow shared-memory/*`, editable in the
|
||||
//! File System panel like any other path rule.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Arc;
|
||||
@@ -327,31 +328,52 @@ impl ApprovalManager {
|
||||
/// Priority `5` places these path-scoped rules *before* the `*` catch-all (999999),
|
||||
/// so an unmatched path still falls through to the default `require`.
|
||||
///
|
||||
/// - `memory/*` → **allow** (the LLM manages its own memory; replaces the former
|
||||
/// hardcoded `is_memory_path` bypass).
|
||||
/// - `user-memory/*` → **allow** (the caller's private memory namespace, blueprint §5;
|
||||
/// the LLM manages its own memory). Reads *and* writes: `@fs_any`.
|
||||
/// - `shared-memory/*` → reads **allow** (`@fs_read`), writes **require** (`@fs_write`):
|
||||
/// shared memory is visible to everyone, so a write is a deliberate, human-confirmed
|
||||
/// act — the agent must not silently push one person's information into it.
|
||||
/// - `data/*` → **allow** (scratch/data workspace).
|
||||
/// - `secrets/*` → **deny** (`@fs_any` denies reads *and* writes; a read would leak
|
||||
/// the secret into the LLM context / history / WS stream, and `Deny` is
|
||||
/// non-bypassable). The `/*` pattern also matches the `secrets` dir node itself, so
|
||||
/// recursive `list_files`/`grep_files` rooted at it are covered.
|
||||
/// - `memory_search` → **allow**, path-less: it searches note *content* (arg `query`,
|
||||
/// not `path`), so it needs a tool-scoped rule rather than a path pattern.
|
||||
pub async fn seed_fs_path_rules(&self) -> Result<()> {
|
||||
// (tool_pattern, path_pattern, action, note)
|
||||
let rules: &[(&str, &str, &str, &str)] = &[
|
||||
("@fs_any", "memory/*", "allow", "auto-allow memory/"),
|
||||
("@fs_any", "data/*", "allow", "auto-allow data/"),
|
||||
("@fs_any", "secrets/*", "deny", "deny secrets/ access"),
|
||||
// (tool_pattern, path_pattern, action, note). `path_pattern = None` is a
|
||||
// tool-scoped rule that matches regardless of args.
|
||||
let rules: &[(&str, Option<&str>, &str, &str)] = &[
|
||||
("@fs_any", Some("user-memory/*"), "allow", "auto-allow user-memory/"),
|
||||
("@fs_read", Some("shared-memory/*"), "allow", "auto-allow read shared-memory/"),
|
||||
("@fs_write", Some("shared-memory/*"), "require", "require write shared-memory/"),
|
||||
("@fs_any", Some("data/*"), "allow", "auto-allow data/"),
|
||||
("@fs_any", Some("secrets/*"), "deny", "deny secrets/ access"),
|
||||
("memory_search", None, "allow", "allow memory_search"),
|
||||
];
|
||||
|
||||
let mut seeded = 0;
|
||||
for (tool_pattern, path_pattern, action, note) in rules {
|
||||
let exists: i64 = sqlx::query_scalar(
|
||||
"SELECT COUNT(*) FROM approval_rules
|
||||
WHERE tool_pattern = ? AND path_pattern = ? AND group_id = 'default'",
|
||||
)
|
||||
.bind(tool_pattern)
|
||||
.bind(path_pattern)
|
||||
.fetch_one(self.db.as_ref())
|
||||
.await?;
|
||||
for &(tool_pattern, path_pattern, action, note) in rules {
|
||||
// A NULL path can't be matched with `=` (NULL comparisons are never true),
|
||||
// so the existence check branches on it — otherwise the row would re-insert
|
||||
// on every boot.
|
||||
let exists: i64 = match path_pattern {
|
||||
Some(p) => sqlx::query_scalar(
|
||||
"SELECT COUNT(*) FROM approval_rules
|
||||
WHERE tool_pattern = ? AND path_pattern = ? AND group_id = 'default'",
|
||||
)
|
||||
.bind(tool_pattern)
|
||||
.bind(p)
|
||||
.fetch_one(self.db.as_ref())
|
||||
.await?,
|
||||
None => sqlx::query_scalar(
|
||||
"SELECT COUNT(*) FROM approval_rules
|
||||
WHERE tool_pattern = ? AND path_pattern IS NULL AND group_id = 'default'",
|
||||
)
|
||||
.bind(tool_pattern)
|
||||
.fetch_one(self.db.as_ref())
|
||||
.await?,
|
||||
};
|
||||
if exists > 0 {
|
||||
continue;
|
||||
}
|
||||
@@ -360,7 +382,7 @@ impl ApprovalManager {
|
||||
VALUES (?, ?, ?, ?, 5, 'default')",
|
||||
)
|
||||
.bind(tool_pattern)
|
||||
.bind(path_pattern)
|
||||
.bind(path_pattern) // Option<&str> → NULL when None
|
||||
.bind(action)
|
||||
.bind(note)
|
||||
.execute(self.db.as_ref())
|
||||
@@ -383,7 +405,10 @@ impl ApprovalManager {
|
||||
/// - the per-tool write `require` defaults (`note = 'default rule'`, no path) — fs
|
||||
/// gating now lives in the File System panel + the `*` catch-all;
|
||||
/// - the old `data/*` allow rows (`note = 'auto-allow data/ writes'`);
|
||||
/// - the old `secrets` deny rows (`note = 'deny reading secrets/'`).
|
||||
/// - the old `secrets` deny rows (`note = 'deny reading secrets/'`);
|
||||
/// - the old single `memory/*` allow row (`note = 'auto-allow memory/'`) — the memory
|
||||
/// namespace split into `user-memory/` + `shared-memory/`, so the `memory/*` pattern
|
||||
/// no longer routes and would otherwise linger as a stale allow on a disk `./memory/`.
|
||||
///
|
||||
/// Idempotent: a no-op once the legacy rows are gone. Run before `seed_fs_path_rules`.
|
||||
pub async fn migrate_legacy_fs_rules(&self) -> Result<()> {
|
||||
@@ -406,7 +431,20 @@ impl ApprovalManager {
|
||||
.await?
|
||||
.rows_affected();
|
||||
|
||||
let total = n1 + n2 + n3;
|
||||
let n4 = sqlx::query("DELETE FROM approval_rules WHERE note = 'auto-allow memory/'")
|
||||
.execute(self.db.as_ref())
|
||||
.await?
|
||||
.rows_affected();
|
||||
|
||||
// An earlier build seeded `@fs_any allow shared-memory/*`; shared writes now
|
||||
// require approval, so that blanket allow must go (the new @fs_read/@fs_write
|
||||
// rows are seeded fresh by `seed_fs_path_rules`).
|
||||
let n5 = sqlx::query("DELETE FROM approval_rules WHERE note = 'auto-allow shared-memory/'")
|
||||
.execute(self.db.as_ref())
|
||||
.await?
|
||||
.rows_affected();
|
||||
|
||||
let total = n1 + n2 + n3 + n4 + n5;
|
||||
if total > 0 {
|
||||
info!("approval_rules: migrated {total} legacy filesystem rules to @fs_* File System panel");
|
||||
}
|
||||
@@ -419,8 +457,8 @@ impl ApprovalManager {
|
||||
///
|
||||
/// Evaluation order:
|
||||
/// 1. Rules for `group_id` first, then "default" group as fallback, sorted by priority ASC.
|
||||
/// First match wins. (`memory/` auto-allow is a seeded `@fs_any allow memory/*` rule,
|
||||
/// not a hardcoded exception — see `seed_fs_path_rules`.)
|
||||
/// First match wins. (The `user-memory/` and `shared-memory/` auto-allows are seeded
|
||||
/// `@fs_any allow …/*` rules, not a hardcoded exception — see `seed_fs_path_rules`.)
|
||||
/// 2. Session bypass: if a matching bypass is active, `Require` → `Allow`.
|
||||
/// `Deny` is never bypassed.
|
||||
/// 3. No match → `Require` (default-closed policy).
|
||||
@@ -1119,21 +1157,31 @@ mod tests {
|
||||
.unwrap();
|
||||
assert_eq!(legacy, 0, "legacy fs rules should be removed by migration");
|
||||
|
||||
// …and replaced by exactly the three @fs_* token rows.
|
||||
// …and replaced by exactly the five @fs_* token rows (shared-memory has two:
|
||||
// read-allow and write-require).
|
||||
let fs_rows: i64 = sqlx::query_scalar(
|
||||
"SELECT COUNT(*) FROM approval_rules WHERE tool_pattern LIKE '@fs%'",
|
||||
)
|
||||
.fetch_one(db.as_ref())
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(fs_rows, 3, "memory/data/secrets @fs_* rules should be seeded");
|
||||
assert_eq!(fs_rows, 5, "user-memory + shared-memory(r/w) + data + secrets @fs_* rules should be seeded");
|
||||
|
||||
// Gate decisions through the real check() path.
|
||||
async fn decide(mgr: &ApprovalManager, tool: &str, path: &str) -> GateResult {
|
||||
mgr.check(1, None, "main", "web", tool, &json!({ "path": path }), Some("default")).await
|
||||
}
|
||||
assert!(matches!(decide(&mgr, "write_file", "memory/notes.md").await, GateResult::Allow));
|
||||
assert!(matches!(decide(&mgr, "read_file", "data/x.txt").await, GateResult::Allow));
|
||||
// user-memory auto-allows reads and writes; the old `memory/*` no longer matches.
|
||||
assert!(matches!(decide(&mgr, "write_file", "user-memory/notes.md").await, GateResult::Allow));
|
||||
assert!(matches!(decide(&mgr, "list_files", "user-memory").await, GateResult::Allow));
|
||||
assert!(matches!(decide(&mgr, "write_file", "memory/notes.md").await, GateResult::Require));
|
||||
// shared-memory: reads allowed, writes require approval.
|
||||
assert!(matches!(decide(&mgr, "read_file", "shared-memory/casa.md").await, GateResult::Allow));
|
||||
assert!(matches!(decide(&mgr, "write_file", "shared-memory/casa.md").await, GateResult::Require));
|
||||
assert!(matches!(decide(&mgr, "edit_file", "shared-memory/casa.md").await, GateResult::Require));
|
||||
assert!(matches!(decide(&mgr, "read_file", "data/x.txt").await, GateResult::Allow));
|
||||
// memory_search is allowed by a path-less tool rule (it has `query`, not `path`).
|
||||
assert!(matches!(decide(&mgr, "memory_search", "ignored").await, GateResult::Allow));
|
||||
// Improvement over legacy: secrets *writes* are now denied too, not just reads.
|
||||
assert!(matches!(decide(&mgr, "write_file", "secrets/key").await, GateResult::Deny));
|
||||
assert!(matches!(decide(&mgr, "read_file", "secrets/key").await, GateResult::Deny));
|
||||
|
||||
@@ -0,0 +1,203 @@
|
||||
//! Accessor for `memory_docs` — the backing store of the virtual `memory/`
|
||||
//! namespace (blueprint §5).
|
||||
//!
|
||||
//! The **pool is the namespace**: a user pool holds that user's private notes
|
||||
//! (`memory/{userid}`), the system pool holds shared notes (`memory/shared`).
|
||||
//! Callers pass a `path` already stripped of the `memory/…` prefix — the file
|
||||
//! it lands in decides the namespace, the row keeps only the tail. `path` is
|
||||
//! UNIQUE, so [`upsert`] is the single write path for both create and edit, and
|
||||
//! the `memory_docs_fts` triggers keep the full-text index in step underneath.
|
||||
|
||||
use anyhow::Result;
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
#[derive(Debug, Clone, sqlx::FromRow)]
|
||||
pub struct MemoryDoc {
|
||||
pub id: i64,
|
||||
pub path: String,
|
||||
pub content: String,
|
||||
pub created_at: String,
|
||||
pub updated_at: String,
|
||||
}
|
||||
|
||||
/// One row of a directory-style listing: metadata only, no `content` body.
|
||||
#[derive(Debug, Clone, sqlx::FromRow)]
|
||||
pub struct MemoryEntry {
|
||||
pub path: String,
|
||||
pub updated_at: String,
|
||||
}
|
||||
|
||||
/// One full-text hit: the matching note's path and a highlighted excerpt.
|
||||
#[derive(Debug, Clone, sqlx::FromRow)]
|
||||
pub struct MemoryHit {
|
||||
pub path: String,
|
||||
pub snippet: String,
|
||||
}
|
||||
|
||||
const SELECT: &str = "SELECT id, path, content, created_at, updated_at FROM memory_docs";
|
||||
|
||||
/// Fetch one note by its exact path.
|
||||
pub async fn get(pool: &SqlitePool, path: &str) -> Result<Option<MemoryDoc>> {
|
||||
let row = sqlx::query_as::<_, MemoryDoc>(sqlx::AssertSqlSafe(format!("{SELECT} WHERE path = ?")))
|
||||
.bind(path)
|
||||
.fetch_optional(pool)
|
||||
.await?;
|
||||
Ok(row)
|
||||
}
|
||||
|
||||
/// Create the note at `path`, or overwrite it if it already exists. `created_at`
|
||||
/// survives an overwrite; `updated_at` is bumped. Returns the stored row.
|
||||
pub async fn upsert(pool: &SqlitePool, path: &str, content: &str) -> Result<MemoryDoc> {
|
||||
sqlx::query(
|
||||
"INSERT INTO memory_docs (path, content)
|
||||
VALUES (?, ?)
|
||||
ON CONFLICT(path) DO UPDATE SET
|
||||
content = excluded.content,
|
||||
updated_at = datetime('now')",
|
||||
)
|
||||
.bind(path)
|
||||
.bind(content)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
let row = sqlx::query_as::<_, MemoryDoc>(sqlx::AssertSqlSafe(format!("{SELECT} WHERE path = ?")))
|
||||
.bind(path)
|
||||
.fetch_one(pool)
|
||||
.await?;
|
||||
Ok(row)
|
||||
}
|
||||
|
||||
/// List notes whose path starts with `prefix` (pass `""` for all), most recently
|
||||
/// edited first. Metadata only — the `content` body is not loaded.
|
||||
pub async fn list(pool: &SqlitePool, prefix: &str) -> Result<Vec<MemoryEntry>> {
|
||||
// Escaped LIKE prefix: a literal `%`/`_` in the caller's path must match as
|
||||
// itself, not as a wildcard. `\` is the escape character.
|
||||
let pattern = format!("{}%", escape_like(prefix));
|
||||
let rows = sqlx::query_as::<_, MemoryEntry>(
|
||||
"SELECT path, updated_at FROM memory_docs
|
||||
WHERE path LIKE ? ESCAPE '\\'
|
||||
ORDER BY updated_at DESC",
|
||||
)
|
||||
.bind(pattern)
|
||||
.fetch_all(pool)
|
||||
.await?;
|
||||
Ok(rows)
|
||||
}
|
||||
|
||||
/// Full-text search over note bodies and paths, best match first. `query` is
|
||||
/// FTS5 MATCH syntax; `snippet` is a short excerpt of the body with the matched
|
||||
/// terms wrapped in `[` … `]`.
|
||||
pub async fn search(pool: &SqlitePool, query: &str, limit: i64) -> Result<Vec<MemoryHit>> {
|
||||
let rows = sqlx::query_as::<_, MemoryHit>(
|
||||
// `memory_docs_fts` is external-content, so its rowid is `memory_docs.id`;
|
||||
// join back for the path, and read the excerpt from content column 1.
|
||||
"SELECT d.path AS path,
|
||||
snippet(memory_docs_fts, 1, '[', ']', '…', 12) AS snippet
|
||||
FROM memory_docs_fts
|
||||
JOIN memory_docs d ON d.id = memory_docs_fts.rowid
|
||||
WHERE memory_docs_fts MATCH ?
|
||||
ORDER BY bm25(memory_docs_fts)
|
||||
LIMIT ?",
|
||||
)
|
||||
.bind(query)
|
||||
.bind(limit)
|
||||
.fetch_all(pool)
|
||||
.await?;
|
||||
Ok(rows)
|
||||
}
|
||||
|
||||
/// Delete the note at `path`. Returns whether a row was removed.
|
||||
pub async fn delete(pool: &SqlitePool, path: &str) -> Result<bool> {
|
||||
let n = sqlx::query("DELETE FROM memory_docs WHERE path = ?")
|
||||
.bind(path)
|
||||
.execute(pool)
|
||||
.await?
|
||||
.rows_affected();
|
||||
Ok(n > 0)
|
||||
}
|
||||
|
||||
/// Escapes `%`, `_` and `\` so a caller-supplied string is a literal LIKE prefix.
|
||||
fn escape_like(s: &str) -> String {
|
||||
let mut out = String::with_capacity(s.len());
|
||||
for c in s.chars() {
|
||||
if matches!(c, '%' | '_' | '\\') {
|
||||
out.push('\\');
|
||||
}
|
||||
out.push(c);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::path::PathBuf;
|
||||
|
||||
/// A standalone owner-schema database in a throwaway temp dir. `tag` plus an
|
||||
/// atomic counter keep parallel tests from colliding on the same file.
|
||||
/// Returns the pool and the dir so the caller can wipe it (SQLite leaves
|
||||
/// `-wal`/`-shm` sidecars beside the file).
|
||||
async fn owner_pool(tag: &str) -> (SqlitePool, PathBuf) {
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
static SEQ: AtomicU64 = AtomicU64::new(0);
|
||||
let n = SEQ.fetch_add(1, Ordering::Relaxed);
|
||||
let dir = std::env::temp_dir()
|
||||
.join(format!("skald-memdocs-{}-{tag}-{n}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let pool = crate::db::create_user_pool(&dir.join("owner.db"), None).await.unwrap();
|
||||
(pool, dir)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn upsert_is_create_then_overwrite_and_fts_follows() {
|
||||
let (pool, dir) = owner_pool("upsert").await;
|
||||
|
||||
// create
|
||||
let doc = upsert(&pool, "notes/spesa.md", "latte e pane").await.unwrap();
|
||||
assert_eq!(doc.path, "notes/spesa.md");
|
||||
assert_eq!(doc.content, "latte e pane");
|
||||
let first_id = doc.id;
|
||||
|
||||
// get by exact path
|
||||
assert_eq!(get(&pool, "notes/spesa.md").await.unwrap().unwrap().content, "latte e pane");
|
||||
assert!(get(&pool, "notes/altro.md").await.unwrap().is_none());
|
||||
|
||||
// overwrite: same row, new content, created_at preserved
|
||||
let doc2 = upsert(&pool, "notes/spesa.md", "latte, pane, uova").await.unwrap();
|
||||
assert_eq!(doc2.id, first_id, "upsert must update in place, not insert a new row");
|
||||
assert_eq!(doc2.content, "latte, pane, uova");
|
||||
assert_eq!(doc2.created_at, doc.created_at, "created_at survives an overwrite");
|
||||
assert_eq!(list(&pool, "").await.unwrap().len(), 1, "still one row for that path");
|
||||
|
||||
// FTS follows the update: the removed token is gone, the new one is found
|
||||
assert!(search(&pool, "uova", 10).await.unwrap().iter().any(|h| h.path == "notes/spesa.md"));
|
||||
assert!(search(&pool, "latte", 10).await.unwrap().iter().any(|h| h.path == "notes/spesa.md"));
|
||||
|
||||
pool.close().await;
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn list_by_prefix_and_delete_deindexes() {
|
||||
let (pool, dir) = owner_pool("list").await;
|
||||
|
||||
upsert(&pool, "notes/spesa.md", "latte pane uova").await.unwrap();
|
||||
upsert(&pool, "notes/idee.md", "un'idea brillante").await.unwrap();
|
||||
upsert(&pool, "diary/2026.md", "oggi e' successo").await.unwrap();
|
||||
|
||||
let notes = list(&pool, "notes/").await.unwrap();
|
||||
assert_eq!(notes.len(), 2, "prefix listing is scoped to the subtree");
|
||||
assert!(notes.iter().all(|e| e.path.starts_with("notes/")));
|
||||
assert_eq!(list(&pool, "").await.unwrap().len(), 3, "empty prefix lists everything");
|
||||
|
||||
// delete removes the row and de-indexes it from FTS
|
||||
assert!(delete(&pool, "notes/spesa.md").await.unwrap());
|
||||
assert!(get(&pool, "notes/spesa.md").await.unwrap().is_none());
|
||||
assert!(search(&pool, "uova", 10).await.unwrap().is_empty(), "delete must de-index");
|
||||
assert!(!delete(&pool, "notes/spesa.md").await.unwrap(), "second delete is a no-op");
|
||||
|
||||
pool.close().await;
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
}
|
||||
@@ -13,6 +13,7 @@ pub mod llm_requests;
|
||||
pub mod llm_request_payloads;
|
||||
pub mod mcp_events;
|
||||
pub mod mcp_servers;
|
||||
pub mod memory_docs;
|
||||
pub mod plugins;
|
||||
pub mod roles;
|
||||
pub mod scheduled_jobs;
|
||||
@@ -710,6 +711,60 @@ pub async fn create_owner_tables(pool: &SqlitePool) -> Result<()> {
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
// Backing store for the virtual `memory/` namespace (blueprint §5). MD-only
|
||||
// notes keyed by a path *relative to the namespace root* — the file is the
|
||||
// namespace, so no `memory/{userid}` / `memory/shared` prefix is stored:
|
||||
// routing picks the pool, the row keeps only the tail. Because this is an
|
||||
// owner table, the same schema backs private memory in each `{userid}.db`
|
||||
// (behind SQLCipher) and shared memory in `system.db` (cleartext, the
|
||||
// household owner) — §5.1. `path` is UNIQUE, so a write is an upsert.
|
||||
sqlx::query(
|
||||
"CREATE TABLE IF NOT EXISTS memory_docs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
path TEXT NOT NULL UNIQUE,
|
||||
content TEXT NOT NULL DEFAULT '',
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
// Full-text index over memory notes: the payoff of a SQLite backing over
|
||||
// opaque file blobs (§5) — a decrypted session can search / RAG its own
|
||||
// memory. External-content FTS5 keeps no second copy of `content`; the
|
||||
// triggers below mirror every change from `memory_docs`. FTS5 is compiled
|
||||
// into the bundled SQLCipher build, so this works inside encrypted user
|
||||
// files too.
|
||||
sqlx::query(
|
||||
"CREATE VIRTUAL TABLE IF NOT EXISTS memory_docs_fts USING fts5(
|
||||
path, content,
|
||||
content='memory_docs',
|
||||
content_rowid='id'
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
for trigger in [
|
||||
"CREATE TRIGGER IF NOT EXISTS memory_docs_ai AFTER INSERT ON memory_docs BEGIN
|
||||
INSERT INTO memory_docs_fts(rowid, path, content)
|
||||
VALUES (new.id, new.path, new.content);
|
||||
END",
|
||||
"CREATE TRIGGER IF NOT EXISTS memory_docs_ad AFTER DELETE ON memory_docs BEGIN
|
||||
INSERT INTO memory_docs_fts(memory_docs_fts, rowid, path, content)
|
||||
VALUES ('delete', old.id, old.path, old.content);
|
||||
END",
|
||||
"CREATE TRIGGER IF NOT EXISTS memory_docs_au AFTER UPDATE ON memory_docs BEGIN
|
||||
INSERT INTO memory_docs_fts(memory_docs_fts, rowid, path, content)
|
||||
VALUES ('delete', old.id, old.path, old.content);
|
||||
INSERT INTO memory_docs_fts(rowid, path, content)
|
||||
VALUES (new.id, new.path, new.content);
|
||||
END",
|
||||
] {
|
||||
sqlx::query(trigger).execute(pool).await?;
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -764,6 +819,15 @@ mod tests {
|
||||
one("INSERT INTO projects (id, name, path) VALUES (1, 'p', '/tmp')").await.unwrap();
|
||||
one("INSERT INTO project_tickets (project_id, title, job_id) VALUES (1, 't', 1)").await.unwrap();
|
||||
one("INSERT INTO llm_request_payloads (request_id, request_json) VALUES ('r1', '{}')").await.unwrap();
|
||||
// Fires the AFTER INSERT trigger into the external-content FTS5 table.
|
||||
one("INSERT INTO memory_docs (path, content) VALUES ('notes/x.md', 'hello world')").await.unwrap();
|
||||
|
||||
// ...and the FTS index actually answers a MATCH.
|
||||
let (hits,): (i64,) = sqlx::query_as(
|
||||
"SELECT count(*) FROM memory_docs_fts WHERE memory_docs_fts MATCH 'world'",
|
||||
)
|
||||
.fetch_one(&pool).await.unwrap();
|
||||
assert_eq!(hits, 1, "memory_docs_fts must index inserted notes");
|
||||
|
||||
pool.close().await;
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
|
||||
@@ -34,6 +34,9 @@ fn system_timezone() -> Option<&'static str> {
|
||||
/// without needing the full handler and all its dependencies.
|
||||
pub struct MessageBuilder {
|
||||
pub pool: Arc<SqlitePool>,
|
||||
/// The shared (`system.db`) pool, for injecting `shared-memory/` notes. The
|
||||
/// owner `pool` above backs `user-memory/`.
|
||||
pub shared_pool: Arc<SqlitePool>,
|
||||
pub session_id: i64,
|
||||
pub mcp: Arc<McpManager>,
|
||||
pub datetime_config: DatetimeConfig,
|
||||
@@ -93,9 +96,7 @@ impl MessageBuilder {
|
||||
You can edit them with `edit_file` or `write_file` using the path shown.\n"
|
||||
);
|
||||
for mem_path in &meta.inject_memory {
|
||||
// Resolve the entry to (absolute path to read, path to show the agent).
|
||||
let (abs, display) = self.resolve_memory_path(mem_path);
|
||||
let content = tokio::fs::read_to_string(&abs).await.ok();
|
||||
let (content, display) = self.load_inject_memory(mem_path).await;
|
||||
match content {
|
||||
Some(c) => static_content.push_str(&format!(
|
||||
"\n<memory_file path=\"{display}\">\n{c}\n</memory_file>\n"
|
||||
@@ -405,6 +406,27 @@ impl MessageBuilder {
|
||||
/// when the file lives under it, absolute otherwise** — so when the agent references
|
||||
/// it back via `edit_file`/`write_file`, the loop's working-directory injection
|
||||
/// (which rewrites relative paths against the WD) resolves to the very same file.
|
||||
/// Loads an `inject_memory` entry, returning `(content, display_path)`.
|
||||
///
|
||||
/// Virtual memory paths are read from SQLite: `user-memory/…` from the owner
|
||||
/// `pool`, `shared-memory/…` from the `shared_pool` (`system.db`). Everything
|
||||
/// else (`data/…`, `$WD/…`) is an ordinary disk read. A missing note / file
|
||||
/// yields `None`, rendered as "(file not created yet)".
|
||||
async fn load_inject_memory(&self, mem_path: &str) -> (Option<String>, String) {
|
||||
use crate::tools::fs::{classify_memory, MemScope};
|
||||
if let Some(m) = classify_memory(mem_path) {
|
||||
let pool = match m.scope {
|
||||
MemScope::User => &self.pool,
|
||||
MemScope::Shared => &self.shared_pool,
|
||||
};
|
||||
let content = crate::db::memory_docs::get(pool, &m.rel)
|
||||
.await.ok().flatten().map(|d| d.content);
|
||||
return (content, mem_path.to_string());
|
||||
}
|
||||
let (abs, display) = self.resolve_memory_path(mem_path);
|
||||
(tokio::fs::read_to_string(&abs).await.ok(), display)
|
||||
}
|
||||
|
||||
fn resolve_memory_path(&self, mem_path: &str) -> (std::path::PathBuf, String) {
|
||||
let wd = self.working_directory.clone()
|
||||
.unwrap_or_else(|| std::env::current_dir().unwrap_or_default());
|
||||
|
||||
@@ -28,6 +28,7 @@ impl ChatSessionHandler {
|
||||
.map(|rc| rc.effective_working_dir());
|
||||
let builder = MessageBuilder {
|
||||
pool: Arc::clone(&self.db),
|
||||
shared_pool: Arc::clone(&self.shared_pool),
|
||||
session_id: self.scratchpad_sid(),
|
||||
mcp: Arc::clone(&self.mcp),
|
||||
datetime_config: self.datetime_config.clone(),
|
||||
|
||||
@@ -262,6 +262,9 @@ impl ApprovalDecision {
|
||||
pub struct ChatSessionHandler {
|
||||
pub session_id: i64,
|
||||
pub(super) db: Arc<SqlitePool>,
|
||||
/// The shared (`system.db`) pool. Owner-bound work uses `db`; this is only for
|
||||
/// cross-owner reads, e.g. injecting `shared-memory/` notes into the prompt.
|
||||
pub(super) shared_pool: Arc<SqlitePool>,
|
||||
/// The authenticated user who owns this session. Threaded into `ChatOptions`
|
||||
/// so the telemetry metadata row in `system.db` carries `user_id`.
|
||||
pub(super) user_id: String,
|
||||
@@ -332,6 +335,7 @@ impl ChatSessionHandler {
|
||||
pub fn new(
|
||||
session_id: i64,
|
||||
db: Arc<SqlitePool>,
|
||||
shared_pool: Arc<SqlitePool>,
|
||||
user_id: String,
|
||||
llm_manager: Arc<LlmManager>,
|
||||
max_history_messages: usize,
|
||||
@@ -357,6 +361,7 @@ impl ChatSessionHandler {
|
||||
Self {
|
||||
session_id,
|
||||
db,
|
||||
shared_pool,
|
||||
user_id,
|
||||
llm_manager,
|
||||
max_history_messages,
|
||||
|
||||
@@ -22,6 +22,9 @@ use super::handler::ChatSessionHandler;
|
||||
|
||||
pub struct ChatSessionManager {
|
||||
db: Arc<SqlitePool>,
|
||||
/// The shared (`system.db`) pool, threaded to each handler for cross-owner
|
||||
/// reads such as injecting `shared-memory/` notes.
|
||||
shared_pool: Arc<SqlitePool>,
|
||||
user_id: String,
|
||||
llm_manager: Arc<LlmManager>,
|
||||
max_history_messages: usize,
|
||||
@@ -48,6 +51,7 @@ pub struct ChatSessionManager {
|
||||
impl ChatSessionManager {
|
||||
pub fn new(
|
||||
db: Arc<SqlitePool>,
|
||||
shared_pool: Arc<SqlitePool>,
|
||||
user_id: String,
|
||||
llm_manager: Arc<LlmManager>,
|
||||
max_history_messages: usize,
|
||||
@@ -68,6 +72,7 @@ impl ChatSessionManager {
|
||||
) -> Self {
|
||||
Self {
|
||||
db,
|
||||
shared_pool,
|
||||
user_id,
|
||||
llm_manager,
|
||||
max_history_messages,
|
||||
@@ -155,6 +160,7 @@ impl ChatSessionManager {
|
||||
let handler = Arc::new(ChatSessionHandler::new(
|
||||
session_id,
|
||||
self.db.clone(),
|
||||
self.shared_pool.clone(),
|
||||
self.user_id.clone(),
|
||||
Arc::clone(&self.llm_manager),
|
||||
self.max_history_messages,
|
||||
|
||||
@@ -204,9 +204,9 @@ impl Tools {
|
||||
/// Captures sibling managers (mcp, plugins, cron, secrets) into the tool
|
||||
/// registry. `execute_task` is deliberately NOT registered here — it is injected
|
||||
/// per interactive session by `ChatHub::send_message`.
|
||||
pub(super) fn build(integrations: &Integrations, tasks: &Tasks, models: &Models) -> Self {
|
||||
pub(super) fn build(rt: &Runtime, integrations: &Integrations, tasks: &Tasks, models: &Models) -> Self {
|
||||
let mut tool_registry = ToolRegistry::new();
|
||||
crate::tools::fs::register_all(&mut tool_registry);
|
||||
crate::tools::fs::register_all(&mut tool_registry, Arc::clone(&rt.db));
|
||||
tool_registry.register(crate::tools::ast_outline::AstOutline::new());
|
||||
tool_registry.register(crate::tools::exec::ExecuteCmd);
|
||||
tool_registry.register(crate::tools::read_notification::ReadNotification);
|
||||
@@ -338,6 +338,7 @@ impl Conversation {
|
||||
|
||||
let manager = Arc::new(ChatSessionManager::new(
|
||||
Arc::clone(&rt.db),
|
||||
Arc::clone(&rt.db), // shared pool == system.db (this is the ownerless manager)
|
||||
String::new(),
|
||||
Arc::clone(&models.llm_manager),
|
||||
config.llm.max_history_messages,
|
||||
|
||||
@@ -66,7 +66,7 @@ impl Skald {
|
||||
let media = Media::build(&rt, &models).await?;
|
||||
let integrations = Integrations::build(&rt, plugins);
|
||||
let tasks = Tasks::build(&rt, config);
|
||||
let tools = Tools::build(&integrations, &tasks, &models);
|
||||
let tools = Tools::build(&rt, &integrations, &tasks, &models);
|
||||
let interaction = Interaction::build(&rt, &tools).await?;
|
||||
let conversation = Conversation::build(&rt, &models, &media, &tools, &integrations, &interaction, config).await?;
|
||||
let infra = Infra::build();
|
||||
|
||||
@@ -159,6 +159,7 @@ impl UserContextFactory {
|
||||
|
||||
let manager = Arc::new(ChatSessionManager::new(
|
||||
Arc::clone(&pool),
|
||||
Arc::clone(&self.registry_pool), // shared pool = system.db, for shared-memory injection
|
||||
user_id.to_string(),
|
||||
Arc::clone(&self.llm_manager),
|
||||
self.max_history_messages,
|
||||
|
||||
@@ -1,8 +1,55 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use anyhow::Result;
|
||||
use serde_json::{Value, json};
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT};
|
||||
use super::{read_to_string, write_string};
|
||||
use crate::tools::{
|
||||
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
|
||||
truncate_label, MAX_LABEL_SHORT,
|
||||
};
|
||||
use super::{classify_memory, read_to_string, write_string, MemScope};
|
||||
|
||||
/// Applies the substring edit to `content`, returning the new content. Shared by
|
||||
/// the on-disk [`EditFile::execute`] and the `memory/` routing in `run_with`;
|
||||
/// `display` is the path used in error messages.
|
||||
fn apply_edit(content: &str, args: &Value, display: &str) -> Result<String> {
|
||||
let old = args["old"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: old"))?;
|
||||
let new = args["new"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: new"))?;
|
||||
let replace_all = args["replace_all"].as_bool().unwrap_or(false);
|
||||
|
||||
let updated = if replace_all {
|
||||
if !content.contains(old) {
|
||||
anyhow::bail!(
|
||||
"Text not found in {display}. \
|
||||
Call read_file first and copy the text exactly as shown after the '| ' prefix."
|
||||
);
|
||||
}
|
||||
content.replace(old, new)
|
||||
} else {
|
||||
let exact_count = content.matches(old).count();
|
||||
if exact_count > 1 {
|
||||
anyhow::bail!(
|
||||
"Text found {exact_count} times in {display}. \
|
||||
Include more surrounding context in `old` to make it unique, or set replace_all=true."
|
||||
);
|
||||
}
|
||||
if exact_count == 1 {
|
||||
content.replacen(old, new, 1)
|
||||
} else {
|
||||
let normalized_old = normalize_ws(old);
|
||||
let (start, end) = find_normalized(content, &normalized_old)
|
||||
.ok_or_else(|| anyhow::anyhow!(
|
||||
"Text not found in {display}. \
|
||||
Call read_file first and copy the text exactly as shown after the '| ' prefix."
|
||||
))?;
|
||||
format!("{}{}{}", &content[..start], new, &content[end..])
|
||||
}
|
||||
};
|
||||
Ok(updated)
|
||||
}
|
||||
|
||||
fn normalize_ws(s: &str) -> String {
|
||||
s.lines()
|
||||
@@ -56,10 +103,13 @@ fn find_normalized(haystack: &str, normalized_needle: &str) -> Option<(usize, us
|
||||
None
|
||||
}
|
||||
|
||||
pub struct EditFile;
|
||||
pub struct EditFile {
|
||||
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
|
||||
shared_pool: Arc<SqlitePool>,
|
||||
}
|
||||
|
||||
impl EditFile {
|
||||
pub fn new() -> Self { Self }
|
||||
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
|
||||
}
|
||||
|
||||
impl Tool for EditFile {
|
||||
@@ -102,46 +152,32 @@ impl Tool for EditFile {
|
||||
truncate_label(&format!("edit_file `{path}`"), MAX_LABEL_SHORT)
|
||||
}
|
||||
|
||||
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
|
||||
/// path falls through to the on-disk [`execute`](Self::execute).
|
||||
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
|
||||
let path = super::path_arg(&args).unwrap_or_default();
|
||||
let Some(m) = classify_memory(&path) else { return self.run(args); };
|
||||
let pool = match m.scope {
|
||||
MemScope::User => Arc::clone(&ctx.pool),
|
||||
MemScope::Shared => Arc::clone(&self.shared_pool),
|
||||
};
|
||||
let rel = m.rel;
|
||||
|
||||
Box::new(SimpleExecution::new(Box::pin(async move {
|
||||
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
|
||||
anyhow::bail!("No note at {path}");
|
||||
};
|
||||
let updated = apply_edit(&doc.content, &args, &path)?;
|
||||
crate::db::memory_docs::upsert(&pool, &rel, &updated).await?;
|
||||
Ok(ToolResult::Text(format!("Edited {path}.")))
|
||||
})))
|
||||
}
|
||||
|
||||
fn execute(&self, args: Value) -> Result<String> {
|
||||
let user_path = args["path"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
|
||||
let old = args["old"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: old"))?;
|
||||
let new = args["new"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: new"))?;
|
||||
|
||||
let replace_all = args["replace_all"].as_bool().unwrap_or(false);
|
||||
let content = read_to_string(user_path)?;
|
||||
|
||||
let updated = if replace_all {
|
||||
if !content.contains(old) {
|
||||
anyhow::bail!(
|
||||
"Text not found in {user_path}. \
|
||||
Call read_file first and copy the text exactly as shown after the '| ' prefix."
|
||||
);
|
||||
}
|
||||
content.replace(old, new)
|
||||
} else {
|
||||
let exact_count = content.matches(old).count();
|
||||
if exact_count > 1 {
|
||||
anyhow::bail!(
|
||||
"Text found {exact_count} times in {user_path}. \
|
||||
Include more surrounding context in `old` to make it unique, or set replace_all=true."
|
||||
);
|
||||
}
|
||||
if exact_count == 1 {
|
||||
content.replacen(old, new, 1)
|
||||
} else {
|
||||
let normalized_old = normalize_ws(old);
|
||||
let (start, end) = find_normalized(&content, &normalized_old)
|
||||
.ok_or_else(|| anyhow::anyhow!(
|
||||
"Text not found in {user_path}. \
|
||||
Call read_file first and copy the text exactly as shown after the '| ' prefix."
|
||||
))?;
|
||||
format!("{}{}{}", &content[..start], new, &content[end..])
|
||||
}
|
||||
};
|
||||
|
||||
let updated = apply_edit(&content, &args, user_path)?;
|
||||
write_string(user_path, &updated)?;
|
||||
Ok(format!("Edited {user_path}."))
|
||||
}
|
||||
|
||||
@@ -1,13 +1,49 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use anyhow::Result;
|
||||
use serde_json::{Value, json};
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL};
|
||||
use super::{read_to_string, write_string};
|
||||
use crate::tools::{
|
||||
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
|
||||
truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL,
|
||||
};
|
||||
use super::{classify_memory, read_to_string, write_string, MemScope};
|
||||
|
||||
pub struct InsertAtLine;
|
||||
pub struct InsertAtLine {
|
||||
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
|
||||
shared_pool: Arc<SqlitePool>,
|
||||
}
|
||||
|
||||
impl InsertAtLine {
|
||||
pub fn new() -> Self { Self }
|
||||
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
|
||||
}
|
||||
|
||||
/// Inserts `content` before/after `line` in `text`, returning the new text and a
|
||||
/// result message. Shared by the on-disk `execute` and the `memory/` routing;
|
||||
/// `display` is the path used in the message.
|
||||
fn apply_insert(text: &str, args: &Value, display: &str) -> Result<(String, String)> {
|
||||
let line_num = args["line"].as_u64()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: line"))? as usize;
|
||||
let new_text = args["content"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: content"))?;
|
||||
let placement = args["placement"].as_str().unwrap_or("after");
|
||||
|
||||
anyhow::ensure!(line_num >= 1, "line must be >= 1");
|
||||
|
||||
let mut lines: Vec<&str> = text.split('\n').collect();
|
||||
let idx = (line_num - 1).min(lines.len().saturating_sub(1));
|
||||
let insert_idx = if placement == "before" { idx } else { idx + 1 };
|
||||
let new_lines: Vec<&str> = new_text.split('\n').collect();
|
||||
for (i, l) in new_lines.iter().enumerate() {
|
||||
lines.insert(insert_idx + i, l);
|
||||
}
|
||||
let updated = lines.join("\n");
|
||||
let msg = format!(
|
||||
"Inserted {} line(s) {} line {} in {display}.",
|
||||
new_lines.len(), placement, line_num
|
||||
);
|
||||
Ok((updated, msg))
|
||||
}
|
||||
|
||||
impl Tool for InsertAtLine {
|
||||
@@ -53,32 +89,33 @@ impl Tool for InsertAtLine {
|
||||
}
|
||||
}
|
||||
|
||||
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
|
||||
/// path falls through to the on-disk [`execute`](Self::execute).
|
||||
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
|
||||
let path = super::path_arg(&args).unwrap_or_default();
|
||||
let Some(m) = classify_memory(&path) else { return self.run(args); };
|
||||
let pool = match m.scope {
|
||||
MemScope::User => Arc::clone(&ctx.pool),
|
||||
MemScope::Shared => Arc::clone(&self.shared_pool),
|
||||
};
|
||||
let rel = m.rel;
|
||||
|
||||
Box::new(SimpleExecution::new(Box::pin(async move {
|
||||
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
|
||||
anyhow::bail!("No note at {path}");
|
||||
};
|
||||
let (updated, msg) = apply_insert(&doc.content, &args, &path)?;
|
||||
crate::db::memory_docs::upsert(&pool, &rel, &updated).await?;
|
||||
Ok(ToolResult::Text(msg))
|
||||
})))
|
||||
}
|
||||
|
||||
fn execute(&self, args: Value) -> Result<String> {
|
||||
let user_path = args["path"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
|
||||
let line_num = args["line"].as_u64()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: line"))? as usize;
|
||||
let new_text = args["content"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: content"))?;
|
||||
let placement = args["placement"].as_str().unwrap_or("after");
|
||||
|
||||
anyhow::ensure!(line_num >= 1, "line must be >= 1");
|
||||
|
||||
let text = read_to_string(user_path)?;
|
||||
let mut lines: Vec<&str> = text.split('\n').collect();
|
||||
let idx = (line_num - 1).min(lines.len().saturating_sub(1));
|
||||
let insert_idx = if placement == "before" { idx } else { idx + 1 };
|
||||
let new_lines: Vec<&str> = new_text.split('\n').collect();
|
||||
for (i, l) in new_lines.iter().enumerate() {
|
||||
lines.insert(insert_idx + i, l);
|
||||
}
|
||||
let updated = lines.join("\n");
|
||||
|
||||
let (updated, msg) = apply_insert(&text, &args, user_path)?;
|
||||
write_string(user_path, &updated)?;
|
||||
|
||||
Ok(format!(
|
||||
"Inserted {} line(s) {} line {} in {user_path}.",
|
||||
new_lines.len(), placement, line_num
|
||||
))
|
||||
Ok(msg)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,20 +1,28 @@
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
|
||||
use anyhow::Result;
|
||||
use serde_json::{Value, json};
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT};
|
||||
use super::resolve;
|
||||
use crate::tools::{
|
||||
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
|
||||
truncate_label, MAX_LABEL_SHORT,
|
||||
};
|
||||
use super::{classify_memory, resolve, MemScope};
|
||||
|
||||
/// Directories to skip unconditionally when walking.
|
||||
/// `secrets` is skipped so a recursive listing rooted at a parent (e.g. the auto-read
|
||||
/// working directory) never reveals the contents of the secrets store.
|
||||
const SKIP_DIRS: &[&str] = &[".git", "target", "node_modules", ".cache", "secrets"];
|
||||
|
||||
pub struct ListFiles;
|
||||
pub struct ListFiles {
|
||||
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
|
||||
shared_pool: Arc<SqlitePool>,
|
||||
}
|
||||
|
||||
impl ListFiles {
|
||||
pub fn new() -> Self { Self }
|
||||
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
|
||||
}
|
||||
|
||||
impl Tool for ListFiles {
|
||||
@@ -27,7 +35,8 @@ impl Tool for ListFiles {
|
||||
Relative paths are resolved from the project root; absolute paths (starting with /) are used as-is. \
|
||||
Skips .git, target, node_modules, .cache. \
|
||||
Returns a JSON array of paths relative to the requested directory. \
|
||||
Use depth=1 for immediate contents only, depth=2-3 for moderate exploration."
|
||||
Use depth=1 for immediate contents only, depth=2-3 for moderate exploration. \
|
||||
Listing under user-memory/ (private) or shared-memory/ (shared) lists your memory notes instead of disk."
|
||||
}
|
||||
|
||||
fn parameters_schema(&self) -> Value {
|
||||
@@ -56,6 +65,32 @@ impl Tool for ListFiles {
|
||||
truncate_label(&format!("list_files `{path}`"), MAX_LABEL_SHORT)
|
||||
}
|
||||
|
||||
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
|
||||
/// path falls through to the on-disk [`execute`](Self::execute). Memory is a
|
||||
/// flat key space, so `depth`/`dirs_only` don't apply — the whole subtree
|
||||
/// under the prefix is returned, keyed relative to the requested directory.
|
||||
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
|
||||
let path = args["path"].as_str().unwrap_or("").to_string();
|
||||
let Some(m) = classify_memory(&path) else { return self.run(args); };
|
||||
let pool = match m.scope {
|
||||
MemScope::User => Arc::clone(&ctx.pool),
|
||||
MemScope::Shared => Arc::clone(&self.shared_pool),
|
||||
};
|
||||
let rel = m.rel;
|
||||
|
||||
Box::new(SimpleExecution::new(Box::pin(async move {
|
||||
// Treat `rel` as a directory prefix: match `rel/…` (or everything at
|
||||
// the root), then strip it so results are relative to what was asked.
|
||||
let prefix = if rel.is_empty() || rel.ends_with('/') { rel } else { format!("{rel}/") };
|
||||
let entries = crate::db::memory_docs::list(&pool, &prefix).await?;
|
||||
let mut paths: Vec<String> = entries.into_iter()
|
||||
.map(|e| e.path.strip_prefix(&prefix).unwrap_or(&e.path).to_string())
|
||||
.collect();
|
||||
paths.sort();
|
||||
Ok(ToolResult::Text(serde_json::to_string(&paths)?))
|
||||
})))
|
||||
}
|
||||
|
||||
fn execute(&self, args: Value) -> Result<String> {
|
||||
let user_path = args["path"].as_str().unwrap_or(".");
|
||||
let max_depth = args["depth"].as_u64().unwrap_or(3) as usize;
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
//! `memory_search` — full-text search over the virtual memory namespace (§5).
|
||||
//!
|
||||
//! Unlike the fs-tools, this does **not** route a path: it searches note *content*
|
||||
//! through the `memory_docs` FTS5 index (`memory_docs::search`, bm25-ranked with a
|
||||
//! highlighted snippet). `user-memory` is the caller's own pool (`ToolContext::pool`);
|
||||
//! `shared-memory` is the system pool captured at registration. Kept a distinct tool
|
||||
//! rather than folding FTS into `grep_files`: grep is regex-per-line over a tree,
|
||||
//! this is ranked keyword recall — different semantics, so different names.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use serde_json::{Value, json};
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
use crate::db::memory_docs::{self, MemoryHit};
|
||||
use crate::tools::{
|
||||
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
|
||||
truncate_label, MAX_LABEL_SHORT,
|
||||
};
|
||||
|
||||
pub struct MemorySearch {
|
||||
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
|
||||
shared_pool: Arc<SqlitePool>,
|
||||
}
|
||||
|
||||
impl MemorySearch {
|
||||
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
|
||||
}
|
||||
|
||||
/// Turns free text into a robust FTS5 MATCH query: each whitespace token becomes a
|
||||
/// quoted term (AND-combined), so arbitrary input — colons, dashes, punctuation —
|
||||
/// can't trip an FTS5 syntax error. Returns `None` when there are no tokens.
|
||||
fn fts_query(input: &str) -> Option<String> {
|
||||
let terms: Vec<String> = input
|
||||
.split_whitespace()
|
||||
.map(|t| format!("\"{}\"", t.replace('"', "\"\"")))
|
||||
.collect();
|
||||
(!terms.is_empty()).then(|| terms.join(" "))
|
||||
}
|
||||
|
||||
fn render_hits(store: &str, hits: &[MemoryHit], out: &mut String) {
|
||||
for h in hits {
|
||||
out.push_str(&format!("[{store}] {} — {}\n", h.path, h.snippet));
|
||||
}
|
||||
}
|
||||
|
||||
impl Tool for MemorySearch {
|
||||
fn name(&self) -> &str { "memory_search" }
|
||||
fn category(&self) -> crate::tools::ToolCategory { crate::tools::ToolCategory::Introspection }
|
||||
|
||||
fn description(&self) -> &str {
|
||||
"Full-text search across your memory notes by keyword, ranked by relevance. \
|
||||
Searches user-memory/ (private) and shared-memory/ (shared); set scope to narrow it. \
|
||||
Returns matching note paths with a short highlighted snippet — open one with read_file. \
|
||||
Use this to recall where you wrote something instead of listing and reading notes one by one."
|
||||
}
|
||||
|
||||
fn parameters_schema(&self) -> Value {
|
||||
json!({
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"query": { "type": "string", "description": "Keywords to search for. Plain words; all must appear." },
|
||||
"scope": {
|
||||
"type": "string",
|
||||
"enum": ["all", "private", "shared"],
|
||||
"description": "Which store to search: 'private' (user-memory), 'shared' (shared-memory), or 'all' (default)."
|
||||
},
|
||||
"limit": { "type": "integer", "description": "Max results per store (default 10, max 50).", "default": 10 }
|
||||
},
|
||||
"required": ["query"]
|
||||
})
|
||||
}
|
||||
|
||||
fn describe(&self, args: &Value, _length: ToolDescriptionLength) -> String {
|
||||
let q = args["query"].as_str().unwrap_or("?");
|
||||
truncate_label(&format!("memory_search \"{q}\""), MAX_LABEL_SHORT)
|
||||
}
|
||||
|
||||
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
|
||||
let user_pool = Arc::clone(&ctx.pool);
|
||||
let shared_pool = Arc::clone(&self.shared_pool);
|
||||
|
||||
Box::new(SimpleExecution::new(Box::pin(async move {
|
||||
let raw = args["query"].as_str().unwrap_or("");
|
||||
let Some(q) = fts_query(raw) else {
|
||||
anyhow::bail!("memory_search needs a non-empty query");
|
||||
};
|
||||
let scope = args["scope"].as_str().unwrap_or("all");
|
||||
let limit = args["limit"].as_u64().unwrap_or(10).clamp(1, 50) as i64;
|
||||
|
||||
let mut out = String::new();
|
||||
let mut total = 0usize;
|
||||
if scope == "all" || scope == "private" {
|
||||
let hits = memory_docs::search(&user_pool, &q, limit).await?;
|
||||
total += hits.len();
|
||||
render_hits("user-memory", &hits, &mut out);
|
||||
}
|
||||
if scope == "all" || scope == "shared" {
|
||||
let hits = memory_docs::search(&shared_pool, &q, limit).await?;
|
||||
total += hits.len();
|
||||
render_hits("shared-memory", &hits, &mut out);
|
||||
}
|
||||
|
||||
if total == 0 {
|
||||
return Ok(ToolResult::Text(format!("No memory notes match {raw:?}.")));
|
||||
}
|
||||
Ok(ToolResult::Text(format!("{total} result(s):\n{out}")))
|
||||
})))
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn fts_query_quotes_tokens_and_survives_punctuation() {
|
||||
assert_eq!(fts_query("spesa settimana").unwrap(), "\"spesa\" \"settimana\"");
|
||||
// colons / dashes would be FTS5 operators unquoted; quoting makes them literal
|
||||
assert_eq!(fts_query("budget: 2026-07").unwrap(), "\"budget:\" \"2026-07\"");
|
||||
// an embedded quote is escaped by doubling
|
||||
assert_eq!(fts_query("say \"hi\"").unwrap(), "\"say\" \"\"\"hi\"\"\"");
|
||||
assert!(fts_query(" ").is_none());
|
||||
}
|
||||
}
|
||||
@@ -2,15 +2,18 @@ mod edit_file;
|
||||
mod grep_files;
|
||||
mod insert_at_line;
|
||||
mod list_files;
|
||||
mod memory_search;
|
||||
mod read_file;
|
||||
mod replace_lines;
|
||||
mod search_file;
|
||||
mod write_file;
|
||||
|
||||
use std::path::{Component, Path, PathBuf};
|
||||
use std::sync::Arc;
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use serde_json::Value;
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
use crate::tools::ToolRegistry;
|
||||
|
||||
@@ -25,11 +28,65 @@ pub use edit_file::EditFile;
|
||||
pub use grep_files::GrepFiles;
|
||||
pub use insert_at_line::InsertAtLine;
|
||||
pub use list_files::ListFiles;
|
||||
pub use memory_search::MemorySearch;
|
||||
pub use read_file::ReadFile;
|
||||
pub use replace_lines::ReplaceLines;
|
||||
pub use search_file::SearchFile;
|
||||
pub use write_file::WriteFile;
|
||||
|
||||
// ── Virtual memory namespace (blueprint §5) ───────────────────────────────────
|
||||
//
|
||||
// Two sibling top-level roots, each backed by the `memory_docs` table in SQLite
|
||||
// rather than the disk. The fs-tools intercept these prefixes in `run_with` and
|
||||
// route reads/writes to the `memory_docs` accessor on the right pool, so the LLM
|
||||
// uses ordinary read/write/list against what looks like two folders.
|
||||
|
||||
/// The current user's **private** memory — routed to `ctx.pool` (`{userid}.db`,
|
||||
/// behind SQLCipher).
|
||||
pub const USER_MEMORY_ROOT: &str = "user-memory";
|
||||
|
||||
/// The instance-wide **shared** memory — routed to the system pool (`system.db`,
|
||||
/// cleartext, readable by every member).
|
||||
pub const SHARED_MEMORY_ROOT: &str = "shared-memory";
|
||||
|
||||
/// Which memory store a path resolves to.
|
||||
pub(crate) enum MemScope {
|
||||
/// `user-memory/…` → the caller's own pool (`ToolContext::pool`).
|
||||
User,
|
||||
/// `shared-memory/…` → the shared system pool.
|
||||
Shared,
|
||||
}
|
||||
|
||||
/// A path that falls inside the virtual memory namespace: the store it belongs to
|
||||
/// and the note key **relative to that store's root** (the root prefix stripped).
|
||||
pub(crate) struct MemRef {
|
||||
pub scope: MemScope,
|
||||
pub rel: String,
|
||||
}
|
||||
|
||||
/// Classifies a user-supplied path. Returns `Some` when it lands under one of the
|
||||
/// virtual memory roots — to be routed to SQLite — and `None` for an ordinary
|
||||
/// disk path.
|
||||
///
|
||||
/// The **first** component decides the store, taken raw *before* normalization, so
|
||||
/// a `..` in the tail can never drop the memory root and silently fall back to a
|
||||
/// disk path. The tail is then normalized (resolving `.`/`..`) and clamped at the
|
||||
/// store root, so a memory path stays within its store and an absolute path is
|
||||
/// always disk.
|
||||
pub(crate) fn classify_memory(user_path: &str) -> Option<MemRef> {
|
||||
let mut parts = user_path.trim_start_matches("./").splitn(2, ['/', '\\']);
|
||||
let scope = match parts.next()? {
|
||||
USER_MEMORY_ROOT => MemScope::User,
|
||||
SHARED_MEMORY_ROOT => MemScope::Shared,
|
||||
_ => return None,
|
||||
};
|
||||
// Normalize the tail within the store (empty = the root itself). `..` clamps
|
||||
// at the root rather than escaping upward.
|
||||
let tail = parts.next().unwrap_or("");
|
||||
let rel = lexical_normalize(Path::new(tail)).to_string_lossy().replace('\\', "/");
|
||||
Some(MemRef { scope, rel })
|
||||
}
|
||||
|
||||
/// Resolve a user-supplied path:
|
||||
/// - starts with `/` → absolute path, used as-is
|
||||
/// - otherwise → relative to the process working directory (project root)
|
||||
@@ -127,13 +184,209 @@ pub(super) fn write_string(user_path: &str, content: &str) -> Result<()> {
|
||||
.with_context(|| format!("Failed to write: {}", abs.display()))
|
||||
}
|
||||
|
||||
pub fn register_all(registry: &mut ToolRegistry) {
|
||||
registry.register(EditFile::new());
|
||||
registry.register(GrepFiles::new());
|
||||
registry.register(InsertAtLine::new());
|
||||
registry.register(ListFiles::new());
|
||||
registry.register(ReadFile::new());
|
||||
registry.register(ReplaceLines::new());
|
||||
registry.register(SearchFile::new());
|
||||
registry.register(WriteFile::new());
|
||||
/// Registers the filesystem tools. `shared_pool` is the system (`shared-memory`)
|
||||
/// pool captured once here — a global singleton — and handed to the memory-aware
|
||||
/// tools; each still resolves the per-user (`user-memory`) pool per call from the
|
||||
/// `ToolContext`.
|
||||
pub fn register_all(registry: &mut ToolRegistry, shared_pool: Arc<SqlitePool>) {
|
||||
registry.register(EditFile::new(Arc::clone(&shared_pool)));
|
||||
registry.register(GrepFiles::new()); // not memory-aware yet — see blueprint Prossimi passi
|
||||
registry.register(InsertAtLine::new(Arc::clone(&shared_pool)));
|
||||
registry.register(ListFiles::new(Arc::clone(&shared_pool)));
|
||||
registry.register(ReadFile::new(Arc::clone(&shared_pool)));
|
||||
registry.register(ReplaceLines::new(Arc::clone(&shared_pool)));
|
||||
registry.register(SearchFile::new(Arc::clone(&shared_pool)));
|
||||
registry.register(MemorySearch::new(Arc::clone(&shared_pool)));
|
||||
registry.register(WriteFile::new(shared_pool));
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::path::PathBuf;
|
||||
|
||||
use serde_json::json;
|
||||
|
||||
use crate::tools::{ExecutionOutcome, Tool, ToolContext};
|
||||
|
||||
#[test]
|
||||
fn classify_memory_splits_root_from_key() {
|
||||
let u = classify_memory("user-memory/notes/x.md").unwrap();
|
||||
assert!(matches!(u.scope, MemScope::User));
|
||||
assert_eq!(u.rel, "notes/x.md");
|
||||
|
||||
let s = classify_memory("./shared-memory/casa.md").unwrap();
|
||||
assert!(matches!(s.scope, MemScope::Shared));
|
||||
assert_eq!(s.rel, "casa.md");
|
||||
|
||||
// bare roots (with/without trailing slash) resolve to the empty key
|
||||
assert_eq!(classify_memory("user-memory").unwrap().rel, "");
|
||||
assert_eq!(classify_memory("shared-memory/").unwrap().rel, "");
|
||||
|
||||
// `..` clamps inside the store instead of falling back to a disk path
|
||||
assert_eq!(classify_memory("user-memory/../secret.md").unwrap().rel, "secret.md");
|
||||
|
||||
// ordinary, absolute, and look-alike paths are disk (None)
|
||||
assert!(classify_memory("src/main.rs").is_none());
|
||||
assert!(classify_memory("/etc/hosts").is_none());
|
||||
assert!(classify_memory("user-memoryish/x").is_none());
|
||||
}
|
||||
|
||||
/// A throwaway owner-schema pool (as `Arc`, ready for a `ToolContext`), plus its
|
||||
/// dir for cleanup. `tag` + a counter keep parallel tests off the same file.
|
||||
async fn store(tag: &str) -> (Arc<SqlitePool>, PathBuf) {
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
static SEQ: AtomicU64 = AtomicU64::new(0);
|
||||
let n = SEQ.fetch_add(1, Ordering::Relaxed);
|
||||
let dir = std::env::temp_dir().join(format!("skald-fsmem-{}-{tag}-{n}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let pool = crate::db::create_user_pool(&dir.join("owner.db"), None).await.unwrap();
|
||||
(Arc::new(pool), dir)
|
||||
}
|
||||
|
||||
/// Drives a tool through the context-aware path and returns its text result.
|
||||
async fn drive(tool: &dyn Tool, ctx: &ToolContext, args: Value) -> Result<String, String> {
|
||||
let exec = tool.run_with(ctx, args);
|
||||
match exec.wait().await {
|
||||
ExecutionOutcome::Completed(r) => Ok(r.to_wire()),
|
||||
ExecutionOutcome::Failed(e) => Err(e),
|
||||
ExecutionOutcome::Cancelled => Err("cancelled".into()),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn memory_tools_route_and_isolate_user_vs_shared() {
|
||||
let (user, udir) = store("user").await;
|
||||
let (shared, sdir) = store("shared").await;
|
||||
|
||||
// The shared pool is captured by the tools; the user pool arrives per call.
|
||||
let write = WriteFile::new(Arc::clone(&shared));
|
||||
let read = ReadFile::new(Arc::clone(&shared));
|
||||
let list = ListFiles::new(Arc::clone(&shared));
|
||||
let ctx = ToolContext { session_id: 1, pool: Arc::clone(&user) };
|
||||
|
||||
// Private write lands in the user pool — and never in the shared one.
|
||||
let out = drive(&write, &ctx, json!({"path":"user-memory/spesa.md","content":"latte\npane"}))
|
||||
.await.unwrap();
|
||||
assert!(out.starts_with("Created user-memory/spesa.md"), "{out}");
|
||||
assert!(crate::db::memory_docs::get(&user, "spesa.md").await.unwrap().is_some());
|
||||
assert!(crate::db::memory_docs::get(&shared, "spesa.md").await.unwrap().is_none(),
|
||||
"a user-memory write must not touch the shared store");
|
||||
|
||||
// Shared write lands in the shared pool — and never in the user one.
|
||||
drive(&write, &ctx, json!({"path":"shared-memory/casa.md","content":"wifi 1234"}))
|
||||
.await.unwrap();
|
||||
assert!(crate::db::memory_docs::get(&shared, "casa.md").await.unwrap().is_some());
|
||||
assert!(crate::db::memory_docs::get(&user, "casa.md").await.unwrap().is_none());
|
||||
|
||||
// Read back with 1-based line numbers; a missing note errors.
|
||||
let r = drive(&read, &ctx, json!({"path":"user-memory/spesa.md"})).await.unwrap();
|
||||
assert!(r.contains("| latte") && r.contains("| pane"), "{r}");
|
||||
assert!(drive(&read, &ctx, json!({"path":"user-memory/nope.md"})).await.is_err());
|
||||
|
||||
// A second write to the same key overwrites (and says so).
|
||||
let out = drive(&write, &ctx, json!({"path":"user-memory/spesa.md","content":"latte"}))
|
||||
.await.unwrap();
|
||||
assert!(out.starts_with("Overwrote user-memory/spesa.md"), "{out}");
|
||||
|
||||
// Listing returns keys relative to the requested directory.
|
||||
drive(&write, &ctx, json!({"path":"user-memory/notes/idee.md","content":"x"}))
|
||||
.await.unwrap();
|
||||
let l = drive(&list, &ctx, json!({"path":"user-memory"})).await.unwrap();
|
||||
assert_eq!(serde_json::from_str::<Vec<String>>(&l).unwrap(),
|
||||
vec!["notes/idee.md".to_string(), "spesa.md".to_string()]);
|
||||
let l = drive(&list, &ctx, json!({"path":"user-memory/notes"})).await.unwrap();
|
||||
assert_eq!(serde_json::from_str::<Vec<String>>(&l).unwrap(),
|
||||
vec!["idee.md".to_string()]);
|
||||
|
||||
let _ = std::fs::remove_dir_all(&udir);
|
||||
let _ = std::fs::remove_dir_all(&sdir);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn memory_edit_insert_replace_search_route_to_the_note() {
|
||||
let (user, udir) = store("edit-user").await;
|
||||
let (shared, sdir) = store("edit-shared").await;
|
||||
|
||||
let write = WriteFile::new(Arc::clone(&shared));
|
||||
let edit = EditFile::new(Arc::clone(&shared));
|
||||
let insert = InsertAtLine::new(Arc::clone(&shared));
|
||||
let replace = ReplaceLines::new(Arc::clone(&shared));
|
||||
let search = SearchFile::new(Arc::clone(&shared));
|
||||
let ctx = ToolContext { session_id: 1, pool: Arc::clone(&user) };
|
||||
|
||||
async fn note(pool: &SqlitePool, path: &str) -> String {
|
||||
crate::db::memory_docs::get(pool, path).await.unwrap().unwrap().content
|
||||
}
|
||||
|
||||
drive(&write, &ctx, json!({"path":"user-memory/todo.md","content":"latte\npane\nuvoa"}))
|
||||
.await.unwrap();
|
||||
|
||||
// edit_file: fix the typo, in place
|
||||
let out = drive(&edit, &ctx, json!({"path":"user-memory/todo.md","old":"uvoa","new":"uova"}))
|
||||
.await.unwrap();
|
||||
assert_eq!(out, "Edited user-memory/todo.md.");
|
||||
assert_eq!(note(&user, "todo.md").await, "latte\npane\nuova");
|
||||
|
||||
// insert_at_line: add a line after line 1
|
||||
drive(&insert, &ctx, json!({"path":"user-memory/todo.md","line":1,"content":"burro","placement":"after"}))
|
||||
.await.unwrap();
|
||||
assert_eq!(note(&user, "todo.md").await, "latte\nburro\npane\nuova");
|
||||
|
||||
// replace_lines: collapse lines 2–3 into one
|
||||
drive(&replace, &ctx, json!({"path":"user-memory/todo.md","from_line":2,"to_line":3,"new":"olio"}))
|
||||
.await.unwrap();
|
||||
assert_eq!(note(&user, "todo.md").await, "latte\nolio\nuova");
|
||||
|
||||
// search_file: find a line inside the note
|
||||
let s = drive(&search, &ctx, json!({"path":"user-memory/todo.md","query":"olio"})).await.unwrap();
|
||||
assert!(s.contains("match(es) in user-memory/todo.md"), "{s}");
|
||||
assert!(s.contains("| olio"), "{s}");
|
||||
|
||||
// editing a note that doesn't exist errors, not creates
|
||||
assert!(drive(&edit, &ctx, json!({"path":"user-memory/ghost.md","old":"a","new":"b"}))
|
||||
.await.is_err());
|
||||
|
||||
let _ = std::fs::remove_dir_all(&udir);
|
||||
let _ = std::fs::remove_dir_all(&sdir);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn memory_search_scopes_to_user_shared_or_all() {
|
||||
let (user, udir) = store("search-user").await;
|
||||
let (shared, sdir) = store("search-shared").await;
|
||||
|
||||
let write = WriteFile::new(Arc::clone(&shared));
|
||||
let search = MemorySearch::new(Arc::clone(&shared));
|
||||
let ctx = ToolContext { session_id: 1, pool: Arc::clone(&user) };
|
||||
|
||||
// one note in each store, both mentioning "wifi"
|
||||
drive(&write, &ctx, json!({"path":"user-memory/rete.md","content":"la mia wifi privata"}))
|
||||
.await.unwrap();
|
||||
drive(&write, &ctx, json!({"path":"shared-memory/casa.md","content":"wifi di casa 1234"}))
|
||||
.await.unwrap();
|
||||
|
||||
// scope=private → only the user store
|
||||
let r = drive(&search, &ctx, json!({"query":"wifi","scope":"private"})).await.unwrap();
|
||||
assert!(r.contains("[user-memory] rete.md"), "{r}");
|
||||
assert!(!r.contains("shared-memory"), "{r}");
|
||||
|
||||
// scope=shared → only the shared store
|
||||
let r = drive(&search, &ctx, json!({"query":"wifi","scope":"shared"})).await.unwrap();
|
||||
assert!(r.contains("[shared-memory] casa.md"), "{r}");
|
||||
assert!(!r.contains("[user-memory]"), "{r}");
|
||||
|
||||
// scope=all (default) → both, and the snippet highlights the term
|
||||
let r = drive(&search, &ctx, json!({"query":"wifi"})).await.unwrap();
|
||||
assert!(r.contains("[user-memory] rete.md") && r.contains("[shared-memory] casa.md"), "{r}");
|
||||
assert!(r.contains("[wifi]"), "snippet should highlight the match: {r}");
|
||||
|
||||
// no match → a friendly message, not an error
|
||||
let r = drive(&search, &ctx, json!({"query":"inesistente"})).await.unwrap();
|
||||
assert!(r.starts_with("No memory notes match"), "{r}");
|
||||
|
||||
let _ = std::fs::remove_dir_all(&udir);
|
||||
let _ = std::fs::remove_dir_all(&sdir);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,13 +1,50 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use anyhow::Result;
|
||||
use serde_json::{Value, json};
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL};
|
||||
use super::read_to_string;
|
||||
use crate::tools::{
|
||||
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
|
||||
truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL,
|
||||
};
|
||||
use super::{classify_memory, read_to_string, MemScope};
|
||||
|
||||
pub struct ReadFile;
|
||||
pub struct ReadFile {
|
||||
/// The `shared-memory` (system) pool. `user-memory` resolves per call from the
|
||||
/// `ToolContext`; only the shared store is a global singleton captured here.
|
||||
shared_pool: Arc<SqlitePool>,
|
||||
}
|
||||
|
||||
impl ReadFile {
|
||||
pub fn new() -> Self { Self }
|
||||
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
|
||||
}
|
||||
|
||||
/// Render `content` with 1-based line numbers, honouring the same
|
||||
/// `start`/`end_line`/`limit` windowing as the disk path. Shared by the on-disk
|
||||
/// [`ReadFile::execute`] and the `memory/` routing in [`ReadFile::run_with`].
|
||||
fn number_lines(content: &str, start: usize, end_line: Option<usize>, limit: Option<usize>) -> String {
|
||||
let lines: Vec<&str> = content.lines().collect();
|
||||
let total = lines.len();
|
||||
|
||||
let end = match (end_line, limit) {
|
||||
(Some(e), _) => e.min(total),
|
||||
(None, Some(l)) => (start + l).min(total),
|
||||
(None, None) => total,
|
||||
};
|
||||
|
||||
if start >= total && total > 0 {
|
||||
return format!("(file has only {total} lines; start_line {} is out of range)", start + 1);
|
||||
}
|
||||
|
||||
let end = end.max(start);
|
||||
let width = total.to_string().len().max(3);
|
||||
lines[start..end]
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, line)| format!("{:>width$} | {line}", start + i + 1))
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n")
|
||||
}
|
||||
|
||||
impl Tool for ReadFile {
|
||||
@@ -19,7 +56,8 @@ impl Tool for ReadFile {
|
||||
Use instead of cat/head/tail in the terminal. \
|
||||
Returns text prefixed as ' N | line'. When calling edit_file, copy the text after '| ' exactly. \
|
||||
For large files use start_line/end_line to read in chunks — files over ~2000 lines should never be read whole. \
|
||||
Use limit to cap output when end_line is unknown."
|
||||
Use limit to cap output when end_line is unknown. \
|
||||
Paths under user-memory/ (private) or shared-memory/ (shared) read a note from your memory instead of disk."
|
||||
}
|
||||
|
||||
fn parameters_schema(&self) -> Value {
|
||||
@@ -70,37 +108,35 @@ impl Tool for ReadFile {
|
||||
}
|
||||
}
|
||||
|
||||
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
|
||||
/// path falls through to the on-disk [`execute`](Self::execute).
|
||||
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
|
||||
let path = super::path_arg(&args).unwrap_or_default();
|
||||
let Some(m) = classify_memory(&path) else { return self.run(args); };
|
||||
let pool = match m.scope {
|
||||
MemScope::User => Arc::clone(&ctx.pool),
|
||||
MemScope::Shared => Arc::clone(&self.shared_pool),
|
||||
};
|
||||
let rel = m.rel;
|
||||
let start = args["start_line"].as_u64().map(|n| (n as usize).saturating_sub(1)).unwrap_or(0);
|
||||
let end_line = args["end_line"].as_u64().map(|n| n as usize);
|
||||
let limit = args["limit"].as_u64().map(|n| n.min(2000) as usize);
|
||||
|
||||
Box::new(SimpleExecution::new(Box::pin(async move {
|
||||
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
|
||||
anyhow::bail!("No note at {path}");
|
||||
};
|
||||
Ok(ToolResult::Text(number_lines(&doc.content, start, end_line, limit)))
|
||||
})))
|
||||
}
|
||||
|
||||
fn execute(&self, args: Value) -> Result<String> {
|
||||
let user_path = args["path"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
|
||||
let content = read_to_string(user_path)?;
|
||||
let lines: Vec<&str> = content.lines().collect();
|
||||
let total = lines.len();
|
||||
|
||||
let start = args["start_line"].as_u64().map(|n| (n as usize).saturating_sub(1)).unwrap_or(0);
|
||||
let end_line = args["end_line"].as_u64().map(|n| n as usize);
|
||||
let limit = args["limit"].as_u64().map(|n| n.min(2000) as usize);
|
||||
let start = args["start_line"].as_u64()
|
||||
.map(|n| (n as usize).saturating_sub(1))
|
||||
.unwrap_or(0);
|
||||
let end = match (args["end_line"].as_u64(), limit) {
|
||||
(Some(e), _) => (e as usize).min(total),
|
||||
(None, Some(l)) => (start + l).min(total),
|
||||
(None, None) => total,
|
||||
};
|
||||
|
||||
if start >= total && total > 0 {
|
||||
return Ok(format!("(file has only {total} lines; start_line {start_line} is out of range)",
|
||||
start_line = start + 1));
|
||||
}
|
||||
|
||||
let end = end.max(start);
|
||||
|
||||
let width = total.to_string().len().max(3);
|
||||
let numbered = lines[start..end]
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, line)| format!("{:>width$} | {line}", start + i + 1))
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n");
|
||||
Ok(numbered)
|
||||
Ok(number_lines(&content, start, end_line, limit))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,13 +1,56 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use anyhow::Result;
|
||||
use serde_json::{Value, json};
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL};
|
||||
use super::{read_to_string, write_string};
|
||||
use crate::tools::{
|
||||
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
|
||||
truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL,
|
||||
};
|
||||
use super::{classify_memory, read_to_string, write_string, MemScope};
|
||||
|
||||
pub struct ReplaceLines;
|
||||
pub struct ReplaceLines {
|
||||
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
|
||||
shared_pool: Arc<SqlitePool>,
|
||||
}
|
||||
|
||||
impl ReplaceLines {
|
||||
pub fn new() -> Self { Self }
|
||||
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
|
||||
}
|
||||
|
||||
/// Replaces the inclusive 1-based line range with `new`, returning the new content
|
||||
/// and a result message. Shared by the on-disk `execute` and the `memory/` routing;
|
||||
/// `display` is the path used in the message.
|
||||
fn apply_replace(content: &str, args: &Value, display: &str) -> Result<(String, String)> {
|
||||
let from_line = args["from_line"].as_u64()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: from_line"))? as usize;
|
||||
let to_line = args["to_line"].as_u64()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: to_line"))? as usize;
|
||||
let new = args["new"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: new"))?;
|
||||
|
||||
if from_line == 0 { anyhow::bail!("from_line must be >= 1"); }
|
||||
if to_line < from_line { anyhow::bail!("to_line must be >= from_line"); }
|
||||
|
||||
let mut lines: Vec<&str> = content.lines().collect();
|
||||
let total = lines.len();
|
||||
if from_line > total {
|
||||
anyhow::bail!("from_line {from_line} exceeds file length ({total} lines)");
|
||||
}
|
||||
let to_clamped = to_line.min(total);
|
||||
let new_lines: Vec<&str> = new.lines().collect();
|
||||
lines.splice((from_line - 1)..to_clamped, new_lines);
|
||||
|
||||
let has_trailing = content.ends_with('\n');
|
||||
let mut updated = lines.join("\n");
|
||||
if has_trailing { updated.push('\n'); }
|
||||
|
||||
let msg = format!(
|
||||
"Replaced lines {from_line}–{to_clamped} in {display} with {} new lines.",
|
||||
new.lines().count()
|
||||
);
|
||||
Ok((updated, msg))
|
||||
}
|
||||
|
||||
impl Tool for ReplaceLines {
|
||||
@@ -51,38 +94,33 @@ impl Tool for ReplaceLines {
|
||||
}
|
||||
}
|
||||
|
||||
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
|
||||
/// path falls through to the on-disk [`execute`](Self::execute).
|
||||
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
|
||||
let path = super::path_arg(&args).unwrap_or_default();
|
||||
let Some(m) = classify_memory(&path) else { return self.run(args); };
|
||||
let pool = match m.scope {
|
||||
MemScope::User => Arc::clone(&ctx.pool),
|
||||
MemScope::Shared => Arc::clone(&self.shared_pool),
|
||||
};
|
||||
let rel = m.rel;
|
||||
|
||||
Box::new(SimpleExecution::new(Box::pin(async move {
|
||||
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
|
||||
anyhow::bail!("No note at {path}");
|
||||
};
|
||||
let (updated, msg) = apply_replace(&doc.content, &args, &path)?;
|
||||
crate::db::memory_docs::upsert(&pool, &rel, &updated).await?;
|
||||
Ok(ToolResult::Text(msg))
|
||||
})))
|
||||
}
|
||||
|
||||
fn execute(&self, args: Value) -> Result<String> {
|
||||
let user_path = args["path"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
|
||||
let from_line = args["from_line"].as_u64()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: from_line"))? as usize;
|
||||
let to_line = args["to_line"].as_u64()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: to_line"))? as usize;
|
||||
let new = args["new"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: new"))?;
|
||||
|
||||
if from_line == 0 { anyhow::bail!("from_line must be >= 1"); }
|
||||
if to_line < from_line { anyhow::bail!("to_line must be >= from_line"); }
|
||||
|
||||
let content = read_to_string(user_path)?;
|
||||
let mut lines: Vec<&str> = content.lines().collect();
|
||||
let total = lines.len();
|
||||
if from_line > total {
|
||||
anyhow::bail!("from_line {from_line} exceeds file length ({total} lines)");
|
||||
}
|
||||
let to_clamped = to_line.min(total);
|
||||
let new_lines: Vec<&str> = new.lines().collect();
|
||||
lines.splice((from_line - 1)..to_clamped, new_lines);
|
||||
|
||||
let has_trailing = content.ends_with('\n');
|
||||
let mut updated = lines.join("\n");
|
||||
if has_trailing { updated.push('\n'); }
|
||||
|
||||
let (updated, msg) = apply_replace(&content, &args, user_path)?;
|
||||
write_string(user_path, &updated)?;
|
||||
|
||||
Ok(format!(
|
||||
"Replaced lines {from_line}–{to_clamped} in {user_path} with {} new lines.",
|
||||
new.lines().count()
|
||||
))
|
||||
Ok(msg)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,13 +1,67 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use anyhow::Result;
|
||||
use serde_json::{Value, json};
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL};
|
||||
use super::read_to_string;
|
||||
use crate::tools::{
|
||||
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
|
||||
truncate_label, MAX_LABEL_SHORT, MAX_LABEL_FULL,
|
||||
};
|
||||
use super::{classify_memory, read_to_string, MemScope};
|
||||
|
||||
pub struct SearchFile;
|
||||
pub struct SearchFile {
|
||||
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
|
||||
shared_pool: Arc<SqlitePool>,
|
||||
}
|
||||
|
||||
impl SearchFile {
|
||||
pub fn new() -> Self { Self }
|
||||
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
|
||||
}
|
||||
|
||||
/// Renders the case-insensitive substring search over `text` with context lines.
|
||||
/// Shared by the on-disk `execute` and the `memory/` routing; `display` is the
|
||||
/// path shown in the output.
|
||||
fn render_search(text: &str, args: &Value, display: &str) -> Result<String> {
|
||||
let query = args["query"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: query"))?;
|
||||
let context = args["context_lines"].as_u64().unwrap_or(3).min(10) as usize;
|
||||
|
||||
let lines: Vec<&str> = text.lines().collect();
|
||||
let lower_query = query.to_lowercase();
|
||||
let width = lines.len().to_string().len().max(3);
|
||||
|
||||
let matches: Vec<usize> = lines.iter().enumerate()
|
||||
.filter(|(_, l)| l.to_lowercase().contains(&lower_query))
|
||||
.map(|(i, _)| i)
|
||||
.collect();
|
||||
|
||||
if matches.is_empty() {
|
||||
return Ok(format!("No matches found for {:?} in {display}.", query));
|
||||
}
|
||||
|
||||
let mut chunks: Vec<(usize, usize)> = Vec::new();
|
||||
for &m in &matches {
|
||||
let start = m.saturating_sub(context);
|
||||
let end = (m + context).min(lines.len() - 1);
|
||||
if let Some(last) = chunks.last_mut() {
|
||||
if start <= last.1 + 1 { last.1 = last.1.max(end); continue; }
|
||||
}
|
||||
chunks.push((start, end));
|
||||
}
|
||||
|
||||
let match_set: std::collections::HashSet<usize> = matches.into_iter().collect();
|
||||
let mut out = format!("{} match(es) in {display}:\n", match_set.len());
|
||||
|
||||
for (ci, (start, end)) in chunks.iter().enumerate() {
|
||||
if ci > 0 { out.push_str(" ···\n"); }
|
||||
for idx in *start..=*end {
|
||||
let marker = if match_set.contains(&idx) { ">" } else { " " };
|
||||
out.push_str(&format!("{marker}{:>width$} | {}\n", idx + 1, lines[idx]));
|
||||
}
|
||||
}
|
||||
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
impl Tool for SearchFile {
|
||||
@@ -53,48 +107,29 @@ impl Tool for SearchFile {
|
||||
}
|
||||
}
|
||||
|
||||
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
|
||||
/// path falls through to the on-disk [`execute`](Self::execute).
|
||||
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
|
||||
let path = super::path_arg(&args).unwrap_or_default();
|
||||
let Some(m) = classify_memory(&path) else { return self.run(args); };
|
||||
let pool = match m.scope {
|
||||
MemScope::User => Arc::clone(&ctx.pool),
|
||||
MemScope::Shared => Arc::clone(&self.shared_pool),
|
||||
};
|
||||
let rel = m.rel;
|
||||
|
||||
Box::new(SimpleExecution::new(Box::pin(async move {
|
||||
let Some(doc) = crate::db::memory_docs::get(&pool, &rel).await? else {
|
||||
anyhow::bail!("No note at {path}");
|
||||
};
|
||||
Ok(ToolResult::Text(render_search(&doc.content, &args, &path)?))
|
||||
})))
|
||||
}
|
||||
|
||||
fn execute(&self, args: Value) -> Result<String> {
|
||||
let user_path = args["path"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
|
||||
let query = args["query"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: query"))?;
|
||||
let context = args["context_lines"].as_u64().unwrap_or(3).min(10) as usize;
|
||||
|
||||
let text = read_to_string(user_path)?;
|
||||
let lines: Vec<&str> = text.lines().collect();
|
||||
let lower_query = query.to_lowercase();
|
||||
let width = lines.len().to_string().len().max(3);
|
||||
|
||||
let matches: Vec<usize> = lines.iter().enumerate()
|
||||
.filter(|(_, l)| l.to_lowercase().contains(&lower_query))
|
||||
.map(|(i, _)| i)
|
||||
.collect();
|
||||
|
||||
if matches.is_empty() {
|
||||
return Ok(format!("No matches found for {:?} in {user_path}.", query));
|
||||
}
|
||||
|
||||
let mut chunks: Vec<(usize, usize)> = Vec::new();
|
||||
for &m in &matches {
|
||||
let start = m.saturating_sub(context);
|
||||
let end = (m + context).min(lines.len() - 1);
|
||||
if let Some(last) = chunks.last_mut() {
|
||||
if start <= last.1 + 1 { last.1 = last.1.max(end); continue; }
|
||||
}
|
||||
chunks.push((start, end));
|
||||
}
|
||||
|
||||
let match_set: std::collections::HashSet<usize> = matches.into_iter().collect();
|
||||
let mut out = format!("{} match(es) in {user_path}:\n", match_set.len());
|
||||
|
||||
for (ci, (start, end)) in chunks.iter().enumerate() {
|
||||
if ci > 0 { out.push_str(" ···\n"); }
|
||||
for idx in *start..=*end {
|
||||
let marker = if match_set.contains(&idx) { ">" } else { " " };
|
||||
out.push_str(&format!("{marker}{:>width$} | {}\n", idx + 1, lines[idx]));
|
||||
}
|
||||
}
|
||||
|
||||
Ok(out)
|
||||
render_search(&text, &args, user_path)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,13 +1,22 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use anyhow::Result;
|
||||
use serde_json::{Value, json};
|
||||
use sqlx::SqlitePool;
|
||||
|
||||
use crate::tools::{Tool, ToolDescriptionLength, truncate_label, MAX_LABEL_SHORT};
|
||||
use super::{resolve, write_string};
|
||||
use crate::tools::{
|
||||
SimpleExecution, Tool, ToolContext, ToolDescriptionLength, ToolExecution, ToolResult,
|
||||
truncate_label, MAX_LABEL_SHORT,
|
||||
};
|
||||
use super::{classify_memory, resolve, write_string, MemScope};
|
||||
|
||||
pub struct WriteFile;
|
||||
pub struct WriteFile {
|
||||
/// The `shared-memory` (system) pool; see [`ReadFile`](super::ReadFile).
|
||||
shared_pool: Arc<SqlitePool>,
|
||||
}
|
||||
|
||||
impl WriteFile {
|
||||
pub fn new() -> Self { Self }
|
||||
pub fn new(shared_pool: Arc<SqlitePool>) -> Self { Self { shared_pool } }
|
||||
}
|
||||
|
||||
impl Tool for WriteFile {
|
||||
@@ -18,7 +27,8 @@ impl Tool for WriteFile {
|
||||
"Create a new file or fully overwrite an existing one. \
|
||||
Use instead of echo/cat heredoc in the terminal. \
|
||||
Relative paths are resolved from the project root; absolute paths (starting with /) are used as-is. \
|
||||
OVERWRITES the entire file — for targeted edits to an existing file use edit_file instead."
|
||||
OVERWRITES the entire file — for targeted edits to an existing file use edit_file instead. \
|
||||
Write Markdown under user-memory/ (private to you) or shared-memory/ (shared with everyone) to save a durable note in your memory instead of on disk."
|
||||
}
|
||||
|
||||
fn parameters_schema(&self) -> Value {
|
||||
@@ -48,6 +58,30 @@ impl Tool for WriteFile {
|
||||
truncate_label(&format!("write_file `{path}`"), MAX_LABEL_SHORT)
|
||||
}
|
||||
|
||||
/// Routes `user-memory/…` / `shared-memory/…` to the note store; every other
|
||||
/// path falls through to the on-disk [`execute`](Self::execute).
|
||||
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
|
||||
let path = super::path_arg(&args).unwrap_or_default();
|
||||
let Some(m) = classify_memory(&path) else { return self.run(args); };
|
||||
let pool = match m.scope {
|
||||
MemScope::User => Arc::clone(&ctx.pool),
|
||||
MemScope::Shared => Arc::clone(&self.shared_pool),
|
||||
};
|
||||
let rel = m.rel;
|
||||
let content = args["content"].as_str().map(str::to_string);
|
||||
|
||||
Box::new(SimpleExecution::new(Box::pin(async move {
|
||||
let content = content.ok_or_else(|| anyhow::anyhow!("Missing required argument: content"))?;
|
||||
if rel.is_empty() {
|
||||
anyhow::bail!("{path} is a memory root, not a note — write to a path like {path}/notes.md");
|
||||
}
|
||||
let existed = crate::db::memory_docs::get(&pool, &rel).await?.is_some();
|
||||
crate::db::memory_docs::upsert(&pool, &rel, &content).await?;
|
||||
let verb = if existed { "Overwrote" } else { "Created" };
|
||||
Ok(ToolResult::Text(format!("{verb} {path} ({} bytes).", content.len())))
|
||||
})))
|
||||
}
|
||||
|
||||
fn execute(&self, args: Value) -> Result<String> {
|
||||
let user_path = args["path"].as_str()
|
||||
.ok_or_else(|| anyhow::anyhow!("Missing required argument: path"))?;
|
||||
|
||||
Reference in New Issue
Block a user