Skald-Circle/dev-docs/filesystem-and-containers.md

*Skald dev-docs — architectural reference for coding agents. Index: [README.md](README.md) · Entry point: [../CLAUDE.md](../CLAUDE.md)*

**Read this when:** you touch `container/`, the fs-tools, mounts, skills, or anything about where a path physically lives.

---

# Filesystem & containers (blueprint §6)

Each user has one **permanent Docker container** (`skald-{userid}`, our own `skald-runtime` image with python+node and a preinstalled shell toolbelt), created on user creation and started at boot (`ContainerManager`, `crates/skald-core/src/container/`). Docker is **required**: a missing daemon fails `Skald::new` and the process exits. **What goes in the image vs. what the agent installs on demand** is a real trade, and the Dockerfile states its rule: `sudo apt-get install` works in the sandbox but re-runs on **every container recreate**, inside a task, where it costs latency and can fail — while the image is **one, shared by every container**, so preinstalling costs its size once for the whole box. Anything an agent reaches for repeatedly is therefore baked in; `build-essential`/`python3-dev` and `pandoc` are deliberately left out as big *and* self-recoverable. The container runs as the **host `uid:gid`** (not root) so files created in-container and by the host-side fs-tools share ownership on the bind mounts (matters on native Linux; masked on macOS Docker Desktop). Because that user isn't root, the image ships passwordless `sudo` (a passwd/shadow entry is injected at create) so an agent can still `sudo apt-get install …`; `--init` runs tini as pid 1 to reap zombies.

The agent sees **one namespace**, routed on the first path component. The choke point is `UserFs` (`core-api/src/user_fs.rs`, a pure value type carried in `ToolContext.fs`), plus `resolve_host_path()` in `tools/fs/mod.rs`:

| Agent path | Backing | Routed by |
| ---- | ---- | ---- |
| `user-memory/…` | SQLite `ctx.pool` (`{userid}.db`) | `classify_memory` → `memory_docs` |
| `shared-memory/…` | SQLite `system.db` | `classify_memory` → `memory_docs` |
| `shared/{X}/…` | host `{WD}/shared/{X}` (if a member) | `UserFs::host_base_and_tail` |
| `projects/{O}/{S}/…` | host `{WD}/projects/{owner_userid}/{S}` (if a member) | `UserFs::host_base_and_tail` |
| `skills/shared/{id}/…` | host `{WD}/skills/{id}` — **read-only** | `UserFs::host_base_and_tail` |
| `skills/{username}/{id}/…` | host `{WD}/skills-users/{userid}/{id}` — **read-only** | `UserFs::host_base_and_tail` |
| `~/…`, relative | host `{WD}/homes/{userid}` | `UserFs::host_base_and_tail` |
| any other absolute path (`/tmp/…`, `/etc/…`) | the **container's own** filesystem | `resolve_target` → `container::exec_fs` |

Two views, **one storage**: for the mounted subtree the fs-tools run **host-side** in the Skald process on `{WD}/homes/{userid}` + `{WD}/shared/{X}`; `execute_cmd` runs **inside the container** (`docker exec -w <container-path> skald-{userid} sh -c …`, via `ExecuteCmd::run_with`) on the same paths bind-mounted (`homes/{userid}`→`/root`, `shared/{X}`→`/root/shared/{X}`, read-only when `can_write=0`). A file written in the container appears to the host fs-tools and vice versa.

**The security boundary is the container, not the mounted subtree — the mount is the *fast* path, not the only one.** An agent already reaches every corner of its container through `execute_cmd`, which runs there with passwordless `sudo`; fs-tools that stopped at the mounts were not protecting anything, they were offering a poorer view of the same sandbox, and the model answered that by shelling out (the observed failure: `read_file /tmp/cv.txt` → *"path escapes your workspace"* → the agent re-read it with `cat`). So `resolve_target` routes a physical path to one of two backings. An **absolute** path is container vocabulary — it is what `execute_cmd` prints — so it is reverse-mapped through `UserFs::container_to_agent` first: landing on a mount takes the host path (**`/root/x` *is* `~/x`**, which the tools used to reject outright, since `PathBuf::join` with an absolute tail silently discards the base and the result then failed the prefix check); landing nowhere means it exists only in the container, and `container::exec_fs` acts there over `docker exec` (paths passed **positionally** as `$1`, so a path containing `$(…)` is data, not syntax). Membership is not bypassed: `/root/shared/{X}` for a non-member still resolves to the same error as `shared/{X}`.

**One implementation per tool, not two.** Every single-file fs-tool already funnels through the same shape — resolve, then run a sync `execute` over one absolute host path — so the container branch is a **shuttle** (`fs::Shuttle`, behind `fs::run_physical`): pull the file out of the container, run the *unchanged* tool on the copy, push it back if the content changed (compared by bytes, not mtime, whose one-second resolution would miss a fast edit). Nothing about a tool's messages, diffs or pure transforms is duplicated. A missing remote file is deliberately **not** pre-created — `write_file` reports "Created" vs "Overwrote" from whether the path existed, and a placeholder would make every creation lie. Three tools opt out of the shuttle because a single file is the wrong unit: `list_files` lists in place via `exec_fs::list` (`find -printf`; `line_count` is omitted, since counting lines would turn a listing into a `docker exec` per file), `read_file` reads container paths as text (a shuttled copy is gone by the time the projection would inline a `MediaRef`, so media stays a mount-only feature), and `grep_files` **refuses** container paths with a pointer to `execute_cmd` + `rg` — its regex flavour, glob, windowing and offset would all have to be re-derived from ripgrep's flags, and a grep that answers *almost* the same is worse than one that says where to go. The viewer follows the same routing through `resolve_view_target` (`GET /api/file` and `show_file_to_user` open container paths; served without an ETag, so the editor stays read-only there). One read-only shuttle lives here too: `GET /api/file?compile-docx=true` on a **container-only** word document pulls the bytes out with `exec_fs::read` and converts the host-side copy via `DocxConverter::convert_bytes` — correct because the format is self-contained. LaTeX deliberately gets no such branch: a shuttled `.tex` would silently lose its relative `\input`/`\includegraphics` dependencies.

**The memory roots are signposted inside the container, not merely absent.** `user-memory/`/`shared-memory/` are virtual, so nothing of them existed on disk — and the nothing was worse than it sounds: `cat user-memory/x.md` returned a bare ENOENT (which reads as *the note is missing*, not *wrong door*), while `mkdir -p user-memory && echo … > user-memory/x.md` **succeeded**, writing a real file into the home that no reader ever visits and that the next `ls` then confirms as if it had worked. Each root is therefore a **read-only bind mount** (`{WD}/.memory-signpost/{root}` → `{container_home}/{root}:ro`, gitignored, rewritten from consts on every `ensure`) holding a README that names the tools. Read-only *as a mount*, not as a mode: the container user has passwordless `sudo`, so a `chmod` would be a suggestion, whereas `:ro` holds — remounting needs `CAP_SYS_ADMIN` (verified: write, `sudo` write, `sudo chmod`, `sudo mount -o remount,rw` and `sudo rm` all fail). A README rather than an empty dir because `Permission denied` is an error, not an instruction — models answer it by reaching for `sudo`; the README puts the correction in the directory the failing command just named. These mounts are deliberately **not** in `UserFs`: they back no agent path and the host-side fs-tools must never resolve into them. They are the **fourth self-heal axis** in `reusable()` (`signposts_mounted`) rather than an `IMAGE_TAG` bump, since the image is unchanged and a bump would make every box rebuild it to fix a mount. The matching half is in `classify_memory`, which now strips the home spellings (`./`, `~/`, `/root/`) before matching the root — without it `~/user-memory/x.md` missed the match, fell through to the disk router, and became exactly the invisible physical file the signpost exists to prevent.

**Skills are a read-only tree with two scopes, and the space *between* them is closed too.** `skills/shared/{id}` is the group's, `skills/{username}/{id}` is one member's own (`core-api`'s `SkillMounts`; agent path on the username like `projects/`, host path on the stable userid). Everything under `skills/` is read-only in **both** directions — `:ro` bind mounts and `can_write_to → false` — because these hold installed artefacts, not working files: a skill body is *read as instruction* by whoever it is visible to, so writing one is a decision that must pass a gate, not a file write — the one door is `skill_register` (with `skill_delete` and `list_items(type="skills")`), called from the chat and gated `require`; a public repo is fetched with `fetch_repo` and then registered. Two traps, both closed together and neither covering the other's half. **Host-side**, the `skills` arm of `can_write_to`/`host_base_and_tail` spans the **whole root**, not the two known scopes: the fallthrough answers `true`/home, so an invented scope segment (`skills/pippo/SKILL.md` — the *likely* guess, not the lucky one) would land in a physical directory under the home that no indexer ever reads. That is the memory-signpost failure exactly. **In-container**, the defect is structural rather than name-dependent: the scope mounts nest inside `container_home`, so `/root/skills` would be a real directory inside the *writable* home mount and `mkdir -p ~/skills/pippo` would succeed. Hence a third mount: `{WD}/.skills-root/{userid}` → `{container_home}/skills:ro`, holding the signpost README plus the two scope mountpoints. That root is **per-user and materialized whole** (`container::ensure_skills_root`) because Docker refuses to create a mountpoint inside a `:ro` mount — `shared/` and `{username}/` must already exist in the root's own source, and one of those names is the member's — which is also why the three host paths are one `SkillMounts` field rather than three `Option<PathBuf>`. `mounts()` emits them root-first; `skills_mounted` is the **fifth self-heal axis**, for the signposts' reason. A stale scope dir left by a rename is pruned at each `ensure`. The bare-id alias `skills/{id}` (the shortest spelling, so the one a model writes unprompted) resolves in `resolve_skill_alias` — **only** when the id is unique across the two trees, failing loudly with both full paths otherwise, since a personal skill silently shadowing a group one is a divergence nobody chose. `UserFs` stays pure: it returns `RouteError::SkillAlias` and skald-core does the probe.

**The skills index is generated, and the sentinel is the knob.** What reaches the model is not a file anyone maintains but a **function of the two trees** (`crates/skald-core/src/skills/`, pure functions in the shape of `LlmCommandManager`): each skill's `SKILL.md` **path** plus its frontmatter `description`, truncated to 200 chars, under an imperative header ("you MUST read its SKILL.md") — the countermeasure to the real failure mode, which is the model *under*-triggering. Printing the full path rather than an id plus a composition rule is what makes a read tool unnecessary: `read_file` on the printed path is one call, and there is no step left for the model to get wrong. Injection is the placeholder `<!-- SKILLS_LIST -->` (normally `<!-- INCLUDE: common/skills.md -->`, a fragment that holds **only** the sentinel), substituted in `AgentSystemContext::build_base` beside `__MCP_LIST__`; `resolve_includes` needs no branch, its generic `<!-- KEY -->` → `__KEY__` arm already covers it. There is **no `meta.json` flag** — the sentinel *is* the switch, so the four `type: system` agents opt out by not including the fragment (an imperative "read it with read_file" is exactly wrong in an unattended turn, and some of those run with `allow_tools: false`). All eleven `chat`/`task` agents carry the include, sub-agents included: in a delegation the one doing the work is the child. Three rendering rules are load-bearing and each closes a specific failure: a **stable order** (scope, then id) because the index sits inside the provider's cache key; a **deterministic tail cut** at an 8 KB budget, announced by a `[N more skills omitted]` line, because a silently truncated index has the model conclude in good faith that a skill does not exist; and **empty in, empty out** — every word of prose lives inside the render, so an instance with no skills spends zero tokens and leaves no orphan sentence (the MCP list is the counter-example: its prose sits *around* the placeholder, and the empty state once had the model inventing a discovery tool). A colliding id is marked `[name collision]` on **both** lines, never shadowed. A malformed skill is skipped with a `warn!`, never fatal — the index is built while assembling a prompt. Freshness has two doors, one per writer. The in-process tools invalidate directly (`Skald::invalidate_prompt_prefix`, called by `skill_register`/`skill_delete`); a hand edit on the box is caught by the **skills watcher** (`skills/watch.rs`, spawned from `spawn_background`): a recursive `notify` on `{WD}/skills` + `{WD}/skills-users`, debounced ~800 ms, that re-digests each touched tree (`skills::tree_digest` — the (id, description) pairs the index is made of) and emits `SystemEvent::SkillsChanged { scope }` only when the digest moved. The subscriber `spawn_skills_freshness` (next to `spawn_user_lifecycle`, same `Weak` shape) maps the scope and calls the same invalidate accessor. Editing a script leaves the digest byte-identical and announces nothing — which is exactly the §6 rule, so an invisible change costs nobody a cache miss. Two gotchas the code carries comments for: the watcher **canonicalizes `{WD}`** (FSEvents reports real paths, and `/var` is a symlink on macOS), and it creates the two trees if absent (a box before its first user has neither).

**The sandbox command list is a discovery hint, and the tool — not the sentinel — is the knob.** `container/commands.rs` probes the user's container at login (`UserContextFactory::build`, right after `ensure()`, **non-fatal**) with one `docker exec` running `command -v` over a curated ~35-entry `PROBE_ALLOWLIST`, and the result rides `LoopConfig.sandbox_commands` → `AgentSystemContext` → `__SANDBOX_COMMANDS__`. Three decisions carry it and each is the answer to an obvious-looking alternative. **The allowlist is the curation, and the probe is there so the list cannot lie** — not the other way round: a full `PATH` dump is 800 entries of coreutils noise, so what is worth tokens is decided by hand, and `command -v` exists only so we never announce something a container recreate threw away. A tool outside the list therefore never appears, which is fine because **the rendered prose says the list is partial and names `command -v`** — an inventory the model reads as exhaustive is the failure this shape avoids, the same one the skills index's `[N more skills omitted]` line closes. Order is the allowlist's own (grouped by kind of work), never sorted: the grouping *is* the curation, and the reader is a model, not a `grep`. **Staleness is cheap in both directions**, which is why there is no refresh machinery at all: a mid-session install is known to the agent that ran it, and a container recreate costs one `not found` plus the `apt-get install` the agent was already able to do. Gating is the one part that is not the skills pattern: every `AGENT.md` carries `<!-- INCLUDE: common/sandbox.md -->`, **including the four `type: system` ones**, and the section is emitted iff the turn's model is shown `execute_cmd` — computed from `allow_tools` plus the security group's visibility filter (`session/handler/config.rs`) for a root turn, and from `child_defs` for a sub-agent, i.e. always from *the same definitions the model will see*. Hence `has_execute_cmd` is in the `PrefixCache` key: the group is switchable mid-conversation from the chat's shield pill, and keying on it costs nothing because that switch already rewrites the tool payload sitting in the same provider cache. The fragment holds only the heading and one stable sentence; **every conditional claim lives in the renderer** (a departure from the `__MCP_LIST__` shape it otherwise follows), because prose promising `sudo apt-get install` is not the renderer's to retract when the tool is absent. Three rendered cases, and the middle one is why this is not a one-liner: the list, the *unreadable-probe* line (empty ≠ bare sandbox — rendering nothing under a heading that promises a list is how the MCP section once had a model invent a discovery tool), and the no-`execute_cmd` line. `execute_cmd`'s own description deliberately carries **no** capability advertisement — its `(python + node available)` was removed when this landed, since its job is steering the model *away* from the shell for work a file tool does better, and the two messages dilute each other.

**Containment** (`resolve_host_path`) is unchanged and still guards **the host branch**: every path that lands on a mount is canonicalized (following symlinks) and prefix-checked against its mount base, **fail-closed**. That check is what it always was — the defence against a symlink planted from inside the container pointing at the **host's** `/etc`, which the host-side tool would otherwise follow off the box. Opening the container branch does not weaken it: that branch never touches the host filesystem, so there is no host to escape from, and the check keeps applying to everything mounted. `grep_files` stays disk-only (regex ≠ FTS; memory → `memory_search`) but resolves its root the same way. `execute_cmd`'s `workdir` is an agent path mapped to its container path via `UserFs::to_container`.

The threading: `UserContext.fs` (built by `container::build_user_fs` at login, snapshotting shared memberships) → `ChatSessionManager` → `ChatSessionHandler.fs` → `ToolContext.fs`. **Admin CRUD is wired** (`src/frontend/api/shared_folders.rs` — `GET/POST /api/shared-folders`, `PATCH/DELETE /api/shared-folders/{id}`, `POST`/`DELETE .../members[/{user_id}]`; UI `shared-folders.js`): a create/describe/delete + per-member `can_write` surface, and each mutation emits `SystemEvent::UserMountsChanged`, on which the lifecycle reconciler runs `Skald::refresh_user_mounts` — rebuilding the affected user's fs + container mounts **in place**, so a membership change lands without a re-login (blueprint §6's "admin CRUD" + "membership refresh without re-login" TODOs, now closed; it still settles at next login/boot if the live remount fails). `execute_cmd` /stop is robust: the command runs under `setsid -w` in its own process-group (leader pid recorded in a container pidfile), and a `KillReaper` drop-guard reaps that group on /stop **or** timeout via a detached `docker exec` that walks `/proc` and kills members by **positive pid** (the container's dash mishandles `kill -<pgid>`); the pidfile is passed positionally (`$1`), and the container's `--init` (tini) reaps the killed processes so no zombies accumulate. **Per-user MCP connectors now run inside this container** (§7) — the container infra enabled it; see [mcp-connectors.md](mcp-connectors.md).


## `crates/skald-core/src/container/`

`ContainerManager` (§6): per-user Docker containers (the execution sandbox). Docker is a **hard requirement** — `check_docker()` fails `Skald::new` (→ shell exits) if the daemon is unreachable. Builds our own `skald-runtime` image (python+node+**sudo**, plus a shell-work toolbelt — `jq`/`ripgrep`/`unzip`/`ffmpeg`/`poppler-utils`/`tesseract`/`procps`…; tag is **versioned** `skald-runtime:v3` so a `Dockerfile` change forces a rebuild) once from the embedded `Dockerfile`, then `reconcile_all()` at boot ensures one running container `skald-{userid}` per active user. Each container runs as the **host `uid:gid`** (`--user`, §6 UID coherence) with `--init` (tini reaps zombies); `ensure()` **self-heals** a container that is stale on any of three axes — `--user` (e.g. an old root one), `--init`, or the **image tag** — by recreating it, and injects a passwd/shadow entry post-create so `sudo` (NOPASSWD, in the image) resolves the arbitrary uid. The image check is what makes a tag bump reach *existing* users: a container pins the image it was created from, so without it a rebuild would only ever equip new users. `build_user_fs()` assembles a user's `UserFs` (home `{WD}/homes/{userid}` → `/root`, plus each `shared/{name}` they belong to). Shells the `docker` CLI (no client crate)

## Built-in tools (`crates/skald-core/src/tools/`)

Built-in tools: `exec` (**runs inside the caller's per-user Docker container** via `docker exec`, as the non-root host uid — `sudo` for system installs — with a robust /stop that reaps the command's process-group; see `container/`; the only live path is `run_with` (needs `ToolContext`) — the context-free `Tool::execute`/`execute_async` now **error** (`HOST_PATH_ERROR`) instead of the old host `sh -c`, so nothing can run a command outside the sandbox), `list_agents`, `fs/*` (route `user-memory/`/`shared-memory/` to `memory_docs`, and every other **physical** path through `ctx.fs` to the caller's per-user host workspace — see DB tables + container), `notify`, `ast_outline`, `image_generate`, MCP tools, plugin tools, cron tools