Skald-Circle/dev-docs/filesystem-and-containers.md

Skald dev-docs — architectural reference for coding agents. Index: README.md · Entry point: ../CLAUDE.md

Read this when: you touch container/, the fs-tools, mounts, skills, or anything about where a path physically lives.

---

Filesystem & containers (blueprint §6)

Each user has one permanent Docker container (skald-{userid}, our own skald-runtime image with python+node and a preinstalled shell toolbelt), created on user creation and started at boot (ContainerManager, crates/skald-core/src/container/). Docker is required: a missing daemon fails Skald::new and the process exits. What goes in the image vs. what the agent installs on demand is a real trade, and the Dockerfile states its rule: sudo apt-get install works in the sandbox but re-runs on every container recreate, inside a task, where it costs latency and can fail — while the image is one, shared by every container, so preinstalling costs its size once for the whole box. Anything an agent reaches for repeatedly is therefore baked in; build-essential/python3-dev and pandoc are deliberately left out as big and self-recoverable. The container runs as the host uid:gid (not root) so files created in-container and by the host-side fs-tools share ownership on the bind mounts (matters on native Linux; masked on macOS Docker Desktop). Because that user isn't root, the image ships passwordless sudo (a passwd/shadow entry is injected at create) so an agent can still sudo apt-get install …; --init runs tini as pid 1 to reap zombies.

The agent sees one namespace, routed on the first path component. The choke point is UserFs (core-api/src/user_fs.rs, a pure value type carried in ToolContext.fs), plus resolve_host_path() in tools/fs/mod.rs:

Agent path	Backing	Routed by
user-memory/…	SQLite ctx.pool ({userid}.db)	classify_memory → memory_docs
shared-memory/…	SQLite system.db	classify_memory → memory_docs
shared/{X}/…	host {WD}/shared/{X} (if a member)	UserFs::host_base_and_tail
projects/{O}/{S}/…	host {WD}/projects/{owner_userid}/{S} (if a member)	UserFs::host_base_and_tail
skills/shared/{id}/…	host {WD}/skills/{id} — read-only	UserFs::host_base_and_tail
skills/{username}/{id}/…	host {WD}/skills-users/{userid}/{id} — read-only	UserFs::host_base_and_tail
~/…, relative	host {WD}/homes/{userid}	UserFs::host_base_and_tail
any other absolute path (/tmp/…, /etc/…)	the container's own filesystem	resolve_target → container::exec_fs

Two views, one storage: for the mounted subtree the fs-tools run host-side in the Skald process on {WD}/homes/{userid} + {WD}/shared/{X}; execute_cmd runs inside the container (docker exec -w <container-path> skald-{userid} sh -c …, via ExecuteCmd::run_with) on the same paths bind-mounted (homes/{userid}→/root, shared/{X}→/root/shared/{X}, read-only when can_write=0). A file written in the container appears to the host fs-tools and vice versa.

The security boundary is the container, not the mounted subtree — the mount is the fast path, not the only one. An agent already reaches every corner of its container through execute_cmd, which runs there with passwordless sudo; fs-tools that stopped at the mounts were not protecting anything, they were offering a poorer view of the same sandbox, and the model answered that by shelling out (the observed failure: read_file /tmp/cv.txt → "path escapes your workspace" → the agent re-read it with cat). So resolve_target routes a physical path to one of two backings. An absolute path is container vocabulary — it is what execute_cmd prints — so it is reverse-mapped through UserFs::container_to_agent first: landing on a mount takes the host path (/root/x is ~/x, which the tools used to reject outright, since PathBuf::join with an absolute tail silently discards the base and the result then failed the prefix check); landing nowhere means it exists only in the container, and container::exec_fs acts there over docker exec (paths passed positionally as $1, so a path containing $(…) is data, not syntax). Membership is not bypassed: /root/shared/{X} for a non-member still resolves to the same error as shared/{X}.

One implementation per tool, not two. Every single-file fs-tool already funnels through the same shape — resolve, then run a sync execute over one absolute host path — so the container branch is a shuttle (fs::Shuttle, behind fs::run_physical): pull the file out of the container, run the unchanged tool on the copy, push it back if the content changed (compared by bytes, not mtime, whose one-second resolution would miss a fast edit). Nothing about a tool's messages, diffs or pure transforms is duplicated. A missing remote file is deliberately not pre-created — write_file reports "Created" vs "Overwrote" from whether the path existed, and a placeholder would make every creation lie. Three tools opt out of the shuttle because a single file is the wrong unit: list_files lists in place via exec_fs::list (find -printf; line_count is omitted, since counting lines would turn a listing into a docker exec per file), read_file reads container paths as text (a shuttled copy is gone by the time the projection would inline a MediaRef, so media stays a mount-only feature), and grep_files refuses container paths with a pointer to execute_cmd + rg — its regex flavour, glob, windowing and offset would all have to be re-derived from ripgrep's flags, and a grep that answers almost the same is worse than one that says where to go. The viewer follows the same routing through resolve_view_target (GET /api/file and show_file_to_user open container paths; served without an ETag, so the editor stays read-only there). One read-only shuttle lives here too: GET /api/file?compile-docx=true on a container-only word document pulls the bytes out with exec_fs::read and converts the host-side copy via DocxConverter::convert_bytes — correct because the format is self-contained. LaTeX deliberately gets no such branch: a shuttled .tex would silently lose its relative \input/\includegraphics dependencies.

The memory roots are signposted inside the container, not merely absent. user-memory//shared-memory/ are virtual, so nothing of them existed on disk — and the nothing was worse than it sounds: cat user-memory/x.md returned a bare ENOENT (which reads as the note is missing, not wrong door), while mkdir -p user-memory && echo … > user-memory/x.md succeeded, writing a real file into the home that no reader ever visits and that the next ls then confirms as if it had worked. Each root is therefore a read-only bind mount ({WD}/.memory-signpost/{root} → {container_home}/{root}:ro, gitignored, rewritten from consts on every ensure) holding a README that names the tools. Read-only as a mount, not as a mode: the container user has passwordless sudo, so a chmod would be a suggestion, whereas :ro holds — remounting needs CAP_SYS_ADMIN (verified: write, sudo write, sudo chmod, sudo mount -o remount,rw and sudo rm all fail). A README rather than an empty dir because Permission denied is an error, not an instruction — models answer it by reaching for sudo; the README puts the correction in the directory the failing command just named. These mounts are deliberately not in UserFs: they back no agent path and the host-side fs-tools must never resolve into them. They are the fourth self-heal axis in reusable() (signposts_mounted) rather than an IMAGE_TAG bump, since the image is unchanged and a bump would make every box rebuild it to fix a mount. The matching half is in classify_memory, which now strips the home spellings (./, ~/, /root/) before matching the root — without it ~/user-memory/x.md missed the match, fell through to the disk router, and became exactly the invisible physical file the signpost exists to prevent.

Skills are a read-only tree with two scopes, and the space between them is closed too. skills/shared/{id} is the group's, skills/{username}/{id} is one member's own (core-api's SkillMounts; agent path on the username like projects/, host path on the stable userid). Everything under skills/ is read-only in both directions — :ro bind mounts and can_write_to → false — because these hold installed artefacts, not working files: a skill body is read as instruction by whoever it is visible to, so writing one is a decision that must pass a gate, not a file write — the one door is skill_register (with skill_delete and list_items(type="skills")), called from the chat and gated require; a public repo is fetched with fetch_repo and then registered. Two traps, both closed together and neither covering the other's half. Host-side, the skills arm of can_write_to/host_base_and_tail spans the whole root, not the two known scopes: the fallthrough answers true/home, so an invented scope segment (skills/pippo/SKILL.md — the likely guess, not the lucky one) would land in a physical directory under the home that no indexer ever reads. That is the memory-signpost failure exactly. In-container, the defect is structural rather than name-dependent: the scope mounts nest inside container_home, so /root/skills would be a real directory inside the writable home mount and mkdir -p ~/skills/pippo would succeed. Hence a third mount: {WD}/.skills-root/{userid} → {container_home}/skills:ro, holding the signpost README plus the two scope mountpoints. That root is per-user and materialized whole (container::ensure_skills_root) because Docker refuses to create a mountpoint inside a :ro mount — shared/ and {username}/ must already exist in the root's own source, and one of those names is the member's — which is also why the three host paths are one SkillMounts field rather than three Option<PathBuf>. mounts() emits them root-first; skills_mounted is the fifth self-heal axis, for the signposts' reason. A stale scope dir left by a rename is pruned at each ensure. The bare-id alias skills/{id} (the shortest spelling, so the one a model writes unprompted) resolves in resolve_skill_alias — only when the id is unique across the two trees, failing loudly with both full paths otherwise, since a personal skill silently shadowing a group one is a divergence nobody chose. UserFs stays pure: it returns RouteError::SkillAlias and skald-core does the probe.

The skills index is generated, and the sentinel is the knob. What reaches the model is not a file anyone maintains but a function of the two trees (crates/skald-core/src/skills/, pure functions in the shape of LlmCommandManager): each skill's SKILL.md path plus its frontmatter description, truncated to 200 chars, under an imperative header ("you MUST read its SKILL.md") — the countermeasure to the real failure mode, which is the model under-triggering. Printing the full path rather than an id plus a composition rule is what makes a read tool unnecessary: read_file on the printed path is one call, and there is no step left for the model to get wrong. Injection is the placeholder <!-- SKILLS_LIST --> (normally <!-- INCLUDE: common/skills.md -->, a fragment that holds only the sentinel), substituted in AgentSystemContext::build_base beside __MCP_LIST__; resolve_includes needs no branch, its generic <!-- KEY --> → __KEY__ arm already covers it. There is no meta.json flag — the sentinel is the switch, so the four type: system agents opt out by not including the fragment (an imperative "read it with read_file" is exactly wrong in an unattended turn, and some of those run with allow_tools: false). All eleven chat/task agents carry the include, sub-agents included: in a delegation the one doing the work is the child. Three rendering rules are load-bearing and each closes a specific failure: a stable order (scope, then id) because the index sits inside the provider's cache key; a deterministic tail cut at an 8 KB budget, announced by a [N more skills omitted] line, because a silently truncated index has the model conclude in good faith that a skill does not exist; and empty in, empty out — every word of prose lives inside the render, so an instance with no skills spends zero tokens and leaves no orphan sentence (the MCP list is the counter-example: its prose sits around the placeholder, and the empty state once had the model inventing a discovery tool). A colliding id is marked [name collision] on both lines, never shadowed. A malformed skill is skipped with a warn!, never fatal — the index is built while assembling a prompt. Freshness has two doors, one per writer. The in-process tools invalidate directly (Skald::invalidate_prompt_prefix, called by skill_register/skill_delete); a hand edit on the box is caught by the skills watcher (skills/watch.rs, spawned from spawn_background): a recursive notify on {WD}/skills + {WD}/skills-users, debounced ~800 ms, that re-digests each touched tree (skills::tree_digest — the (id, description) pairs the index is made of) and emits SystemEvent::SkillsChanged { scope } only when the digest moved. The subscriber spawn_skills_freshness (next to spawn_user_lifecycle, same Weak shape) maps the scope and calls the same invalidate accessor. Editing a script leaves the digest byte-identical and announces nothing — which is exactly the §6 rule, so an invisible change costs nobody a cache miss. Two gotchas the code carries comments for: the watcher canonicalizes {WD} (FSEvents reports real paths, and /var is a symlink on macOS), and it creates the two trees if absent (a box before its first user has neither).

The sandbox command list is a discovery hint, and the tool — not the sentinel — is the knob. container/commands.rs probes the user's container at login (UserContextFactory::build, right after ensure(), non-fatal) with one docker exec running command -v over a curated ~35-entry PROBE_ALLOWLIST, and the result rides LoopConfig.sandbox_commands → AgentSystemContext → __SANDBOX_COMMANDS__. Three decisions carry it and each is the answer to an obvious-looking alternative. The allowlist is the curation, and the probe is there so the list cannot lie — not the other way round: a full PATH dump is 800 entries of coreutils noise, so what is worth tokens is decided by hand, and command -v exists only so we never announce something a container recreate threw away. A tool outside the list therefore never appears, which is fine because the rendered prose says the list is partial and names command -v — an inventory the model reads as exhaustive is the failure this shape avoids, the same one the skills index's [N more skills omitted] line closes. Order is the allowlist's own (grouped by kind of work), never sorted: the grouping is the curation, and the reader is a model, not a grep. Staleness is cheap in both directions, which is why there is no refresh machinery at all: a mid-session install is known to the agent that ran it, and a container recreate costs one not found plus the apt-get install the agent was already able to do. Gating is the one part that is not the skills pattern: every AGENT.md carries <!-- INCLUDE: common/sandbox.md -->, including the four type: system ones, and the section is emitted iff the turn's model is shown execute_cmd — computed from allow_tools plus the security group's visibility filter (session/handler/config.rs) for a root turn, and from child_defs for a sub-agent, i.e. always from the same definitions the model will see. Hence has_execute_cmd is in the PrefixCache key: the group is switchable mid-conversation from the chat's shield pill, and keying on it costs nothing because that switch already rewrites the tool payload sitting in the same provider cache. The fragment holds only the heading and one stable sentence; every conditional claim lives in the renderer (a departure from the __MCP_LIST__ shape it otherwise follows), because prose promising sudo apt-get install is not the renderer's to retract when the tool is absent. Three rendered cases, and the middle one is why this is not a one-liner: the list, the unreadable-probe line (empty ≠ bare sandbox — rendering nothing under a heading that promises a list is how the MCP section once had a model invent a discovery tool), and the no-execute_cmd line. execute_cmd's own description deliberately carries no capability advertisement — its (python + node available) was removed when this landed, since its job is steering the model away from the shell for work a file tool does better, and the two messages dilute each other.

Containment (resolve_host_path) is unchanged and still guards the host branch: every path that lands on a mount is canonicalized (following symlinks) and prefix-checked against its mount base, fail-closed. That check is what it always was — the defence against a symlink planted from inside the container pointing at the host's /etc, which the host-side tool would otherwise follow off the box. Opening the container branch does not weaken it: that branch never touches the host filesystem, so there is no host to escape from, and the check keeps applying to everything mounted. grep_files stays disk-only (regex ≠ FTS; memory → memory_search) but resolves its root the same way. execute_cmd's workdir is an agent path mapped to its container path via UserFs::to_container.

The threading: UserContext.fs (built by container::build_user_fs at login, snapshotting shared memberships) → ChatSessionManager → ChatSessionHandler.fs → ToolContext.fs. Admin CRUD is wired (src/frontend/api/shared_folders.rs — GET/POST /api/shared-folders, PATCH/DELETE /api/shared-folders/{id}, POST/DELETE .../members[/{user_id}]; UI shared-folders.js): a create/describe/delete + per-member can_write surface, and each mutation emits SystemEvent::UserMountsChanged, on which the lifecycle reconciler runs Skald::refresh_user_mounts — rebuilding the affected user's fs + container mounts in place, so a membership change lands without a re-login (blueprint §6's "admin CRUD" + "membership refresh without re-login" TODOs, now closed; it still settles at next login/boot if the live remount fails). execute_cmd /stop is robust: the command runs under setsid -w in its own process-group (leader pid recorded in a container pidfile), and a KillReaper drop-guard reaps that group on /stop or timeout via a detached docker exec that walks /proc and kills members by positive pid (the container's dash mishandles kill -<pgid>); the pidfile is passed positionally ($1), and the container's --init (tini) reaps the killed processes so no zombies accumulate. Per-user MCP connectors now run inside this container (§7) — the container infra enabled it; see mcp-connectors.md.

crates/skald-core/src/container/

ContainerManager (§6): per-user Docker containers (the execution sandbox). Docker is a hard requirement — check_docker() fails Skald::new (→ shell exits) if the daemon is unreachable. Builds our own skald-runtime image (python+node+sudo, plus a shell-work toolbelt — jq/ripgrep/unzip/ffmpeg/poppler-utils/tesseract/procps…; tag is versioned skald-runtime:v3 so a Dockerfile change forces a rebuild) once from the embedded Dockerfile, then reconcile_all() at boot ensures one running container skald-{userid} per active user. Each container runs as the host uid:gid (--user, §6 UID coherence) with --init (tini reaps zombies); ensure() self-heals a container that is stale on any of three axes — --user (e.g. an old root one), --init, or the image tag — by recreating it, and injects a passwd/shadow entry post-create so sudo (NOPASSWD, in the image) resolves the arbitrary uid. The image check is what makes a tag bump reach existing users: a container pins the image it was created from, so without it a rebuild would only ever equip new users. build_user_fs() assembles a user's UserFs (home {WD}/homes/{userid} → /root, plus each shared/{name} they belong to). Shells the docker CLI (no client crate)

Built-in tools (crates/skald-core/src/tools/)

Built-in tools: exec (runs inside the caller's per-user Docker container via docker exec, as the non-root host uid — sudo for system installs — with a robust /stop that reaps the command's process-group; see container/; the only live path is run_with (needs ToolContext) — the context-free Tool::execute/execute_async now error (HOST_PATH_ERROR) instead of the old host sh -c, so nothing can run a command outside the sandbox), list_agents, fs/* (route user-memory//shared-memory/ to memory_docs, and every other physical path through ctx.fs to the caller's per-user host workspace — see DB tables + container), notify, ast_outline, image_generate, MCP tools, plugin tools, cron tools