Skald-Circle/dev-docs/system-agents.md

Skald dev-docs — architectural reference for coding agents. Index: README.md · Entry point: ../CLAUDE.md

Read this when: you touch background agents — event triage, the memory lints, the conversation review — or their scheduler.

---

System agents (event triage, memory lints)

A system agent runs on a user's behalf without being asked. There are three — event triage (the background event processor) and the two memory lints — behind one scheduler, and the machinery is deliberately shaped so a fourth is a trait impl plus one line in a registry.

The unit of work is one agent for one user, and every part of the design falls out of that. the triage agent's events (mcp_events) are in the caller's own encrypted database, pushed there by connectors in the caller's container; the notification goes to the caller's hub; the trace (system_agent_runs) is in that same file. So an agent owns no timer and no user list: it implements SystemAgent (crates/skald-core/src/system_agents/) — has_work + run over an AgentRunCtx unpacked from that user's UserContext — and skald::wiring::spawn_system_agents decides who and when. Building it against the ownerless Conversation bundle was exactly what made the pre-multi-user version inert: it wrote sessions into system.db, notified a hub with no subscribers, and resolved tool paths against a container that does not exist.

One loop for cadences three orders of magnitude apart. Event triage runs every few minutes, a lint weekly — the case that tempts a second loop. It stays one because the wake-up decides nothing: base_tick (min enabled interval, clamped to [60s, 15min]) only picks how often to look, and whether an agent runs for a given user is system_agents::is_due against persisted state. A second scheduler would be a fourth global bus in disguise.

Due-ness is persisted, not counted from boot — the new owner table system_agent_state(agent_id, last_attempt_at) (accessor db/system_agent_state.rs). It is deliberately not system_agent_runs: the run log is a history for the human and skips idle ticks, while scheduling needs every attempt, so reading due-ness off the log would re-run an idle agent every tick and never bring a weekly one due once its last productive run aged out. Persisting it is also what makes a long interval survive a restart — an in-memory deadline is fine at event triage's scale but a weekly agent on a box rebooted every few days would have it re-armed before it ever fired, and would simply never run. Side benefit: a user who logs in after a long absence is picked up on the next pass.

run_and_record orders the three steps, once, for everybody: mark the attempt (always, even for an idle pass) → has_work (false writes nothing at all, or the run log becomes a heartbeat) → open the run row, then work. The start/finish split (unlike job_runs, written once at the end) leaves a visible running row when the process dies mid-pass, swept to failed by the next start for that agent — safe precisely because the scheduler is sequential and single-instance, at both levels (agents in order, then users in order).

AgentScope::PerSubject is the scope where "whose data" and "whose runtime" come apart — the conversation review (system_agents/conversation_review.rs, wiring subject_pass) is the first and the reason it exists. The pass reads the subject's database and runs inside a supervisor's runtime, so everything it leaves behind (ephemeral session, run row) lands in the watcher's file and nothing in the watched one's; the report crosses between them via system.db. Three things fall out and each is load-bearing: (a) iteration is over subjects, not supervisors — two parents watching one child must yield one review, so whichever of them is unlocked lends a runtime and the report is filed against the subject; (b) is_due is not consulted — it keys state by agent within one file, which would collapse every subject sharing a supervisor into one clock, so due-ness lives in system_agent_coverage and is answered inside has_work (and run_and_record skips mark_attempt for this scope for the same reason); (c) the subject need not be logged in, via the new UserManager::open_unencrypted — for a user with no key the password guards the session, not the data, so this makes that explicit in one place and refuses an encrypted user, not as policy but because there is no key to be had. The rule that falls out is neutral by construction and worth quoting: work over somebody else's history runs unattended for a user who is not encrypted, and only while they are logged in for one who is. The returned pool is deliberately not registered as unlocked (that map is what "logged in" means to everything else). Authorization is the caller's: subject_pass is behind the supervision edge, never a role check.

meta.json: "allow_tools": false empties the turn's tool set (AgentMeta::allow_tools → loop_adapters/runtime.rs::turn_params swaps in an empty ToolRegistry): built-ins, MCP, plugin and interface tools alike, notify included. Distinct from a restrictive security group — a group decides whether a call is allowed, this decides whether the model is shown anything to call. For an agent whose input is other people's text, that is also the prompt-injection answer: the round an injected instruction would act in has no tools in it. The conversation review declares it, and consequently produces its report as the turn's final assistant message (read back with chat_history::last_assistant_for_session, parsed shallowly by parse_report: leading # heading → title, opening paragraph → summary, NOTHING_TO_REPORT sentinel → no row) rather than through a save_report tool, which would have needed whitelisting past the approval gate that an unattended pass auto-denies. The cost is that severity cannot come from the model; every report it files is notice.

Per-pass prompt substitutions. run_ephemeral_turn takes a system_substitutions map. The two the system context resolves by itself (__USER_PROFILE__, __SHARED_FOLDERS__) describe the session owner, which for a pass about somebody else is the wrong person — so the review passes the subject's profile under its own <!-- SUBJECT_PROFILE --> key (rendered by the shared loop_adapters::system::render_user_profile_section). It goes in the system prompt rather than the trigger message because age, name and sex change what counts as worth reporting, and the model needs them before it reads a word of the transcript.

A locked user is skipped, and that is the normal case, not an error. The pool is the unlock token (§9): a user who has not logged in since the last restart has no readable events, no session store — and no place to record the skip, since the only file that could hold it is the one we cannot open. Hence system_agent_runs has no skipped status: the skip is an INFO log line and nothing else.

AgentScope::Instance is the ownerless-work escape hatch, and there is exactly one user of it. The shared memory store belongs to nobody, but a pass over it still has to run somewhere: an ownerless run would write its trace into system.db, which GET /api/system-agents/runs shows to nobody (scoped on the caller's own pool, by design), and its notify() would have no recipient. So instance_pass runs it as the first active unlocked admin (users::list order, so the choice is stable across passes), and the whole per-user surface keeps working unchanged. Cost: it needs an admin who has logged in since the restart.

The run log is theirs, not the admin's (db/system_agent_runs.rs, owner table, no user_id column — the file is the owner). GET /api/system-agents/runs is scoped through require_context with no admin override: everyone, admin included, sees their own runs. stats is a JSON blob of the agent's own counters, never contents.

The configured security group is not applied verbatim. <agent>.security_group is an instance-wide admin setting; handing it to a restricted member's run would give their background agent a tool set their role never granted. system_agents::configured_run_context puts it through run_context::reconcile_group_for_user — the same seam a persisted group takes — degrading to the role default when the role disallows it. With nothing configured the run still starts from role_default_run_context, never None, because None means the catch-all group, which is wider.

The conversation review

system_agents/conversation_review.rs — nightly, one report per supervised subject, covering every conversation in the window rather than one report per session (the useful signal is often across conversations). The window is [covered_through, now) and due-ness is "the watermark stops before the most recent occurrence of run_at_hour local" (default 4am), which is also why downtime needs no catch-up mechanism: a machine off for three days finds a three-day-old watermark and covers it in one pass. most_recent_occurrence is generic over the timezone so it is testable without depending on where the box is, and resolves through the timezone (not UTC arithmetic) so a DST-skipped hour is handled.

chat_history::conversation_window is the transcript query, and its four filters each exist because of a specific way the result would otherwise be wrong: is_ephemeral = 0 (or a pass reads the transcript its previous pass was given and reports on itself), depth = 0 (sub-agent frames are machine-to-machine), is_synthetic = 0 (machinery-injected turns are not things the person said), content <> '' (an assistant row that was only a tool call). Tool calls are absent by construction, not by filter — they live in chat_llm_tools — so the review sees what was said, never what was done, and the prompt says so plainly because a model shown a gap narrates over it. Rendering is prose grouped by conversation, never JSON: a dialogue read as a dialogue is what models are best at, and nothing machine-readable comes back this way — the structured artefact is the report at the other end.

The memory lints

system_agents/memory_lint.rs — one struct, two instances differing only by fields: MemoryLintAgent::private (PerUser, over user-memory/ in the caller's pool) and ::shared (Instance, over shared-memory/ in the system pool — the same routing classify_memory gives the fs-tools). Prompts are two AGENT.mds sharing agents/common/memory-lint.md; the shared one additionally hunts table-rule violations and is told to report which note and what kind of problem without repeating the sensitive line, since restating it is the harm being flagged.

Read-only, enforced twice. The prompt says report-never-repair, and shared-memory/* writes are already @fs_write require — so an agent that tried to fix something would raise an approval card from an unattended pass, which run_ephemeral_turn auto-denies. Read-only is not a convention here, it is the only thing that works. has_work is "the store is non-empty", so a member who never uses memory collects no weekly row and no weekly notification.

Interval units are per-agent: event triage in minutes, the lints in days (interval_from_config takes the unit). Asking an admin to type 10080 for "weekly" would be a worse version of the same field.

The cadence is per user for exactly one agent, and the trait says so in two methods, not one. Event triage fires on inbound events, so how often it has work is a property of the person — someone on a dozen mailing lists triggers it on nearly every tick from the same setting that leaves a quiet account idle for a day. So SystemAgent gained interval_secs_for(user_id) (what is_due measures against) beside the instance-wide interval_secs, both defaulting to the latter so every other agent implements nothing. The second method is the non-obvious half: base_tick sleeps for the shortest interval any enabled agent asks for, so an agent whose overrides can go below its instance value must also implement shortest_interval_secs — without it the wake-up never comes round often enough and the override works when it lengthens and silently does nothing when it shortens. Storage is the registry table system_agent_user_settings(agent_id, user_id, interval_secs) (accessor + interval_for_user/shortest_interval_for helpers in system_agents/mod.rs, both failing open onto the instance value): a row is an override, its absence is inheritance — no sentinel value, no row written at user creation, and clearing the field deletes the row. Registry rather than the user's own user_config for a reason that is not about scope: the writer is the admin, on #users/{id}, and a member's file is unreadable unless they happen to be logged in (§9) — a setting that could only be changed while its subject has a live session would not be a setting. Endpoints GET/PUT /api/users/{id}/event-triage (admin-gated, minutes on the wire, null = inherit), rendered as one section on that person's page next to the grants. Nothing rides the bus: the scheduler re-reads the interval every tick and due-ness is measured from the user's own last attempt, so a change lands on the next wake-up with no push and no subscriber — the ConfigKeyUpdated reschedule stays for the instance key only. Keyed by agent_id though only one agent uses it, because the alternative is a column per agent on users and "a fourth agent is a trait impl plus one registry line" would stop being true the moment its schedule needed a schema change.

Where the settings live

ConfigSet gained owner: Option<String> (core-api): None renders on the general Config page, Some(agent_id) is claimed by the surface that owns it. Placement is data on the set, not a filter that knows set names, so a new owned set lands in the right place without touching either page. system_agents::registry() and ::config_sets() are the single enumeration of the agents — registry_and_config_sets_agree is the test that stops the scheduler's list and the settings surface from drifting.

/api/config serves only owner-less sets and is now admin-gated (caps::require_admin), read and write: before this, both handlers ignored the caller entirely, so any authenticated session could read and change instance config — the sidebar hiding the page is presentation, not authorization. GET /api/system-agents lists the agents, with config resolved (via the shared config::render_sets) only for an admin and Value::Null for everyone else; writes still go through PUT /api/config/{key}, so the gate and the known-key check exist in one place.

UI: #system-agents (web/components/system-agents.js, sidebar group extensions, visible to everyone — the run log is the caller's own). One tab per agent, plus "All", each tab holding that agent's description, its settings (admin only) and its runs — the tab is the agent, not the kind of information, because "why did this do nothing last night?" is half a schedule question and half a log question. The settings form is web/components/shared/config-form.js (ConfigFormController), shared with config-page.js so an owned set renders identically wherever it is edited. It replaced a since-removed debug page (#tic, from when the triage agent was called TIC), which listed chat_sessions WHERE source='tic' and so inferred runs from leftover ephemeral sessions rather than recording them.