Skald-Circle/dev-docs/context-and-compaction.md

Skald dev-docs — architectural reference for coding agents. Index: README.md · Entry point: ../CLAUDE.md

Read this when: you touch compaction, the history window, or the cached system prompt prefix.

---

Context window & compaction

agent_loop::compaction owns the mechanics: split point (never between an assistant turn and its tool results), transcript, prompt (SUMMARY_PREFIX / preamble / template live there now), the single no-tools model call, the saved summary row. skald-core/src/compactor.rs owns the policy: the token threshold, the ephemeral guard, which model summarises (compaction_model from Settings, else AUTO by compaction.strength), and publishing CompactionEvent on the chat bus. The DTL re-anchor is the on_compacted hook (loop_adapters/hooks.rs::DtlReanchorHook). The next turn needs nothing: the assembler reads the latest summary from the store.

Context size: both automatic guards are off by default

Nothing shrinks a conversation unless a human asks. llm.max_history_messages and llm.compaction.threshold_tokens are both Option, both unset in default.config.yaml, and the only remaining reducer is the user typing /compact. The reason is the prompt cache: every provider that caches (Anthropic breakpoints, OpenAI automatic prefix caching) keys on the longest common prefix, so anything that rewrites history mid-conversation costs a full miss on the next request.

The two guards are not equally bad at that, and the difference is why one is merely off and the other is close to a trap. max_history_messages is a sliding tail window (agent_loop::projection::window — drain(..len - max)): past the cap it drops from the head on every turn, so it is a cache miss per request, forever, and it drops messages with no summary standing in for them — silent amnesia. Compaction rewrites the prefix once per compaction and leaves a summary behind. So the previous default — window on, compaction off — was the worse of the two in both dimensions, and the window's own doc-comment already said the two were mutually exclusive.

Three consequences worth not re-deriving:

• The compactor is built unconditionally, in both bundles.rs and user_context.rs. It used to be Option<Arc<ContextCompactor>>, keyed on the config section existing — which meant that commenting out compaction: also silently disabled manual /compact (force_compact returned Ok(false) and the chat answered "compaction disabled"). Manual compaction is a command a user types; it must not depend on an admin having filled in a token threshold. try_compact early-returns on threshold_tokens: None; force_compact deliberately does not consult it — the human is the trigger.
• The projection yields to the automatic pass, not to the compactor's existence: LoopConfig.auto_compaction_enabled (= ContextCompactor::auto_enabled()), so a configured message cap is not silently voided by the mere availability of /compact. Expressed as max_history_messages.filter(|_| !auto_compaction_enabled) in projection_cfg.rs.
• CompactionConfig's Default is hand-written, same trap as RoleAttrs: a derived one gives keep_recent: 0, which would compact away every recent message on any box omitting the section — now the shipped default.

The future automatic pass should trigger off the resolved model's own context window, not a hand-tuned threshold_tokens that has no idea which model is answering.

The system prefix is frozen per conversation

Same economics, other end of the request. AgentSystemContext::system_context is called once per round, and it reassembled base from disk and SQLite every time — so an agent writing user-memory/index.md in round 3 made round 4, seconds later and with the cache certainly warm, a full miss. Since base is the head of every provider's cache key, that is the most expensive string in the request to touch. loop_adapters/prefix_cache.rs::PrefixCache builds it once per (conversation, agent) — the agent is in the key because a sub-agent shares its parent's conversation but has a prompt of its own — and holds it on UserLoopRuntime, so it outlives the turn.

The refresh rule is the only one that is free: rebuild once the conversation has been idle longer than a provider's cache could survive (PREFIX_TTL, 20 min). The clock is therefore idle time of this conversation, not time since a file changed, and reading restarts it — every get is a request about to go out. The asymmetry that sets the constant: below a provider's window you pay misses that buy nothing, above it you only pay freshness.

Writes are deliberately not reacted to, and there is no bus variant for this. When the agent itself edits an injected file the content is already in the context — its tool call and result sit two messages downstream — so refreshing would repeat what the model just said. A write from elsewhere (the same user's Telegram session, a cron job, another member editing shared-memory/) is genuinely invisible until the TTL: that is the case where an immediate rebuild costs the most, since a conversation that would notice is by definition a warm one, and the cheaper freshness path already exists — the agent can read_file, and a tool result appends, which invalidates nothing. The injection header says so in words. Cross-user invalidation of a file write would need a SystemEventBus variant plus a subscriber per user (the writer lives in a different UserContext); it is future work, and this type's key is the seam for it. Note base is frozen whole: freezing the memory files while letting __USER_PROFILE__ move would invalidate just as much. The cost is that an AGENT.md edit lands at the next rebuild rather than the next round.

What is invalidated eagerly: the two generated lists, because a stale one makes the model deny a tool it has. The TTL is right for injected content the agent can re-read on demand and wrong for an inventory — a model that reads "no such connector" in the ## MCP servers table does not go looking, it answers the question. So Skald::invalidate_prompt_prefix (the skills door, called straight from skill_register/skill_delete) has two MCP siblings, both looping the all_live() they already had: refresh_global_mcp_access — the admin enabling or re-granting a global connector, where refreshing the access snapshot alone fixed what mcp.tools() offers while leaving the table describing the world before it — and refresh_connector_after_reinstall, where a reinstall's new llm_short_description reached the runtime but not the prompt. Order is load-bearing and opposite to the intuition: render_mcp_list renders the live runtime's in-RAM state, not the DB, so the invalidation goes last, after the snapshot refresh and after the servers restart — rebuild the prefix first and it is repopulated from the very descriptions being replaced, with nothing left to invalidate it again. In the reinstall that means waiting out a global dependency install that can take minutes; correct anyway, since those users were already reading a stale table and an early rebuild would only freeze the stale one in place. The price is a provider cache miss on the next turn of every open conversation of every live user — cross-user by nature, since one admin is changing something for other people, and there is no cheaper direct path the way there is for a user editing their own memory. It buys back the failure the skills doc-comment already describes word for word.

The compaction policy (crates/skald-core/src/compactor.rs)

Context compaction policy — when to compact and with which model; the mechanics are agent_loop::compaction. The compactor is always constructed (manual /compact must work with no config); compaction.threshold_tokens is Option and arms only the automatic pass, and is unset by default — see the context-size section above. Model for the summary call: the instance-wide Settings pick (compaction_model, a PropertyType::LlmModel config property declared by compactor::config_set) wins; else AUTO by compaction.strength (config.yml); a missing configured model degrades to the same AUTO path