fix: stop shrinking conversations behind the user's back — both automatic context guards ship off
Nightly Build / build (push) Successful in 7m35s
Nightly Build / build (push) Successful in 7m35s
The shipped default combined a sliding history window with no compaction, which is the worse of the two available trades in both directions it is measured on. `max_history_messages: 30` is a sliding tail window (`projection::window` — `drain(..len - max)`). Past 30 messages it drops from the head on *every* turn, so the prompt prefix changes on every single request and every provider that caches one (Anthropic breakpoints, OpenAI automatic prefix caching) misses every time. It also drops those messages with no summary standing in for them: silent amnesia, not just a cold cache. Compaction rewrites the prefix once per compaction and leaves a summary behind — yet it was the half that was commented out, while the window's own doc-comment already said the two were exclusive. Both are now `Option` and both ship unset, so nothing shrinks a conversation unless a human types `/compact`. Which surfaced the real bug: `/compact` did not work either. The compactor was `Option<Arc<ContextCompactor>>` keyed on the config section existing, so commenting out `compaction:` disabled the manual command too — `force_compact` returned `Ok(false)` and the chat answered "compaction disabled". Manual compaction is a command a user types; it cannot depend on an admin having filled in a token threshold. The compactor is now built unconditionally and `threshold_tokens: Option<u32>` arms only the automatic pass; `try_compact` early-returns without it, `force_compact` deliberately never consults it. The projection accordingly yields to the *automatic* pass rather than to the compactor's existence (`LoopConfig.auto_compaction_enabled`), so a configured message cap is not silently voided by `/compact` merely being available. `CompactionConfig::Default` is hand-written for the same reason `RoleAttrs`'s is: a derived one gives `keep_recent: 0`, which would compact away every recent message on any box omitting the section — now the default. Also fixes two documentation bugs in the same file: `event_triage` was documented nested under `llm:`, where it parses fine and is then silently ignored (it is a top-level field), and `datetime` was documented twice with conflicting examples. A new test asserts the shipped default actually deserializes and that both guards are off — a field the default omits must be genuinely optional, or a brand-new install fails to boot. Automatic compaction returns later, triggered off the resolved model's own context window instead of a hand-tuned token count that cannot know which model is answering.
This commit is contained in:
@@ -345,7 +345,7 @@ async fn handle_compact(bot: &Bot, chat_id: ChatId, hub: &Arc<dyn core_api::chat
|
||||
bot.send_message(chat_id, "✅ Context compacted.").await.ok();
|
||||
}
|
||||
Ok(false) => {
|
||||
bot.send_message(chat_id, "⏩ Compaction skipped (no messages to summarise or compaction disabled).").await.ok();
|
||||
bot.send_message(chat_id, "⏩ Compaction skipped (nothing to summarise).").await.ok();
|
||||
}
|
||||
Err(e) => {
|
||||
error!(error = %e, "telegram: manual compaction failed");
|
||||
|
||||
@@ -102,6 +102,13 @@ impl ContextCompactor {
|
||||
Self { config, llm_manager, event_bus, config_store }
|
||||
}
|
||||
|
||||
/// Whether the **automatic** trigger is armed. The compactor always exists
|
||||
/// (manual `/compact` needs no config), so this — not its presence — is what
|
||||
/// tells the projection that a summary bounds the context.
|
||||
pub fn auto_enabled(&self) -> bool {
|
||||
self.config.threshold_tokens.is_some()
|
||||
}
|
||||
|
||||
/// Attempt to compact the conversation history for `stack_id`.
|
||||
///
|
||||
/// * `last_input_tokens` — input tokens from the **previous** turn.
|
||||
@@ -123,10 +130,16 @@ impl ContextCompactor {
|
||||
if is_ephemeral {
|
||||
return Ok(false);
|
||||
}
|
||||
// No threshold configured ⇒ automatic compaction is off and history stays
|
||||
// append-only. `force_compact` deliberately does not consult this: the
|
||||
// human asking for `/compact` *is* the trigger.
|
||||
let Some(threshold) = self.config.threshold_tokens else {
|
||||
return Ok(false);
|
||||
};
|
||||
|
||||
// A provider that reported no usage leaves only the character estimate.
|
||||
let estimated = chat_history::estimate_tokens_for_stack(pool, stack_id).await?;
|
||||
if !should_compact(Some(last_input_tokens), estimated, self.config.threshold_tokens) {
|
||||
if !should_compact(Some(last_input_tokens), estimated, threshold) {
|
||||
return Ok(false);
|
||||
}
|
||||
let effective_tokens = if last_input_tokens > 0 { last_input_tokens } else { estimated };
|
||||
@@ -134,7 +147,7 @@ impl ContextCompactor {
|
||||
info!(
|
||||
stack_id,
|
||||
effective_tokens,
|
||||
threshold = self.config.threshold_tokens,
|
||||
threshold,
|
||||
"compactor: threshold exceeded, starting compaction"
|
||||
);
|
||||
|
||||
|
||||
@@ -7,7 +7,14 @@ pub use core_api::provider::LlmStrength;
|
||||
/// LLM runtime settings (clients are managed via LlmManager / DB, not here).
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct LlmConfig {
|
||||
pub max_history_messages: usize,
|
||||
/// Hard cap on the number of history messages projected into the context,
|
||||
/// applied as a **sliding tail window**. Omit (the default) to disable it:
|
||||
/// once history exceeds the cap, every turn shifts the window's start, which
|
||||
/// changes the prompt prefix and costs a full prompt-cache miss on every
|
||||
/// single request — while dropping the oldest messages with no summary to
|
||||
/// stand in for them. Set it only when a hard message bound is worth both.
|
||||
#[serde(default)]
|
||||
pub max_history_messages: Option<usize>,
|
||||
pub max_tool_rounds: Option<usize>,
|
||||
/// Maximum number of synchronous sub-agents run concurrently when the LLM emits
|
||||
/// a homogeneous batch of sub-agent calls in one response. Omit to use the
|
||||
@@ -21,8 +28,11 @@ pub struct LlmConfig {
|
||||
pub max_tool_result_chars: Option<usize>,
|
||||
/// Request/response logging configuration. Omit or set `enabled: false` to disable.
|
||||
pub requests_log: Option<LlmRequestsLogConfig>,
|
||||
/// Context compaction settings. Omit to disable automatic compaction.
|
||||
pub compaction: Option<CompactionConfig>,
|
||||
/// Context compaction settings. Omitting the section leaves manual `/compact`
|
||||
/// working on defaults — only the automatic trigger is opt-in, see
|
||||
/// [`CompactionConfig::threshold_tokens`].
|
||||
#[serde(default)]
|
||||
pub compaction: CompactionConfig,
|
||||
/// Controls how the current date/time is injected into each LLM request.
|
||||
#[serde(default)]
|
||||
pub datetime: DatetimeConfig,
|
||||
@@ -48,12 +58,21 @@ impl Default for DatetimeConfig {
|
||||
}
|
||||
}
|
||||
|
||||
/// Context compaction: summarises conversation history when the LLM context
|
||||
/// exceeds `threshold_tokens`.
|
||||
/// Context compaction: summarises conversation history so the context stops
|
||||
/// growing.
|
||||
///
|
||||
/// The compactor is **always built** — `/compact` is a manual command and must
|
||||
/// work out of the box. This struct only tunes it, and `threshold_tokens` is
|
||||
/// the one switch that arms the *automatic* trigger.
|
||||
#[derive(Debug, Clone, Deserialize)]
|
||||
pub struct CompactionConfig {
|
||||
/// Trigger compaction when the previous turn consumed more than this many input tokens.
|
||||
pub threshold_tokens: u32,
|
||||
/// Trigger compaction when the previous turn consumed more than this many
|
||||
/// input tokens. Omit (the default) to leave automatic compaction **off**:
|
||||
/// history is then append-only, which is what keeps the prompt prefix — and
|
||||
/// so the provider's prompt cache — stable across a whole conversation.
|
||||
/// Manual `/compact` is unaffected either way.
|
||||
#[serde(default)]
|
||||
pub threshold_tokens: Option<u32>,
|
||||
/// Number of recent messages to keep outside the summary. Defaults to 6.
|
||||
#[serde(default = "default_keep_recent")]
|
||||
pub keep_recent: usize,
|
||||
@@ -61,6 +80,19 @@ pub struct CompactionConfig {
|
||||
pub strength: Option<LlmStrength>,
|
||||
}
|
||||
|
||||
/// Hand-written rather than derived: a derived `Default` would give
|
||||
/// `keep_recent: 0`, silently compacting away every recent message on any box
|
||||
/// that omits the section — which is now the shipped default.
|
||||
impl Default for CompactionConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
threshold_tokens: None,
|
||||
keep_recent: default_keep_recent(),
|
||||
strength: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Event-triage background processor settings.
|
||||
#[derive(Debug, Clone, Deserialize)]
|
||||
pub struct EventTriageConfig {
|
||||
|
||||
@@ -236,7 +236,7 @@ impl AgentCatalog for SkaldAgentCatalog {
|
||||
)),
|
||||
Some(self.fs.load()),
|
||||
self.config.max_history_messages,
|
||||
self.config.compaction_enabled,
|
||||
self.config.auto_compaction_enabled,
|
||||
self.config.max_tool_result_chars,
|
||||
),
|
||||
);
|
||||
|
||||
@@ -26,20 +26,24 @@ const INTERRUPTED: &str = "Error: tool call was interrupted (connection lost bef
|
||||
|
||||
/// The knobs Skald's model fleet needs.
|
||||
///
|
||||
/// - `max_history_messages` applies **only without compaction**: with the
|
||||
/// compactor on, the summary is what bounds the context, and a window on top
|
||||
/// of it would silently drop messages the summary does not cover.
|
||||
/// - `max_history_messages` is **off by default** (`None`), leaving history
|
||||
/// append-only so the prompt prefix — and the provider's cache of it — stays
|
||||
/// stable for the whole conversation. When set, it applies **only without
|
||||
/// automatic compaction**: with the automatic pass on, the summary is what
|
||||
/// bounds the context, and a window on top of it would silently drop messages
|
||||
/// the summary does not cover. Manual `/compact` does not disarm it — a cap
|
||||
/// the admin typed should not vanish because a user ran a command.
|
||||
/// - tool results are shrunk for previous turns only, so the in-flight turn
|
||||
/// always sees its own output in full.
|
||||
pub fn skald_projection(
|
||||
max_history_messages: usize,
|
||||
compaction_enabled: bool,
|
||||
max_tool_result_chars: Option<usize>,
|
||||
max_history_messages: Option<usize>,
|
||||
auto_compaction_enabled: bool,
|
||||
max_tool_result_chars: Option<usize>,
|
||||
) -> Projection {
|
||||
Projection {
|
||||
summary_prefix: SUMMARY_PREFIX.to_string(),
|
||||
summary_suffix: Some(SUMMARY_SUFFIX.to_string()),
|
||||
max_messages: (!compaction_enabled).then_some(max_history_messages),
|
||||
max_messages: max_history_messages.filter(|_| !auto_compaction_enabled),
|
||||
max_tool_result: max_tool_result_chars.map(|max_chars| ResultLimit {
|
||||
max_chars,
|
||||
previous_turns_only: true,
|
||||
@@ -66,16 +70,16 @@ pub fn skald_projection(
|
||||
/// never inlined (nothing can be authorized), which is the right default for a
|
||||
/// context with no user workspace.
|
||||
pub fn skald_assembler(
|
||||
activation: Arc<dyn ActivationSource>,
|
||||
fs: Option<Arc<UserFs>>,
|
||||
max_history_messages: usize,
|
||||
compaction_enabled: bool,
|
||||
max_tool_result_chars: Option<usize>,
|
||||
activation: Arc<dyn ActivationSource>,
|
||||
fs: Option<Arc<UserFs>>,
|
||||
max_history_messages: Option<usize>,
|
||||
auto_compaction_enabled: bool,
|
||||
max_tool_result_chars: Option<usize>,
|
||||
) -> LinearAssembler {
|
||||
let mut assembler = LinearAssembler::new()
|
||||
.with_projection(skald_projection(
|
||||
max_history_messages,
|
||||
compaction_enabled,
|
||||
auto_compaction_enabled,
|
||||
max_tool_result_chars,
|
||||
))
|
||||
.with_activation(activation)
|
||||
|
||||
@@ -60,10 +60,12 @@ use crate::tools::tool_names as tn;
|
||||
pub struct LoopConfig {
|
||||
pub max_rounds: usize,
|
||||
pub max_parallel_calls: usize,
|
||||
pub max_history_messages: usize,
|
||||
/// Sliding-window cap on projected history. `None` (the default) leaves
|
||||
/// history append-only — see `LlmConfig::max_history_messages`.
|
||||
pub max_history_messages: Option<usize>,
|
||||
pub max_tool_result_chars: Option<usize>,
|
||||
/// Compaction bounds the context instead of a message window.
|
||||
pub compaction_enabled: bool,
|
||||
/// Automatic compaction bounds the context instead of a message window.
|
||||
pub auto_compaction_enabled: bool,
|
||||
pub datetime: DatetimeConfig,
|
||||
pub max_agent_depth: u32,
|
||||
}
|
||||
@@ -272,7 +274,7 @@ impl UserLoopRuntime {
|
||||
)),
|
||||
Some(self.fs.load()),
|
||||
self.config.max_history_messages,
|
||||
self.config.compaction_enabled,
|
||||
self.config.auto_compaction_enabled,
|
||||
self.config.max_tool_result_chars,
|
||||
));
|
||||
|
||||
|
||||
@@ -244,8 +244,8 @@ pub async fn project(db: &Db, agent: &AgentFixture, case: &Case) -> Vec<Value> {
|
||||
None,
|
||||
)),
|
||||
case.fs.clone(),
|
||||
HISTORY_LIMIT,
|
||||
// `compaction_enabled: false` mirrors the builder's `compactor: None`.
|
||||
Some(HISTORY_LIMIT),
|
||||
// The snapshots exercise the window, so automatic compaction stays off.
|
||||
false,
|
||||
Some(TOOL_RESULT_LIMIT),
|
||||
);
|
||||
|
||||
@@ -277,9 +277,10 @@ pub struct ChatSessionHandler {
|
||||
/// (no live oneshot to unblock). The next resume's approval gate skips re-gating
|
||||
/// these so a post-restart approve dispatches the tool without a second prompt.
|
||||
pub(super) pre_approved: Arc<std::sync::Mutex<std::collections::HashSet<i64>>>,
|
||||
/// Context compactor, shared across all sessions. `None` when compaction
|
||||
/// is disabled (no `compaction` section in config).
|
||||
pub(super) compactor: Option<Arc<ContextCompactor>>,
|
||||
/// Context compactor, shared across all sessions. Always present: `/compact`
|
||||
/// works with no configuration, and the automatic pass below is what
|
||||
/// `CompactionConfig::threshold_tokens` gates.
|
||||
pub(super) compactor: Arc<ContextCompactor>,
|
||||
/// This user's loop stack (manager, store, gate, catalog, delegate), built
|
||||
/// once per `ChatSessionManager` and shared by every session of the owner.
|
||||
pub(super) loop_runtime: Arc<crate::loop_adapters::runtime::UserLoopRuntime>,
|
||||
@@ -315,7 +316,7 @@ impl ChatSessionHandler {
|
||||
event_bus: Arc<ChatEventBus>,
|
||||
memory_manager: Arc<MemoryManager>,
|
||||
image_generator_manager: Arc<ImageGeneratorManager>,
|
||||
compactor: Option<Arc<ContextCompactor>>,
|
||||
compactor: Arc<ContextCompactor>,
|
||||
run_context: Option<RunContext>,
|
||||
loop_runtime: Arc<crate::loop_adapters::runtime::UserLoopRuntime>,
|
||||
) -> Self {
|
||||
@@ -449,15 +450,10 @@ impl ChatSessionHandler {
|
||||
Some(s) => s,
|
||||
None => return Ok(false),
|
||||
};
|
||||
match self.compactor {
|
||||
Some(ref compactor) => {
|
||||
compactor.force_compact(
|
||||
self.loop_runtime.manager(), pool, &self.user_id,
|
||||
self.session_id, stack.id, self.is_ephemeral,
|
||||
).await
|
||||
}
|
||||
None => Ok(false),
|
||||
}
|
||||
self.compactor.force_compact(
|
||||
self.loop_runtime.manager(), pool, &self.user_id,
|
||||
self.session_id, stack.id, self.is_ephemeral,
|
||||
).await
|
||||
}
|
||||
|
||||
/// Processes a user message end-to-end:
|
||||
@@ -553,9 +549,10 @@ impl ChatSessionHandler {
|
||||
// threshold. If so, summarise the old history before processing the
|
||||
// new message. This keeps latency transparent to the user — the wait
|
||||
// happens here, before the LLM loop, and is not a separate turn.
|
||||
if let Some(ref compactor) = self.compactor {
|
||||
// A no-op unless `compaction.threshold_tokens` is configured.
|
||||
{
|
||||
let last_tokens = self.last_input_tokens.load(Ordering::Relaxed);
|
||||
match compactor.try_compact(
|
||||
match self.compactor.try_compact(
|
||||
self.loop_runtime.manager(), pool, &self.user_id,
|
||||
self.session_id, stack.id, last_tokens, self.is_ephemeral,
|
||||
).await {
|
||||
|
||||
@@ -44,8 +44,10 @@ pub struct ChatSessionManager {
|
||||
event_bus: Arc<ChatEventBus>,
|
||||
memory_manager: Arc<MemoryManager>,
|
||||
image_generator_manager: Arc<ImageGeneratorManager>,
|
||||
/// Shared compactor instance, `None` when compaction is disabled.
|
||||
compactor: Option<Arc<ContextCompactor>>,
|
||||
/// Shared compactor instance. Always present — manual `/compact` needs no
|
||||
/// configuration; `CompactionConfig::threshold_tokens` arms the automatic
|
||||
/// trigger on top of it.
|
||||
compactor: Arc<ContextCompactor>,
|
||||
run_context_manager: Arc<RunContextManager>,
|
||||
/// This user's loop stack (blueprint D12): built once here and shared by
|
||||
/// every session of the owner, so the manager keeps a global view of what
|
||||
@@ -61,7 +63,7 @@ impl ChatSessionManager {
|
||||
user_id: String,
|
||||
user_fs: SharedFs,
|
||||
llm_manager: Arc<LlmManager>,
|
||||
max_history_messages: usize,
|
||||
max_history_messages: Option<usize>,
|
||||
max_tool_rounds: usize,
|
||||
max_parallel_subagents: usize,
|
||||
max_tool_result_chars: Option<usize>,
|
||||
@@ -73,7 +75,7 @@ impl ChatSessionManager {
|
||||
event_bus: Arc<ChatEventBus>,
|
||||
memory_manager: Arc<MemoryManager>,
|
||||
image_generator_manager: Arc<ImageGeneratorManager>,
|
||||
compactor: Option<Arc<ContextCompactor>>,
|
||||
compactor: Arc<ContextCompactor>,
|
||||
run_context_manager: Arc<RunContextManager>,
|
||||
tool_discovery: Arc<ToolDiscovery>,
|
||||
) -> anyhow::Result<Self> {
|
||||
@@ -93,7 +95,10 @@ impl ChatSessionManager {
|
||||
max_parallel_calls: max_parallel_subagents,
|
||||
max_history_messages,
|
||||
max_tool_result_chars,
|
||||
compaction_enabled: compactor.is_some(),
|
||||
// The window yields to the *automatic* compactor, not to its mere
|
||||
// existence: manual `/compact` alone must not silently disable a
|
||||
// configured message cap.
|
||||
auto_compaction_enabled: compactor.auto_enabled(),
|
||||
datetime: datetime_config.clone(),
|
||||
max_agent_depth: crate::session::handler::MAX_AGENT_DEPTH as u32,
|
||||
},
|
||||
|
||||
@@ -329,23 +329,28 @@ impl Conversation {
|
||||
}
|
||||
info!("run_context manager ready");
|
||||
|
||||
let compactor = config.llm.compaction.as_ref().map(|cfg| {
|
||||
info!(
|
||||
threshold_tokens = cfg.threshold_tokens,
|
||||
keep_recent = cfg.keep_recent,
|
||||
?cfg.strength,
|
||||
"context compactor enabled"
|
||||
);
|
||||
// Always built: `/compact` is a manual command and must work with no
|
||||
// configuration. Only the automatic trigger is opt-in (`threshold_tokens`).
|
||||
let compactor = {
|
||||
let cfg = &config.llm.compaction;
|
||||
match cfg.threshold_tokens {
|
||||
Some(threshold_tokens) => info!(
|
||||
threshold_tokens,
|
||||
keep_recent = cfg.keep_recent,
|
||||
?cfg.strength,
|
||||
"context compactor ready (automatic compaction enabled)"
|
||||
),
|
||||
None => info!(
|
||||
"context compactor ready (automatic compaction off — /compact only)"
|
||||
),
|
||||
}
|
||||
Arc::new(ContextCompactor::new(
|
||||
cfg.clone(),
|
||||
Arc::clone(&models.llm_manager),
|
||||
Arc::clone(&rt.event_bus),
|
||||
Arc::clone(&rt.config),
|
||||
))
|
||||
});
|
||||
if compactor.is_none() {
|
||||
info!("context compactor disabled (no compaction config)");
|
||||
}
|
||||
};
|
||||
|
||||
// The ownerless manager is inert (no loops, no consumers — see §19): it takes
|
||||
// a placeholder UserFs purely to satisfy the type, never used to resolve a path.
|
||||
|
||||
@@ -139,12 +139,12 @@ pub(super) struct UserContextFactory {
|
||||
supervisor: Arc<super::supervisor::TaskSupervisor>,
|
||||
shutdown_token: CancellationToken,
|
||||
config_store: Arc<GlobalConfigManager>,
|
||||
max_history_messages: usize,
|
||||
max_history_messages: Option<usize>,
|
||||
max_tool_rounds: usize,
|
||||
max_parallel_subagents: usize,
|
||||
max_tool_result_chars: Option<usize>,
|
||||
datetime_config: DatetimeConfig,
|
||||
compaction: Option<CompactionConfig>,
|
||||
compaction: CompactionConfig,
|
||||
cron_tz: Option<Tz>,
|
||||
}
|
||||
|
||||
@@ -217,14 +217,13 @@ impl UserContextFactory {
|
||||
Arc::clone(&self.tools),
|
||||
);
|
||||
|
||||
let compactor = self.compaction.as_ref().map(|cfg| {
|
||||
Arc::new(ContextCompactor::new(
|
||||
cfg.clone(),
|
||||
Arc::clone(&self.llm_manager),
|
||||
Arc::clone(&event_bus),
|
||||
Arc::clone(&self.config_store),
|
||||
))
|
||||
});
|
||||
// Always built (see `bundles.rs`): manual `/compact` needs no config.
|
||||
let compactor = Arc::new(ContextCompactor::new(
|
||||
self.compaction.clone(),
|
||||
Arc::clone(&self.llm_manager),
|
||||
Arc::clone(&event_bus),
|
||||
Arc::clone(&self.config_store),
|
||||
));
|
||||
|
||||
// Per-user MCP runtime (blueprint §7/§9): the connectors this user has
|
||||
// activated, run INSIDE their container. Started here on first login and
|
||||
|
||||
Reference in New Issue
Block a user