agent-loop: projection, recovery, compaction into the crate (phase 3)
Nightly Build / build (push) Successful in 6m49s
Nightly Build / build (push) Successful in 6m49s
The session handler is now a thin shell: three entry points in kernel_turn.rs (run_kernel_turn / recover_turn / resolve_pending_call) and the ChatSessionHandler. Everything that shaped a Value — projection, recovery, compaction mechanics, the LLM loop, message building — lives in agent-loop or behind a loop_adapters trait. agent-loop: - projection/ (mod + media): stored history -> wire messages, the one place provider divergence lives; well-formedness contract, DTL injections (append-only), media parts. LinearAssembler is now a Projection + ProjectionHooks config, not its own implementation - recovery.rs: reap interrupted batches -> resolve the deepest frame's non-terminal calls (Running by policy + RestartHint, AwaitingHuman re-asked) -> un-wedge finished children -> cascade up, every frame on its own agent (B3) - compaction.rs: split point (never assistant+tool group), transcript, SUMMARY_PREFIX/preamble/template, the no-tools model call, summary row - manager: resolve_pending (gate skipped, real ToolContext, then continue incl. sub-agent); start_loop used by recovery; LiveInput - delegate: AsyncExecutor + StoreSink for mode:async (durable cron row, result delivered back into the parent conversation) - kernel/context/store: support the above (TurnScope via Extensions, frame lookups, aligned result-text semantics) skald-core: - loop_adapters: UserLoopRuntime (D12 - one LoopManager per user), TurnScope (per-turn state in the Extensions type-map; no scope is denied), projection_cfg/media_source/tool_digest (Skald's projection knobs without owning projection code), async_task (CronExecutor + DurableSink) - session/handler: stripped to mod.rs + kernel_turn.rs + config.rs + interface_tools.rs + media.rs; deleted agent_dispatch, approval, dispatch, emitter, gate, llm_call, llm_loop, message_builder, messages, outcome, resume - compactor.rs: policy only (threshold, model pick, CompactionEvent); mechanics are the crate's CLAUDE.md updated (recovery, compaction, sub-agents, approval gate, projection sections now describe the crate-owned flow).
This commit is contained in:
@@ -9,6 +9,7 @@ license = "MIT"
|
||||
tokio = { version = "1", features = ["sync", "rt", "time", "macros"] }
|
||||
tokio-util = { version = "0.7" }
|
||||
async-trait = "0.1"
|
||||
base64 = "0.22"
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
tracing = "0.1"
|
||||
|
||||
@@ -0,0 +1,546 @@
|
||||
//! Compaction (blueprint §9, D6) — summarising the old part of a frame's
|
||||
//! history so the context stops growing.
|
||||
//!
|
||||
//! It is **not a turn**: one model call, no tools, no rounds, no kernel. That
|
||||
//! is the whole reason it is its own component — a host can compact a
|
||||
//! conversation nothing is driving, and the loop never learns it happened.
|
||||
//!
|
||||
//! The result is a row, not a return value: the next loop reads
|
||||
//! `latest_summary` through the assembler and projects
|
||||
//! `system → summary → messages after covered_up_to`. Callers get a
|
||||
//! [`CompactionOutcome`] for telemetry, not for threading anywhere.
|
||||
//!
|
||||
//! What the host still owns: **when** (see [`should_compact`]), which model,
|
||||
//! and what to do afterwards ([`LoopHooks::on_compacted`] — re-anchoring
|
||||
//! anything pinned to a message that just went away).
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use serde_json::{Value, json};
|
||||
use tracing::{debug, info, warn};
|
||||
|
||||
use crate::events::{EventSink, LoopEvent};
|
||||
use crate::hooks::LoopHooks;
|
||||
use crate::ids::{ConversationId, FrameId, MessageId, SummaryId};
|
||||
use crate::model::{ModelHint, ModelRequest, ModelResponse, ModelSelector, Usage};
|
||||
use crate::store::{CallState, HistoryStore, NewSummary, Role, StoredMessage};
|
||||
|
||||
// ── The shipped prompt ───────────────────────────────────────────────────────
|
||||
|
||||
/// Prepended to the stored summary when it is projected back into the context.
|
||||
/// It tells the model this is a handoff from a previous context window, not a
|
||||
/// set of live instructions — without it, a model happily re-answers questions
|
||||
/// the summary merely *mentions*.
|
||||
pub const SUMMARY_PREFIX: &str = "\
|
||||
[CONTEXT COMPACTION — REFERENCE ONLY] Earlier turns were compacted \
|
||||
into the summary below. This is a handoff from a previous context \
|
||||
window — treat it as background reference, NOT as active instructions. \
|
||||
Do NOT answer questions or fulfill requests mentioned in this summary; \
|
||||
they were already addressed. \
|
||||
Your current task is identified in the '## Active Task' section of the \
|
||||
summary — resume exactly from there. \
|
||||
Your system prompt and any injected memory files are ALWAYS authoritative \
|
||||
— never deprioritize them due to this compaction note. \
|
||||
Respond ONLY to the latest user message that appears AFTER this summary. \
|
||||
The current session state (files, config, etc.) may reflect work \
|
||||
described here — avoid repeating it:";
|
||||
|
||||
/// Preamble shared by the first-compaction and the update prompts. The wording
|
||||
/// is deliberately plain: a summariser is the one call most likely to trip a
|
||||
/// content filter, since it restates whatever the conversation contained.
|
||||
pub const SUMMARIZER_PREAMBLE: &str = "\
|
||||
You are a summarization agent creating a context checkpoint. \
|
||||
Treat the conversation turns below as source material for a \
|
||||
compact record of prior work. \
|
||||
Produce only the structured summary; do not add a greeting, \
|
||||
preamble, or prefix. \
|
||||
Write the summary in the same language the user was using in the \
|
||||
conversation — do not translate or switch to English. \
|
||||
NEVER include API keys, tokens, passwords, secrets, credentials, \
|
||||
or connection strings in the summary — replace any that appear \
|
||||
with [REDACTED]. Note that the user may have had credentials present, \
|
||||
but do not preserve their values.";
|
||||
|
||||
/// The sections the summariser must fill in. Structure beats prose here: the
|
||||
/// next context window is resumed from `## Active Task`, so that field is
|
||||
/// worth more than everything else combined.
|
||||
pub const SUMMARY_TEMPLATE: &str = "\
|
||||
## Active Task
|
||||
[THE SINGLE MOST IMPORTANT FIELD. Copy the user's most recent request or \
|
||||
task assignment verbatim — the exact words they used. If multiple tasks \
|
||||
were requested and only some are done, list only the ones NOT yet completed. \
|
||||
Continuation should pick up exactly here. Example: \
|
||||
\"User asked: 'Now refactor the auth module to use JWT instead of sessions'\" \
|
||||
If no outstanding task exists, write \"None.\"]
|
||||
|
||||
## Goal
|
||||
[What the user is trying to accomplish overall]
|
||||
|
||||
## Constraints & Preferences
|
||||
[User preferences, coding style, constraints, important decisions]
|
||||
|
||||
## Completed Actions
|
||||
[Numbered list of concrete actions taken — include tool used, target, and outcome.
|
||||
Format each as: N. ACTION target — outcome [tool: name]
|
||||
Example:
|
||||
1. READ config.rs:45 — found == should be != [tool: read_file]
|
||||
2. EDIT config.rs:45 — changed == to != [tool: write_file]
|
||||
3. BUILD `cargo build` — succeeded, 0 errors [tool: execute_cmd]
|
||||
Be specific with file paths, commands, line numbers, and results.]
|
||||
|
||||
## Active State
|
||||
[Current working state — include:
|
||||
- Working directory and branch (if applicable)
|
||||
- Modified/created files with brief note on each
|
||||
- Build/test status
|
||||
- Any running processes or servers
|
||||
- Environment details that matter]
|
||||
|
||||
## In Progress
|
||||
[Work currently underway — what was being done when compaction fired]
|
||||
|
||||
## Blocked
|
||||
[Any blockers, errors, or issues not yet resolved. Include exact error messages.]
|
||||
|
||||
## Key Decisions
|
||||
[Important technical decisions and WHY they were made]
|
||||
|
||||
## Resolved Questions
|
||||
[Questions the user asked that were ALREADY answered — include the answer so it is not repeated]
|
||||
|
||||
## Pending User Asks
|
||||
[Questions or requests from the user that have NOT yet been answered or fulfilled. If none, write \"None.\"]
|
||||
|
||||
## Relevant Files
|
||||
[Files read, modified, or created — with brief note on each]
|
||||
|
||||
## Remaining Work
|
||||
[What remains to be done — framed as context, not instructions]
|
||||
|
||||
## Critical Context
|
||||
[Any specific values, error messages, configuration details, or data that would \
|
||||
be lost without explicit preservation. NEVER include API keys, tokens, passwords, \
|
||||
or credentials — write [REDACTED] instead.]
|
||||
|
||||
Write only the summary body. Do not include any preamble or prefix.";
|
||||
|
||||
/// How the summariser is asked. Override to change the wording or the sections
|
||||
/// without touching the mechanics.
|
||||
pub trait CompactionPrompt: Send + Sync {
|
||||
/// The single user message sent to the summariser. `prior` is the previous
|
||||
/// summary's body (without [`SUMMARY_PREFIX`]) when this is an update, so
|
||||
/// summaries never nest.
|
||||
fn build(&self, transcript: &str, prior: Option<&str>) -> String;
|
||||
}
|
||||
|
||||
/// The shipped prompt: preamble + transcript + template, in an update or a
|
||||
/// first-time shape.
|
||||
pub struct DefaultPrompt;
|
||||
|
||||
impl CompactionPrompt for DefaultPrompt {
|
||||
fn build(&self, transcript: &str, prior: Option<&str>) -> String {
|
||||
match prior {
|
||||
Some(prev) => format!(
|
||||
"{SUMMARIZER_PREAMBLE}\n\n\
|
||||
You are updating a context compaction summary. A previous compaction produced \
|
||||
the summary below. New conversation turns have occurred since then and need \
|
||||
to be incorporated.\n\n\
|
||||
PREVIOUS SUMMARY:\n{prev}\n\n\
|
||||
NEW TURNS TO INCORPORATE:\n{transcript}\n\n\
|
||||
Update the summary using this exact structure. PRESERVE all existing information \
|
||||
that is still relevant. ADD new completed actions to the numbered list (continue \
|
||||
numbering). Move items from \"In Progress\" to \"Completed Actions\" when done. \
|
||||
Move answered questions to \"Resolved Questions\". Update \"Active State\" to \
|
||||
reflect current state. Remove information only if it is clearly obsolete. \
|
||||
CRITICAL: Update \"## Active Task\" to reflect the user's most recent unfulfilled \
|
||||
request — this is the most important field for task continuity.\n\n\
|
||||
{SUMMARY_TEMPLATE}"
|
||||
),
|
||||
None => format!(
|
||||
"{SUMMARIZER_PREAMBLE}\n\n\
|
||||
Create a structured checkpoint summary for the conversation after earlier turns \
|
||||
are compacted. The summary should preserve enough detail for continuity without \
|
||||
re-reading the original turns.\n\n\
|
||||
TURNS TO SUMMARIZE:\n{transcript}\n\n\
|
||||
Use this exact structure:\n\n\
|
||||
{SUMMARY_TEMPLATE}"
|
||||
),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ── Mode / outcome ───────────────────────────────────────────────────────────
|
||||
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub enum CompactionMode {
|
||||
/// Summarise everything except the last `keep_tail` messages, cutting on a
|
||||
/// user/agent boundary so an assistant turn is never split from its tool
|
||||
/// results.
|
||||
Auto { keep_tail: usize },
|
||||
/// Summarise up to an explicit message (a UI that lets the user pick).
|
||||
UpTo(MessageId),
|
||||
}
|
||||
|
||||
impl Default for CompactionMode {
|
||||
fn default() -> Self {
|
||||
Self::Auto { keep_tail: 6 }
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CompactionOutcome {
|
||||
pub summary_id: SummaryId,
|
||||
pub covered_up_to: MessageId,
|
||||
/// The first message the summary does NOT cover — what anything pinned to
|
||||
/// a compacted message must be re-anchored onto.
|
||||
pub first_surviving: MessageId,
|
||||
pub summary_text: String,
|
||||
pub messages_covered: usize,
|
||||
pub usage: Usage,
|
||||
}
|
||||
|
||||
/// Is it time? `usage` is the previous turn's reported input tokens; when the
|
||||
/// provider reported none, `estimated` (the host's own count) decides.
|
||||
pub fn should_compact(usage: Option<u32>, estimated: u32, threshold: u32) -> bool {
|
||||
usage.filter(|t| *t > 0).unwrap_or(estimated) >= threshold
|
||||
}
|
||||
|
||||
// ── Compaction ───────────────────────────────────────────────────────────────
|
||||
|
||||
/// One compaction, ready to run. Built via
|
||||
/// [`LoopManager::new_compaction`](crate::manager::LoopManager::new_compaction)
|
||||
/// so it shares the manager's store, hooks and event bus.
|
||||
pub struct Compaction {
|
||||
pub(crate) store: Arc<dyn HistoryStore>,
|
||||
pub(crate) selector: Arc<dyn ModelSelector>,
|
||||
pub(crate) hooks: Vec<Arc<dyn LoopHooks>>,
|
||||
pub(crate) events: EventSink,
|
||||
pub(crate) conversation: ConversationId,
|
||||
pub(crate) frame: FrameId,
|
||||
pub(crate) mode: CompactionMode,
|
||||
pub(crate) hint: ModelHint,
|
||||
pub(crate) prompt: Arc<dyn CompactionPrompt>,
|
||||
pub(crate) temperature: Option<f32>,
|
||||
/// Host free-form, forwarded on the request (payload logging).
|
||||
pub(crate) log: Option<Value>,
|
||||
}
|
||||
|
||||
impl Compaction {
|
||||
pub fn mode(mut self, mode: CompactionMode) -> Self {
|
||||
self.mode = mode;
|
||||
self
|
||||
}
|
||||
|
||||
/// Pin the summariser's model. Default: whatever the selector picks.
|
||||
pub fn model(mut self, hint: ModelHint) -> Self {
|
||||
self.hint = hint;
|
||||
self
|
||||
}
|
||||
|
||||
/// Override the selector for this call (a cheaper tier, say).
|
||||
pub fn selector(mut self, selector: Arc<dyn ModelSelector>) -> Self {
|
||||
self.selector = selector;
|
||||
self
|
||||
}
|
||||
|
||||
pub fn prompt(mut self, prompt: Arc<dyn CompactionPrompt>) -> Self {
|
||||
self.prompt = prompt;
|
||||
self
|
||||
}
|
||||
|
||||
pub fn log(mut self, log: Value) -> Self {
|
||||
self.log = Some(log);
|
||||
self
|
||||
}
|
||||
|
||||
/// Summarise and save. `Ok(None)` means there was nothing worth compacting
|
||||
/// — not an error: too few messages, no clean split point, or a summariser
|
||||
/// that came back empty.
|
||||
pub async fn run(&self) -> crate::Result<Option<CompactionOutcome>> {
|
||||
let prior = self.store.latest_summary(self.frame).await?;
|
||||
let messages = match &prior {
|
||||
Some(s) => self.store.load_since(self.frame, s.covered_up_to).await?,
|
||||
None => self.store.load(self.frame).await?,
|
||||
};
|
||||
|
||||
let Some(split) = self.split_point(&messages) else {
|
||||
debug!(frame = %self.frame, "compaction: nothing to summarise");
|
||||
return Ok(None);
|
||||
};
|
||||
let (to_summarise, surviving) = messages.split_at(split);
|
||||
let covered_up_to = to_summarise.last().expect("split > 0").id;
|
||||
let first_surviving = surviving.first().expect("split < len").id;
|
||||
|
||||
let transcript = transcript(to_summarise);
|
||||
let body = self.prompt.build(&transcript, prior.as_ref().map(|s| s.text.as_str()));
|
||||
|
||||
let handle = self.selector.select(&self.hint, &[]).await?;
|
||||
info!(
|
||||
frame = %self.frame,
|
||||
model = %handle.id,
|
||||
messages = to_summarise.len(),
|
||||
"compaction: summarising"
|
||||
);
|
||||
let request = ModelRequest {
|
||||
messages: vec![json!({ "role": "user", "content": body })],
|
||||
tools: Vec::new(),
|
||||
model: handle.id.clone(),
|
||||
max_tokens: None,
|
||||
temperature: self.temperature,
|
||||
request_id: uuid_like(),
|
||||
conversation: self.conversation.clone(),
|
||||
frame: self.frame,
|
||||
extras: handle.info.extras.clone(),
|
||||
log: self.log.clone(),
|
||||
};
|
||||
let response = handle.model.complete(&request, None).await.map_err(|e| {
|
||||
warn!(frame = %self.frame, error = %e, "compaction: the summariser failed");
|
||||
anyhow::anyhow!("compaction: {e}")
|
||||
})?;
|
||||
|
||||
let (summary_text, usage) = match response {
|
||||
ModelResponse::Message { content, usage, .. } => (content, usage),
|
||||
// A summariser has no tools; if one hallucinates a call, its text is
|
||||
// still the summary.
|
||||
ModelResponse::ToolCalls { content, usage, .. } => {
|
||||
warn!(frame = %self.frame, "compaction: unexpected tool calls, using the content");
|
||||
(content, usage)
|
||||
}
|
||||
};
|
||||
if summary_text.trim().is_empty() {
|
||||
warn!(frame = %self.frame, "compaction: empty summary, nothing saved");
|
||||
return Ok(None);
|
||||
}
|
||||
|
||||
let summary_id = self
|
||||
.store
|
||||
.save_summary(self.frame, NewSummary { text: summary_text.clone(), covered_up_to })
|
||||
.await?;
|
||||
|
||||
self.events.emit(self.frame, None, LoopEvent::Compacted {
|
||||
frame: self.frame,
|
||||
covered_up_to,
|
||||
});
|
||||
for h in &self.hooks {
|
||||
h.on_compacted(self.frame, covered_up_to, first_surviving).await;
|
||||
}
|
||||
|
||||
info!(frame = %self.frame, %summary_id, %covered_up_to, "compaction: summary saved");
|
||||
Ok(Some(CompactionOutcome {
|
||||
summary_id,
|
||||
covered_up_to,
|
||||
first_surviving,
|
||||
summary_text,
|
||||
messages_covered: to_summarise.len(),
|
||||
usage,
|
||||
}))
|
||||
}
|
||||
|
||||
/// Where to cut. Never between an assistant message and its tool results —
|
||||
/// the surviving half would be a tool result answering a call the model
|
||||
/// cannot see, which strict APIs reject outright.
|
||||
fn split_point(&self, messages: &[StoredMessage]) -> Option<usize> {
|
||||
match self.mode {
|
||||
CompactionMode::UpTo(id) => {
|
||||
let idx = messages.iter().position(|m| m.id == id)? + 1;
|
||||
(idx < messages.len()).then_some(idx)
|
||||
}
|
||||
CompactionMode::Auto { keep_tail } => {
|
||||
if messages.len() <= keep_tail {
|
||||
return None;
|
||||
}
|
||||
let raw = messages.len() - keep_tail;
|
||||
let split = (0..=raw)
|
||||
.rev()
|
||||
.find(|&i| i == 0 || matches!(messages[i].role, Role::User | Role::Agent))
|
||||
.unwrap_or(0);
|
||||
(split > 0).then_some(split)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ── Transcript ───────────────────────────────────────────────────────────────
|
||||
|
||||
/// Head+tail truncation: a summariser needs both how a long output started and
|
||||
/// how it ended; a prefix cut throws the conclusion away.
|
||||
fn truncate_head_tail(s: &str, head_chars: usize, tail_chars: usize) -> String {
|
||||
let s = s.trim();
|
||||
let char_count = s.chars().count();
|
||||
if char_count <= head_chars + tail_chars {
|
||||
return s.to_string();
|
||||
}
|
||||
let head_end = s.char_indices().nth(head_chars).map(|(i, _)| i).unwrap_or(s.len());
|
||||
let tail_start = s
|
||||
.char_indices()
|
||||
.nth(char_count - tail_chars)
|
||||
.map(|(i, _)| i)
|
||||
.unwrap_or(0);
|
||||
format!("{}\n...[truncated]...\n{}", &s[..head_end], &s[tail_start..])
|
||||
}
|
||||
|
||||
fn truncate(s: &str, max_chars: usize) -> String {
|
||||
let s = s.trim();
|
||||
if s.chars().count() <= max_chars {
|
||||
return s.to_string();
|
||||
}
|
||||
let end = s.char_indices().nth(max_chars).map(|(i, _)| i).unwrap_or(s.len());
|
||||
format!("{}…", &s[..end])
|
||||
}
|
||||
|
||||
/// The messages as labeled text. Not the wire projection: a summariser reads
|
||||
/// better prose than JSON, and tool results are worth more than tool schemas.
|
||||
fn transcript(messages: &[StoredMessage]) -> String {
|
||||
let mut parts: Vec<String> = Vec::new();
|
||||
for msg in messages {
|
||||
match msg.role {
|
||||
Role::User | Role::Agent => {
|
||||
parts.push(format!("[USER]: {}", truncate_head_tail(&msg.content, 6000, 1500)));
|
||||
}
|
||||
Role::Assistant => {
|
||||
let mut content = truncate_head_tail(&msg.content, 6000, 1500);
|
||||
if !msg.calls.is_empty() {
|
||||
let lines: Vec<String> = msg
|
||||
.calls
|
||||
.iter()
|
||||
.map(|c| {
|
||||
let args = c
|
||||
.arguments_raw
|
||||
.clone()
|
||||
.unwrap_or_else(|| c.arguments.to_string());
|
||||
format!(" {}({})", c.name, truncate(&args, 1200))
|
||||
})
|
||||
.collect();
|
||||
content.push_str(&format!("\n[Tool calls:\n{}\n]", lines.join("\n")));
|
||||
}
|
||||
parts.push(format!("[ASSISTANT]: {content}"));
|
||||
|
||||
for call in &msg.calls {
|
||||
let result = match call.state {
|
||||
CallState::Done => call
|
||||
.result
|
||||
.as_deref()
|
||||
.map(|r| truncate_head_tail(r, 4000, 1500))
|
||||
.unwrap_or_default(),
|
||||
_ => "(failed or interrupted)".to_string(),
|
||||
};
|
||||
parts.push(format!("[TOOL RESULT tc_{}]: {result}", call.id));
|
||||
}
|
||||
}
|
||||
// System messages are built per turn, never stored (see `store`).
|
||||
Role::System => {}
|
||||
}
|
||||
}
|
||||
parts.join("\n\n")
|
||||
}
|
||||
|
||||
/// Correlation id for the summariser call (the crate carries no uuid crate).
|
||||
fn uuid_like() -> String {
|
||||
let nanos = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.map(|d| d.as_nanos())
|
||||
.unwrap_or(0);
|
||||
format!("compaction-{nanos:032x}")
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::store::{CallOutcome, NewCall, NewMessage};
|
||||
use crate::store_memory::InMemoryStore;
|
||||
use crate::tool::ToolOutput;
|
||||
|
||||
#[test]
|
||||
fn the_threshold_falls_back_to_the_estimate_when_usage_is_missing() {
|
||||
assert!(should_compact(Some(120), 0, 100));
|
||||
assert!(!should_compact(Some(80), 999, 100));
|
||||
// No usage reported (or zero) → the host's own estimate decides.
|
||||
assert!(should_compact(None, 120, 100));
|
||||
assert!(should_compact(Some(0), 120, 100));
|
||||
assert!(!should_compact(None, 80, 100));
|
||||
}
|
||||
|
||||
async fn seeded() -> (Arc<InMemoryStore>, FrameId, Vec<StoredMessage>) {
|
||||
let store = Arc::new(InMemoryStore::new());
|
||||
let conv = ConversationId::new("c");
|
||||
let frame = store
|
||||
.open_frame(&conv, None, crate::store::FrameSpec::root("a"))
|
||||
.await
|
||||
.unwrap();
|
||||
for i in 0..4 {
|
||||
store.append(frame, NewMessage::user(format!("q{i}"))).await.unwrap();
|
||||
let m = store
|
||||
.append(frame, NewMessage::assistant(format!("a{i}"), None))
|
||||
.await
|
||||
.unwrap();
|
||||
let c = store
|
||||
.append_call(m, NewCall::new("read_file", json!({ "path": "x" })))
|
||||
.await
|
||||
.unwrap();
|
||||
store
|
||||
.resolve_call(c, &CallOutcome::Completed(ToolOutput::Text("body".into())))
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
let msgs = store.load(frame).await.unwrap();
|
||||
(store, frame, msgs)
|
||||
}
|
||||
|
||||
fn compaction(store: Arc<InMemoryStore>, frame: FrameId, mode: CompactionMode) -> Compaction {
|
||||
let (bus, _) = tokio::sync::broadcast::channel(16);
|
||||
Compaction {
|
||||
store,
|
||||
// The split-point tests never reach the model.
|
||||
selector: Arc::new(crate::model::SingleModel::new(crate::testing::FakeModel::new(
|
||||
"unused",
|
||||
Vec::new(),
|
||||
))),
|
||||
hooks: Vec::new(),
|
||||
events: EventSink::new(ConversationId::new("c"), bus),
|
||||
conversation: ConversationId::new("c"),
|
||||
frame,
|
||||
mode,
|
||||
hint: ModelHint::default(),
|
||||
prompt: Arc::new(DefaultPrompt),
|
||||
temperature: None,
|
||||
log: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn the_cut_never_splits_an_assistant_turn_from_its_tool_results() {
|
||||
let (store, frame, msgs) = seeded().await;
|
||||
// 8 messages: user/assistant × 4. keep_tail = 3 would cut at index 5 —
|
||||
// an assistant message — so it must walk back to the user before it.
|
||||
let c = compaction(store, frame, CompactionMode::Auto { keep_tail: 3 });
|
||||
let split = c.split_point(&msgs).unwrap();
|
||||
assert!(matches!(msgs[split].role, Role::User), "cut at {split}: {:?}", msgs[split].role);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn there_is_nothing_to_compact_in_a_short_conversation() {
|
||||
let (store, frame, msgs) = seeded().await;
|
||||
let c = compaction(store, frame, CompactionMode::Auto { keep_tail: 99 });
|
||||
assert!(c.split_point(&msgs).is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn an_explicit_cut_point_covers_it_and_keeps_the_rest() {
|
||||
let (store, frame, msgs) = seeded().await;
|
||||
let c = compaction(store.clone(), frame, CompactionMode::UpTo(msgs[2].id));
|
||||
assert_eq!(c.split_point(&msgs), Some(3));
|
||||
// Cutting at the very last message would leave nothing surviving.
|
||||
let c = compaction(store, frame, CompactionMode::UpTo(msgs.last().unwrap().id));
|
||||
assert_eq!(c.split_point(&msgs), None);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn the_transcript_carries_calls_and_their_results() {
|
||||
let (_store, _frame, msgs) = seeded().await;
|
||||
let text = transcript(&msgs[..2]);
|
||||
assert!(text.contains("[USER]: q0"), "{text}");
|
||||
assert!(text.contains("[ASSISTANT]: a0"), "{text}");
|
||||
assert!(text.contains("read_file({\"path\":\"x\"})"), "{text}");
|
||||
assert!(text.contains("[TOOL RESULT tc_1]: body"), "{text}");
|
||||
}
|
||||
}
|
||||
@@ -1,27 +1,23 @@
|
||||
//! The system context (layered) and the `ContextAssembler` — from system +
|
||||
//! history to wire messages.
|
||||
//!
|
||||
//! **Well-formedness contract** (every assembler MUST honor it):
|
||||
//!
|
||||
//! 1. Order: static system → compaction summary (if any) → messages after
|
||||
//! `covered_up_to` → dynamic tail → tail reminder.
|
||||
//! 2. Every assistant `tool_call` has a tool-result: `Done`→result,
|
||||
//! `Failed`→error, `Cancelled`/`Rejected`→note, **`Running`/`AwaitingHuman`
|
||||
//! surviving a crash → synthetic "interrupted" result**.
|
||||
//! 3. No `failed` messages (orphans) — already filtered by the store.
|
||||
//! 4. DTL injection (§4.10 of the blueprint): when `model.tool_rendering` is
|
||||
//! not `Inline` and an `ActivationSource` is present, each activation is
|
||||
//! projected at its anchor (marker vs system+tools block, append-only).
|
||||
//! The projection itself lives in [`crate::projection`], which owns the
|
||||
//! well-formedness contract and every provider-shaped decision. This module is
|
||||
//! the seam: hosts implement [`SystemContextSource`] to say *what* goes in the
|
||||
//! system prompt, and [`LinearAssembler`] configures the projection.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use serde_json::{Value, json};
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::activation::{ActivationSource, ToolRendering};
|
||||
use crate::activation::ActivationSource;
|
||||
use crate::ids::{ConversationId, FrameId};
|
||||
use crate::model::ModelInfo;
|
||||
use crate::store::{HistoryStore, Role, StoredMessage};
|
||||
use crate::projection::{
|
||||
MediaSource, Projection, ProjectionHooks, ResultLimit, ToolResultDigest,
|
||||
};
|
||||
use crate::store::HistoryStore;
|
||||
|
||||
// ── SystemContext ────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -115,36 +111,55 @@ pub trait ContextAssembler: Send + Sync {
|
||||
|
||||
// ── LinearAssembler ──────────────────────────────────────────────────────────
|
||||
|
||||
/// The shipped assembler: system + summary + history, with an optional
|
||||
/// message window and tool-result truncation. Honors the DTL injection
|
||||
/// contract when given an `ActivationSource`.
|
||||
/// The shipped assembler: a [`Projection`] plus the host hooks it may use.
|
||||
///
|
||||
/// Out of the box it produces a correct OpenAI-shaped conversation. A host with
|
||||
/// stricter models overrides the projection (`with_projection`) and plugs in its
|
||||
/// media authorization and result-digest policy.
|
||||
pub struct LinearAssembler {
|
||||
/// Keep at most this many history messages (cut at a User/Agent boundary,
|
||||
/// never mid assistant+tool group).
|
||||
pub max_messages: Option<usize>,
|
||||
/// Truncate each tool result to this many chars.
|
||||
pub max_tool_result_chars: Option<usize>,
|
||||
/// DTL activations (only consulted when `tool_rendering != Inline`).
|
||||
pub activation: Option<Arc<dyn ActivationSource>>,
|
||||
pub projection: Projection,
|
||||
pub hooks: ProjectionHooks,
|
||||
}
|
||||
|
||||
impl LinearAssembler {
|
||||
pub fn new() -> Self {
|
||||
Self { max_messages: None, max_tool_result_chars: None, activation: None }
|
||||
Self { projection: Projection::default(), hooks: ProjectionHooks::default() }
|
||||
}
|
||||
|
||||
/// Replace the whole protocol configuration.
|
||||
pub fn with_projection(mut self, projection: Projection) -> Self {
|
||||
self.projection = projection;
|
||||
self
|
||||
}
|
||||
|
||||
/// Keep at most this many history messages (cut boundary-safely).
|
||||
pub fn with_max_messages(mut self, n: usize) -> Self {
|
||||
self.max_messages = Some(n);
|
||||
self.projection.max_messages = Some(n);
|
||||
self
|
||||
}
|
||||
|
||||
/// Shrink every tool result longer than `n` chars.
|
||||
pub fn with_tool_result_limit(mut self, n: usize) -> Self {
|
||||
self.max_tool_result_chars = Some(n);
|
||||
self.projection.max_tool_result =
|
||||
Some(ResultLimit { max_chars: n, previous_turns_only: false });
|
||||
self
|
||||
}
|
||||
|
||||
/// DTL activations (consulted only when `tool_rendering != Inline`).
|
||||
pub fn with_activation(mut self, src: Arc<dyn ActivationSource>) -> Self {
|
||||
self.activation = Some(src);
|
||||
self.hooks.activation = Some(src);
|
||||
self
|
||||
}
|
||||
|
||||
/// Which media a message may inline.
|
||||
pub fn with_media(mut self, src: Arc<dyn MediaSource>) -> Self {
|
||||
self.hooks.media = Some(src);
|
||||
self
|
||||
}
|
||||
|
||||
/// How an over-long tool result is condensed.
|
||||
pub fn with_digest(mut self, digest: Arc<dyn ToolResultDigest>) -> Self {
|
||||
self.hooks.digest = Some(digest);
|
||||
self
|
||||
}
|
||||
}
|
||||
@@ -153,9 +168,8 @@ impl Default for LinearAssembler {
|
||||
fn default() -> Self { Self::new() }
|
||||
}
|
||||
|
||||
/// The summary block is prefixed so the model understands what it is (Skald
|
||||
/// keeps its own SUMMARY_PREFIX in its assembler).
|
||||
pub const SUMMARY_PREFIX: &str = "[CONTEXT SUMMARY — earlier messages were compacted into this summary]";
|
||||
/// Re-exported for hosts that only need the default summary header.
|
||||
pub use crate::projection::SUMMARY_PREFIX;
|
||||
|
||||
#[async_trait]
|
||||
impl ContextAssembler for LinearAssembler {
|
||||
@@ -164,183 +178,6 @@ impl ContextAssembler for LinearAssembler {
|
||||
store: &Arc<dyn HistoryStore>,
|
||||
input: &AssembleInput,
|
||||
) -> crate::Result<Vec<Value>> {
|
||||
let mut out: Vec<Value> = Vec::new();
|
||||
|
||||
// 1. static system
|
||||
if !input.system.base.is_empty() {
|
||||
out.push(json!({ "role": "system", "content": input.system.base }));
|
||||
}
|
||||
for s in &input.system.extra_static {
|
||||
out.push(json!({ "role": "system", "content": s }));
|
||||
}
|
||||
|
||||
// 2. summary + surviving history
|
||||
let summary = store.latest_summary(input.frame).await?;
|
||||
if let Some(s) = &summary {
|
||||
out.push(json!({
|
||||
"role": "system",
|
||||
"content": format!("{SUMMARY_PREFIX}\n\n{}", s.text),
|
||||
}));
|
||||
}
|
||||
let mut history = match &summary {
|
||||
Some(s) => store.load_since(input.frame, s.covered_up_to).await?,
|
||||
None => store.load(input.frame).await?,
|
||||
};
|
||||
if let Some(max) = self.max_messages {
|
||||
history = window(history, max);
|
||||
}
|
||||
|
||||
// 3. DTL activations (consulted only in non-Inline modes)
|
||||
let activations = match (&self.activation, input.model.tool_rendering) {
|
||||
(Some(src), ToolRendering::Inline) => {
|
||||
let _ = src;
|
||||
Vec::new()
|
||||
}
|
||||
(Some(src), _) => src.activations(input.frame).await.unwrap_or_default(),
|
||||
(None, _) => Vec::new(),
|
||||
};
|
||||
|
||||
for msg in &history {
|
||||
project_message(&mut out, msg, self.max_tool_result_chars);
|
||||
inject_activations(&mut out, msg, &activations, &input.model.tool_rendering);
|
||||
}
|
||||
|
||||
// 4. dynamic tail + reminder
|
||||
for s in &input.system.dynamic_tail {
|
||||
out.push(json!({ "role": "system", "content": s }));
|
||||
}
|
||||
if let Some(r) = &input.system.tail_reminder {
|
||||
out.push(json!({ "role": "system", "content": r }));
|
||||
}
|
||||
|
||||
Ok(out)
|
||||
}
|
||||
}
|
||||
|
||||
/// Cut the history to at most `max` messages, at a User/Agent boundary so an
|
||||
/// assistant+tool group is never split.
|
||||
fn window(history: Vec<StoredMessage>, max: usize) -> Vec<StoredMessage> {
|
||||
if history.len() <= max {
|
||||
return history;
|
||||
}
|
||||
let start = history.len() - max;
|
||||
let cut = history[start..]
|
||||
.iter()
|
||||
.position(|m| matches!(m.role, Role::User | Role::Agent))
|
||||
.map(|p| start + p)
|
||||
.unwrap_or(start);
|
||||
history[cut..].to_vec()
|
||||
}
|
||||
|
||||
/// Project one stored message (and its tool results) to wire messages.
|
||||
fn project_message(out: &mut Vec<Value>, msg: &StoredMessage, result_limit: Option<usize>) {
|
||||
match msg.role {
|
||||
Role::System => {
|
||||
out.push(json!({ "role": "system", "content": msg.content }));
|
||||
}
|
||||
Role::User | Role::Agent => {
|
||||
out.push(json!({ "role": "user", "content": msg.content }));
|
||||
}
|
||||
Role::Assistant => {
|
||||
let mut wire = json!({ "role": "assistant", "content": msg.content });
|
||||
if let Some(r) = &msg.reasoning {
|
||||
// Echoed under both names: DeepSeek expects reasoning_content,
|
||||
// others reasoning (the clients normalize on read).
|
||||
wire["reasoning_content"] = json!(r);
|
||||
}
|
||||
if !msg.calls.is_empty() {
|
||||
let calls: Vec<Value> = msg
|
||||
.calls
|
||||
.iter()
|
||||
.map(|c| {
|
||||
json!({
|
||||
"id": c.provider_id,
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": c.name,
|
||||
"arguments": serde_json::to_string(&c.arguments)
|
||||
.unwrap_or_else(|_| "{}".into()),
|
||||
},
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
wire["tool_calls"] = Value::Array(calls);
|
||||
}
|
||||
out.push(wire);
|
||||
|
||||
for call in &msg.calls {
|
||||
let mut content = match call.state {
|
||||
crate::store::CallState::Running | crate::store::CallState::AwaitingHuman => {
|
||||
"[interrupted: this tool call did not complete — the session restarted \
|
||||
before a result was recorded]"
|
||||
.to_string()
|
||||
}
|
||||
crate::store::CallState::Failed => {
|
||||
format!("Error: {}", call.result.as_deref().unwrap_or("unknown error"))
|
||||
}
|
||||
_ => call.result.clone().unwrap_or_default(),
|
||||
};
|
||||
if let Some(limit) = result_limit
|
||||
&& content.chars().count() > limit
|
||||
{
|
||||
content = format!(
|
||||
"{}… [truncated]",
|
||||
content.chars().take(limit).collect::<String>()
|
||||
);
|
||||
}
|
||||
out.push(json!({
|
||||
"role": "tool",
|
||||
"tool_call_id": call.provider_id,
|
||||
"content": content,
|
||||
}));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// DTL injection at an activation anchor (blueprint §4.10):
|
||||
/// - `DeferredToolReference`: `_tool_references` marker on the FIRST tool
|
||||
/// result of the anchored assistant message (the client converts it).
|
||||
/// - `SystemToolBlock`: a `{role:"system", tools:[defs]}` message appended
|
||||
/// right after the anchored message's tool-result group.
|
||||
fn inject_activations(
|
||||
out: &mut Vec<Value>,
|
||||
msg: &StoredMessage,
|
||||
activations: &[crate::activation::Activation],
|
||||
mode: &ToolRendering,
|
||||
) {
|
||||
let acts: Vec<&crate::activation::Activation> =
|
||||
activations.iter().filter(|a| a.anchor == msg.id).collect();
|
||||
if acts.is_empty() {
|
||||
return;
|
||||
}
|
||||
match mode {
|
||||
ToolRendering::Inline => {}
|
||||
ToolRendering::DeferredToolReference => {
|
||||
let names: Vec<Value> = acts
|
||||
.iter()
|
||||
.flat_map(|a| &a.defs)
|
||||
.filter_map(|d| d["function"]["name"].as_str())
|
||||
.map(|n| json!(n))
|
||||
.collect();
|
||||
if names.is_empty() {
|
||||
return;
|
||||
}
|
||||
// Attach to the first tool result just emitted for this message.
|
||||
if let Some(tool_msg) = out
|
||||
.iter_mut()
|
||||
.rev()
|
||||
.take(msg.calls.len())
|
||||
.find(|m| m["role"].as_str() == Some("tool"))
|
||||
{
|
||||
tool_msg["_tool_references"] = Value::Array(names);
|
||||
}
|
||||
}
|
||||
ToolRendering::SystemToolBlock => {
|
||||
let defs: Vec<Value> = acts.iter().flat_map(|a| a.defs.clone()).collect();
|
||||
if !defs.is_empty() {
|
||||
out.push(json!({ "role": "system", "tools": defs }));
|
||||
}
|
||||
}
|
||||
crate::projection::project(store, input, &self.projection, &self.hooks).await
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,9 +4,10 @@
|
||||
//! parent awaits; a homogeneous batch of sync delegates fans out through the
|
||||
//! kernel's generic concurrency (`concurrency_safe`).
|
||||
//!
|
||||
//! The crate ships the SYNC flow. Async delegation rides the host's
|
||||
//! `AsyncExecutor` (phase-3 concern: Skald wires its durable cron executor
|
||||
//! there); calling it here fails with a clear error.
|
||||
//! Both flows ship. A SYNC child is awaited in place; an ASYNC one is handed to
|
||||
//! the host's [`AsyncExecutor`] and its result comes back later through an
|
||||
//! [`AsyncResultSink`] — a tool call the model already has an id for, resolved
|
||||
//! whenever the work finishes.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
@@ -15,11 +16,11 @@ use serde_json::{Value, json};
|
||||
use crate::async_trait;
|
||||
use crate::context::SystemContextSource;
|
||||
use crate::events::{EventSink, LoopEvent};
|
||||
use crate::ids::FrameId;
|
||||
use crate::ids::{ConversationId, FrameId, TaskId, ToolCallId};
|
||||
use crate::manager::{LoopManager, LoopParams, TurnMeta};
|
||||
use crate::model::{ModelHint, ModelSelector};
|
||||
use crate::store::{FrameSpec, HistoryStore, NewMessage};
|
||||
use crate::tool::{SharedToolSet, Tool, ToolCtx, ToolFailure, ToolOutput, ToolSet};
|
||||
use crate::store::{CallOutcome, FrameSpec, HistoryStore, NewCall, NewMessage};
|
||||
use crate::tool::{Extensions, SharedToolSet, Tool, ToolCtx, ToolFailure, ToolOutput, ToolSet};
|
||||
|
||||
// ── AgentCatalog ─────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -84,7 +85,16 @@ pub trait AgentCatalog: Send + Sync {
|
||||
/// Load a dispatchable profile, built for `child_frame` (already opened by
|
||||
/// the DelegateTool — frame-scoped pieces like grants/activation anchor to
|
||||
/// it). MUST reject non-`Task` kinds and unknown ids.
|
||||
async fn get(&self, id: &str, child_frame: FrameId) -> crate::Result<AgentProfile>;
|
||||
///
|
||||
/// `ctx` is the delegating call's context: a catalog that lives as long as
|
||||
/// the tenant reads the turn's own state (session, source, permissions)
|
||||
/// from `ctx.extensions` instead of having captured it at construction.
|
||||
async fn get(
|
||||
&self,
|
||||
id: &str,
|
||||
child_frame: FrameId,
|
||||
ctx: &ToolCtx,
|
||||
) -> crate::Result<AgentProfile>;
|
||||
async fn list(&self, kind: AgentKind) -> Vec<AgentSummary>;
|
||||
/// Frame-exit hook (host cleanup, e.g. deleting stack-scoped activations).
|
||||
async fn on_child_closed(&self, _frame: crate::ids::FrameId) {}
|
||||
@@ -99,6 +109,18 @@ pub struct FilteredToolSet {
|
||||
add: Vec<Arc<dyn Tool>>,
|
||||
}
|
||||
|
||||
impl FilteredToolSet {
|
||||
/// A child's set derived from the parent's. Used by the delegate at
|
||||
/// dispatch and by [`crate::recovery`] when it rebuilds a resumed frame.
|
||||
pub fn derive(inner: Arc<dyn ToolSet>, selection: &ToolSelection) -> Self {
|
||||
Self {
|
||||
inner,
|
||||
remove: selection.remove.clone(),
|
||||
add: selection.add.clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl ToolSet for FilteredToolSet {
|
||||
fn defs(&self, model: &crate::model::ModelInfo) -> Vec<Value> {
|
||||
let mut defs: Vec<Value> = self
|
||||
@@ -125,6 +147,255 @@ impl ToolSet for FilteredToolSet {
|
||||
}
|
||||
}
|
||||
|
||||
// ── Async delegation ─────────────────────────────────────────────────────────
|
||||
|
||||
/// What the host is asked to run out of band (blueprint §7.2).
|
||||
///
|
||||
/// The parent's turn does **not** wait for it: `delegate` returns a receipt and
|
||||
/// the loop moves on. Everything needed to run the work later is in here, so an
|
||||
/// executor backed by a durable queue can pick it up after a restart.
|
||||
#[derive(Clone)]
|
||||
pub struct AsyncSpec {
|
||||
pub conversation: ConversationId,
|
||||
/// The delegating frame — where the result is delivered.
|
||||
pub parent_frame: FrameId,
|
||||
/// The delegating call, so a host can correlate its own record with ours.
|
||||
pub parent_call: ToolCallId,
|
||||
/// The agent that delegated (the child's is `agent`).
|
||||
pub parent_agent: String,
|
||||
pub agent: String,
|
||||
pub prompt: String,
|
||||
pub title: Option<String>,
|
||||
pub description: Option<String>,
|
||||
/// The delegating turn's extensions (the host's own context).
|
||||
pub extensions: Extensions,
|
||||
}
|
||||
|
||||
/// The host's receipt for a submitted task.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct TaskHandle {
|
||||
pub id: TaskId,
|
||||
pub title: String,
|
||||
}
|
||||
|
||||
/// Runs a delegated task out of band. **Durability is the host's**: the crate's
|
||||
/// [`InProcessExecutor`] is lossy across restarts, a queue-backed one is not.
|
||||
#[async_trait]
|
||||
pub trait AsyncExecutor: Send + Sync {
|
||||
async fn submit(&self, spec: AsyncSpec) -> crate::Result<TaskHandle>;
|
||||
}
|
||||
|
||||
/// A task that finished, whatever ran it.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CompletedTask {
|
||||
pub id: TaskId,
|
||||
pub title: String,
|
||||
pub result: String,
|
||||
}
|
||||
|
||||
/// Where a finished task's result goes.
|
||||
#[async_trait]
|
||||
pub trait AsyncResultSink: Send + Sync {
|
||||
async fn deliver(&self, parent: ConversationId, task: CompletedTask) -> crate::Result<()>;
|
||||
}
|
||||
|
||||
/// The wire name of the synthetic call carrying a delivered result. The model
|
||||
/// sees it as a tool call it never made — which is exactly what it is: the
|
||||
/// system reporting back.
|
||||
pub const DELIVERY_CALL: &str = "task_completed";
|
||||
|
||||
/// The shipped sink: writes the delivery into the store, as a synthetic
|
||||
/// assistant message plus one completed call.
|
||||
///
|
||||
/// Durable by construction — it is a normal state transition, so the result is
|
||||
/// in the history the instant it lands, whether or not anything is driving the
|
||||
/// conversation. **Waking the parent is the host's job**: a live loop picks the
|
||||
/// result up on its own (it reads the store each round), and an idle
|
||||
/// conversation needs a resume, which only the host knows how to trigger for
|
||||
/// its surfaces. Wrap this sink to add that.
|
||||
pub struct StoreSink {
|
||||
store: Arc<dyn HistoryStore>,
|
||||
call_name: String,
|
||||
}
|
||||
|
||||
impl StoreSink {
|
||||
pub fn new(store: Arc<dyn HistoryStore>) -> Self {
|
||||
Self { store, call_name: DELIVERY_CALL.to_string() }
|
||||
}
|
||||
|
||||
/// Rename the synthetic call (hosts with their own legacy name).
|
||||
pub fn with_call_name(mut self, name: impl Into<String>) -> Self {
|
||||
self.call_name = name.into();
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl AsyncResultSink for StoreSink {
|
||||
async fn deliver(&self, parent: ConversationId, task: CompletedTask) -> crate::Result<()> {
|
||||
// The deepest active frame is where the conversation currently is: a
|
||||
// result delivered to a closed frame would never be read.
|
||||
let frame = self
|
||||
.store
|
||||
.deepest_active(&parent)
|
||||
.await?
|
||||
.ok_or_else(|| anyhow::anyhow!("deliver: no active frame on conversation {parent}"))?;
|
||||
|
||||
let reasoning = format!(
|
||||
"The system is notifying me that async task #{} ('{}') has completed. \
|
||||
Let me process the result via {}.",
|
||||
task.id, task.title, self.call_name,
|
||||
);
|
||||
let msg = self
|
||||
.store
|
||||
.append(
|
||||
frame.id,
|
||||
NewMessage {
|
||||
role: crate::store::Role::Assistant,
|
||||
content: String::new(),
|
||||
synthetic: true,
|
||||
reasoning: Some(reasoning),
|
||||
metadata: None,
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
|
||||
let call = self
|
||||
.store
|
||||
.append_call(msg, NewCall::new(&self.call_name, json!({ "task_id": task.id.get() })))
|
||||
.await?;
|
||||
let payload = json!({
|
||||
"task_id": task.id.get(),
|
||||
"title": task.title,
|
||||
"result": task.result,
|
||||
});
|
||||
self.store
|
||||
.resolve_call(call, &CallOutcome::Completed(ToolOutput::Text(payload.to_string())))
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// The lossy executor: runs the task on the current process, on the same
|
||||
/// manager, and delivers through the given sink.
|
||||
///
|
||||
/// **A restart loses in-flight tasks** — nothing records that the work was
|
||||
/// owed. Fine for a single-process host that treats async delegation as
|
||||
/// best-effort; a host that must not lose one wires an executor over its own
|
||||
/// durable queue (Skald: a `scheduled_jobs` row).
|
||||
pub struct InProcessExecutor {
|
||||
manager: Arc<LoopManager>,
|
||||
catalog: Arc<dyn AgentCatalog>,
|
||||
store: Arc<dyn HistoryStore>,
|
||||
sink: Arc<dyn AsyncResultSink>,
|
||||
tools: Arc<dyn ToolSet>,
|
||||
next_id: std::sync::atomic::AtomicI64,
|
||||
}
|
||||
|
||||
impl InProcessExecutor {
|
||||
pub fn new(
|
||||
manager: Arc<LoopManager>,
|
||||
catalog: Arc<dyn AgentCatalog>,
|
||||
store: Arc<dyn HistoryStore>,
|
||||
sink: Arc<dyn AsyncResultSink>,
|
||||
tools: Arc<dyn ToolSet>,
|
||||
) -> Self {
|
||||
Self { manager, catalog, store, sink, tools, next_id: std::sync::atomic::AtomicI64::new(1) }
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl AsyncExecutor for InProcessExecutor {
|
||||
async fn submit(&self, spec: AsyncSpec) -> crate::Result<TaskHandle> {
|
||||
let id = TaskId(self.next_id.fetch_add(1, std::sync::atomic::Ordering::Relaxed));
|
||||
let title = spec.title.clone().unwrap_or_else(|| spec.agent.clone());
|
||||
|
||||
// Its own frame, child of the delegating one: the task is a sub-agent
|
||||
// that nobody awaits.
|
||||
let parent = self
|
||||
.store
|
||||
.get_frame(spec.parent_frame)
|
||||
.await?
|
||||
.ok_or_else(|| anyhow::anyhow!("submit: parent frame not found"))?;
|
||||
let frame = self
|
||||
.store
|
||||
.open_frame(&spec.conversation, Some(spec.parent_frame), FrameSpec {
|
||||
agent: spec.agent.clone(),
|
||||
prompt: Some(spec.prompt.clone()),
|
||||
depth: parent.spec.depth + 1,
|
||||
// NOT the delegating call: that one is already resolved with the
|
||||
// receipt, and recovery must not try to complete it twice.
|
||||
parent_call: None,
|
||||
meta: Value::Null,
|
||||
})
|
||||
.await?;
|
||||
// The delegating call's context, minus its cancellation: the profile is
|
||||
// resolved against the turn that asked for the work.
|
||||
let ctx = ToolCtx {
|
||||
conversation: spec.conversation.clone(),
|
||||
frame: spec.parent_frame,
|
||||
agent: spec.parent_agent.clone(),
|
||||
call_id: spec.parent_call,
|
||||
cancel: tokio_util::sync::CancellationToken::new(),
|
||||
extensions: spec.extensions.clone(),
|
||||
};
|
||||
let profile = self.catalog.get(&spec.agent, frame, &ctx).await?;
|
||||
self.store.append(frame, NewMessage::agent(&spec.prompt)).await?;
|
||||
|
||||
let manager = self.manager.clone();
|
||||
let store = self.store.clone();
|
||||
let catalog = self.catalog.clone();
|
||||
let sink = self.sink.clone();
|
||||
let tools = profile.toolset.clone().unwrap_or_else(|| self.tools.clone());
|
||||
let task_title = title.clone();
|
||||
tokio::spawn(async move {
|
||||
let outcome = match manager
|
||||
.start_loop(LoopParams {
|
||||
conversation: spec.conversation.clone(),
|
||||
frame,
|
||||
parent_frame: Some(spec.parent_frame),
|
||||
agent: spec.agent.clone(),
|
||||
system: profile.context,
|
||||
tools,
|
||||
model_hint: profile.model.unwrap_or_default(),
|
||||
selector: profile.selector,
|
||||
// Detached from the parent turn: the point of async is that
|
||||
// the parent's /stop does not kill the background work.
|
||||
token: None,
|
||||
live_input: None,
|
||||
extensions: spec.extensions.clone(),
|
||||
meta: TurnMeta::default(),
|
||||
assembler: profile.assembler,
|
||||
})
|
||||
.await
|
||||
{
|
||||
Ok(handle) => handle.join().await,
|
||||
Err(e) => Err(anyhow::anyhow!("{e}")),
|
||||
};
|
||||
|
||||
catalog.on_child_closed(frame).await;
|
||||
let _ = store.close_frame(frame).await;
|
||||
|
||||
let result = match outcome {
|
||||
Ok(crate::kernel::TurnOutcome::Final { content, .. }) => content,
|
||||
Ok(crate::kernel::TurnOutcome::Cancelled) => "(cancelled)".to_string(),
|
||||
Ok(crate::kernel::TurnOutcome::Exhausted) => {
|
||||
"(no output: tool-call round budget exhausted)".to_string()
|
||||
}
|
||||
Err(e) => format!("(failed: {e})"),
|
||||
};
|
||||
if let Err(e) = sink
|
||||
.deliver(spec.conversation.clone(), CompletedTask { id, title: task_title, result })
|
||||
.await
|
||||
{
|
||||
tracing::error!(task = %id, "async task delivery failed: {e}");
|
||||
}
|
||||
});
|
||||
|
||||
Ok(TaskHandle { id, title })
|
||||
}
|
||||
}
|
||||
|
||||
// ── DelegateTool ─────────────────────────────────────────────────────────────
|
||||
|
||||
/// The shipped `delegate` tool. The parent loop simply awaits a slow tool —
|
||||
@@ -137,6 +408,8 @@ pub struct DelegateTool {
|
||||
max_depth: u32,
|
||||
name: String,
|
||||
definition_override: Option<Value>,
|
||||
/// `None` → `mode: "async"` is refused instead of silently running sync.
|
||||
async_exec: Option<Arc<dyn AsyncExecutor>>,
|
||||
}
|
||||
|
||||
impl DelegateTool {
|
||||
@@ -146,7 +419,23 @@ impl DelegateTool {
|
||||
store: Arc<dyn HistoryStore>,
|
||||
max_depth: u32,
|
||||
) -> Self {
|
||||
Self { manager, catalog, store, max_depth, name: "delegate".to_string(), definition_override: None }
|
||||
Self {
|
||||
manager,
|
||||
catalog,
|
||||
store,
|
||||
max_depth,
|
||||
name: "delegate".to_string(),
|
||||
definition_override: None,
|
||||
async_exec: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Wire `mode: "async"` to an executor. Without one the mode is refused —
|
||||
/// running it synchronously instead would block a turn that asked not to
|
||||
/// wait.
|
||||
pub fn with_async(mut self, exec: Arc<dyn AsyncExecutor>) -> Self {
|
||||
self.async_exec = Some(exec);
|
||||
self
|
||||
}
|
||||
|
||||
/// Register under a different wire name (Skald's legacy aliases
|
||||
@@ -182,6 +471,57 @@ impl DelegateTool {
|
||||
})
|
||||
}
|
||||
|
||||
/// Hands the work to the host and returns the receipt immediately. The
|
||||
/// result arrives later as its own call (see [`AsyncResultSink`]), so the
|
||||
/// model is told plainly not to poll for it.
|
||||
async fn run_async(
|
||||
&self,
|
||||
agent_id: &str,
|
||||
prompt: &str,
|
||||
args: &Value,
|
||||
ctx: &ToolCtx,
|
||||
) -> Result<ToolOutput, ToolFailure> {
|
||||
let Some(exec) = &self.async_exec else {
|
||||
return Err(ToolFailure::Failed(
|
||||
"delegate: async mode is not available in this session".to_string(),
|
||||
));
|
||||
};
|
||||
if agent_id == ctx.agent {
|
||||
return Err(ToolFailure::Failed(format!(
|
||||
"delegate: an agent cannot call itself (`{agent_id}`)"
|
||||
)));
|
||||
}
|
||||
|
||||
let handle = exec
|
||||
.submit(AsyncSpec {
|
||||
conversation: ctx.conversation.clone(),
|
||||
parent_frame: ctx.frame,
|
||||
parent_call: ctx.call_id,
|
||||
parent_agent: ctx.agent.clone(),
|
||||
agent: agent_id.to_string(),
|
||||
prompt: prompt.to_string(),
|
||||
title: args["title"].as_str().map(str::to_string),
|
||||
description: args["description"].as_str().map(str::to_string),
|
||||
extensions: ctx.extensions.clone(),
|
||||
})
|
||||
.await
|
||||
.map_err(|e| ToolFailure::Failed(format!("delegate: async submit failed: {e}")))?;
|
||||
|
||||
Ok(ToolOutput::Text(
|
||||
json!({
|
||||
"task_id": handle.id.get(),
|
||||
"status": "started",
|
||||
"message": format!(
|
||||
"Task {} ('{}') is running in the background. \
|
||||
The system will automatically deliver the result to this conversation when complete. \
|
||||
Do NOT poll for it. Continue the conversation normally.",
|
||||
handle.id, handle.title,
|
||||
),
|
||||
})
|
||||
.to_string(),
|
||||
))
|
||||
}
|
||||
|
||||
async fn run_sync(&self, agent_id: &str, prompt: &str, ctx: &ToolCtx) -> Result<ToolOutput, ToolFailure> {
|
||||
if agent_id == ctx.agent {
|
||||
return Err(ToolFailure::Failed(format!(
|
||||
@@ -218,7 +558,7 @@ impl DelegateTool {
|
||||
|
||||
// Profile AFTER the frame exists (frame-scoped pieces anchor to it).
|
||||
// On rejection the frame is closed so nothing dangles.
|
||||
let profile = match self.catalog.get(agent_id, child_frame).await {
|
||||
let profile = match self.catalog.get(agent_id, child_frame, ctx).await {
|
||||
Ok(p) => p,
|
||||
Err(e) => {
|
||||
let _ = self.store.close_frame(child_frame).await;
|
||||
@@ -258,11 +598,7 @@ impl DelegateTool {
|
||||
.extensions
|
||||
.get::<SharedToolSet>()
|
||||
.ok_or_else(|| ToolFailure::Failed("delegate: no ToolSet in extensions".into()))?;
|
||||
Arc::new(FilteredToolSet {
|
||||
inner: parent_tools.0.clone(),
|
||||
remove: profile.tools.remove.clone(),
|
||||
add: profile.tools.add.clone(),
|
||||
})
|
||||
Arc::new(FilteredToolSet::derive(parent_tools.0.clone(), &profile.tools))
|
||||
}
|
||||
};
|
||||
|
||||
@@ -361,11 +697,8 @@ impl Tool for DelegateTool {
|
||||
.ok_or_else(|| ToolFailure::Failed("delegate: missing required argument `prompt`".into()))?;
|
||||
|
||||
match args["mode"].as_str() {
|
||||
Some("async") => Err(ToolFailure::Failed(
|
||||
"delegate: async mode rides the host's AsyncExecutor, which is not wired on this path"
|
||||
.to_string(),
|
||||
)),
|
||||
_ => self.run_sync(agent_id, prompt, ctx).await,
|
||||
Some("async") => self.run_async(agent_id, prompt, &args, ctx).await,
|
||||
_ => self.run_sync(agent_id, prompt, ctx).await,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -399,7 +732,12 @@ impl Default for StaticCatalog {
|
||||
|
||||
#[async_trait]
|
||||
impl AgentCatalog for StaticCatalog {
|
||||
async fn get(&self, id: &str, _child_frame: FrameId) -> crate::Result<AgentProfile> {
|
||||
async fn get(
|
||||
&self,
|
||||
id: &str,
|
||||
_child_frame: FrameId,
|
||||
_ctx: &ToolCtx,
|
||||
) -> crate::Result<AgentProfile> {
|
||||
self.profiles
|
||||
.iter()
|
||||
.find(|p| p.id == id)
|
||||
|
||||
@@ -86,12 +86,7 @@ pub(crate) async fn run(
|
||||
// ToolCtx extensions: host extensions + the event sink + the turn's tool
|
||||
// set, so shipped tools (ask_user, activate_tools, delegate) reach what
|
||||
// they need.
|
||||
let tool_extensions = || {
|
||||
let mut ext = params.extensions.clone();
|
||||
ext.insert(Arc::new(events.clone()));
|
||||
ext.insert(Arc::new(crate::tool::SharedToolSet(params.tools.clone())));
|
||||
ext
|
||||
};
|
||||
let tool_extensions = || tool_extensions(¶ms, &events);
|
||||
|
||||
events.emit(frame, parent, LoopEvent::TurnStarted);
|
||||
|
||||
@@ -289,6 +284,20 @@ pub(crate) async fn run(
|
||||
finish(TurnOutcome::Exhausted, &deps, &hook_ctx(), &events, frame, parent).await
|
||||
}
|
||||
|
||||
/// What a tool call sees: the host's extensions plus the event sink and the
|
||||
/// turn's tool set (shipped tools — ask_user, activate_tools, delegate — reach
|
||||
/// what they need through them). Shared with [`crate::recovery`], which
|
||||
/// re-executes a call outside a round and must hand it the same context.
|
||||
pub(crate) fn tool_extensions(
|
||||
params: &LoopParams,
|
||||
events: &EventSink,
|
||||
) -> crate::tool::Extensions {
|
||||
let mut ext = params.extensions.clone();
|
||||
ext.insert(Arc::new(events.clone()));
|
||||
ext.insert(Arc::new(crate::tool::SharedToolSet(params.tools.clone())));
|
||||
ext
|
||||
}
|
||||
|
||||
/// Terminal helper: hooks.on_turn_end (+ Cancelled event) then return.
|
||||
async fn finish(
|
||||
outcome: TurnOutcome,
|
||||
@@ -510,7 +519,7 @@ async fn record_call(
|
||||
})
|
||||
}
|
||||
|
||||
enum PreExecution {
|
||||
pub(crate) enum PreExecution {
|
||||
Run(Arc<dyn crate::tool::Tool>),
|
||||
Resolved(CallOutcome),
|
||||
TurnCancelled,
|
||||
@@ -519,8 +528,9 @@ enum PreExecution {
|
||||
Suspended,
|
||||
}
|
||||
|
||||
/// Gate + hooks.pre + tool lookup — shared by sequential and fan-out paths.
|
||||
async fn pre_execution(
|
||||
/// Gate + hooks.pre + tool lookup — shared by the sequential path, the
|
||||
/// fan-out and [`crate::recovery`]'s re-execution of an interrupted call.
|
||||
pub(crate) async fn pre_execution(
|
||||
deps: &Arc<KernelDeps>,
|
||||
params: &LoopParams,
|
||||
events: &EventSink,
|
||||
@@ -572,8 +582,8 @@ async fn pre_execution(
|
||||
}
|
||||
}
|
||||
|
||||
/// Phase-3 shared by both paths: hooks.post → resolve → emit.
|
||||
async fn record_outcome(
|
||||
/// Phase-3 shared by both paths (and by recovery): hooks.post → resolve → emit.
|
||||
pub(crate) async fn record_outcome(
|
||||
deps: &Arc<KernelDeps>,
|
||||
params: &LoopParams,
|
||||
events: &EventSink,
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
//! Design document: `blueprint/project-loop.md` (Skald workspace).
|
||||
|
||||
pub mod activation;
|
||||
pub mod compaction;
|
||||
pub mod context;
|
||||
pub mod delegate;
|
||||
pub mod events;
|
||||
@@ -23,6 +24,8 @@ pub mod kernel;
|
||||
pub mod manager;
|
||||
pub mod model;
|
||||
pub mod models;
|
||||
pub mod projection;
|
||||
pub mod recovery;
|
||||
pub mod store;
|
||||
pub mod store_memory;
|
||||
pub mod testing;
|
||||
@@ -43,13 +46,17 @@ pub mod prelude {
|
||||
pub use crate::activation::{
|
||||
ActivateToolsTool, Activation, ActivationSource, ToolActivator, ToolRendering,
|
||||
};
|
||||
pub use crate::compaction::{
|
||||
Compaction, CompactionMode, CompactionOutcome, CompactionPrompt, should_compact,
|
||||
};
|
||||
pub use crate::context::{
|
||||
AssembleInput, ContextAssembler, LinearAssembler, StaticSystemContext, SystemContext,
|
||||
SystemContextSource, TurnInfo,
|
||||
};
|
||||
pub use crate::delegate::{
|
||||
AgentCatalog, AgentKind, AgentProfile, AgentSummary, DelegateTool, FilteredToolSet,
|
||||
StaticCatalog, ToolSelection,
|
||||
AgentCatalog, AgentKind, AgentProfile, AgentSummary, AsyncExecutor, AsyncResultSink,
|
||||
AsyncSpec, CompletedTask, DelegateTool, FilteredToolSet, InProcessExecutor, StaticCatalog,
|
||||
StoreSink, TaskHandle, ToolSelection,
|
||||
};
|
||||
pub use crate::events::{DeltaKind, Event, EventSink, LoopEvent};
|
||||
pub use crate::gate::{AllowAll, DenyList, Gate, GateDecision, PendingCall};
|
||||
@@ -67,6 +74,13 @@ pub mod prelude {
|
||||
ModelSelector, RawMeta, RetryPolicy, SingleModel, StaticModels, StreamDelta, ToolCall,
|
||||
Usage,
|
||||
};
|
||||
pub use crate::recovery::{
|
||||
HumanDecision, PendingPolicy, Recovery, RecoveryPolicy, RecoveryReport, RunningPolicy,
|
||||
};
|
||||
pub use crate::projection::{
|
||||
MediaBlob, MediaBudget, MediaKind, MediaSource, Projection, ProjectionHooks,
|
||||
ReasoningEcho, ResultLimit, ToolResultDigest,
|
||||
};
|
||||
pub use crate::store::{
|
||||
CallOutcome, CallState, FrameRecord, FrameSpec, HistoryStore, NewCall, NewMessage,
|
||||
NewSummary, Role, StoredCall, StoredMessage, StoredSummary,
|
||||
|
||||
@@ -58,6 +58,10 @@ pub struct TurnParams {
|
||||
/// Already filtered (visibility/approval).
|
||||
pub tools: Arc<dyn ToolSet>,
|
||||
pub model_hint: ModelHint,
|
||||
/// Per-turn selector override — e.g. this agent's required strength, which
|
||||
/// is host policy (D14) and varies turn to turn while the manager lives as
|
||||
/// long as the tenant. `None` = the manager's.
|
||||
pub selector: Option<Arc<dyn ModelSelector>>,
|
||||
/// None for sub-agents / cron / resume.
|
||||
pub live_input: Option<Arc<dyn LiveInput>>,
|
||||
/// Flows into `ToolCtx.extensions`.
|
||||
@@ -139,6 +143,29 @@ struct RunningEntry {
|
||||
cancel: CancellationToken,
|
||||
}
|
||||
|
||||
/// Holds a conversation in the live registry for work that is not one spawned
|
||||
/// loop (see [`LoopManager::claim`]). Releases on drop, including on an early
|
||||
/// return or a panic — a leaked claim would lock the conversation for the
|
||||
/// process's lifetime.
|
||||
pub(crate) struct ConversationClaim {
|
||||
conversation: ConversationId,
|
||||
registry: Arc<Mutex<HashMap<ConversationId, RunningEntry>>>,
|
||||
token: CancellationToken,
|
||||
}
|
||||
|
||||
impl ConversationClaim {
|
||||
/// The claim's cancellation token — `/stop` cancels it through the registry.
|
||||
pub(crate) fn token(&self) -> CancellationToken {
|
||||
self.token.clone()
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for ConversationClaim {
|
||||
fn drop(&mut self) {
|
||||
self.registry.lock().unwrap().remove(&self.conversation);
|
||||
}
|
||||
}
|
||||
|
||||
// ── LoopManager ──────────────────────────────────────────────────────────────
|
||||
|
||||
pub struct LoopManager {
|
||||
@@ -212,7 +239,7 @@ impl LoopManager {
|
||||
system: params.system,
|
||||
tools: params.tools,
|
||||
model_hint: params.model_hint,
|
||||
selector: None,
|
||||
selector: params.selector,
|
||||
token: None,
|
||||
live_input: params.live_input,
|
||||
extensions: params.extensions,
|
||||
@@ -287,6 +314,108 @@ impl LoopManager {
|
||||
self.registry.lock().unwrap().contains_key(conv)
|
||||
}
|
||||
|
||||
/// Take the conversation for something that is not a single spawned loop —
|
||||
/// a recovery pass, an out-of-band tool resolution. `None` when another
|
||||
/// loop already holds it (anti double-driving, same rule as `start_turn`).
|
||||
///
|
||||
/// The claim registers in the live registry, so `/stop` cancels it and
|
||||
/// `list_running` shows it; dropping the guard releases it.
|
||||
pub(crate) fn claim(
|
||||
&self,
|
||||
conv: &ConversationId,
|
||||
frame: FrameId,
|
||||
agent: &str,
|
||||
) -> Option<ConversationClaim> {
|
||||
let token = CancellationToken::new();
|
||||
let mut registry = self.registry.lock().unwrap();
|
||||
if registry.contains_key(conv) {
|
||||
return None;
|
||||
}
|
||||
registry.insert(conv.clone(), RunningEntry {
|
||||
frame,
|
||||
agent: agent.to_string(),
|
||||
cancel: token.clone(),
|
||||
});
|
||||
Some(ConversationClaim {
|
||||
conversation: conv.clone(),
|
||||
registry: self.registry.clone(),
|
||||
token,
|
||||
})
|
||||
}
|
||||
|
||||
// ── recovery (blueprint §8) ──
|
||||
|
||||
/// A [`Recovery`](crate::recovery::Recovery) bound to this manager.
|
||||
pub fn recovery(
|
||||
self: &Arc<Self>,
|
||||
catalog: Arc<dyn crate::delegate::AgentCatalog>,
|
||||
policy: crate::recovery::RecoveryPolicy,
|
||||
) -> crate::recovery::Recovery {
|
||||
crate::recovery::Recovery::new(self.clone(), catalog, policy)
|
||||
}
|
||||
|
||||
/// Resume a conversation left mid-turn: recovery with the default policy.
|
||||
pub async fn resume(
|
||||
self: &Arc<Self>,
|
||||
conv: &ConversationId,
|
||||
catalog: Arc<dyn crate::delegate::AgentCatalog>,
|
||||
root: &TurnParams,
|
||||
) -> crate::Result<crate::recovery::RecoveryReport> {
|
||||
self.recovery(catalog, crate::recovery::RecoveryPolicy::default())
|
||||
.run(conv, root)
|
||||
.await
|
||||
}
|
||||
|
||||
/// Resolve a call a human answered out of band — the approval card clicked
|
||||
/// after a restart, when no loop is left holding the oneshot.
|
||||
///
|
||||
/// On approval the tool runs with the **gate skipped**: the human just
|
||||
/// decided, and asking the rules again would either re-prompt or overturn
|
||||
/// them. The conversation is then recovered, so the model sees the result
|
||||
/// and continues.
|
||||
pub async fn resolve_pending(
|
||||
self: &Arc<Self>,
|
||||
call: crate::ids::ToolCallId,
|
||||
decision: crate::recovery::HumanDecision,
|
||||
catalog: Arc<dyn crate::delegate::AgentCatalog>,
|
||||
root: &TurnParams,
|
||||
) -> crate::Result<crate::recovery::RecoveryReport> {
|
||||
crate::recovery::resolve_pending(self, call, decision, catalog, root).await
|
||||
}
|
||||
|
||||
// ── compaction (blueprint §9) ──
|
||||
|
||||
/// A [`Compaction`](crate::compaction::Compaction) on one frame, sharing
|
||||
/// this manager's store, hooks and event bus. Configure it with the
|
||||
/// builder methods, then `run()`.
|
||||
pub fn new_compaction(
|
||||
&self,
|
||||
conv: ConversationId,
|
||||
frame: FrameId,
|
||||
) -> crate::compaction::Compaction {
|
||||
crate::compaction::Compaction {
|
||||
store: self.deps.store.clone(),
|
||||
selector: self.deps.models.clone(),
|
||||
hooks: self.deps.hooks.clone(),
|
||||
events: self.sink(conv.clone()),
|
||||
conversation: conv,
|
||||
frame,
|
||||
mode: crate::compaction::CompactionMode::default(),
|
||||
hint: ModelHint::default(),
|
||||
prompt: Arc::new(crate::compaction::DefaultPrompt),
|
||||
temperature: None,
|
||||
log: None,
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn deps(&self) -> &Arc<KernelDeps> {
|
||||
&self.deps
|
||||
}
|
||||
|
||||
pub(crate) fn sink_for(&self, conv: ConversationId) -> EventSink {
|
||||
self.sink(conv)
|
||||
}
|
||||
|
||||
/// Global view (UI "running agents").
|
||||
pub fn list_running(&self) -> Vec<RunningInfo> {
|
||||
self.registry
|
||||
|
||||
@@ -0,0 +1,416 @@
|
||||
//! The wire half of multimodal media: which files a model can take, in which
|
||||
//! content-part shape, within which budgets.
|
||||
//!
|
||||
//! The host supplies **blobs** it has already authorized (containment, upload
|
||||
//! rules, ownership — its policy); this module decides whether a blob reaches
|
||||
//! the model and in what shape. The split is deliberate: the part shapes and
|
||||
//! the byte ceilings are protocol (`MAX_DOCUMENT_BYTES` is literally
|
||||
//! Anthropic's per-request document ceiling), the authorization is not.
|
||||
//!
|
||||
//! Promotion is strict: a blob is inlined only when the model declares the
|
||||
//! modality's capability, the **sniffed magic bytes** match an allowed MIME (a
|
||||
//! host-claimed MIME is never trusted — there is no seam to pass one), and the
|
||||
//! per-file / per-turn budgets hold. Anything failing a check is reported back
|
||||
//! as skipped so the host can keep it on its textual path.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use base64::Engine as _;
|
||||
use serde_json::{Value, json};
|
||||
use tracing::debug;
|
||||
|
||||
/// Max media parts inlined per turn.
|
||||
pub const MAX_MEDIA_PER_TURN: usize = 4;
|
||||
/// Max bytes for one inlined image.
|
||||
pub const MAX_IMAGE_BYTES: u64 = 10 * 1024 * 1024;
|
||||
/// Max bytes for one inlined video.
|
||||
pub const MAX_VIDEO_BYTES: u64 = 32 * 1024 * 1024;
|
||||
/// Max bytes for one inlined document (Anthropic's per-request ceiling).
|
||||
pub const MAX_DOCUMENT_BYTES: u64 = 32 * 1024 * 1024;
|
||||
/// Max combined media bytes inlined per turn.
|
||||
pub const MAX_TOTAL_MEDIA_BYTES: u64 = 48 * 1024 * 1024;
|
||||
|
||||
// ── MediaKind ────────────────────────────────────────────────────────────────
|
||||
|
||||
/// A model-input modality: the capability that unlocks it and the content-part
|
||||
/// shape it maps to.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum MediaKind {
|
||||
Image,
|
||||
Video,
|
||||
/// PDFs, as the OpenAI file-input part (`{"type":"file","file":{…}}`) —
|
||||
/// forwarded verbatim by OpenAI-compatible clients and translated to a
|
||||
/// native `document` block by the Anthropic client.
|
||||
Document,
|
||||
}
|
||||
|
||||
impl MediaKind {
|
||||
/// The `ModelInfo::capabilities` entry that unlocks this modality.
|
||||
pub fn capability(self) -> &'static str {
|
||||
match self {
|
||||
Self::Image => "vision",
|
||||
Self::Video => "video",
|
||||
Self::Document => "document",
|
||||
}
|
||||
}
|
||||
|
||||
/// The OpenAI content-part type.
|
||||
pub fn part_type(self) -> &'static str {
|
||||
match self {
|
||||
Self::Image => "image_url",
|
||||
Self::Video => "video_url",
|
||||
Self::Document => "file",
|
||||
}
|
||||
}
|
||||
|
||||
/// Human-readable format list (hosts use it in tool descriptions).
|
||||
pub fn formats(self) -> &'static str {
|
||||
match self {
|
||||
Self::Image => "images (PNG, JPEG, GIF, WebP)",
|
||||
Self::Video => "video (MP4, WebM, MOV, …)",
|
||||
Self::Document => "PDF documents",
|
||||
}
|
||||
}
|
||||
|
||||
/// The modality a sniffed MIME belongs to.
|
||||
pub fn for_mime(mime: &str) -> Option<Self> {
|
||||
match mime {
|
||||
"image/png" | "image/jpeg" | "image/gif" | "image/webp" => Some(Self::Image),
|
||||
"video/mp4" | "video/mpeg" | "video/quicktime" | "video/webm" | "video/x-msvideo"
|
||||
| "video/x-flv" | "video/3gpp" => Some(Self::Video),
|
||||
"application/pdf" => Some(Self::Document),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// The modalities a model with these capabilities can take, in a stable order.
|
||||
pub fn enabled(capabilities: &[String]) -> Vec<Self> {
|
||||
[Self::Image, Self::Video, Self::Document]
|
||||
.into_iter()
|
||||
.filter(|k| capabilities.iter().any(|c| c == k.capability()))
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
|
||||
// ── MediaBudget ──────────────────────────────────────────────────────────────
|
||||
|
||||
/// Per-file and per-turn ceilings.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct MediaBudget {
|
||||
pub max_per_turn: usize,
|
||||
pub max_image_bytes: u64,
|
||||
pub max_video_bytes: u64,
|
||||
pub max_document_bytes: u64,
|
||||
pub max_total_bytes: u64,
|
||||
}
|
||||
|
||||
impl Default for MediaBudget {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
max_per_turn: MAX_MEDIA_PER_TURN,
|
||||
max_image_bytes: MAX_IMAGE_BYTES,
|
||||
max_video_bytes: MAX_VIDEO_BYTES,
|
||||
max_document_bytes: MAX_DOCUMENT_BYTES,
|
||||
max_total_bytes: MAX_TOTAL_MEDIA_BYTES,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl MediaBudget {
|
||||
pub fn max_bytes(&self, kind: MediaKind) -> u64 {
|
||||
match kind {
|
||||
MediaKind::Image => self.max_image_bytes,
|
||||
MediaKind::Video => self.max_video_bytes,
|
||||
MediaKind::Document => self.max_document_bytes,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ── MediaBlob ────────────────────────────────────────────────────────────────
|
||||
|
||||
/// A candidate medium the host has already authorized. Reads are lazy so a
|
||||
/// blob rejected on capability or size is never fully loaded.
|
||||
#[async_trait]
|
||||
pub trait MediaBlob: Send + Sync {
|
||||
/// Display name (the `filename` of a `file` part).
|
||||
fn name(&self) -> &str;
|
||||
/// Byte length; `None` (unknown) means "do not inline".
|
||||
async fn size(&self) -> Option<u64>;
|
||||
/// The first bytes, for magic-byte sniffing (16 are enough).
|
||||
async fn head(&self) -> Option<Vec<u8>>;
|
||||
/// The whole content.
|
||||
async fn read_all(&self) -> Option<Vec<u8>>;
|
||||
}
|
||||
|
||||
// ── projection ───────────────────────────────────────────────────────────────
|
||||
|
||||
/// The OpenAI-wire content part for one inlined medium.
|
||||
pub fn media_part(kind: MediaKind, mime: &str, bytes: &[u8], filename: &str) -> Value {
|
||||
let b64 = base64::engine::general_purpose::STANDARD.encode(bytes);
|
||||
let url = format!("data:{mime};base64,{b64}");
|
||||
match kind {
|
||||
MediaKind::Document => {
|
||||
json!({ "type": "file", "file": { "filename": filename, "file_data": url } })
|
||||
}
|
||||
k => {
|
||||
let t = k.part_type();
|
||||
json!({ "type": t, t: { "url": url } })
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Splits blobs into inline content parts and the indices left out.
|
||||
///
|
||||
/// Skipped blobs are the host's business: it typically renders them as a
|
||||
/// textual path list so the agent can still read them with a tool.
|
||||
pub async fn partition(
|
||||
blobs: &[Arc<dyn MediaBlob>],
|
||||
capabilities: &[String],
|
||||
budget: &MediaBudget,
|
||||
) -> (Vec<Value>, Vec<usize>) {
|
||||
if blobs.is_empty() {
|
||||
return (Vec::new(), Vec::new());
|
||||
}
|
||||
if MediaKind::enabled(capabilities).is_empty() {
|
||||
return (Vec::new(), (0..blobs.len()).collect());
|
||||
}
|
||||
|
||||
let mut parts: Vec<Value> = Vec::new();
|
||||
let mut skipped: Vec<usize> = Vec::new();
|
||||
let mut total: u64 = 0;
|
||||
|
||||
for (idx, blob) in blobs.iter().enumerate() {
|
||||
if parts.len() >= budget.max_per_turn {
|
||||
debug!(name = blob.name(), "media not inlined: per-turn count budget exhausted");
|
||||
skipped.push(idx);
|
||||
continue;
|
||||
}
|
||||
match promote(blob.as_ref(), capabilities, budget, total).await {
|
||||
Some((part, bytes)) => {
|
||||
total += bytes;
|
||||
parts.push(part);
|
||||
}
|
||||
None => skipped.push(idx),
|
||||
}
|
||||
}
|
||||
(parts, skipped)
|
||||
}
|
||||
|
||||
/// Sniff + capability + budget + build, for one blob. `None` (logged at debug)
|
||||
/// when it is not a recognized medium, the model lacks the modality, or a byte
|
||||
/// budget is exhausted. The per-turn **count** budget is the caller's.
|
||||
async fn promote(
|
||||
blob: &dyn MediaBlob,
|
||||
capabilities: &[String],
|
||||
budget: &MediaBudget,
|
||||
used_total: u64,
|
||||
) -> Option<(Value, u64)> {
|
||||
let head = blob.head().await?;
|
||||
let mime = sniff_mime(&head)?;
|
||||
let kind = MediaKind::for_mime(mime)?;
|
||||
if !capabilities.iter().any(|c| c == kind.capability()) {
|
||||
debug!(name = blob.name(), mime, "media not inlined: model lacks the capability");
|
||||
return None;
|
||||
}
|
||||
|
||||
let size = blob.size().await?;
|
||||
if size > budget.max_bytes(kind) {
|
||||
debug!(name = blob.name(), size, "media not inlined: file too large");
|
||||
return None;
|
||||
}
|
||||
if used_total + size > budget.max_total_bytes {
|
||||
debug!(name = blob.name(), "media not inlined: per-turn byte budget exhausted");
|
||||
return None;
|
||||
}
|
||||
|
||||
let bytes = blob.read_all().await?;
|
||||
Some((media_part(kind, mime, &bytes, blob.name()), size))
|
||||
}
|
||||
|
||||
/// Sniffs the magic bytes of a medium we know how to inline, returning its
|
||||
/// canonical MIME type. `None` = not a recognized medium (not an error —
|
||||
/// ordinary files simply are not model input).
|
||||
pub fn sniff_mime(head: &[u8]) -> Option<&'static str> {
|
||||
if head.starts_with(b"\x89PNG\r\n\x1a\n") {
|
||||
return Some("image/png");
|
||||
}
|
||||
if head.starts_with(b"\xff\xd8\xff") {
|
||||
return Some("image/jpeg");
|
||||
}
|
||||
if head.starts_with(b"GIF87a") || head.starts_with(b"GIF89a") {
|
||||
return Some("image/gif");
|
||||
}
|
||||
if head.len() >= 12 && &head[0..4] == b"RIFF" && &head[8..12] == b"WEBP" {
|
||||
return Some("image/webp");
|
||||
}
|
||||
if head.len() >= 12 && &head[4..8] == b"ftyp" {
|
||||
let brand = &head[8..12];
|
||||
if brand.starts_with(b"3gp") || brand.starts_with(b"3g2") {
|
||||
return Some("video/3gpp");
|
||||
}
|
||||
if brand == b"qt " {
|
||||
return Some("video/quicktime");
|
||||
}
|
||||
// isom / mp41 / mp42 / avc1 / M4V …
|
||||
return Some("video/mp4");
|
||||
}
|
||||
// EBML header — WebM (and Matroska, close enough for the video models).
|
||||
if head.starts_with(&[0x1A, 0x45, 0xDF, 0xA3]) {
|
||||
return Some("video/webm");
|
||||
}
|
||||
if head.len() >= 12 && &head[0..4] == b"RIFF" && &head[8..12] == b"AVI " {
|
||||
return Some("video/x-msvideo");
|
||||
}
|
||||
if head.starts_with(b"FLV\x01") {
|
||||
return Some("video/x-flv");
|
||||
}
|
||||
if head.starts_with(&[0x00, 0x00, 0x01, 0xBA]) || head.starts_with(&[0x00, 0x00, 0x01, 0xB3]) {
|
||||
return Some("video/mpeg");
|
||||
}
|
||||
if head.starts_with(b"%PDF-") {
|
||||
return Some("application/pdf");
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// An in-memory blob.
|
||||
struct Blob {
|
||||
name: String,
|
||||
bytes: Vec<u8>,
|
||||
}
|
||||
|
||||
/// A blob as the trait object the engine takes.
|
||||
fn blob(name: &str, bytes: Vec<u8>) -> Arc<dyn MediaBlob> {
|
||||
Arc::new(Blob { name: name.to_string(), bytes })
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl MediaBlob for Blob {
|
||||
fn name(&self) -> &str { &self.name }
|
||||
async fn size(&self) -> Option<u64> { Some(self.bytes.len() as u64) }
|
||||
async fn head(&self) -> Option<Vec<u8>> {
|
||||
Some(self.bytes.iter().copied().take(16).collect())
|
||||
}
|
||||
async fn read_all(&self) -> Option<Vec<u8>> { Some(self.bytes.clone()) }
|
||||
}
|
||||
|
||||
fn png() -> Vec<u8> {
|
||||
let mut v = b"\x89PNG\r\n\x1a\n".to_vec();
|
||||
v.extend_from_slice(&[0xAA; 64]);
|
||||
v
|
||||
}
|
||||
|
||||
fn pdf() -> Vec<u8> {
|
||||
let mut v = b"%PDF-1.7\n".to_vec();
|
||||
v.extend_from_slice(&[0x00; 64]);
|
||||
v
|
||||
}
|
||||
|
||||
fn caps(xs: &[&str]) -> Vec<String> {
|
||||
xs.iter().map(|s| s.to_string()).collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sniff_known_signatures() {
|
||||
assert_eq!(sniff_mime(b"\x89PNG\r\n\x1a\n...."), Some("image/png"));
|
||||
assert_eq!(sniff_mime(b"\xff\xd8\xff\xe0...."), Some("image/jpeg"));
|
||||
assert_eq!(sniff_mime(b"GIF89a...."), Some("image/gif"));
|
||||
assert_eq!(sniff_mime(b"RIFF\x00\x00\x00\x00WEBP"), Some("image/webp"));
|
||||
assert_eq!(sniff_mime(b"\x00\x00\x00\x18ftypisom"), Some("video/mp4"));
|
||||
assert_eq!(sniff_mime(b"\x00\x00\x00\x18ftypqt "), Some("video/quicktime"));
|
||||
assert_eq!(sniff_mime(b"\x00\x00\x00\x18ftyp3gp4"), Some("video/3gpp"));
|
||||
assert_eq!(sniff_mime(&[0x1A, 0x45, 0xDF, 0xA3, 0, 0]), Some("video/webm"));
|
||||
assert_eq!(sniff_mime(b"RIFF\x00\x00\x00\x00AVI "), Some("video/x-msvideo"));
|
||||
assert_eq!(sniff_mime(b"FLV\x01\x05"), Some("video/x-flv"));
|
||||
assert_eq!(sniff_mime(&[0x00, 0x00, 0x01, 0xBA]), Some("video/mpeg"));
|
||||
assert_eq!(sniff_mime(b"%PDF-1.7"), Some("application/pdf"));
|
||||
assert_eq!(sniff_mime(b""), None);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn inlines_png_for_a_vision_model() {
|
||||
let (parts, skipped) =
|
||||
partition(&[blob("a.png", png())], &caps(&["vision"]), &MediaBudget::default()).await;
|
||||
assert!(skipped.is_empty());
|
||||
assert_eq!(parts.len(), 1);
|
||||
assert_eq!(parts[0]["type"], "image_url");
|
||||
assert!(
|
||||
parts[0]["image_url"]["url"].as_str().unwrap().starts_with("data:image/png;base64,")
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn inlines_pdf_as_a_file_part_for_a_document_model() {
|
||||
let (parts, skipped) =
|
||||
partition(&[blob("a.pdf", pdf())], &caps(&["document"]), &MediaBudget::default()).await;
|
||||
assert!(skipped.is_empty());
|
||||
assert_eq!(parts[0]["type"], "file");
|
||||
assert_eq!(parts[0]["file"]["filename"], "a.pdf");
|
||||
assert!(
|
||||
parts[0]["file"]["file_data"].as_str().unwrap().starts_with("data:application/pdf;base64,")
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn gates_on_capability_per_modality() {
|
||||
let b = |bytes: Vec<u8>| vec![blob("x", bytes)];
|
||||
let budget = MediaBudget::default();
|
||||
|
||||
// No capability at all.
|
||||
let (parts, skipped) = partition(&b(png()), &caps(&[]), &budget).await;
|
||||
assert!(parts.is_empty() && skipped == vec![0]);
|
||||
|
||||
// vision does not unlock PDFs, document does not unlock images.
|
||||
let (parts, skipped) = partition(&b(pdf()), &caps(&["vision"]), &budget).await;
|
||||
assert!(parts.is_empty() && skipped == vec![0]);
|
||||
let (parts, skipped) = partition(&b(png()), &caps(&["document"]), &budget).await;
|
||||
assert!(parts.is_empty() && skipped == vec![0]);
|
||||
|
||||
// An unrecognized medium is never inlined.
|
||||
let (parts, skipped) = partition(&b(b"plain text".to_vec()), &caps(&["vision"]), &budget).await;
|
||||
assert!(parts.is_empty() && skipped == vec![0]);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn enforces_count_per_file_and_total_budgets() {
|
||||
let budget = MediaBudget::default();
|
||||
let blobs: Vec<Arc<dyn MediaBlob>> = (0..budget.max_per_turn + 2)
|
||||
.map(|i| blob(&format!("{i}.png"), png()))
|
||||
.collect();
|
||||
let (parts, skipped) = partition(&blobs, &caps(&["vision"]), &budget).await;
|
||||
assert_eq!(parts.len(), budget.max_per_turn);
|
||||
assert_eq!(skipped.len(), 2);
|
||||
|
||||
// Per-file ceiling.
|
||||
let tight = MediaBudget { max_image_bytes: 8, ..MediaBudget::default() };
|
||||
let (parts, skipped) = partition(&[blob("a.png", png())], &caps(&["vision"]), &tight).await;
|
||||
assert!(parts.is_empty() && skipped == vec![0]);
|
||||
|
||||
// Per-turn total: the first fits, the second does not.
|
||||
let total = MediaBudget { max_total_bytes: 100, ..MediaBudget::default() };
|
||||
let (parts, skipped) = partition(
|
||||
&[blob("a.png", png()), blob("b.png", png())],
|
||||
&caps(&["vision"]),
|
||||
&total,
|
||||
)
|
||||
.await;
|
||||
assert_eq!(parts.len(), 1);
|
||||
assert_eq!(skipped, vec![1]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn enabled_modalities_are_capability_driven() {
|
||||
assert!(MediaKind::enabled(&caps(&[])).is_empty());
|
||||
assert_eq!(MediaKind::enabled(&caps(&["vision"])), vec![MediaKind::Image]);
|
||||
assert_eq!(
|
||||
MediaKind::enabled(&caps(&["document", "vision"])),
|
||||
vec![MediaKind::Image, MediaKind::Document],
|
||||
"the order is the enum's, not the capability list's"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,609 @@
|
||||
//! The projection: stored history → wire messages. **This is where provider
|
||||
//! divergence lives**, so it belongs to the crate rather than to any host.
|
||||
//!
|
||||
//! What the crate owns here: the shape of every message (string content vs
|
||||
//! content-part array, `cache_control` placement, `tool_calls`/`tool` shapes,
|
||||
//! media parts), the well-formedness rules (a result for every tool call, no
|
||||
//! orphans, role alternation, boundary-safe windowing), the dynamic-tool-loading
|
||||
//! injections, and the byte fidelity of what goes back on the wire.
|
||||
//!
|
||||
//! What the host owns: the **content** — the system prompt layers
|
||||
//! ([`crate::context::SystemContextSource`]), which media a message may inline
|
||||
//! ([`MediaSource`]) and how an over-long tool result is condensed
|
||||
//! ([`ToolResultDigest`]). Everything is optional: with no hooks at all the
|
||||
//! projection is a complete, correct OpenAI-shaped conversation.
|
||||
//!
|
||||
//! **Well-formedness contract** (the reason a resumed turn can just re-run):
|
||||
//!
|
||||
//! 1. Order: static system → extra static → summary → history after
|
||||
//! `covered_up_to` → dynamic tail → tail reminder.
|
||||
//! 2. Every assistant `tool_call` has a tool result: `Done` → the result,
|
||||
//! `Failed` → an error, `Cancelled`/`Rejected` → a note, and a `Running` /
|
||||
//! `AwaitingHuman` call that survived a crash → a synthetic "interrupted"
|
||||
//! result. A model must never see a call it gets no answer for.
|
||||
//! 3. No `failed` messages (orphans of cancelled turns) — the store filters them.
|
||||
//! 4. DTL injections are **append-only**: the cacheable prefix stays
|
||||
//! byte-identical, so activating a tool never invalidates the prompt cache.
|
||||
|
||||
pub mod media;
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Arc;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use serde_json::{Value, json};
|
||||
|
||||
use crate::activation::{Activation, ActivationSource, ToolRendering};
|
||||
use crate::context::AssembleInput;
|
||||
use crate::ids::MessageId;
|
||||
use crate::store::{CallState, HistoryStore, Role, StoredCall, StoredMessage};
|
||||
|
||||
pub use media::{MediaBlob, MediaBudget, MediaKind};
|
||||
|
||||
// ── Configuration ────────────────────────────────────────────────────────────
|
||||
|
||||
/// How a stored `reasoning_content` is echoed back.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||||
pub enum ReasoningEcho {
|
||||
/// `reasoning_content` only (DeepSeek).
|
||||
#[default]
|
||||
ContentOnly,
|
||||
/// Both `reasoning_content` and `reasoning` — some OpenAI-compatible
|
||||
/// endpoints read one, some the other, and neither rejects the extra key.
|
||||
Both,
|
||||
}
|
||||
|
||||
/// When and how far tool results are shrunk.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct ResultLimit {
|
||||
/// Gate: results longer than this (in bytes — cheap and stable) are shrunk.
|
||||
/// The fallback truncation cuts on a **char** boundary, never mid-codepoint.
|
||||
pub max_chars: usize,
|
||||
/// Shrink only results of turns before the current one, so the in-flight
|
||||
/// turn always sees its own tool output in full.
|
||||
pub previous_turns_only: bool,
|
||||
}
|
||||
|
||||
/// The protocol-shaped knobs of the projection. [`Default`] is a correct
|
||||
/// OpenAI-shaped conversation; a host overrides only what its models need.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Projection {
|
||||
/// Header of the compaction summary block.
|
||||
pub summary_prefix: String,
|
||||
/// Optional trailer, to mark where the summary ends and full history resumes.
|
||||
pub summary_suffix: Option<String>,
|
||||
/// Keep at most this many history messages (cut boundary-safely).
|
||||
pub max_messages: Option<usize>,
|
||||
pub max_tool_result: Option<ResultLimit>,
|
||||
/// Result text for a call that was still `Running`/`AwaitingHuman` when the
|
||||
/// process died.
|
||||
pub interrupted_text: String,
|
||||
/// Result text for a `Rejected` call that recorded none.
|
||||
pub rejected_default: String,
|
||||
/// Result text for a `Cancelled` call that recorded none.
|
||||
pub cancelled_default: String,
|
||||
/// Some models (DeepSeek thinking mode) reject a replayed tool-calling turn
|
||||
/// whose `reasoning_content` is empty: this stands in when none was stored.
|
||||
pub reasoning_placeholder: Option<String>,
|
||||
pub reasoning_echo: ReasoningEcho,
|
||||
/// Joins the dynamic-tail layers into the single trailing system message.
|
||||
pub tail_separator: String,
|
||||
pub media: MediaBudget,
|
||||
/// In `DeferredToolReference` mode, the tool whose result carries the
|
||||
/// `_tool_references` marker (the activation tool's name). `None` = the
|
||||
/// first result of the anchored message.
|
||||
pub activation_anchor_tool: Option<String>,
|
||||
}
|
||||
|
||||
/// The default summary header — enough for a model to know what it is reading.
|
||||
pub const SUMMARY_PREFIX: &str =
|
||||
"[CONTEXT SUMMARY — earlier messages were compacted into this summary]";
|
||||
|
||||
impl Default for Projection {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
summary_prefix: SUMMARY_PREFIX.to_string(),
|
||||
summary_suffix: None,
|
||||
max_messages: None,
|
||||
max_tool_result: None,
|
||||
interrupted_text: "[interrupted: this tool call did not complete — the session \
|
||||
restarted before a result was recorded]"
|
||||
.to_string(),
|
||||
rejected_default: String::new(),
|
||||
cancelled_default: String::new(),
|
||||
reasoning_placeholder: None,
|
||||
reasoning_echo: ReasoningEcho::default(),
|
||||
tail_separator: "\n\n---\n".to_string(),
|
||||
media: MediaBudget::default(),
|
||||
activation_anchor_tool: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ── Host hooks ───────────────────────────────────────────────────────────────
|
||||
|
||||
/// Which media a message may inline. The host authorizes (containment,
|
||||
/// ownership, upload rules); the crate decides shape and budget.
|
||||
#[async_trait]
|
||||
pub trait MediaSource: Send + Sync {
|
||||
/// Media attached to a user/agent message.
|
||||
async fn message_media(&self, _msg: &StoredMessage) -> Vec<Arc<dyn MediaBlob>> {
|
||||
Vec::new()
|
||||
}
|
||||
/// Media produced by an assistant turn's tool calls.
|
||||
async fn call_media(&self, _calls: &[StoredCall]) -> Vec<Arc<dyn MediaBlob>> {
|
||||
Vec::new()
|
||||
}
|
||||
/// Text appended to the message for the media that did NOT make it (a path
|
||||
/// list, so the agent can still reach them with a tool).
|
||||
///
|
||||
/// `skipped` are **positions in the vector `message_media` just returned**
|
||||
/// for this message, so the host can map them back to whatever it built
|
||||
/// them from.
|
||||
fn skipped_text(&self, _msg: &StoredMessage, _skipped: &[usize]) -> Option<String> {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// How an over-long tool result is condensed. The crate decides *when*
|
||||
/// (the [`ResultLimit`] gate); the host decides *what to say*, because a good
|
||||
/// summary knows what the tool does.
|
||||
#[async_trait]
|
||||
pub trait ToolResultDigest: Send + Sync {
|
||||
/// `None` → the crate applies its generic char-boundary truncation.
|
||||
async fn condense(&self, name: &str, args: &Value, result: &str) -> Option<String>;
|
||||
}
|
||||
|
||||
/// The host hooks, all optional.
|
||||
#[derive(Default, Clone)]
|
||||
pub struct ProjectionHooks {
|
||||
pub activation: Option<Arc<dyn ActivationSource>>,
|
||||
pub media: Option<Arc<dyn MediaSource>>,
|
||||
pub digest: Option<Arc<dyn ToolResultDigest>>,
|
||||
}
|
||||
|
||||
// ── The engine ───────────────────────────────────────────────────────────────
|
||||
|
||||
/// Project a frame's stored history into wire messages.
|
||||
pub async fn project(
|
||||
store: &Arc<dyn HistoryStore>,
|
||||
input: &AssembleInput,
|
||||
cfg: &Projection,
|
||||
hooks: &ProjectionHooks,
|
||||
) -> crate::Result<Vec<Value>> {
|
||||
let mut out: Vec<Value> = Vec::new();
|
||||
|
||||
// 1. Static system message — the cacheable prefix. With prompt caching the
|
||||
// content becomes a one-part array carrying the cache breakpoint.
|
||||
if !input.system.base.is_empty() {
|
||||
out.push(if input.model.prompt_cache {
|
||||
json!({
|
||||
"role": "system",
|
||||
"content": [{
|
||||
"type": "text",
|
||||
"text": input.system.base,
|
||||
"cache_control": { "type": "ephemeral" },
|
||||
}],
|
||||
})
|
||||
} else {
|
||||
json!({ "role": "system", "content": input.system.base })
|
||||
});
|
||||
}
|
||||
|
||||
// 2. Extra static layers (per-interface rules, session-scoped blocks).
|
||||
for s in &input.system.extra_static {
|
||||
out.push(json!({ "role": "system", "content": s }));
|
||||
}
|
||||
|
||||
// 3. Compaction summary, then the history it did not cover.
|
||||
let summary = store.latest_summary(input.frame).await?;
|
||||
if let Some(s) = &summary {
|
||||
let mut content = format!("{}\n\n{}", cfg.summary_prefix, s.text);
|
||||
if let Some(suffix) = &cfg.summary_suffix {
|
||||
content.push_str("\n\n");
|
||||
content.push_str(suffix);
|
||||
}
|
||||
out.push(json!({ "role": "system", "content": content }));
|
||||
}
|
||||
let mut history = match &summary {
|
||||
Some(s) => store.load_since(input.frame, s.covered_up_to).await?,
|
||||
None => store.load(input.frame).await?,
|
||||
};
|
||||
if let Some(max) = cfg.max_messages {
|
||||
window(&mut history, max);
|
||||
}
|
||||
|
||||
// 4. The conversation.
|
||||
let ctx = HistoryCtx::new(&history, cfg, hooks, input).await?;
|
||||
for (idx, entry) in history.iter().enumerate() {
|
||||
ctx.project_message(&mut out, idx, entry).await;
|
||||
}
|
||||
|
||||
// 5. Dynamic tail — the fresh layers, as ONE trailing system message so a
|
||||
// model reads them as a single "current state" block.
|
||||
if !input.system.dynamic_tail.is_empty() {
|
||||
let tail = input.system.dynamic_tail.join(&cfg.tail_separator);
|
||||
if !tail.is_empty() {
|
||||
out.push(json!({ "role": "system", "content": tail }));
|
||||
}
|
||||
}
|
||||
|
||||
// 6. Tail reminder.
|
||||
if let Some(r) = &input.system.tail_reminder {
|
||||
out.push(json!({ "role": "system", "content": r }));
|
||||
}
|
||||
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Cut the history to at most `max` messages. A leading assistant message is
|
||||
/// dropped as well: a window must not open on half an exchange.
|
||||
fn window(history: &mut Vec<StoredMessage>, max: usize) {
|
||||
if history.len() <= max {
|
||||
return;
|
||||
}
|
||||
history.drain(..history.len() - max);
|
||||
if matches!(history.first().map(|m| m.role), Some(Role::Assistant)) {
|
||||
history.drain(..1);
|
||||
}
|
||||
}
|
||||
|
||||
/// Per-build state shared by every message projection.
|
||||
struct HistoryCtx<'a> {
|
||||
cfg: &'a Projection,
|
||||
hooks: &'a ProjectionHooks,
|
||||
model: &'a crate::model::ModelInfo,
|
||||
/// Activated tool defs by anchor message (empty in `Inline` mode).
|
||||
activations: HashMap<MessageId, Vec<Value>>,
|
||||
/// Index of the last `User`/`Agent` message: everything before it belongs
|
||||
/// to a previous turn.
|
||||
boundary: Option<usize>,
|
||||
/// First index of the current turn's group — media is inlined only from
|
||||
/// here on, so images are not re-sent (and re-billed) every round.
|
||||
media_turn_start: usize,
|
||||
}
|
||||
|
||||
impl<'a> HistoryCtx<'a> {
|
||||
async fn new(
|
||||
history: &[StoredMessage],
|
||||
cfg: &'a Projection,
|
||||
hooks: &'a ProjectionHooks,
|
||||
input: &'a AssembleInput,
|
||||
) -> crate::Result<Self> {
|
||||
let activations = match (&hooks.activation, input.model.tool_rendering) {
|
||||
// Inline mode renders activated tools in the `tools` array itself:
|
||||
// nothing to inject, so the source is not even consulted.
|
||||
(_, ToolRendering::Inline) | (None, _) => HashMap::new(),
|
||||
(Some(src), _) => src
|
||||
.activations(input.frame)
|
||||
.await
|
||||
.unwrap_or_default()
|
||||
.into_iter()
|
||||
.fold(HashMap::<MessageId, Vec<Value>>::new(), |mut acc, a: Activation| {
|
||||
acc.entry(a.anchor).or_default().extend(a.defs);
|
||||
acc
|
||||
}),
|
||||
};
|
||||
|
||||
let boundary = history
|
||||
.iter()
|
||||
.rposition(|e| matches!(e.role, Role::User | Role::Agent));
|
||||
|
||||
// Trailing assistant rows are the in-flight turn's own rounds; the
|
||||
// current turn's user messages sit just before them.
|
||||
let mut media_turn_start = history.len();
|
||||
while media_turn_start > 0
|
||||
&& matches!(history[media_turn_start - 1].role, Role::Assistant)
|
||||
{
|
||||
media_turn_start -= 1;
|
||||
}
|
||||
while media_turn_start > 0
|
||||
&& matches!(history[media_turn_start - 1].role, Role::User | Role::Agent)
|
||||
{
|
||||
media_turn_start -= 1;
|
||||
}
|
||||
|
||||
Ok(Self {
|
||||
cfg,
|
||||
hooks,
|
||||
model: &input.model,
|
||||
activations,
|
||||
boundary,
|
||||
media_turn_start,
|
||||
})
|
||||
}
|
||||
|
||||
async fn project_message(&self, out: &mut Vec<Value>, idx: usize, entry: &StoredMessage) {
|
||||
match entry.role {
|
||||
// System messages are BUILT (layers 1-2), never replayed from the
|
||||
// store; a host that stores them gets them back verbatim.
|
||||
Role::System => out.push(json!({ "role": "system", "content": entry.content })),
|
||||
Role::User | Role::Agent => self.push_user(out, idx, entry).await,
|
||||
Role::Assistant => self.push_assistant(out, idx, entry).await,
|
||||
}
|
||||
}
|
||||
|
||||
/// A user/agent message: text plus, for the current turn, inlined media.
|
||||
async fn push_user(&self, out: &mut Vec<Value>, idx: usize, entry: &StoredMessage) {
|
||||
let mut text = entry.content.clone();
|
||||
let mut parts: Vec<Value> = Vec::new();
|
||||
|
||||
if let Some(src) = &self.hooks.media {
|
||||
let blobs = src.message_media(entry).await;
|
||||
if !blobs.is_empty() {
|
||||
// Older turns keep the textual path: everything is "skipped".
|
||||
let (inlined, skipped) = if idx >= self.media_turn_start {
|
||||
media::partition(&blobs, &self.model.capabilities, &self.cfg.media).await
|
||||
} else {
|
||||
(Vec::new(), (0..blobs.len()).collect())
|
||||
};
|
||||
if let Some(extra) = src.skipped_text(entry, &skipped) {
|
||||
text.push_str(&extra);
|
||||
}
|
||||
parts = inlined;
|
||||
}
|
||||
}
|
||||
|
||||
push_user_chunk(out, text, parts);
|
||||
}
|
||||
|
||||
/// An assistant message: the turn itself, then a result for every call, then
|
||||
/// the append-only DTL injections.
|
||||
async fn push_assistant(&self, out: &mut Vec<Value>, idx: usize, entry: &StoredMessage) {
|
||||
let stored_reasoning = entry.reasoning.as_deref().filter(|s| !s.is_empty());
|
||||
|
||||
if entry.calls.is_empty() {
|
||||
let mut msg = json!({ "role": "assistant", "content": entry.content });
|
||||
if let Some(r) = stored_reasoning {
|
||||
self.set_reasoning(&mut msg, r);
|
||||
}
|
||||
out.push(msg);
|
||||
return;
|
||||
}
|
||||
|
||||
let calls: Vec<Value> = entry
|
||||
.calls
|
||||
.iter()
|
||||
.map(|c| {
|
||||
json!({
|
||||
"id": c.provider_id,
|
||||
"type": "function",
|
||||
"function": { "name": c.name, "arguments": wire_arguments(c) },
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
let mut msg = json!({
|
||||
"role": "assistant",
|
||||
"content": entry.content,
|
||||
"tool_calls": calls,
|
||||
});
|
||||
// A tool-calling turn may need a non-empty reasoning on replay even when
|
||||
// none was recorded.
|
||||
if let Some(r) = stored_reasoning.or(self.cfg.reasoning_placeholder.as_deref()) {
|
||||
self.set_reasoning(&mut msg, r);
|
||||
}
|
||||
out.push(msg);
|
||||
|
||||
// One result per call, in call order — the model matches them by id.
|
||||
let is_previous_turn = self.boundary.is_some_and(|b| idx < b);
|
||||
let anchored = self.activations.get(&entry.id);
|
||||
let mut marked = false;
|
||||
|
||||
for call in &entry.calls {
|
||||
let mut tool_msg = json!({
|
||||
"role": "tool",
|
||||
"tool_call_id": call.provider_id,
|
||||
"content": self.result_content(call, is_previous_turn).await,
|
||||
});
|
||||
// Anthropic DTL: the activation's result carries the marker its
|
||||
// client turns into `tool_reference` blocks.
|
||||
if self.model.tool_rendering == ToolRendering::DeferredToolReference
|
||||
&& !marked
|
||||
&& let Some(defs) = anchored
|
||||
&& self.is_anchor(call)
|
||||
{
|
||||
let names: Vec<Value> = defs
|
||||
.iter()
|
||||
.filter_map(|d| d["function"]["name"].as_str())
|
||||
.map(|n| json!(n))
|
||||
.collect();
|
||||
if !names.is_empty() {
|
||||
tool_msg["_tool_references"] = Value::Array(names);
|
||||
marked = true;
|
||||
}
|
||||
}
|
||||
out.push(tool_msg);
|
||||
}
|
||||
|
||||
// Media a tool produced, as a synthetic user message right after the
|
||||
// result group (the current turn only).
|
||||
if idx >= self.media_turn_start
|
||||
&& let Some(src) = &self.hooks.media
|
||||
{
|
||||
let blobs = src.call_media(&entry.calls).await;
|
||||
if !blobs.is_empty() {
|
||||
let (parts, _) =
|
||||
media::partition(&blobs, &self.model.capabilities, &self.cfg.media).await;
|
||||
if !parts.is_empty() {
|
||||
out.push(json!({ "role": "user", "content": parts }));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Kimi-style DTL: the activated defs as a `system` message carrying a
|
||||
// `tools` field, appended after the group — the prefix stays identical.
|
||||
if self.model.tool_rendering == ToolRendering::SystemToolBlock
|
||||
&& let Some(defs) = anchored
|
||||
&& !defs.is_empty()
|
||||
{
|
||||
out.push(json!({ "role": "system", "tools": defs }));
|
||||
}
|
||||
}
|
||||
|
||||
fn set_reasoning(&self, msg: &mut Value, reasoning: &str) {
|
||||
msg["reasoning_content"] = json!(reasoning);
|
||||
if self.cfg.reasoning_echo == ReasoningEcho::Both {
|
||||
msg["reasoning"] = json!(reasoning);
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether this call is the DTL anchor within its message.
|
||||
fn is_anchor(&self, call: &StoredCall) -> bool {
|
||||
match &self.cfg.activation_anchor_tool {
|
||||
Some(name) => &call.name == name,
|
||||
None => true, // the first result of the message
|
||||
}
|
||||
}
|
||||
|
||||
/// The tool result text: the well-formedness rule of contract point 2, then
|
||||
/// the size gate.
|
||||
async fn result_content(&self, call: &StoredCall, is_previous_turn: bool) -> String {
|
||||
let content = match call.state {
|
||||
CallState::Done => call.result.clone().unwrap_or_default(),
|
||||
CallState::Failed => {
|
||||
format!("Error: {}", call.result.as_deref().unwrap_or("unknown error"))
|
||||
}
|
||||
// A recorded reason wins; an absent or empty one falls back to the
|
||||
// configured note — a model must never read an empty tool result
|
||||
// and have to guess what happened.
|
||||
CallState::Rejected => non_empty(&call.result)
|
||||
.unwrap_or_else(|| self.cfg.rejected_default.clone()),
|
||||
CallState::Cancelled => non_empty(&call.result)
|
||||
.unwrap_or_else(|| self.cfg.cancelled_default.clone()),
|
||||
// Running / AwaitingHuman reaching the projection means the process
|
||||
// died mid-flight: the call really was interrupted.
|
||||
CallState::Running | CallState::AwaitingHuman => self.cfg.interrupted_text.clone(),
|
||||
};
|
||||
|
||||
let Some(limit) = self.cfg.max_tool_result else {
|
||||
return content;
|
||||
};
|
||||
if limit.previous_turns_only && !is_previous_turn {
|
||||
return content;
|
||||
}
|
||||
if content.len() <= limit.max_chars {
|
||||
return content;
|
||||
}
|
||||
if let Some(d) = &self.hooks.digest
|
||||
&& let Some(short) = d.condense(&call.name, &call.arguments, &content).await
|
||||
{
|
||||
return short;
|
||||
}
|
||||
format!(
|
||||
"{}… [truncated]",
|
||||
content.chars().take(limit.max_chars).collect::<String>()
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
fn non_empty(s: &Option<String>) -> Option<String> {
|
||||
s.clone().filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
/// The arguments string sent back on the wire. The **raw recorded string** wins:
|
||||
/// re-serializing a parsed `Value` reorders object keys (serde_json's map is
|
||||
/// ordered), which would change the bytes the model produced and break the
|
||||
/// prompt-cache prefix.
|
||||
fn wire_arguments(call: &StoredCall) -> String {
|
||||
match &call.arguments_raw {
|
||||
Some(raw) => raw.clone(),
|
||||
None => serde_json::to_string(&call.arguments).unwrap_or_else(|_| "{}".into()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Append one user/agent chunk, coalescing with a preceding `user` message —
|
||||
/// consecutive user rows are one wire message, so strict-alternation APIs stay
|
||||
/// happy. Media parts keep their position relative to the text.
|
||||
pub fn push_user_chunk(out: &mut Vec<Value>, text: String, media: Vec<Value>) {
|
||||
fn text_part(t: &str) -> Value {
|
||||
json!({ "type": "text", "text": t })
|
||||
}
|
||||
|
||||
if let Some(last) = out.last_mut()
|
||||
&& last["role"] == "user"
|
||||
{
|
||||
if !last["content"].is_array() && media.is_empty() {
|
||||
let prev = last["content"].as_str().unwrap_or("").to_string();
|
||||
last["content"] = Value::String(format!("{prev}\n\n{text}"));
|
||||
return;
|
||||
}
|
||||
let mut parts = match last["content"].take() {
|
||||
Value::Array(a) => a,
|
||||
Value::String(s) => vec![text_part(&s)],
|
||||
_ => Vec::new(),
|
||||
};
|
||||
if let Some(tp) = parts.iter_mut().rev().find(|p| p["type"] == "text") {
|
||||
let prev = tp["text"].as_str().unwrap_or("").to_string();
|
||||
tp["text"] = Value::String(format!("{prev}\n\n{text}"));
|
||||
} else {
|
||||
parts.insert(0, text_part(&text));
|
||||
}
|
||||
parts.extend(media);
|
||||
last["content"] = Value::Array(parts);
|
||||
return;
|
||||
}
|
||||
if media.is_empty() {
|
||||
out.push(json!({ "role": "user", "content": text }));
|
||||
} else {
|
||||
let mut parts = vec![text_part(&text)];
|
||||
parts.extend(media);
|
||||
out.push(json!({ "role": "user", "content": parts }));
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn coalesces_consecutive_user_messages() {
|
||||
let mut out = vec![];
|
||||
push_user_chunk(&mut out, "one".into(), vec![]);
|
||||
push_user_chunk(&mut out, "two".into(), vec![]);
|
||||
assert_eq!(out.len(), 1);
|
||||
assert_eq!(out[0]["content"], "one\n\ntwo");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn media_promotes_the_chunk_to_a_parts_array() {
|
||||
let mut out = vec![];
|
||||
let part = json!({ "type": "image_url", "image_url": { "url": "data:x" } });
|
||||
push_user_chunk(&mut out, "look".into(), vec![part.clone()]);
|
||||
assert_eq!(out[0]["content"][0]["type"], "text");
|
||||
assert_eq!(out[0]["content"][1], part);
|
||||
|
||||
// A following text chunk folds into the LAST text part, keeping the
|
||||
// media after it.
|
||||
push_user_chunk(&mut out, "more".into(), vec![]);
|
||||
assert_eq!(out.len(), 1);
|
||||
assert_eq!(out[0]["content"][0]["text"], "look\n\nmore");
|
||||
assert_eq!(out[0]["content"][1], part);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_non_user_tail_starts_a_new_chunk() {
|
||||
let mut out = vec![json!({ "role": "assistant", "content": "hi" })];
|
||||
push_user_chunk(&mut out, "next".into(), vec![]);
|
||||
assert_eq!(out.len(), 2);
|
||||
assert_eq!(out[1]["role"], "user");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn raw_arguments_win_over_the_parsed_value() {
|
||||
let mut call = StoredCall {
|
||||
id: crate::ids::ToolCallId(1),
|
||||
message_id: MessageId(1),
|
||||
provider_id: "c1".into(),
|
||||
name: "write_file".into(),
|
||||
arguments: json!({ "a": 1, "z": 2 }),
|
||||
arguments_raw: Some(r#"{"z":2,"a":1}"#.to_string()),
|
||||
state: CallState::Done,
|
||||
result: None,
|
||||
result_kind: "text".into(),
|
||||
extras: Value::Null,
|
||||
};
|
||||
assert_eq!(wire_arguments(&call), r#"{"z":2,"a":1}"#);
|
||||
call.arguments_raw = None;
|
||||
assert_eq!(wire_arguments(&call), r#"{"a":1,"z":2}"#);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,727 @@
|
||||
//! Restart recovery (blueprint §8) — turning a half-written conversation back
|
||||
//! into a well-formed one, then running a **normal loop** on it.
|
||||
//!
|
||||
//! There is no "recovery mode" in the kernel. Every state transition is written
|
||||
//! the instant it happens (see [`crate::store`]), so a crash loses RAM — the
|
||||
//! approval oneshot, the cancellation token — never the truth. What it leaves
|
||||
//! behind is a store that a model would choke on: calls with no result, a child
|
||||
//! frame whose answer nobody propagated, a half-run parallel batch. This module
|
||||
//! repairs exactly those, then hands the frame to the same `LlmLoop` a live turn
|
||||
//! uses.
|
||||
//!
|
||||
//! The order matters and mirrors `resume.rs`, the path this replaces:
|
||||
//!
|
||||
//! 1. **Reap** an interrupted parallel batch (≥2 active frames at one depth).
|
||||
//! 2. **Resolve** the deepest active frame's non-terminal calls, by policy and
|
||||
//! by each tool's [`RestartHint`].
|
||||
//! 3. **Un-wedge**: a child that finished but never told its parent.
|
||||
//! 4. **Cascade**: run the frame, resolve its parent's call with the result,
|
||||
//! close it, walk up — every frame with **its own** agent's config (B3), read
|
||||
//! from the catalog, never the root's.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Arc;
|
||||
|
||||
use tokio_util::sync::CancellationToken;
|
||||
use tracing::{info, warn};
|
||||
|
||||
use crate::delegate::{AgentCatalog, FilteredToolSet};
|
||||
use crate::events::{EventSink, LoopEvent, PendingToolCall};
|
||||
use crate::ids::{ConversationId, FrameId};
|
||||
use crate::kernel::{PreExecution, TurnOutcome};
|
||||
use crate::manager::{LoopManager, LoopParams, TurnMeta, TurnParams};
|
||||
use crate::store::{CallOutcome, CallState, FrameRecord, Role, StoredCall};
|
||||
use crate::tool::{ExecutionOutcome, RestartHint, ToolCtx, ToolSet, drive_execution};
|
||||
|
||||
// ── Policy ───────────────────────────────────────────────────────────────────
|
||||
|
||||
/// What to do with a call that was `Running` when the process died.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||||
pub enum RunningPolicy {
|
||||
/// Re-gate and re-execute, unless the tool's own [`RestartHint`] says
|
||||
/// otherwise (which always wins: only the tool knows if it is idempotent).
|
||||
#[default]
|
||||
ReExecute,
|
||||
/// Never re-run: resolve every interrupted call as failed.
|
||||
MarkInterrupted,
|
||||
}
|
||||
|
||||
/// What to do with a call that was waiting on a human.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||||
pub enum PendingPolicy {
|
||||
/// Ask again — the approval card reappears (today's behavior).
|
||||
#[default]
|
||||
ReAsk,
|
||||
/// Leave it pending for an out-of-band decision
|
||||
/// ([`LoopManager::resolve_pending`]), and stop: the frame cannot run with
|
||||
/// an unanswered call in it.
|
||||
LeavePending,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct RecoveryPolicy {
|
||||
pub on_running: RunningPolicy,
|
||||
pub on_awaiting_human: PendingPolicy,
|
||||
/// Recorded on a call that is not re-run.
|
||||
pub interrupted_text: String,
|
||||
/// Recorded on the delegating call of a reaped parallel batch.
|
||||
pub batch_reaped_text: String,
|
||||
}
|
||||
|
||||
impl Default for RecoveryPolicy {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
on_running: RunningPolicy::default(),
|
||||
on_awaiting_human: PendingPolicy::default(),
|
||||
interrupted_text: "Tool call interrupted by a restart.".to_string(),
|
||||
batch_reaped_text: "Sub-agent interrupted by restart (parallel batch).".to_string(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// What a recovery pass did — logged by hosts, asserted by tests.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq)]
|
||||
pub struct RecoveryReport {
|
||||
pub frames_resumed: usize,
|
||||
pub calls_reexecuted: usize,
|
||||
pub calls_failed: usize,
|
||||
pub batches_reaped: usize,
|
||||
/// A call was left `AwaitingHuman`: the conversation waits for a decision.
|
||||
pub left_pending: bool,
|
||||
}
|
||||
|
||||
// ── Recovery ─────────────────────────────────────────────────────────────────
|
||||
|
||||
pub struct Recovery {
|
||||
manager: Arc<LoopManager>,
|
||||
catalog: Arc<dyn AgentCatalog>,
|
||||
policy: RecoveryPolicy,
|
||||
}
|
||||
|
||||
impl Recovery {
|
||||
pub fn new(
|
||||
manager: Arc<LoopManager>,
|
||||
catalog: Arc<dyn AgentCatalog>,
|
||||
policy: RecoveryPolicy,
|
||||
) -> Self {
|
||||
Self { manager, catalog, policy }
|
||||
}
|
||||
|
||||
/// Recover one conversation. `root` is what the **root** frame runs with —
|
||||
/// the host's own turn parameters, since no catalog describes the entry
|
||||
/// agent; `root.frame` must be that root frame, and `root.live_input` is
|
||||
/// ignored (a recovery is not a live turn).
|
||||
///
|
||||
/// Refuses while a loop is already live on the conversation: that loop is
|
||||
/// already the thing driving it.
|
||||
pub async fn run(
|
||||
&self,
|
||||
conv: &ConversationId,
|
||||
root: &TurnParams,
|
||||
) -> crate::Result<RecoveryReport> {
|
||||
let Some(claim) = self.manager.claim(conv, root.frame, &root.agent) else {
|
||||
info!(%conv, "recovery: a loop is already running — nothing to do");
|
||||
return Ok(RecoveryReport::default());
|
||||
};
|
||||
let token = claim.token();
|
||||
let events = self.manager.sink_for(conv.clone());
|
||||
let store = self.manager.store();
|
||||
let mut report = RecoveryReport::default();
|
||||
|
||||
// ── 1. reap an interrupted parallel batch ──
|
||||
self.reap_batches(conv, &mut report).await?;
|
||||
|
||||
// ── 2. the deepest active frame is where the conversation stopped ──
|
||||
let Some(mut frame) = store.deepest_active(conv).await? else {
|
||||
info!(%conv, "recovery: no active frame — nothing to resume");
|
||||
return Ok(report);
|
||||
};
|
||||
|
||||
let mut params = self.params_for(&frame, root, conv).await?;
|
||||
let pending = self
|
||||
.resolve_frame_calls(conv, &frame, ¶ms, &token, &events, &mut report)
|
||||
.await?;
|
||||
if report.left_pending {
|
||||
return Ok(report);
|
||||
}
|
||||
|
||||
// ── 3. un-wedge: a finished child whose result never reached its parent ──
|
||||
let mut outcome = match self.completed_without_propagating(&frame, pending).await? {
|
||||
Some(o) => o,
|
||||
None => {
|
||||
report.frames_resumed += 1;
|
||||
self.run_frame(¶ms, &token, conv, frame.id, frame.parent).await?
|
||||
}
|
||||
};
|
||||
|
||||
// ── 4. cascade to the root ──
|
||||
while let Some(parent_call) = frame.spec.parent_call {
|
||||
let result = child_result(&outcome, &frame.spec.agent);
|
||||
match &result {
|
||||
Ok(text) => store.resolve_call(parent_call, &CallOutcome::Completed(
|
||||
crate::tool::ToolOutput::Text(text.clone()),
|
||||
)).await?,
|
||||
Err(text) => store.resolve_call(parent_call, &CallOutcome::Failed(text.clone())).await?,
|
||||
}
|
||||
let (text, failed) = match result {
|
||||
Ok(t) => (t, false),
|
||||
Err(t) => (t, true),
|
||||
};
|
||||
self.catalog.on_child_closed(frame.id).await;
|
||||
store.close_frame(frame.id).await?;
|
||||
|
||||
let parent = match store.frame_of_call(parent_call).await? {
|
||||
Some(p) => p,
|
||||
None => {
|
||||
warn!(%conv, call = %parent_call, "recovery: the call's frame is gone");
|
||||
break;
|
||||
}
|
||||
};
|
||||
events.emit(frame.id, Some(parent.id), LoopEvent::AgentFinished {
|
||||
frame: frame.id,
|
||||
agent: frame.spec.agent.clone(),
|
||||
result_preview: crate::delegate::preview_truncate(&text, 500),
|
||||
parent_agent: parent.spec.agent.clone(),
|
||||
});
|
||||
events.emit(parent.id, parent.parent, LoopEvent::ToolCallFinished {
|
||||
id: parent_call,
|
||||
outcome: if failed {
|
||||
CallOutcome::Failed(text)
|
||||
} else {
|
||||
CallOutcome::Completed(crate::tool::ToolOutput::Text(text))
|
||||
},
|
||||
});
|
||||
|
||||
frame = parent;
|
||||
params = self.params_for(&frame, root, conv).await?;
|
||||
self.resolve_frame_calls(conv, &frame, ¶ms, &token, &events, &mut report)
|
||||
.await?;
|
||||
if report.left_pending {
|
||||
return Ok(report);
|
||||
}
|
||||
report.frames_resumed += 1;
|
||||
outcome = self.run_frame(¶ms, &token, conv, frame.id, frame.parent).await?;
|
||||
}
|
||||
|
||||
drop(claim);
|
||||
Ok(report)
|
||||
}
|
||||
|
||||
/// Make the store well-formed **without continuing the conversation**: reap
|
||||
/// an interrupted batch, resolve the deepest frame's dangling calls.
|
||||
///
|
||||
/// This is what a host runs before starting a *new* turn on a session that
|
||||
/// died mid-tool: the user has something else to say, so nothing should
|
||||
/// re-drive the old turn, but the model must not be shown a call with no
|
||||
/// result. Unlike [`Self::run`] it does not claim the conversation — the
|
||||
/// caller is already inside its own turn.
|
||||
pub async fn repair(
|
||||
&self,
|
||||
conv: &ConversationId,
|
||||
root: &TurnParams,
|
||||
) -> crate::Result<RecoveryReport> {
|
||||
let mut report = RecoveryReport::default();
|
||||
self.reap_batches(conv, &mut report).await?;
|
||||
if let Some(frame) = self.manager.store().deepest_active(conv).await? {
|
||||
let params = self.params_for(&frame, root, conv).await?;
|
||||
let token = CancellationToken::new();
|
||||
let events = self.manager.sink_for(conv.clone());
|
||||
self.resolve_frame_calls(conv, &frame, ¶ms, &token, &events, &mut report)
|
||||
.await?;
|
||||
}
|
||||
Ok(report)
|
||||
}
|
||||
|
||||
/// Two or more active frames at one depth can only be a concurrent batch
|
||||
/// caught mid-flight (a linear stack has at most one per depth). Recovering
|
||||
/// it properly would mean re-driving several siblings; instead the batch is
|
||||
/// pruned — deliberately lossy — and the parent continues with the failures
|
||||
/// in view.
|
||||
async fn reap_batches(
|
||||
&self,
|
||||
conv: &ConversationId,
|
||||
report: &mut RecoveryReport,
|
||||
) -> crate::Result<()> {
|
||||
let store = self.manager.store();
|
||||
let active = store.active_frames(conv).await?;
|
||||
let Some(d_min) = shallowest_parallel_depth(&active) else {
|
||||
return Ok(());
|
||||
};
|
||||
warn!(%conv, depth = d_min, "recovery: reaping an interrupted parallel batch");
|
||||
for frame in active.iter().filter(|f| f.spec.depth >= d_min) {
|
||||
if let Some(parent_call) = frame.spec.parent_call {
|
||||
let _ = store
|
||||
.resolve_call(
|
||||
parent_call,
|
||||
&CallOutcome::Failed(self.policy.batch_reaped_text.clone()),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
let _ = store.close_frame(frame.id).await;
|
||||
}
|
||||
report.batches_reaped += 1;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Runs one frame's loop to completion, through the manager (so the turn is
|
||||
/// an ordinary loop — same kernel, same events, same rules).
|
||||
async fn run_frame(
|
||||
&self,
|
||||
params: &LoopParams,
|
||||
token: &CancellationToken,
|
||||
conv: &ConversationId,
|
||||
frame: FrameId,
|
||||
parent: Option<FrameId>,
|
||||
) -> crate::Result<TurnOutcome> {
|
||||
let handle = self
|
||||
.manager
|
||||
.start_loop(clone_params(params, conv, frame, parent, Some(token.clone())))
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("recovery: {e}"))?;
|
||||
handle.join().await
|
||||
}
|
||||
|
||||
/// Every non-terminal call of a frame, resolved per policy. Returns whether
|
||||
/// anything at all was pending (the un-wedge check needs to know).
|
||||
async fn resolve_frame_calls(
|
||||
&self,
|
||||
conv: &ConversationId,
|
||||
frame: &FrameRecord,
|
||||
params: &LoopParams,
|
||||
token: &CancellationToken,
|
||||
events: &EventSink,
|
||||
report: &mut RecoveryReport,
|
||||
) -> crate::Result<bool> {
|
||||
let store = self.manager.store();
|
||||
let calls = store
|
||||
.calls_in_state(frame.id, &[CallState::Running, CallState::AwaitingHuman])
|
||||
.await?;
|
||||
if calls.is_empty() {
|
||||
return Ok(false);
|
||||
}
|
||||
|
||||
// A call that spawned a frame is the cascade's business: its result is
|
||||
// the child's answer, not a re-execution. Structural, not by name — a
|
||||
// host may register the delegate under any number of aliases.
|
||||
let children = store.active_frames(conv).await?;
|
||||
let spawned = |call: &StoredCall| {
|
||||
children.iter().any(|f| f.spec.parent_call == Some(call.id))
|
||||
};
|
||||
|
||||
for call in &calls {
|
||||
if spawned(call) {
|
||||
info!(call = %call.id, "recovery: sub-agent call left to the cascade");
|
||||
continue;
|
||||
}
|
||||
|
||||
let hint = params
|
||||
.tools
|
||||
.find(&call.name)
|
||||
.map(|t| t.restart_hint())
|
||||
.unwrap_or_default();
|
||||
let re_execute = match call.state {
|
||||
CallState::AwaitingHuman => match self.policy.on_awaiting_human {
|
||||
PendingPolicy::ReAsk => true,
|
||||
PendingPolicy::LeavePending => {
|
||||
info!(call = %call.id, "recovery: leaving the call pending for a decision");
|
||||
report.left_pending = true;
|
||||
return Ok(true);
|
||||
}
|
||||
},
|
||||
// The tool's own hint wins: only it knows whether re-running is
|
||||
// safe (a shell command may already have had its effect).
|
||||
_ => {
|
||||
self.policy.on_running == RunningPolicy::ReExecute
|
||||
&& hint == RestartHint::ReExecute
|
||||
}
|
||||
};
|
||||
|
||||
if !re_execute {
|
||||
store
|
||||
.resolve_call(
|
||||
call.id,
|
||||
&CallOutcome::Failed(self.policy.interrupted_text.clone()),
|
||||
)
|
||||
.await?;
|
||||
events.emit(frame.id, frame.parent, LoopEvent::ToolCallFinished {
|
||||
id: call.id,
|
||||
outcome: CallOutcome::Failed(self.policy.interrupted_text.clone()),
|
||||
});
|
||||
report.calls_failed += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
if self.re_execute(call, params, token, events, frame).await? {
|
||||
report.calls_reexecuted += 1;
|
||||
} else {
|
||||
// Suspended again (the human is still not there, or the channel
|
||||
// closed): the call stays AwaitingHuman for the next attempt.
|
||||
report.left_pending = true;
|
||||
return Ok(true);
|
||||
}
|
||||
}
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
/// Re-runs one call through the **normal** path — gate, hooks, tool — so a
|
||||
/// rule change since the crash applies and the approval card reappears.
|
||||
/// `Ok(false)` = it suspended again and must be left pending.
|
||||
async fn re_execute(
|
||||
&self,
|
||||
call: &StoredCall,
|
||||
params: &LoopParams,
|
||||
token: &CancellationToken,
|
||||
events: &EventSink,
|
||||
frame: &FrameRecord,
|
||||
) -> crate::Result<bool> {
|
||||
let ptc = PendingToolCall {
|
||||
id: call.id,
|
||||
message_id: call.message_id,
|
||||
provider_id: Some(call.provider_id.clone()).filter(|s| !s.is_empty()),
|
||||
name: call.name.clone(),
|
||||
arguments: call.arguments.clone(),
|
||||
};
|
||||
events.emit(frame.id, frame.parent, LoopEvent::ToolCallStarted {
|
||||
id: ptc.id,
|
||||
message_id: ptc.message_id,
|
||||
name: ptc.name.clone(),
|
||||
args: ptc.arguments.clone(),
|
||||
});
|
||||
|
||||
let deps = self.manager.deps();
|
||||
match crate::kernel::pre_execution(deps, params, events, token, &ptc).await? {
|
||||
PreExecution::Run(tool) => {
|
||||
let ctx = ToolCtx {
|
||||
conversation: params.conversation.clone(),
|
||||
frame: params.frame,
|
||||
agent: params.agent.clone(),
|
||||
call_id: ptc.id,
|
||||
cancel: token.clone(),
|
||||
extensions: crate::kernel::tool_extensions(params, events),
|
||||
};
|
||||
let exec = tool.start(ptc.arguments.clone(), &ctx);
|
||||
match drive_execution(&*exec, token).await {
|
||||
ExecutionOutcome::Suspended => Ok(false),
|
||||
outcome => {
|
||||
crate::kernel::record_outcome(
|
||||
deps,
|
||||
params,
|
||||
events,
|
||||
&self.manager.store(),
|
||||
&ptc,
|
||||
outcome.into_call_outcome(),
|
||||
)
|
||||
.await?;
|
||||
Ok(true)
|
||||
}
|
||||
}
|
||||
}
|
||||
PreExecution::Resolved(outcome) => {
|
||||
crate::kernel::record_outcome(
|
||||
deps, params, events, &self.manager.store(), &ptc, outcome,
|
||||
)
|
||||
.await?;
|
||||
Ok(true)
|
||||
}
|
||||
PreExecution::Suspended => Ok(false),
|
||||
PreExecution::TurnCancelled => Ok(false),
|
||||
}
|
||||
}
|
||||
|
||||
/// The wedge case: nothing was pending and the frame's last message is a
|
||||
/// plain assistant reply — its turn finished, and the process died before
|
||||
/// the result reached the parent. Re-running the model would ask it to
|
||||
/// answer a question it already answered, so the stored answer is used as
|
||||
/// the outcome and only the propagation is redone.
|
||||
///
|
||||
/// On the ROOT frame the same shape means the turn is simply complete.
|
||||
async fn completed_without_propagating(
|
||||
&self,
|
||||
frame: &FrameRecord,
|
||||
had_pending: bool,
|
||||
) -> crate::Result<Option<TurnOutcome>> {
|
||||
if had_pending {
|
||||
return Ok(None);
|
||||
}
|
||||
let Some(last) = self.manager.store().last(frame.id).await? else {
|
||||
return Ok(None);
|
||||
};
|
||||
if last.role != Role::Assistant || !last.calls.is_empty() {
|
||||
return Ok(None);
|
||||
}
|
||||
Ok(Some(TurnOutcome::Final {
|
||||
content: last.content,
|
||||
message_id: last.id,
|
||||
usage: last.usage,
|
||||
reasoning: last.reasoning,
|
||||
}))
|
||||
}
|
||||
|
||||
/// The parameters one frame runs with: the host's for the root, the
|
||||
/// catalog's for every other (B3 — a resumed sub-agent is ITS agent, with
|
||||
/// its prompt, its tools and its model).
|
||||
async fn params_for(
|
||||
&self,
|
||||
frame: &FrameRecord,
|
||||
root: &TurnParams,
|
||||
conv: &ConversationId,
|
||||
) -> crate::Result<LoopParams> {
|
||||
let mut params = clone_params_from_turn(root, conv, frame.id, frame.parent);
|
||||
if frame.spec.parent_call.is_none() {
|
||||
return Ok(params);
|
||||
}
|
||||
|
||||
let ctx = ToolCtx {
|
||||
conversation: conv.clone(),
|
||||
frame: frame.id,
|
||||
agent: frame.spec.agent.clone(),
|
||||
// The call that spawned this frame — the same handle the live
|
||||
// dispatch had.
|
||||
call_id: frame.spec.parent_call.unwrap(),
|
||||
cancel: CancellationToken::new(),
|
||||
extensions: root.extensions.clone(),
|
||||
};
|
||||
let profile = self.catalog.get(&frame.spec.agent, frame.id, &ctx).await?;
|
||||
|
||||
params.agent = frame.spec.agent.clone();
|
||||
params.system = profile.context;
|
||||
params.tools = match profile.toolset {
|
||||
Some(ts) => ts,
|
||||
None => Arc::new(FilteredToolSet::derive(root.tools.clone(), &profile.tools))
|
||||
as Arc<dyn ToolSet>,
|
||||
};
|
||||
params.model_hint = profile.model.unwrap_or_default();
|
||||
params.selector = profile.selector;
|
||||
params.assembler = profile.assembler;
|
||||
params.meta = TurnMeta { user_message: frame.spec.prompt.clone(), ..root.meta.clone() };
|
||||
Ok(params)
|
||||
}
|
||||
}
|
||||
|
||||
// ── resolve_pending (blueprint §8.5) ─────────────────────────────────────────
|
||||
|
||||
/// A human's answer to a call that was waiting for one.
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum HumanDecision {
|
||||
Approved,
|
||||
Rejected { reason: String },
|
||||
}
|
||||
|
||||
/// Apply a human decision to a call nothing is driving anymore — the approval
|
||||
/// card answered after a restart, or from the Inbox.
|
||||
///
|
||||
/// Approval **skips the gate**: the human is the gate, and re-running the rules
|
||||
/// would ask them again. The call is executed through the normal tool path
|
||||
/// (with the frame's own context, so a write lands in the caller's workspace,
|
||||
/// never the server's cwd), then the conversation is recovered so the model
|
||||
/// reads the result.
|
||||
pub(crate) async fn resolve_pending(
|
||||
manager: &Arc<LoopManager>,
|
||||
call_id: crate::ids::ToolCallId,
|
||||
decision: HumanDecision,
|
||||
catalog: Arc<dyn AgentCatalog>,
|
||||
root: &TurnParams,
|
||||
) -> crate::Result<RecoveryReport> {
|
||||
let store = manager.store();
|
||||
let call = store
|
||||
.get_call(call_id)
|
||||
.await?
|
||||
.ok_or_else(|| anyhow::anyhow!("resolve_pending: call {call_id} not found"))?;
|
||||
if call.state.is_terminal() {
|
||||
info!(call = %call_id, state = ?call.state, "resolve_pending: already resolved");
|
||||
return Ok(RecoveryReport::default());
|
||||
}
|
||||
let frame = store
|
||||
.frame_of_call(call_id)
|
||||
.await?
|
||||
.ok_or_else(|| anyhow::anyhow!("resolve_pending: no frame for call {call_id}"))?;
|
||||
let conv = frame.conversation.clone();
|
||||
|
||||
match decision {
|
||||
HumanDecision::Rejected { reason } => {
|
||||
store.resolve_call(call_id, &CallOutcome::Rejected { reason: reason.clone() }).await?;
|
||||
manager.sink_for(conv.clone()).emit(frame.id, frame.parent, LoopEvent::ToolCallFinished {
|
||||
id: call_id,
|
||||
outcome: CallOutcome::Rejected { reason },
|
||||
});
|
||||
}
|
||||
HumanDecision::Approved => {
|
||||
// Claimed for the execution only: the recovery below takes its own.
|
||||
let outcome = {
|
||||
let Some(claim) = manager.claim(&conv, frame.id, &frame.spec.agent) else {
|
||||
anyhow::bail!("resolve_pending: a loop is already running on {conv}");
|
||||
};
|
||||
let token = claim.token();
|
||||
let events = manager.sink_for(conv.clone());
|
||||
let params = clone_params_from_turn(root, &conv, frame.id, frame.parent);
|
||||
let ext = crate::kernel::tool_extensions(¶ms, &events);
|
||||
|
||||
match params.tools.find(&call.name) {
|
||||
Some(tool) => {
|
||||
let ctx = ToolCtx {
|
||||
conversation: conv.clone(),
|
||||
frame: frame.id,
|
||||
agent: frame.spec.agent.clone(),
|
||||
call_id,
|
||||
cancel: token.clone(),
|
||||
extensions: ext,
|
||||
};
|
||||
let exec = tool.start(call.arguments.clone(), &ctx);
|
||||
match drive_execution(&*exec, &token).await {
|
||||
// Suspending again would need another human: leave
|
||||
// it pending rather than resolving it as cancelled.
|
||||
ExecutionOutcome::Suspended => None,
|
||||
outcome => Some(outcome.into_call_outcome()),
|
||||
}
|
||||
}
|
||||
None => Some(CallOutcome::Failed(format!(
|
||||
"unknown tool '{}' (not in this turn's tool set)",
|
||||
call.name
|
||||
))),
|
||||
}
|
||||
};
|
||||
|
||||
let Some(outcome) = outcome else {
|
||||
return Ok(RecoveryReport { left_pending: true, ..RecoveryReport::default() });
|
||||
};
|
||||
store.resolve_call(call_id, &outcome).await?;
|
||||
manager.sink_for(conv.clone()).emit(frame.id, frame.parent, LoopEvent::ToolCallFinished {
|
||||
id: call_id,
|
||||
outcome,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// The history is well-formed again: a normal recovery continues the turn.
|
||||
Recovery::new(manager.clone(), catalog, RecoveryPolicy::default())
|
||||
.run(&conv, root)
|
||||
.await
|
||||
}
|
||||
|
||||
// ── helpers ──────────────────────────────────────────────────────────────────
|
||||
|
||||
/// The text a finished child propagates to its parent's call — `Err` when the
|
||||
/// child did not produce an answer.
|
||||
fn child_result(outcome: &TurnOutcome, agent: &str) -> Result<String, String> {
|
||||
match outcome {
|
||||
TurnOutcome::Final { content, .. } => Ok(content.clone()),
|
||||
TurnOutcome::Cancelled => Err(format!("Sub-agent `{agent}` was cancelled.")),
|
||||
TurnOutcome::Exhausted => Err(format!("Sub-agent `{agent}` exhausted tool-call rounds.")),
|
||||
}
|
||||
}
|
||||
|
||||
fn clone_params_from_turn(
|
||||
root: &TurnParams,
|
||||
conv: &ConversationId,
|
||||
frame: FrameId,
|
||||
parent: Option<FrameId>,
|
||||
) -> LoopParams {
|
||||
LoopParams {
|
||||
conversation: conv.clone(),
|
||||
frame,
|
||||
parent_frame: parent,
|
||||
agent: root.agent.clone(),
|
||||
system: root.system.clone(),
|
||||
tools: root.tools.clone(),
|
||||
model_hint: root.model_hint.clone(),
|
||||
selector: root.selector.clone(),
|
||||
token: None,
|
||||
// A recovery is not a live turn: no live input, and no tail reminder
|
||||
// semantics — the host decides that when it builds `root`.
|
||||
live_input: None,
|
||||
extensions: root.extensions.clone(),
|
||||
meta: root.meta.clone(),
|
||||
assembler: root.assembler.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
fn clone_params(
|
||||
p: &LoopParams,
|
||||
conv: &ConversationId,
|
||||
frame: FrameId,
|
||||
parent: Option<FrameId>,
|
||||
token: Option<CancellationToken>,
|
||||
) -> LoopParams {
|
||||
LoopParams {
|
||||
conversation: conv.clone(),
|
||||
frame,
|
||||
parent_frame: parent,
|
||||
agent: p.agent.clone(),
|
||||
system: p.system.clone(),
|
||||
tools: p.tools.clone(),
|
||||
model_hint: p.model_hint.clone(),
|
||||
selector: p.selector.clone(),
|
||||
token,
|
||||
live_input: None,
|
||||
extensions: p.extensions.clone(),
|
||||
meta: p.meta.clone(),
|
||||
assembler: p.assembler.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Shallowest depth holding more than one active frame — the top of an
|
||||
/// interrupted parallel batch. `None` for a linear stack, where every depth has
|
||||
/// at most one active frame. Pure (see tests).
|
||||
pub fn shallowest_parallel_depth(active: &[FrameRecord]) -> Option<u32> {
|
||||
let mut by_depth: HashMap<u32, usize> = HashMap::new();
|
||||
for f in active {
|
||||
*by_depth.entry(f.spec.depth).or_default() += 1;
|
||||
}
|
||||
by_depth
|
||||
.iter()
|
||||
.filter_map(|(depth, count)| (*count > 1).then_some(*depth))
|
||||
.min()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::ids::ToolCallId;
|
||||
use crate::store::FrameSpec;
|
||||
|
||||
fn frame(id: i64, depth: u32, parent_call: Option<i64>) -> FrameRecord {
|
||||
FrameRecord {
|
||||
id: FrameId(id),
|
||||
conversation: ConversationId::new("c"),
|
||||
parent: None,
|
||||
spec: FrameSpec {
|
||||
agent: "agent".into(),
|
||||
prompt: None,
|
||||
depth,
|
||||
parent_call: parent_call.map(ToolCallId),
|
||||
meta: serde_json::Value::Null,
|
||||
},
|
||||
active: true,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn linear_stack_is_not_a_batch() {
|
||||
let frames = vec![frame(1, 0, None), frame(2, 1, Some(10)), frame(3, 2, Some(20))];
|
||||
assert_eq!(shallowest_parallel_depth(&frames), None);
|
||||
assert_eq!(shallowest_parallel_depth(&[]), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn detects_shallowest_multi_frame_depth() {
|
||||
// Two siblings at depth 1 (parallel batch) plus a grandchild at depth 2.
|
||||
let frames = vec![
|
||||
frame(1, 0, None),
|
||||
frame(2, 1, Some(10)),
|
||||
frame(3, 1, Some(11)),
|
||||
frame(4, 2, Some(30)),
|
||||
];
|
||||
assert_eq!(shallowest_parallel_depth(&frames), Some(1));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn detects_deeper_batch_when_upper_levels_linear() {
|
||||
let frames = vec![
|
||||
frame(1, 0, None),
|
||||
frame(2, 1, Some(10)),
|
||||
frame(3, 2, Some(20)),
|
||||
frame(4, 2, Some(21)),
|
||||
];
|
||||
assert_eq!(shallowest_parallel_depth(&frames), Some(2));
|
||||
}
|
||||
}
|
||||
@@ -221,6 +221,11 @@ pub struct StoredCall {
|
||||
pub provider_id: String,
|
||||
pub name: String,
|
||||
pub arguments: Value,
|
||||
/// The arguments **exactly as the model emitted them**, when the store kept
|
||||
/// the string. The projection replays this verbatim: re-serializing
|
||||
/// [`Self::arguments`] reorders object keys, which changes the bytes the
|
||||
/// model produced and breaks the prompt-cache prefix.
|
||||
pub arguments_raw: Option<String>,
|
||||
pub state: CallState,
|
||||
pub result: Option<String>,
|
||||
pub result_kind: String,
|
||||
@@ -280,6 +285,10 @@ pub trait HistoryStore: Send + Sync {
|
||||
async fn set_call_state(&self, id: ToolCallId, state: CallState) -> crate::Result<()>;
|
||||
/// One call by id (translators enriching finish events, recovery).
|
||||
async fn get_call(&self, id: ToolCallId) -> crate::Result<Option<StoredCall>>;
|
||||
/// The frame a call belongs to. Recovery walks the cascade with it, and an
|
||||
/// out-of-band resolution (an approval answered from a REST endpoint) has
|
||||
/// nothing but a call id to start from.
|
||||
async fn frame_of_call(&self, id: ToolCallId) -> crate::Result<Option<FrameRecord>>;
|
||||
/// Merge host free-form extras into a call (Skald: diff preview, media).
|
||||
/// Keys not understood by the store are ignored.
|
||||
async fn set_call_extras(&self, id: ToolCallId, extras: Value) -> crate::Result<()>;
|
||||
|
||||
@@ -154,6 +154,8 @@ impl HistoryStore for InMemoryStore {
|
||||
provider_id,
|
||||
name: call.name,
|
||||
arguments: call.arguments,
|
||||
// Nothing to replay verbatim: this store never saw a wire string.
|
||||
arguments_raw: None,
|
||||
state: CallState::Running,
|
||||
result: None,
|
||||
result_kind: String::new(),
|
||||
@@ -195,6 +197,25 @@ impl HistoryStore for InMemoryStore {
|
||||
Ok(i.calls.values().flatten().find(|c| c.id == id).cloned())
|
||||
}
|
||||
|
||||
async fn frame_of_call(&self, id: ToolCallId) -> crate::Result<Option<FrameRecord>> {
|
||||
let i = self.inner.lock().unwrap();
|
||||
let Some(msg_id) = i
|
||||
.calls
|
||||
.values()
|
||||
.flatten()
|
||||
.find(|c| c.id == id)
|
||||
.map(|c| c.message_id)
|
||||
else {
|
||||
return Ok(None);
|
||||
};
|
||||
let frame = i
|
||||
.messages
|
||||
.iter()
|
||||
.find(|(_, msgs)| msgs.iter().any(|m| m.id == msg_id))
|
||||
.map(|(frame, _)| *frame);
|
||||
Ok(frame.and_then(|f| i.frames.get(&f).cloned()))
|
||||
}
|
||||
|
||||
async fn set_call_extras(&self, id: ToolCallId, extras: serde_json::Value) -> crate::Result<()> {
|
||||
let mut i = self.inner.lock().unwrap();
|
||||
update_call(&mut i, id, |c| {
|
||||
|
||||
@@ -143,6 +143,7 @@ async fn params(
|
||||
system: Arc::new(StaticSystemContext::new("You are a test agent.")),
|
||||
tools,
|
||||
model_hint: ModelHint::default(),
|
||||
selector: None,
|
||||
live_input: None,
|
||||
extensions: Default::default(),
|
||||
meta: TurnMeta::default(),
|
||||
@@ -502,18 +503,26 @@ async fn second_loop_on_same_conversation_rejected() {
|
||||
|
||||
struct TestCatalog {
|
||||
context: Arc<StaticSystemContext>,
|
||||
/// Pins the child to its own model, so a test can script parent and child
|
||||
/// independently (a shared script would race on who pops which step).
|
||||
model: Option<ModelHint>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl AgentCatalog for TestCatalog {
|
||||
async fn get(&self, id: &str, _child_frame: agent_loop::ids::FrameId) -> agent_loop::Result<AgentProfile> {
|
||||
async fn get(
|
||||
&self,
|
||||
id: &str,
|
||||
_child_frame: agent_loop::ids::FrameId,
|
||||
_ctx: &agent_loop::tool::ToolCtx,
|
||||
) -> agent_loop::Result<AgentProfile> {
|
||||
Ok(AgentProfile {
|
||||
id: id.into(),
|
||||
kind: AgentKind::Task,
|
||||
context: self.context.clone(),
|
||||
tools: ToolSelection::inherit(),
|
||||
toolset: None,
|
||||
model: None,
|
||||
model: self.model.clone(),
|
||||
selector: None,
|
||||
assembler: None,
|
||||
})
|
||||
@@ -540,6 +549,7 @@ async fn sync_delegate_runs_child_loop_and_returns_result() {
|
||||
);
|
||||
let catalog: Arc<dyn AgentCatalog> = Arc::new(TestCatalog {
|
||||
context: Arc::new(StaticSystemContext::new("You are a researcher.")),
|
||||
model: None,
|
||||
});
|
||||
let delegate: Arc<dyn Tool> = Arc::new(DelegateTool::new(manager.clone(), catalog, manager.store(), 5));
|
||||
|
||||
@@ -551,6 +561,7 @@ async fn sync_delegate_runs_child_loop_and_returns_result() {
|
||||
system: Arc::new(StaticSystemContext::new("root")),
|
||||
tools: ToolRegistry::new().with_arc(delegate).into_toolset(),
|
||||
model_hint: ModelHint::default(),
|
||||
selector: None,
|
||||
live_input: None,
|
||||
extensions: Default::default(),
|
||||
meta: TurnMeta::default(),
|
||||
@@ -599,6 +610,7 @@ async fn delegate_batch_fans_out_concurrently() {
|
||||
);
|
||||
let catalog: Arc<dyn AgentCatalog> = Arc::new(TestCatalog {
|
||||
context: Arc::new(StaticSystemContext::new("worker")),
|
||||
model: None,
|
||||
});
|
||||
let delegate: Arc<dyn Tool> = Arc::new(DelegateTool::new(manager.clone(), catalog, manager.store(), 5));
|
||||
|
||||
@@ -610,6 +622,7 @@ async fn delegate_batch_fans_out_concurrently() {
|
||||
system: Arc::new(StaticSystemContext::new("root")),
|
||||
tools: ToolRegistry::new().with_arc(delegate).into_toolset(),
|
||||
model_hint: ModelHint::default(),
|
||||
selector: None,
|
||||
live_input: None,
|
||||
extensions: Default::default(),
|
||||
meta: TurnMeta::default(),
|
||||
@@ -633,4 +646,178 @@ async fn delegate_batch_fans_out_concurrently() {
|
||||
);
|
||||
}
|
||||
|
||||
// ── async delegation ──
|
||||
|
||||
/// Polls until `f` holds, so a background delivery does not need a sleep.
|
||||
async fn eventually<F, Fut>(label: &str, f: F)
|
||||
where
|
||||
F: Fn() -> Fut,
|
||||
Fut: std::future::Future<Output = bool>,
|
||||
{
|
||||
let deadline = std::time::Instant::now() + Duration::from_secs(5);
|
||||
while std::time::Instant::now() < deadline {
|
||||
if f().await {
|
||||
return;
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(10)).await;
|
||||
}
|
||||
panic!("timed out waiting for: {label}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn async_delegate_returns_a_receipt_then_delivers_the_result() {
|
||||
// Parent and child get their own scripted model: the parent does NOT wait
|
||||
// for the child, so one shared script would race on who pops which step.
|
||||
let root = Arc::new(FakeModel::new("root", vec![
|
||||
Step::tool_calls("", vec![testing::call("c1", "delegate", json!({
|
||||
"agent_id": "worker", "prompt": "long job", "mode": "async", "title": "nightly",
|
||||
}))]),
|
||||
Step::message("started it"),
|
||||
]));
|
||||
let child = Arc::new(FakeModel::new("child", vec![Step::message("the long answer")]));
|
||||
|
||||
let store = Arc::new(InMemoryStore::new());
|
||||
let manager = Arc::new(
|
||||
LoopManager::builder()
|
||||
.models(Arc::new(StaticModels::new(vec![
|
||||
testing::handle(&root, "root"),
|
||||
testing::handle(&child, "child"),
|
||||
])))
|
||||
.store(store.clone())
|
||||
.build()
|
||||
.unwrap(),
|
||||
);
|
||||
let catalog: Arc<dyn AgentCatalog> = Arc::new(TestCatalog {
|
||||
context: Arc::new(StaticSystemContext::new("worker")),
|
||||
model: Some(ModelHint::name("child")),
|
||||
});
|
||||
let sink: Arc<dyn AsyncResultSink> = Arc::new(StoreSink::new(manager.store()));
|
||||
let exec: Arc<dyn AsyncExecutor> = Arc::new(InProcessExecutor::new(
|
||||
manager.clone(),
|
||||
catalog.clone(),
|
||||
manager.store(),
|
||||
sink,
|
||||
ToolRegistry::new().into_toolset(),
|
||||
));
|
||||
let delegate: Arc<dyn Tool> = Arc::new(
|
||||
DelegateTool::new(manager.clone(), catalog, manager.store(), 5).with_async(exec),
|
||||
);
|
||||
|
||||
let conv = ConversationId::new("d3");
|
||||
let frame = manager.open_root(&conv, FrameSpec::root("assistant")).await.unwrap();
|
||||
let p = TurnParams {
|
||||
frame,
|
||||
agent: "assistant".into(),
|
||||
system: Arc::new(StaticSystemContext::new("root")),
|
||||
tools: ToolRegistry::new().with_arc(delegate).into_toolset(),
|
||||
model_hint: ModelHint::default(),
|
||||
selector: None,
|
||||
live_input: None,
|
||||
extensions: Default::default(),
|
||||
meta: TurnMeta::default(),
|
||||
assembler: None,
|
||||
};
|
||||
|
||||
let handle = manager.start_turn(conv.clone(), NewMessage::user("run it"), p).await.unwrap();
|
||||
let outcome = tokio::time::timeout(Duration::from_secs(5), handle.join())
|
||||
.await
|
||||
.expect("async delegate must not block the parent turn")
|
||||
.unwrap();
|
||||
let TurnOutcome::Final { content, .. } = outcome else { panic!("got {outcome:?}") };
|
||||
assert_eq!(content, "started it");
|
||||
|
||||
// The delegating call resolved with a receipt, not with the child's answer.
|
||||
let done = store.calls_in_state(frame, &[CallState::Done]).await.unwrap();
|
||||
let receipt: Value =
|
||||
serde_json::from_str(done[0].result.as_deref().unwrap()).expect("receipt is JSON");
|
||||
assert_eq!(receipt["status"], "started");
|
||||
assert_eq!(receipt["task_id"], 1);
|
||||
|
||||
// …and the answer lands later, as its own completed call.
|
||||
let store_c = store.clone();
|
||||
eventually("the delivered result", || {
|
||||
let store = store_c.clone();
|
||||
async move {
|
||||
store
|
||||
.load(frame)
|
||||
.await
|
||||
.unwrap()
|
||||
.iter()
|
||||
.any(|m| m.calls.iter().any(|c| c.name == agent_loop::delegate::DELIVERY_CALL))
|
||||
}
|
||||
})
|
||||
.await;
|
||||
|
||||
let history = store.load(frame).await.unwrap();
|
||||
let delivery = history
|
||||
.iter()
|
||||
.find(|m| m.calls.iter().any(|c| c.name == agent_loop::delegate::DELIVERY_CALL))
|
||||
.unwrap();
|
||||
assert!(delivery.synthetic, "the delivery is not a turn the user drove");
|
||||
let call = &delivery.calls[0];
|
||||
assert_eq!(call.state, CallState::Done);
|
||||
let payload: Value = serde_json::from_str(call.result.as_deref().unwrap()).unwrap();
|
||||
assert_eq!(payload["task_id"], 1);
|
||||
assert_eq!(payload["title"], "nightly");
|
||||
assert_eq!(payload["result"], "the long answer");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn async_delegate_without_an_executor_is_refused() {
|
||||
let script = vec![
|
||||
Step::tool_calls("", vec![testing::call("c1", "delegate", json!({
|
||||
"agent_id": "worker", "prompt": "job", "mode": "async",
|
||||
}))]),
|
||||
Step::message("could not start it"),
|
||||
];
|
||||
let store = Arc::new(InMemoryStore::new());
|
||||
let manager = Arc::new(
|
||||
LoopManager::builder()
|
||||
.models(Arc::new(agent_loop::model::SingleModel::new(FakeModel::new("m", script))))
|
||||
.store(store.clone())
|
||||
.build()
|
||||
.unwrap(),
|
||||
);
|
||||
let catalog: Arc<dyn AgentCatalog> = Arc::new(TestCatalog {
|
||||
context: Arc::new(StaticSystemContext::new("worker")),
|
||||
model: None,
|
||||
});
|
||||
// No `with_async`: the mode must fail, never silently run sync — a turn
|
||||
// that asked not to wait would otherwise block on the child.
|
||||
let delegate: Arc<dyn Tool> =
|
||||
Arc::new(DelegateTool::new(manager.clone(), catalog, manager.store(), 5));
|
||||
|
||||
let conv = ConversationId::new("d4");
|
||||
let frame = manager.open_root(&conv, FrameSpec::root("assistant")).await.unwrap();
|
||||
let p = TurnParams {
|
||||
frame,
|
||||
agent: "assistant".into(),
|
||||
system: Arc::new(StaticSystemContext::new("root")),
|
||||
tools: ToolRegistry::new().with_arc(delegate).into_toolset(),
|
||||
model_hint: ModelHint::default(),
|
||||
selector: None,
|
||||
live_input: None,
|
||||
extensions: Default::default(),
|
||||
meta: TurnMeta::default(),
|
||||
assembler: None,
|
||||
};
|
||||
|
||||
let handle = manager.start_turn(conv.clone(), NewMessage::user("run it"), p).await.unwrap();
|
||||
tokio::time::timeout(Duration::from_secs(5), handle.join())
|
||||
.await
|
||||
.expect("turn hung")
|
||||
.unwrap();
|
||||
|
||||
let failed = store.calls_in_state(frame, &[CallState::Failed]).await.unwrap();
|
||||
assert_eq!(failed.len(), 1);
|
||||
assert!(
|
||||
failed[0].result.as_deref().unwrap().contains("async mode is not available"),
|
||||
"{:?}",
|
||||
failed[0].result
|
||||
);
|
||||
// Nothing was spawned: no child frame was ever opened.
|
||||
assert!(store.active_frames(&conv).await.unwrap().iter().all(|f| f.spec.depth == 0));
|
||||
}
|
||||
|
||||
use agent_loop::delegate::{AsyncExecutor, AsyncResultSink, InProcessExecutor, StoreSink};
|
||||
use std::collections::HashSet;
|
||||
|
||||
@@ -0,0 +1,507 @@
|
||||
//! Golden tests of the projection (blueprint §13): the exact wire shape of
|
||||
//! every layer, for every provider knob. These assert full messages, not just
|
||||
//! properties — a change in what a model receives must show up here.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use agent_loop::activation::{Activation, ActivationSource, ToolRendering};
|
||||
use agent_loop::context::{AssembleInput, ContextAssembler, LinearAssembler, SystemContext};
|
||||
use agent_loop::ids::{ConversationId, FrameId, MessageId};
|
||||
use agent_loop::model::ModelInfo;
|
||||
use agent_loop::prelude::async_trait;
|
||||
use agent_loop::projection::{
|
||||
MediaBlob, MediaSource, Projection, ReasoningEcho, ResultLimit, ToolResultDigest,
|
||||
};
|
||||
use agent_loop::store::{
|
||||
CallOutcome, FrameSpec, HistoryStore, NewCall, NewMessage, NewSummary, StoredCall,
|
||||
StoredMessage,
|
||||
};
|
||||
use agent_loop::store_memory::InMemoryStore;
|
||||
use agent_loop::tool::ToolOutput;
|
||||
use serde_json::{Value, json};
|
||||
|
||||
// ── fixtures ─────────────────────────────────────────────────────────────────
|
||||
|
||||
async fn store_and_frame(name: &str) -> (Arc<dyn HistoryStore>, FrameId) {
|
||||
let store = Arc::new(InMemoryStore::new());
|
||||
let conv = ConversationId::new(name);
|
||||
let frame = store.open_frame(&conv, None, FrameSpec::root("assistant")).await.unwrap();
|
||||
(store, frame)
|
||||
}
|
||||
|
||||
fn input(frame: FrameId, system: SystemContext, model: ModelInfo) -> AssembleInput {
|
||||
AssembleInput { frame, system, model, round: 0 }
|
||||
}
|
||||
|
||||
fn tool_def(name: &str) -> Value {
|
||||
json!({"type":"function","function":{"name":name,"parameters":{"type":"object"}}})
|
||||
}
|
||||
|
||||
/// The Skald-flavoured configuration: every knob off the default, so the test
|
||||
/// exercises the parameterization rather than the defaults.
|
||||
fn strict() -> Projection {
|
||||
Projection {
|
||||
summary_suffix: Some("[End of summary]".into()),
|
||||
interrupted_text: "Error: tool call was interrupted.".into(),
|
||||
rejected_default: "User rejected this tool call.".into(),
|
||||
cancelled_default: "Tool call was cancelled by the user.".into(),
|
||||
reasoning_placeholder: Some("(no reasoning recorded for this step)".into()),
|
||||
reasoning_echo: ReasoningEcho::Both,
|
||||
activation_anchor_tool: Some("activate_tools".into()),
|
||||
..Projection::default()
|
||||
}
|
||||
}
|
||||
|
||||
struct Stub(Vec<Activation>);
|
||||
|
||||
#[async_trait]
|
||||
impl ActivationSource for Stub {
|
||||
async fn activations(&self, _frame: FrameId) -> agent_loop::Result<Vec<Activation>> {
|
||||
Ok(self.0.clone())
|
||||
}
|
||||
}
|
||||
|
||||
// ── system layers ────────────────────────────────────────────────────────────
|
||||
|
||||
#[tokio::test]
|
||||
async fn prompt_cache_turns_the_static_prefix_into_a_cache_breakpoint() {
|
||||
let (store, frame) = store_and_frame("p1").await;
|
||||
|
||||
let plain = LinearAssembler::new()
|
||||
.build(&store, &input(frame, SystemContext::base("BASE"), ModelInfo::default()))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(plain[0], json!({ "role": "system", "content": "BASE" }));
|
||||
|
||||
let cached = LinearAssembler::new()
|
||||
.build(&store, &input(frame, SystemContext::base("BASE"), ModelInfo {
|
||||
prompt_cache: true,
|
||||
..ModelInfo::default()
|
||||
}))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
cached[0],
|
||||
json!({
|
||||
"role": "system",
|
||||
"content": [{ "type": "text", "text": "BASE",
|
||||
"cache_control": { "type": "ephemeral" } }],
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn static_and_dynamic_layers_land_on_their_sides_of_the_history() {
|
||||
let (store, frame) = store_and_frame("p2").await;
|
||||
store.append(frame, NewMessage::user("hi")).await.unwrap();
|
||||
|
||||
let system = SystemContext::base("BASE")
|
||||
.with_static("FORMAT RULES")
|
||||
.with_static("<scratchpad/>")
|
||||
.with_dynamic("MEMORY")
|
||||
.with_dynamic("NOW")
|
||||
.with_reminder("REMEMBER");
|
||||
|
||||
let msgs = LinearAssembler::new()
|
||||
.build(&store, &input(frame, system, ModelInfo::default()))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(msgs, vec![
|
||||
json!({ "role": "system", "content": "BASE" }),
|
||||
json!({ "role": "system", "content": "FORMAT RULES" }),
|
||||
json!({ "role": "system", "content": "<scratchpad/>" }),
|
||||
json!({ "role": "user", "content": "hi" }),
|
||||
// The dynamic layers are ONE trailing block, joined by the separator.
|
||||
json!({ "role": "system", "content": "MEMORY\n\n---\nNOW" }),
|
||||
json!({ "role": "system", "content": "REMEMBER" }),
|
||||
]);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn summary_replaces_covered_history_and_carries_its_suffix() {
|
||||
let (store, frame) = store_and_frame("p3").await;
|
||||
let m1 = store.append(frame, NewMessage::user("old question")).await.unwrap();
|
||||
store.append(frame, NewMessage::assistant("old answer", None)).await.unwrap();
|
||||
store.append(frame, NewMessage::user("new question")).await.unwrap();
|
||||
store
|
||||
.save_summary(frame, NewSummary { text: "They discussed old stuff.".into(), covered_up_to: m1 })
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let msgs = LinearAssembler::new()
|
||||
.with_projection(strict())
|
||||
.build(&store, &input(frame, SystemContext::base("BASE"), ModelInfo::default()))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(
|
||||
msgs[1],
|
||||
json!({
|
||||
"role": "system",
|
||||
"content": "[CONTEXT SUMMARY — earlier messages were compacted into this summary]\n\n\
|
||||
They discussed old stuff.\n\n[End of summary]",
|
||||
})
|
||||
);
|
||||
let joined = msgs.iter().filter_map(|m| m["content"].as_str()).collect::<Vec<_>>().join("|");
|
||||
assert!(joined.contains("old answer"), "history after the cut must survive");
|
||||
assert!(!joined.contains("old question"), "covered history must be gone");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn the_window_never_opens_on_half_an_exchange() {
|
||||
let (store, frame) = store_and_frame("p4").await;
|
||||
store.append(frame, NewMessage::user("first")).await.unwrap();
|
||||
let asst = store.append(frame, NewMessage::assistant("calling", None)).await.unwrap();
|
||||
let call = store.append_call(asst, NewCall::new("t", json!({})).with_provider_id("c1")).await.unwrap();
|
||||
store.resolve_call(call, &CallOutcome::Completed(ToolOutput::Text("r".into()))).await.unwrap();
|
||||
store.append(frame, NewMessage::user("second")).await.unwrap();
|
||||
|
||||
// A window of 2 would start on the assistant+tool group: it is dropped.
|
||||
let msgs = LinearAssembler::new()
|
||||
.with_max_messages(2)
|
||||
.build(&store, &input(frame, SystemContext::base("BASE"), ModelInfo::default()))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let roles: Vec<&str> = msgs.iter().filter_map(|m| m["role"].as_str()).collect();
|
||||
assert_eq!(roles, ["system", "user"]);
|
||||
}
|
||||
|
||||
// ── tool calls and results ───────────────────────────────────────────────────
|
||||
|
||||
/// Seeds one assistant turn with a call in each terminal state, plus a survivor.
|
||||
async fn seed_states(store: &Arc<dyn HistoryStore>, frame: FrameId) -> MessageId {
|
||||
store.append(frame, NewMessage::user("go")).await.unwrap();
|
||||
let msg = store.append(frame, NewMessage::assistant("working", None)).await.unwrap();
|
||||
|
||||
let done = store.append_call(msg, NewCall::new("a", json!({})).with_provider_id("c1")).await.unwrap();
|
||||
store.resolve_call(done, &CallOutcome::Completed(ToolOutput::Text("ok".into()))).await.unwrap();
|
||||
|
||||
let failed = store.append_call(msg, NewCall::new("b", json!({})).with_provider_id("c2")).await.unwrap();
|
||||
store.resolve_call(failed, &CallOutcome::Failed("boom".into())).await.unwrap();
|
||||
|
||||
let rejected = store.append_call(msg, NewCall::new("c", json!({})).with_provider_id("c3")).await.unwrap();
|
||||
store.resolve_call(rejected, &CallOutcome::Rejected { reason: String::new() }).await.unwrap();
|
||||
|
||||
let cancelled = store.append_call(msg, NewCall::new("d", json!({})).with_provider_id("c4")).await.unwrap();
|
||||
store.resolve_call(cancelled, &CallOutcome::Cancelled).await.unwrap();
|
||||
|
||||
// Never resolved: a crash survivor.
|
||||
store.append_call(msg, NewCall::new("e", json!({})).with_provider_id("c5")).await.unwrap();
|
||||
msg
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn every_call_state_gets_a_result_the_model_can_read() {
|
||||
let (store, frame) = store_and_frame("p5").await;
|
||||
seed_states(&store, frame).await;
|
||||
|
||||
let msgs = LinearAssembler::new()
|
||||
.with_projection(strict())
|
||||
.build(&store, &input(frame, SystemContext::base("BASE"), ModelInfo::default()))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let results: Vec<(&str, &str)> = msgs
|
||||
.iter()
|
||||
.filter(|m| m["role"] == "tool")
|
||||
.map(|m| (m["tool_call_id"].as_str().unwrap(), m["content"].as_str().unwrap()))
|
||||
.collect();
|
||||
assert_eq!(results, vec![
|
||||
("c1", "ok"),
|
||||
("c2", "Error: boom"),
|
||||
// The rejection recorded an empty reason: the configured note stands in.
|
||||
("c3", "User rejected this tool call."),
|
||||
// A recorded note wins over the configured default.
|
||||
("c4", "Cancelled by user."),
|
||||
("c5", "Error: tool call was interrupted."),
|
||||
]);
|
||||
|
||||
// The assistant turn itself: calls in order, and a stand-in reasoning
|
||||
// because none was recorded.
|
||||
let asst = msgs.iter().find(|m| m["role"] == "assistant").unwrap();
|
||||
assert_eq!(asst["tool_calls"][0], json!({
|
||||
"id": "c1", "type": "function",
|
||||
"function": { "name": "a", "arguments": "{}" },
|
||||
}));
|
||||
assert_eq!(asst["reasoning_content"], "(no reasoning recorded for this step)");
|
||||
assert_eq!(asst["reasoning"], "(no reasoning recorded for this step)");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn reasoning_echo_is_per_provider_and_never_empty() {
|
||||
let (store, frame) = store_and_frame("p6").await;
|
||||
store.append(frame, NewMessage::user("q")).await.unwrap();
|
||||
store.append(frame, NewMessage::assistant("a", Some("because".into()))).await.unwrap();
|
||||
store.append(frame, NewMessage::user("q2")).await.unwrap();
|
||||
// An empty stored reasoning must not produce an empty field.
|
||||
store.append(frame, NewMessage::assistant("a2", Some(String::new()))).await.unwrap();
|
||||
|
||||
let one = LinearAssembler::new()
|
||||
.build(&store, &input(frame, SystemContext::base("B"), ModelInfo::default()))
|
||||
.await
|
||||
.unwrap();
|
||||
let first = one.iter().find(|m| m["content"] == "a").unwrap();
|
||||
assert_eq!(first["reasoning_content"], "because");
|
||||
assert!(first.get("reasoning").is_none(), "ContentOnly must not echo `reasoning`");
|
||||
let second = one.iter().find(|m| m["content"] == "a2").unwrap();
|
||||
assert!(second.get("reasoning_content").is_none());
|
||||
|
||||
let both = LinearAssembler::new()
|
||||
.with_projection(strict())
|
||||
.build(&store, &input(frame, SystemContext::base("B"), ModelInfo::default()))
|
||||
.await
|
||||
.unwrap();
|
||||
let first = both.iter().find(|m| m["content"] == "a").unwrap();
|
||||
assert_eq!(first["reasoning"], "because");
|
||||
// No placeholder for a plain assistant turn — only tool-calling ones need it.
|
||||
let second = both.iter().find(|m| m["content"] == "a2").unwrap();
|
||||
assert!(second.get("reasoning_content").is_none());
|
||||
}
|
||||
|
||||
struct Digest;
|
||||
|
||||
#[async_trait]
|
||||
impl ToolResultDigest for Digest {
|
||||
async fn condense(&self, name: &str, _args: &Value, result: &str) -> Option<String> {
|
||||
Some(format!("[{name}: {} chars]", result.len()))
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn over_long_results_are_condensed_only_for_previous_turns() {
|
||||
let (store, frame) = store_and_frame("p7").await;
|
||||
|
||||
// Turn 1 (previous), then turn 2 (current), both with a long result.
|
||||
for (user, id) in [("first", "c1"), ("second", "c2")] {
|
||||
store.append(frame, NewMessage::user(user)).await.unwrap();
|
||||
let msg = store.append(frame, NewMessage::assistant("run", None)).await.unwrap();
|
||||
let call = store
|
||||
.append_call(msg, NewCall::new("read_file", json!({})).with_provider_id(id))
|
||||
.await
|
||||
.unwrap();
|
||||
store
|
||||
.resolve_call(call, &CallOutcome::Completed(ToolOutput::Text("x".repeat(100))))
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
let cfg = Projection {
|
||||
max_tool_result: Some(ResultLimit { max_chars: 10, previous_turns_only: true }),
|
||||
..Projection::default()
|
||||
};
|
||||
let msgs = LinearAssembler::new()
|
||||
.with_projection(cfg.clone())
|
||||
.with_digest(Arc::new(Digest))
|
||||
.build(&store, &input(frame, SystemContext::base("B"), ModelInfo::default()))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let results: Vec<&str> = msgs
|
||||
.iter()
|
||||
.filter(|m| m["role"] == "tool")
|
||||
.map(|m| m["content"].as_str().unwrap())
|
||||
.collect();
|
||||
assert_eq!(results[0], "[read_file: 100 chars]", "a previous turn is condensed");
|
||||
assert_eq!(results[1].len(), 100, "the current turn keeps its full output");
|
||||
|
||||
// Without a digest the crate truncates on a char boundary.
|
||||
let msgs = LinearAssembler::new()
|
||||
.with_projection(cfg)
|
||||
.build(&store, &input(frame, SystemContext::base("B"), ModelInfo::default()))
|
||||
.await
|
||||
.unwrap();
|
||||
let first = msgs.iter().find(|m| m["role"] == "tool").unwrap();
|
||||
assert_eq!(first["content"], "xxxxxxxxxx… [truncated]");
|
||||
}
|
||||
|
||||
// ── dynamic tool loading ─────────────────────────────────────────────────────
|
||||
|
||||
/// An assistant turn with two calls, the activation being the SECOND one.
|
||||
async fn seed_two_calls(store: &Arc<dyn HistoryStore>, frame: FrameId) -> MessageId {
|
||||
store.append(frame, NewMessage::user("use gmail")).await.unwrap();
|
||||
let anchor = store.append(frame, NewMessage::assistant("activating", None)).await.unwrap();
|
||||
let other = store
|
||||
.append_call(anchor, NewCall::new("read_file", json!({})).with_provider_id("c1"))
|
||||
.await
|
||||
.unwrap();
|
||||
store.resolve_call(other, &CallOutcome::Completed(ToolOutput::Text("file".into()))).await.unwrap();
|
||||
let act = store
|
||||
.append_call(anchor, NewCall::new("activate_tools", json!({"groups":["gmail"]})).with_provider_id("c2"))
|
||||
.await
|
||||
.unwrap();
|
||||
store.resolve_call(act, &CallOutcome::Completed(ToolOutput::Text("activated".into()))).await.unwrap();
|
||||
anchor
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn deferred_reference_marks_the_activation_result_not_the_first_one() {
|
||||
let (store, frame) = store_and_frame("p8").await;
|
||||
let anchor = seed_two_calls(&store, frame).await;
|
||||
|
||||
let msgs = LinearAssembler::new()
|
||||
.with_projection(strict())
|
||||
.with_activation(Arc::new(Stub(vec![Activation {
|
||||
anchor,
|
||||
defs: vec![tool_def("mcp__gmail__send")],
|
||||
}])))
|
||||
.build(&store, &input(frame, SystemContext::base("B"), ModelInfo {
|
||||
tool_rendering: ToolRendering::DeferredToolReference,
|
||||
..ModelInfo::default()
|
||||
}))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let tools: Vec<&Value> = msgs.iter().filter(|m| m["role"] == "tool").collect();
|
||||
assert!(tools[0].get("_tool_references").is_none(), "the read_file result is not the anchor");
|
||||
assert_eq!(tools[1]["_tool_references"], json!(["mcp__gmail__send"]));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn system_tool_block_is_appended_after_the_result_group() {
|
||||
let (store, frame) = store_and_frame("p9").await;
|
||||
let anchor = seed_two_calls(&store, frame).await;
|
||||
|
||||
let msgs = LinearAssembler::new()
|
||||
.with_projection(strict())
|
||||
.with_activation(Arc::new(Stub(vec![Activation {
|
||||
anchor,
|
||||
defs: vec![tool_def("mcp__gmail__send")],
|
||||
}])))
|
||||
.build(&store, &input(frame, SystemContext::base("B"), ModelInfo {
|
||||
tool_rendering: ToolRendering::SystemToolBlock,
|
||||
..ModelInfo::default()
|
||||
}))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let idx = msgs
|
||||
.iter()
|
||||
.position(|m| m["role"] == "system" && m.get("tools").is_some())
|
||||
.expect("no system+tools block");
|
||||
assert_eq!(msgs[idx]["tools"][0]["function"]["name"], "mcp__gmail__send");
|
||||
assert!(msgs[idx].get("content").is_none(), "the block carries tools, not content");
|
||||
assert_eq!(msgs[idx - 1]["role"], "tool", "it comes right after the group");
|
||||
assert!(!msgs.iter().any(|m| m.get("_tool_references").is_some()));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn inline_mode_injects_nothing_at_all() {
|
||||
let (store, frame) = store_and_frame("p10").await;
|
||||
let anchor = seed_two_calls(&store, frame).await;
|
||||
|
||||
let msgs = LinearAssembler::new()
|
||||
.with_projection(strict())
|
||||
.with_activation(Arc::new(Stub(vec![Activation {
|
||||
anchor,
|
||||
defs: vec![tool_def("mcp__gmail__send")],
|
||||
}])))
|
||||
.build(&store, &input(frame, SystemContext::base("B"), ModelInfo::default()))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(!msgs.iter().any(|m| m.get("tools").is_some()));
|
||||
assert!(!msgs.iter().any(|m| m.get("_tool_references").is_some()));
|
||||
}
|
||||
|
||||
// ── media ────────────────────────────────────────────────────────────────────
|
||||
|
||||
struct Png(&'static str);
|
||||
|
||||
#[async_trait]
|
||||
impl MediaBlob for Png {
|
||||
fn name(&self) -> &str { self.0 }
|
||||
async fn size(&self) -> Option<u64> { Some(72) }
|
||||
async fn head(&self) -> Option<Vec<u8>> { Some(b"\x89PNG\r\n\x1a\n........".to_vec()) }
|
||||
async fn read_all(&self) -> Option<Vec<u8>> {
|
||||
let mut v = b"\x89PNG\r\n\x1a\n".to_vec();
|
||||
v.extend_from_slice(&[0xAA; 64]);
|
||||
Some(v)
|
||||
}
|
||||
}
|
||||
|
||||
/// Every user message has one image; every tool call produces one.
|
||||
struct Media;
|
||||
|
||||
#[async_trait]
|
||||
impl MediaSource for Media {
|
||||
async fn message_media(&self, _msg: &StoredMessage) -> Vec<Arc<dyn MediaBlob>> {
|
||||
vec![Arc::new(Png("shot.png"))]
|
||||
}
|
||||
async fn call_media(&self, _calls: &[StoredCall]) -> Vec<Arc<dyn MediaBlob>> {
|
||||
vec![Arc::new(Png("tool.png"))]
|
||||
}
|
||||
fn skipped_text(&self, _msg: &StoredMessage, skipped: &[usize]) -> Option<String> {
|
||||
(!skipped.is_empty()).then(|| format!("\n[files: {}]", skipped.len()))
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn media_is_inlined_for_the_current_turn_and_textual_before_it() {
|
||||
let (store, frame) = store_and_frame("p11").await;
|
||||
store.append(frame, NewMessage::user("old picture")).await.unwrap();
|
||||
store.append(frame, NewMessage::assistant("seen", None)).await.unwrap();
|
||||
store.append(frame, NewMessage::user("new picture")).await.unwrap();
|
||||
|
||||
let msgs = LinearAssembler::new()
|
||||
.with_media(Arc::new(Media))
|
||||
.build(&store, &input(frame, SystemContext::base("B"), ModelInfo {
|
||||
capabilities: vec!["vision".into()],
|
||||
..ModelInfo::default()
|
||||
}))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
// The previous turn keeps the textual note, no parts.
|
||||
assert_eq!(msgs[1], json!({ "role": "user", "content": "old picture\n[files: 1]" }));
|
||||
// The current turn inlines the bytes.
|
||||
let current = msgs.last().unwrap();
|
||||
assert_eq!(current["content"][0], json!({ "type": "text", "text": "new picture" }));
|
||||
assert!(
|
||||
current["content"][1]["image_url"]["url"]
|
||||
.as_str()
|
||||
.unwrap()
|
||||
.starts_with("data:image/png;base64,")
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_model_without_vision_never_receives_bytes() {
|
||||
let (store, frame) = store_and_frame("p12").await;
|
||||
store.append(frame, NewMessage::user("picture")).await.unwrap();
|
||||
|
||||
let msgs = LinearAssembler::new()
|
||||
.with_media(Arc::new(Media))
|
||||
.build(&store, &input(frame, SystemContext::base("B"), ModelInfo::default()))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(msgs[1], json!({ "role": "user", "content": "picture\n[files: 1]" }));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn tool_produced_media_rides_a_synthetic_user_message_after_the_group() {
|
||||
let (store, frame) = store_and_frame("p13").await;
|
||||
store.append(frame, NewMessage::user("read the image")).await.unwrap();
|
||||
let msg = store.append(frame, NewMessage::assistant("reading", None)).await.unwrap();
|
||||
let call = store
|
||||
.append_call(msg, NewCall::new("read_file", json!({"path":"a.png"})).with_provider_id("c1"))
|
||||
.await
|
||||
.unwrap();
|
||||
store.resolve_call(call, &CallOutcome::Completed(ToolOutput::Text("image".into()))).await.unwrap();
|
||||
|
||||
let msgs = LinearAssembler::new()
|
||||
.with_media(Arc::new(Media))
|
||||
.build(&store, &input(frame, SystemContext::base("B"), ModelInfo {
|
||||
capabilities: vec!["vision".into()],
|
||||
..ModelInfo::default()
|
||||
}))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let last = msgs.last().unwrap();
|
||||
assert_eq!(last["role"], "user");
|
||||
assert_eq!(last["content"][0]["type"], "image_url");
|
||||
assert_eq!(msgs[msgs.len() - 2]["role"], "tool", "it follows the result group");
|
||||
}
|
||||
@@ -0,0 +1,442 @@
|
||||
//! Recovery suite (blueprint §8/§13): the post-crash store is built **by hand**
|
||||
//! on `InMemoryStore` — a call left `Running`, a child frame nobody closed, two
|
||||
//! siblings of an interrupted batch — and recovery is asked to make it
|
||||
//! well-formed again and continue.
|
||||
//!
|
||||
//! No DB, no network: the states a real crash produces are exactly the states a
|
||||
//! test can write, because every transition is a store write.
|
||||
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::Duration;
|
||||
|
||||
use agent_loop::context::StaticSystemContext;
|
||||
use agent_loop::delegate::{AgentCatalog, AgentKind, AgentProfile, AgentSummary, ToolSelection};
|
||||
use agent_loop::ids::{ConversationId, FrameId, ToolCallId};
|
||||
use agent_loop::manager::{LoopManager, TurnMeta, TurnParams};
|
||||
use agent_loop::model::{ModelHint, StaticModels};
|
||||
use agent_loop::prelude::async_trait;
|
||||
use agent_loop::recovery::{HumanDecision, PendingPolicy, RecoveryPolicy, RunningPolicy};
|
||||
use agent_loop::store::{
|
||||
CallState, FrameSpec, HistoryStore, NewCall, NewMessage, StoredCall,
|
||||
};
|
||||
use agent_loop::store_memory::InMemoryStore;
|
||||
use agent_loop::testing::{self, FakeModel, Step};
|
||||
use agent_loop::tool::{
|
||||
RestartHint, Tool, ToolCtx, ToolFailure, ToolOutput, ToolRegistry, ToolSet,
|
||||
};
|
||||
use serde_json::{Value, json};
|
||||
|
||||
// ── tools ────────────────────────────────────────────────────────────────────
|
||||
|
||||
/// Idempotent: safe to re-run after a crash. Counts its executions.
|
||||
struct Counter {
|
||||
runs: Arc<Mutex<usize>>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl Tool for Counter {
|
||||
fn name(&self) -> &str { "counter" }
|
||||
fn definition(&self) -> Value {
|
||||
json!({"type":"function","function":{"name":"counter","parameters":{"type":"object"}}})
|
||||
}
|
||||
async fn call(&self, _args: Value, _ctx: &ToolCtx) -> Result<ToolOutput, ToolFailure> {
|
||||
let mut runs = self.runs.lock().unwrap();
|
||||
*runs += 1;
|
||||
Ok(ToolOutput::Text(format!("run {runs}")))
|
||||
}
|
||||
}
|
||||
|
||||
/// Non-idempotent (a shell command already had its effect): must NOT be re-run.
|
||||
struct SideEffect {
|
||||
runs: Arc<Mutex<usize>>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl Tool for SideEffect {
|
||||
fn name(&self) -> &str { "shell" }
|
||||
fn definition(&self) -> Value {
|
||||
json!({"type":"function","function":{"name":"shell","parameters":{"type":"object"}}})
|
||||
}
|
||||
fn restart_hint(&self) -> RestartHint { RestartHint::MarkInterrupted }
|
||||
async fn call(&self, _args: Value, _ctx: &ToolCtx) -> Result<ToolOutput, ToolFailure> {
|
||||
*self.runs.lock().unwrap() += 1;
|
||||
Ok(ToolOutput::Text("ran".into()))
|
||||
}
|
||||
}
|
||||
|
||||
/// Stands in for the delegate: recovery never calls it (a spawned frame is the
|
||||
/// cascade's business), so running it at all is a bug.
|
||||
struct NeverCalled;
|
||||
|
||||
#[async_trait]
|
||||
impl Tool for NeverCalled {
|
||||
fn name(&self) -> &str { "delegate" }
|
||||
fn definition(&self) -> Value {
|
||||
json!({"type":"function","function":{"name":"delegate","parameters":{"type":"object"}}})
|
||||
}
|
||||
async fn call(&self, _args: Value, _ctx: &ToolCtx) -> Result<ToolOutput, ToolFailure> {
|
||||
panic!("recovery re-ran a sub-agent dispatch instead of cascading its frame");
|
||||
}
|
||||
}
|
||||
|
||||
// ── catalog ──────────────────────────────────────────────────────────────────
|
||||
|
||||
/// Every child agent runs on the `child` model with its own prompt — so a test
|
||||
/// can prove a resumed sub-agent came back as ITSELF (B3), not as the root.
|
||||
struct Catalog;
|
||||
|
||||
#[async_trait]
|
||||
impl AgentCatalog for Catalog {
|
||||
async fn get(
|
||||
&self,
|
||||
id: &str,
|
||||
_child_frame: FrameId,
|
||||
_ctx: &ToolCtx,
|
||||
) -> agent_loop::Result<AgentProfile> {
|
||||
Ok(AgentProfile {
|
||||
id: id.into(),
|
||||
kind: AgentKind::Task,
|
||||
context: Arc::new(StaticSystemContext::new(format!("You are {id}."))),
|
||||
tools: ToolSelection::inherit(),
|
||||
toolset: None,
|
||||
model: Some(ModelHint::name("child")),
|
||||
selector: None,
|
||||
assembler: None,
|
||||
})
|
||||
}
|
||||
async fn list(&self, _kind: AgentKind) -> Vec<AgentSummary> { Vec::new() }
|
||||
}
|
||||
|
||||
// ── harness ──────────────────────────────────────────────────────────────────
|
||||
|
||||
struct H {
|
||||
manager: Arc<LoopManager>,
|
||||
store: Arc<InMemoryStore>,
|
||||
tools: Arc<dyn ToolSet>,
|
||||
conv: ConversationId,
|
||||
root: FrameId,
|
||||
counter: Arc<Mutex<usize>>,
|
||||
shell: Arc<Mutex<usize>>,
|
||||
/// The child's script — a test asserting "the model was NOT called" leaves
|
||||
/// it empty, and `FakeModel` panics if anything pops from it.
|
||||
child: Arc<FakeModel>,
|
||||
}
|
||||
|
||||
impl H {
|
||||
async fn new(root_script: Vec<Step>, child_script: Vec<Step>) -> Self {
|
||||
let store = Arc::new(InMemoryStore::new());
|
||||
let root_model = Arc::new(FakeModel::new("root", root_script));
|
||||
let child = Arc::new(FakeModel::new("child", child_script));
|
||||
let manager = Arc::new(
|
||||
LoopManager::builder()
|
||||
.models(Arc::new(StaticModels::new(vec![
|
||||
testing::handle(&root_model, "root"),
|
||||
testing::handle(&child, "child"),
|
||||
])))
|
||||
.store(store.clone())
|
||||
.build()
|
||||
.unwrap(),
|
||||
);
|
||||
let counter = Arc::new(Mutex::new(0));
|
||||
let shell = Arc::new(Mutex::new(0));
|
||||
let tools: Arc<dyn ToolSet> = ToolRegistry::new()
|
||||
.with(Counter { runs: counter.clone() })
|
||||
.with(SideEffect { runs: shell.clone() })
|
||||
.with(NeverCalled)
|
||||
.into_toolset();
|
||||
|
||||
let conv = ConversationId::new("rec");
|
||||
let root = store
|
||||
.open_frame(&conv, None, FrameSpec::root("assistant"))
|
||||
.await
|
||||
.unwrap();
|
||||
Self { manager, store, tools, conv, root, counter, shell, child }
|
||||
}
|
||||
|
||||
fn params(&self) -> TurnParams {
|
||||
TurnParams {
|
||||
frame: self.root,
|
||||
agent: "assistant".into(),
|
||||
system: Arc::new(StaticSystemContext::new("You are the assistant.")),
|
||||
tools: self.tools.clone(),
|
||||
model_hint: ModelHint::default(),
|
||||
selector: None,
|
||||
live_input: None,
|
||||
extensions: Default::default(),
|
||||
meta: TurnMeta::default(),
|
||||
assembler: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// An assistant message with one call left in flight — what a crash leaves.
|
||||
async fn interrupted_call(&self, frame: FrameId, name: &str) -> ToolCallId {
|
||||
self.store.append(frame, NewMessage::user("do it")).await.unwrap();
|
||||
let msg = self
|
||||
.store
|
||||
.append(frame, NewMessage::assistant("calling", None))
|
||||
.await
|
||||
.unwrap();
|
||||
self.store.append_call(msg, NewCall::new(name, json!({}))).await.unwrap()
|
||||
}
|
||||
|
||||
/// A child frame spawned by `call`, with its prompt already appended.
|
||||
async fn child_frame(&self, agent: &str, call: ToolCallId) -> FrameId {
|
||||
let frame = self
|
||||
.store
|
||||
.open_frame(&self.conv, Some(self.root), FrameSpec {
|
||||
agent: agent.into(),
|
||||
prompt: Some("go find out".into()),
|
||||
depth: 1,
|
||||
parent_call: Some(call),
|
||||
meta: Value::Null,
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
self.store.append(frame, NewMessage::agent("go find out")).await.unwrap();
|
||||
frame
|
||||
}
|
||||
|
||||
async fn call(&self, id: ToolCallId) -> StoredCall {
|
||||
self.store.get_call(id).await.unwrap().unwrap()
|
||||
}
|
||||
|
||||
async fn recover_with(&self, policy: RecoveryPolicy) -> agent_loop::recovery::RecoveryReport {
|
||||
let recovery = self.manager.recovery(Arc::new(Catalog), policy);
|
||||
tokio::time::timeout(Duration::from_secs(5), recovery.run(&self.conv, &self.params()))
|
||||
.await
|
||||
.expect("recovery hung")
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
async fn recover(&self) -> agent_loop::recovery::RecoveryReport {
|
||||
self.recover_with(RecoveryPolicy::default()).await
|
||||
}
|
||||
}
|
||||
|
||||
// ── interrupted calls ────────────────────────────────────────────────────────
|
||||
|
||||
#[tokio::test]
|
||||
async fn an_interrupted_idempotent_call_is_re_executed_then_the_turn_continues() {
|
||||
let h = H::new(vec![Step::message("all done")], vec![]).await;
|
||||
let call = h.interrupted_call(h.root, "counter").await;
|
||||
|
||||
let report = h.recover().await;
|
||||
|
||||
assert_eq!(*h.counter.lock().unwrap(), 1, "the call must run exactly once");
|
||||
let call = h.call(call).await;
|
||||
assert_eq!(call.state, CallState::Done);
|
||||
assert_eq!(call.result.as_deref(), Some("run 1"));
|
||||
assert_eq!(report.calls_reexecuted, 1);
|
||||
assert_eq!(report.frames_resumed, 1, "the frame then ran a normal round");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn an_interrupted_call_with_side_effects_is_failed_not_re_run() {
|
||||
// D7: `shell` declares MarkInterrupted, so re-running it could repeat an
|
||||
// effect that already happened.
|
||||
let h = H::new(vec![Step::message("I stopped mid-command")], vec![]).await;
|
||||
let call = h.interrupted_call(h.root, "shell").await;
|
||||
|
||||
let report = h.recover().await;
|
||||
|
||||
assert_eq!(*h.shell.lock().unwrap(), 0, "a non-idempotent tool must NOT be re-run");
|
||||
let call = h.call(call).await;
|
||||
assert_eq!(call.state, CallState::Failed);
|
||||
assert!(call.result.as_deref().unwrap().contains("interrupted"), "{:?}", call.result);
|
||||
assert_eq!(report.calls_failed, 1);
|
||||
assert_eq!(report.calls_reexecuted, 0);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn the_policy_can_refuse_to_re_run_anything() {
|
||||
let h = H::new(vec![Step::message("continuing")], vec![]).await;
|
||||
let call = h.interrupted_call(h.root, "counter").await;
|
||||
|
||||
h.recover_with(RecoveryPolicy {
|
||||
on_running: RunningPolicy::MarkInterrupted,
|
||||
..RecoveryPolicy::default()
|
||||
})
|
||||
.await;
|
||||
|
||||
assert_eq!(*h.counter.lock().unwrap(), 0, "the policy overrides the tool's hint");
|
||||
assert_eq!(h.call(call).await.state, CallState::Failed);
|
||||
}
|
||||
|
||||
// ── awaiting human ───────────────────────────────────────────────────────────
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_call_awaiting_a_human_is_asked_again() {
|
||||
let h = H::new(vec![Step::message("approved and done")], vec![]).await;
|
||||
let call = h.interrupted_call(h.root, "counter").await;
|
||||
h.store.set_call_state(call, CallState::AwaitingHuman).await.unwrap();
|
||||
|
||||
let report = h.recover().await;
|
||||
|
||||
// ReAsk re-runs it through the gate — here an allowing one, so it executes.
|
||||
assert_eq!(*h.counter.lock().unwrap(), 1);
|
||||
assert_eq!(h.call(call).await.state, CallState::Done);
|
||||
assert_eq!(report.calls_reexecuted, 1);
|
||||
assert!(!report.left_pending);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn leave_pending_stops_and_touches_nothing() {
|
||||
// No model step scripted: running the loop would panic, which is the point —
|
||||
// a frame with an unanswered call must not be driven.
|
||||
let h = H::new(vec![], vec![]).await;
|
||||
let call = h.interrupted_call(h.root, "counter").await;
|
||||
h.store.set_call_state(call, CallState::AwaitingHuman).await.unwrap();
|
||||
|
||||
let report = h.recover_with(RecoveryPolicy {
|
||||
on_awaiting_human: PendingPolicy::LeavePending,
|
||||
..RecoveryPolicy::default()
|
||||
})
|
||||
.await;
|
||||
|
||||
assert!(report.left_pending);
|
||||
assert_eq!(report.frames_resumed, 0);
|
||||
assert_eq!(*h.counter.lock().unwrap(), 0);
|
||||
assert_eq!(h.call(call).await.state, CallState::AwaitingHuman, "still the human's to answer");
|
||||
}
|
||||
|
||||
// ── the cascade ──────────────────────────────────────────────────────────────
|
||||
|
||||
#[tokio::test]
|
||||
async fn an_interrupted_sub_agent_finishes_as_itself_then_the_parent_continues() {
|
||||
let h = H::new(
|
||||
vec![Step::message("the root's final answer")],
|
||||
vec![Step::message("the child's answer")],
|
||||
)
|
||||
.await;
|
||||
let call = h.interrupted_call(h.root, "delegate").await;
|
||||
let child = h.child_frame("researcher", call).await;
|
||||
|
||||
let report = h.recover().await;
|
||||
|
||||
// The child ran under ITS agent's prompt and model (B3), not the root's.
|
||||
let seen = h.child.requests();
|
||||
assert_eq!(seen.len(), 1, "the child model ran exactly once");
|
||||
assert!(
|
||||
serde_json::to_string(&seen[0].messages).unwrap().contains("You are researcher."),
|
||||
"the resumed frame must run its own agent's context: {:?}",
|
||||
seen[0].messages
|
||||
);
|
||||
|
||||
// Its answer became the parent call's result, and the child frame is closed.
|
||||
let call = h.call(call).await;
|
||||
assert_eq!(call.state, CallState::Done);
|
||||
assert_eq!(call.result.as_deref(), Some("the child's answer"));
|
||||
assert!(!h.store.get_frame(child).await.unwrap().unwrap().active);
|
||||
assert_eq!(report.frames_resumed, 2, "child then root");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_child_that_finished_but_never_propagated_is_not_re_run() {
|
||||
// The wedge: the turn died in the instant between the child's last message
|
||||
// and its result reaching the parent. Re-running the model would ask it to
|
||||
// answer a question it already answered — the empty child script asserts
|
||||
// that never happens.
|
||||
let h = H::new(vec![Step::message("root wraps up")], vec![]).await;
|
||||
let call = h.interrupted_call(h.root, "delegate").await;
|
||||
let child = h.child_frame("researcher", call).await;
|
||||
h.store
|
||||
.append(child, NewMessage::assistant("already done", None))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let report = h.recover().await;
|
||||
|
||||
let call = h.call(call).await;
|
||||
assert_eq!(call.state, CallState::Done);
|
||||
assert_eq!(call.result.as_deref(), Some("already done"));
|
||||
assert_eq!(h.child.requests().len(), 0, "the child's LLM must not be called again");
|
||||
assert_eq!(report.frames_resumed, 1, "only the parent ran");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn an_interrupted_parallel_batch_is_reaped_and_the_parent_resumes() {
|
||||
let h = H::new(vec![Step::message("carrying on without them")], vec![]).await;
|
||||
|
||||
// Two delegate calls in one round, two live children: impossible for a
|
||||
// linear stack, so it can only be a batch caught mid-flight.
|
||||
h.store.append(h.root, NewMessage::user("do both")).await.unwrap();
|
||||
let msg = h.store.append(h.root, NewMessage::assistant("", None)).await.unwrap();
|
||||
let c1 = h.store.append_call(msg, NewCall::new("delegate", json!({}))).await.unwrap();
|
||||
let c2 = h.store.append_call(msg, NewCall::new("delegate", json!({}))).await.unwrap();
|
||||
let f1 = h.child_frame("a1", c1).await;
|
||||
let f2 = h.child_frame("a2", c2).await;
|
||||
|
||||
let report = h.recover().await;
|
||||
|
||||
assert_eq!(report.batches_reaped, 1);
|
||||
for (call, frame) in [(c1, f1), (c2, f2)] {
|
||||
let call = h.call(call).await;
|
||||
assert_eq!(call.state, CallState::Failed);
|
||||
assert!(call.result.as_deref().unwrap().contains("parallel batch"), "{:?}", call.result);
|
||||
assert!(!h.store.get_frame(frame).await.unwrap().unwrap().active);
|
||||
}
|
||||
assert_eq!(h.child.requests().len(), 0, "a reaped batch is not re-run");
|
||||
assert_eq!(report.frames_resumed, 1, "the root continues with the failures in view");
|
||||
}
|
||||
|
||||
// ── resolve_pending ──────────────────────────────────────────────────────────
|
||||
|
||||
#[tokio::test]
|
||||
async fn approving_after_a_restart_runs_the_call_and_continues() {
|
||||
let h = H::new(vec![Step::message("done, as approved")], vec![]).await;
|
||||
let call = h.interrupted_call(h.root, "counter").await;
|
||||
h.store.set_call_state(call, CallState::AwaitingHuman).await.unwrap();
|
||||
|
||||
h.manager
|
||||
.resolve_pending(call, HumanDecision::Approved, Arc::new(Catalog), &h.params())
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(*h.counter.lock().unwrap(), 1);
|
||||
let call = h.call(call).await;
|
||||
assert_eq!(call.state, CallState::Done);
|
||||
assert_eq!(call.result.as_deref(), Some("run 1"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn rejecting_after_a_restart_records_the_refusal_and_continues() {
|
||||
let h = H::new(vec![Step::message("understood, I won't")], vec![]).await;
|
||||
let call = h.interrupted_call(h.root, "shell").await;
|
||||
h.store.set_call_state(call, CallState::AwaitingHuman).await.unwrap();
|
||||
|
||||
h.manager
|
||||
.resolve_pending(
|
||||
call,
|
||||
HumanDecision::Rejected { reason: "no thanks".into() },
|
||||
Arc::new(Catalog),
|
||||
&h.params(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(*h.shell.lock().unwrap(), 0);
|
||||
let call = h.call(call).await;
|
||||
assert_eq!(call.state, CallState::Rejected);
|
||||
assert_eq!(call.result.as_deref(), Some("no thanks"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn resolving_an_already_terminal_call_is_a_no_op() {
|
||||
let h = H::new(vec![], vec![]).await;
|
||||
let call = h.interrupted_call(h.root, "counter").await;
|
||||
h.store
|
||||
.resolve_call(call, &agent_loop::store::CallOutcome::Cancelled)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let report = h
|
||||
.manager
|
||||
.resolve_pending(call, HumanDecision::Approved, Arc::new(Catalog), &h.params())
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
// Cancelled is terminal and never re-executed (blueprint §8.2).
|
||||
assert_eq!(*h.counter.lock().unwrap(), 0);
|
||||
assert_eq!(h.call(call).await.state, CallState::Cancelled);
|
||||
assert_eq!(report.frames_resumed, 0);
|
||||
}
|
||||
Reference in New Issue
Block a user