llm: add dynamic tool loading (DTL) — Kimi system-tools + Anthropic tool-reference
Nightly Build / build (push) Successful in 6m51s

Replace the old session_mcp_grants/stack_mcp_grants table pair with
a single activated_tools table that anchors each activation at the
assistant message_id that triggered it. The durable write moves from
the activate_tools tool itself to the round loop (handle_tool_call),
which has the message_id the DTL serializer positions injected tool
blocks against.

Introduce DtlMode (None / AnthropicToolReference / KimiSystemTools),
resolved per model from capabilities (opt-in via tool_search)
combined with the provider's dtl_format(). The message builder inserts
Kimi system {tools} blocks at the activation position, or emits
Anthropic tool_reference markers on the tool result. The tool-def
surface (all_tool_defs) switches shape: Anthropic declares everything
deferred; Kimi omits activated tools from the top-level array (system
takes over); None keeps the old grant-set logic.

Anthropic client: accept structured system arrays (cache_control on
the static block when DTL is active), carry defer_loading through
conversion, emit tool_reference blocks on result messages. Prompt
caching enabled exactly when DTL is active (anthropic provider).

MCP server list in the prompt is now a static catalogue (not split
Available/Active) — the split invalidated the cache on every activation.
Groundwork for providers.yaml dtl: key; Moonshot/Kimi providers wired
with kimi_system_tools and the k3* enrich now adds tool_search.
Compactor re-anchors activations whose message was compacted away.
This commit is contained in:
2026-07-24 20:48:04 +01:00
parent 3c52587dee
commit d1d0a2af26
21 changed files with 579 additions and 226 deletions
+44
View File
@@ -26,6 +26,50 @@ pub struct LlmEntry {
/// Input capabilities of the resolved model (`vision`, `video`, …), from
/// `llm_models.capabilities`. Drives multimodal attachment inlining.
pub capabilities: Vec<String>,
/// Dynamic-tool-loading serialization mode for this model (resolved from
/// `capabilities` + provider type). Selects how a session's *activated* tools
/// are put on the wire so that activating one does not invalidate the
/// provider's prompt-cache prefix.
pub dtl: DtlMode,
}
/// Per-model dynamic-tool-loading (DTL) serialization mode. Resolved in
/// `build_entry` from the model's provider (via [`dtl_mode_from_format`]) gated by
/// the `tool_search` capability. It selects how a session's activated tools are serialized so
/// that an `activate_tools` call does not break the provider's prompt-cache
/// prefix. The persistence layer (`activated_tools`) is model-agnostic; this is
/// the model-aware half that renders that state per provider.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum DtlMode {
/// Today's behaviour: activated tools ride in the top-level `tools` array.
/// Correct, but every activation invalidates the cache from that point on.
/// The fallback for Ollama / LM Studio / generic OpenAI-compat providers.
#[default]
None,
/// Anthropic Messages API: candidate tools are declared `defer_loading:true`
/// and loaded via a custom client-side `tool_reference` expansion emitted at
/// the `activate_tools` result (preserves the cache; no 5-result cap).
AnthropicToolReference,
/// Kimi K3 (OpenAI-compatible): activated tools are injected as `system`
/// messages carrying a `tools` field, appended at the activation position so
/// the prefix stays byte-identical (append-only).
KimiSystemTools,
}
/// Parses a provider-declared DTL format name — from a native provider
/// (`AnthropicProvider::dtl_format`) or from `providers.yaml` (`dtl:` on a declared
/// provider) — into a [`DtlMode`]. Unknown names → [`DtlMode::None`].
///
/// The *format* is a property of the provider (which wire its client speaks);
/// whether a given model *uses* it is gated separately by the `tool_search`
/// capability (see `build_entry`). So there is no hardcoded model list — enabling
/// a new Kimi-compatible provider is a `providers.yaml` edit.
pub fn dtl_mode_from_format(fmt: &str) -> DtlMode {
match fmt {
"anthropic_tool_reference" => DtlMode::AnthropicToolReference,
"kimi_system_tools" => DtlMode::KimiSystemTools,
_ => DtlMode::None,
}
}
// ── Provider ──────────────────────────────────────────────────────────────────