tic: run per-user under a system-agent scheduler, with a run log
Nightly Build / build (push) Successful in 6m58s
Nightly Build / build (push) Successful in 6m58s
Reframe TIC from an ownerless global loop into a per-user system agent.
The events it reads live in each user's own encrypted mcp_events, the
connectors that produced them run in that user's container, and the
notifications go to that user's hub — so the previous design (built
against the ownerless Conversation bundle, writing into system.db and
notifying a hub with no subscribers) was inert by construction.
Core changes
- TicManager owns no timer and no user list. It now exposes
run_for(user_id, pool, sessions, hub): one tick for one user, over
deps unpacked from that user's UserContext. Removed from the
Conversation bundle; Skald::tic_manager() is gone.
- New spawn_system_agents in wiring.rs: one instance-wide loop, spawned
post-construction with a Weak<Skald> (like spawn_user_lifecycle).
Each pass walks the directory and runs TIC for one user at a time —
sequential, because a pass is N container round-trips and N LLM calls
nobody is waiting on. A ConfigKeyUpdated on the interval key cuts the
current wait short; enabled is re-read per pass.
- A user whose database is still locked is skipped (normal, not an
error): the pool is the unlock token, so a user who hasn't logged in
since restart has no readable events and nowhere to record a skip.
- The configured tic.security_group is re-checked per user through
run_context::reconcile_group_for_user — a restricted member never
gets a tool set their role wouldn't grant; unconfigured starts from
role_default_run_context, never None (None = catch-all = wider).
- New system_agent_runs owner table (no user_id column — the file is
the owner): start/finish split so a crash leaves a visible 'running'
row, swept to 'failed' by the next start; safe because the scheduler
is sequential and single-instance. An idle tick writes nothing.
- counting_notify wraps the notify tool so the run log can report
notifications emitted without the tool knowing it's counted.
- The session's event channel is drained by a spawned task instead of
a dropped receiver — the translator awaits its sends and would wedge
at capacity.
EventLog::{Persist,Discard} on McpManager::new
- mcp_events is an owner table and its only reader (TIC) is per-user,
so an event is something that happened to someone. The per-user
runtime gets Persist; the ownerless global runtime gets Discard (its
pool is system.db, rows would be unattributable and unread).
API + UI
- GET /api/system-agents/runs: the caller's own run history, scoped
through require_context with no admin override (same promise as the
rest of the private pool).
- web/components/system-agents.js replaces tic-sessions.js. The old
#tic debug page inferred runs from leftover ephemeral sessions; the
new #system-agents page (sidebar group 'extensions', visible to
everyone — the data is the caller's own) reads the real run log.
- i18n: tic.* keys replaced with system_agents.* in en/it/fr.
Docs
- New docs/system-agents.md (user-facing: what TIC does, why it runs
per person, why a run can be missing). Updated docs/settings.md and
docs/index.md.
- agents/tic/AGENT.md reframed per-user: events are that person's,
memory is user-memory/ (private) — never shared-memory/.
- CLAUDE.md records the system-agents design and the EventLog seam.
This commit is contained in:
@@ -2,19 +2,22 @@
|
||||
//! spawns, each concentrated in one readable place instead of being scattered
|
||||
//! through the constructor.
|
||||
//!
|
||||
//! Owner-bound background loops (cron, session-cancel, ticket-listener, tic) have
|
||||
//! Owner-bound background loops (cron, session-cancel, ticket-listener) have
|
||||
//! moved per-user into `UserContextFactory::build`. What remains here are the
|
||||
//! instance-wide tasks: LLM-log cleanup on the registry pool, MCP server
|
||||
//! initialization, and the user-lifecycle reconciler (which needs the finished
|
||||
//! `Arc<Skald>` and is therefore spawned separately, after construction).
|
||||
//! initialization, and the two that need the finished `Arc<Skald>` and are
|
||||
//! therefore spawned separately, after construction — the user-lifecycle
|
||||
//! reconciler and the system-agent scheduler.
|
||||
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use core_api::system_bus::{RecvError, SystemEvent};
|
||||
use tracing::{info, warn};
|
||||
|
||||
use crate::config::CoreConfig;
|
||||
use crate::config::{CoreConfig, TicConfig};
|
||||
use crate::elicitation::ElicitationBridge;
|
||||
use crate::tic::{TicManager, TIC_INTERVAL_MINUTES_KEY};
|
||||
|
||||
use super::bundles::{Conversation, Integrations, Interaction, Tasks};
|
||||
use super::runtime::Runtime;
|
||||
@@ -41,10 +44,11 @@ pub(super) fn wire(
|
||||
|
||||
/// Spawns the instance-wide background tasks.
|
||||
///
|
||||
/// Owner-bound loops (cron, session-cancel, ticket-listener, tic) are **not**
|
||||
/// spawned here — they run per-user inside `UserContext`. Session cancellation is
|
||||
/// handled directly by the API handlers (which have `AuthUser` and resolve the
|
||||
/// per-user context). TIC is deferred until connectors return (§13).
|
||||
/// Owner-bound loops (cron, session-cancel, ticket-listener) are **not** spawned
|
||||
/// here — they run per-user inside `UserContext`. Session cancellation is handled
|
||||
/// directly by the API handlers (which have `AuthUser` and resolve the per-user
|
||||
/// context). The system-agent scheduler needs the finished instance and lives in
|
||||
/// [`spawn_system_agents`].
|
||||
pub(super) fn spawn_background(
|
||||
rt: &Runtime,
|
||||
_tasks: &Tasks,
|
||||
@@ -169,3 +173,107 @@ pub(super) fn spawn_user_lifecycle(skald: &Arc<super::Skald>) {
|
||||
info!("user-lifecycle: reconciler stopped");
|
||||
});
|
||||
}
|
||||
|
||||
/// Spawns the **system-agent scheduler** — the instance-wide timer that runs the
|
||||
/// background agents nobody asked for (today: TIC).
|
||||
///
|
||||
/// One loop, not one per user. Every pass walks the user directory and runs the
|
||||
/// agent for each user **sequentially**: a pass means N container round-trips and
|
||||
/// N LLM calls, and doing them concurrently would spike the box every interval
|
||||
/// for no gain — nobody is waiting on a background tick.
|
||||
///
|
||||
/// A user whose database is still locked is **skipped**, and that is the normal
|
||||
/// case rather than an error: the pool is the unlock token (§9), so a user who
|
||||
/// has not logged in since the last restart has no readable events, no session
|
||||
/// store, and no place to record the skip. It is logged at INFO and the pass
|
||||
/// moves on; their events keep accumulating and are picked up by the first pass
|
||||
/// after they log in.
|
||||
///
|
||||
/// Spawned after `Skald` is fully built, like [`spawn_user_lifecycle`] and for
|
||||
/// the same reason: it resolves each user's runtime through `Skald::user_context`.
|
||||
/// The back-reference is [`std::sync::Weak`].
|
||||
pub(super) fn spawn_system_agents(skald: &Arc<super::Skald>, tic_config: TicConfig) {
|
||||
let weak = Arc::downgrade(skald);
|
||||
let shutdown = skald.rt.shutdown_token.clone();
|
||||
let mut sys_rx = skald.rt.system_bus.subscribe();
|
||||
|
||||
let tic = TicManager::new(
|
||||
tic_config,
|
||||
Arc::clone(&skald.rt.config),
|
||||
Arc::clone(&skald.rt.db),
|
||||
);
|
||||
|
||||
skald.rt.supervisor.spawn("system-agents", async move {
|
||||
info!("system-agents: scheduler started");
|
||||
|
||||
'outer: loop {
|
||||
// Re-read the interval each pass so a Settings change lands without a
|
||||
// restart; a live change also cuts the current wait short.
|
||||
let wait = Duration::from_secs(tic.interval_secs().await);
|
||||
let deadline = tokio::time::sleep(wait);
|
||||
tokio::pin!(deadline);
|
||||
|
||||
loop {
|
||||
tokio::select! {
|
||||
_ = shutdown.cancelled() => break 'outer,
|
||||
_ = &mut deadline => break,
|
||||
ev = sys_rx.recv() => match ev {
|
||||
Ok(SystemEvent::ConfigKeyUpdated { key, .. })
|
||||
if key == TIC_INTERVAL_MINUTES_KEY =>
|
||||
{
|
||||
info!("system-agents: interval changed, rescheduling");
|
||||
continue 'outer;
|
||||
}
|
||||
Err(RecvError::Closed) => break 'outer,
|
||||
_ => {}
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
let Some(skald) = weak.upgrade() else { break };
|
||||
tic_pass(&skald, &tic).await;
|
||||
}
|
||||
|
||||
info!("system-agents: scheduler stopped");
|
||||
});
|
||||
}
|
||||
|
||||
/// One TIC pass over the whole directory, one user at a time.
|
||||
async fn tic_pass(skald: &Arc<super::Skald>, tic: &Arc<TicManager>) {
|
||||
if !tic.is_enabled().await {
|
||||
return;
|
||||
}
|
||||
|
||||
let users = match skald.users().list().await {
|
||||
Ok(u) => u,
|
||||
Err(e) => {
|
||||
warn!(error = %e, "system-agents: cannot list users, skipping this pass");
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
for user in users.into_iter().filter(|u| u.active) {
|
||||
if skald.rt.shutdown_token.is_cancelled() {
|
||||
break;
|
||||
}
|
||||
|
||||
if !skald.users().is_unlocked(&user.id) {
|
||||
info!(
|
||||
user = %user.id, username = %user.username,
|
||||
"TIC: skipped — the user's database is still encrypted (not logged in since the last restart)",
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
// Unlocked, so this resolves (and is normally already live from their login).
|
||||
let Some(ctx) = skald.user_context(&user.id).await else {
|
||||
warn!(user = %user.id, "TIC: skipped — could not resolve the user's runtime");
|
||||
continue;
|
||||
};
|
||||
|
||||
if let Err(e) = tic.run_for(&user.id, &ctx.pool, &ctx.sessions, &ctx.chat_hub).await {
|
||||
// One user's failure must not end the pass for everyone after them.
|
||||
warn!(user = %user.id, error = %e, "TIC: tick failed");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user