13 Commits

Include renames

Author	SHA1	Message	Date
Daniele	5980bdb5b9	feat(users): unlock and start unencrypted users at boot ... A login is what makes an *encrypted* database readable; for an unencrypted one it gated nothing but the runtime — the file has no key and is already readable by this process. The cost was user-visible and read as a bug: after every restart the Telegram bot answered "your account is locked, log in via the web app", cron fired nothing and no background agent ran, until a human opened the SPA. `Skald::new` now calls `UserManager::unlock_all_unencrypted`, which registers the pools exactly as a login would and refuses an encrypted or inactive user. Unlocking alone only makes the data readable, so `wiring::spawn_unlocked_user_runtimes` then builds a `UserContext` for each — cron, the notify queue, the hub and the per-user MCP runtime all hang off it. That build is a background supervisor task rather than part of `new()`: it starts every member's MCP servers inside their container, and the HTTP listener must not wait behind it. The same two steps run per user off the lifecycle bus (`UserCreated`, `UserActiveChanged{active:true}`, after the container `ensure`), so a member created at runtime does not wait for the next restart. Two boundaries stay where they were. Authentication is untouched: `SessionStore` sits above `UserManager`, so no HTTP request authenticates as anyone because of this. And the auto-unlock is deliberately not on a lazy path such as `Skald::user_context` — `revoke_user_runtime` locks a pool synchronously and expects nothing to re-open it, so the writers of that map stay boot, login and the bus. `open_db` and the two unencrypted openers now share `register_unlocked` and `open_unencrypted_file`; `open_unencrypted` (the supervision path) still does not register its pool.	2026-08-10 12:16:33 +01:00
Daniele	c27da4e6ab	feat(skills): rebuild the skill system for the multi-user model ... Per blueprint/skill-project.md: the old single-namespace, hand-maintained index is gone, replaced by a read-only, two-scope tree whose index is a runtime function of its content. - skills/ index generated at runtime (crates/skald-core/src/skills/: inventory, install, validate, watch), injected through the new <!-- SKILLS_LIST --> placeholder in AGENT.md (agents/common/skills.md); meta.json inject_skills flag removed. 11 chat/task agents carry the include, the 4 system agents do not. - Two trees, both read-only in both directions: skills/shared/{id} (the group's) and skills/{username}/{id} (one member's own, on the stable userid). The root is closed too: UserFs::SkillMounts + RouteError (alias probe, plain-denied paths, no home fallback) and a per-user .skills-root/{userid} container mount with the two scope mounts nested inside, plus the fifth self-heal axis (skills_mounted). - Agent verbs: skill_register/skill_delete (Config group, global scope behind the new skill.manage capability), fetch_repo for public repos, list_items(type="skills"); reads are plain read_file on the printed path. Seeded @fs_read skills/* allow. - Freshness: a digest-gated watcher on the two trees emits SystemEvent::SkillsChanged, whose subscriber rebuilds the frozen prompt prefix via Skald::invalidate_prompt_prefix; in-process writers invalidate directly. - The build ships no skills: the three bundled skills (ics2json, mcp-builder, skill-creator) and skills/index.md are removed, skills/ is instance data (gitignored, not packaged, no longer pruned by update.sh). - Docs: skills.md, agents.md, shared-folders.md added; docs/index.md and agents/README.md updated.	2026-08-08 23:05:35 +01:00
dguiducci	85536755ee	feat: a "Run now" button for the memory lints — one pass, for whoever asked ... The two memory lints run weekly, which is right for maintenance and wrong for the moment somebody has just reorganised their notes and wants to know what the lint makes of them. Each agent's tab now carries a button that starts one pass immediately, for the caller. It runs as the caller — their pool, their sessions, their hub — so the report lands with the person who asked. The shared lint is the interesting case: its scheduled pass runs as the admin because the shared store belongs to nobody, but a member pressing the button reads the same store and gets the report themselves, which is coherent with shared memory being readable by every member anyway. Two settings are treated differently on purpose. Due-ness is skipped, exactly as manual /compact skips the compactor's token threshold: the interval answers *when*, and a human asking is a good enough answer to that. The Enabled switch is honoured: it answers *whether*, and that one is the admin's. The conversation review gets no button (AgentScope::PerSubject): it is about somebody else and picks its own subjects, so "run it for me" has no meaning. The frontend reads that from the agent's scope, not from a list of ids. A second starter breaks an invariant the scheduler used to hold for free. system_agent_runs::start sweeps any leftover `running` row of the same agent to `failed` before inserting, which was safe only because one sequential loop was the only thing that ever started a pass; a manual run overlapping a scheduled one would have marked a healthy run as interrupted and duplicated its work. So the agent list moves out of the scheduler and onto Skald as SystemAgents, which holds the registry plus an in-flight guard both paths claim through — keyed on what the pass is *about*, so an instance-wide agent is one slot no matter who runs it, and a per-subject review is keyed on the subject rather than on the supervisor lending the runtime. has_work is answered synchronously, before anything is spawned: it leaves no run row, so without that the button would say "started" over a log that never gains a row. Everything after it is spawned — a pass is an LLM turn, and no HTTP request should be held open for one. The run row exists before the browser is answered, so the log itself is the progress surface; the page polls it quietly until the pass leaves `running`.	2026-08-02 21:40:21 +01:00
dguiducci	4f10528368	feat: conversation review — a nightly report on a supervised person's conversations ... The first AgentScope::PerSubject system agent, and the reason that scope exists. Once a night, for each person with a supervision edge, it reads every message that person and the assistant exchanged since the previous review — across all their conversations — and writes one report for the people who supervise them. Schema (all registry except reports): - supervision(subject_user_id, supervisor_user_id): the generic §0.1 edge, answering both 'whom does a background agent look at' and 'who may read what it produced', with real FKs so deleting a user cascades both ways - system_agent_coverage(agent_id, subject_user_id, covered_through): the per-subject watermark that makes 'everything since last time' a window — neither system_agent_runs (history for humans) nor system_agent_state (advances before the work), and advanced only on a completed pass so a crash re-covers instead of skipping - reports (owner schema, the second two-homes table after memory_docs): instance rows land in system.db, deliberately cleartext to the box owner, who is the intended reader (§2); the subject cannot see them structurally The pass reads the subject's database inside a supervisor's runtime, so the ephemeral session and run row land in the watcher's file; iteration is over subjects, so two parents watching one child get one review; and the subject need not be logged in when their space is unencrypted — via the new UserManager::open_unencrypted, which refuses an encrypted user outright (no key to be had) and never registers the pool as unlocked. The agent declares the new AgentMeta flag allow_tools: false, so its turn gets an empty tool registry — nothing for a prompt injection in the transcript to call — and produces its report as its final assistant message, read back from chat_history and parsed (NOTHING_TO_REPORT sentinel, no row on quiet days). chat_history::conversation_window is the transcript query; its four filters (non-ephemeral, depth 0, non-synthetic, non-empty) each guard a specific way the review would otherwise be wrong, and tool calls are absent by construction. Cadence is Run at (hour) rather than Interval — 4am local by default — with due-ness answered inside has_work against the coverage watermark, so a machine off for three days covers the whole stretch in one pass. Reports announce ReportCreated on the system bus (no subscriber yet). run_ephemeral_turn gains a per-pass system_substitutions map, which the review uses to hand the model the subject's profile under __SUBJECT_PROFILE__ — the system-context substitutions describe the session owner, the wrong person here. docs/system-agents.md gains the conversation review section; CLAUDE.md documents the scope, the tables and the tool-less design.	2026-08-02 20:30:27 +01:00
dguiducci	046f060fcd	rename the TIC system agent to event triage ... TIC said nothing about what the agent does, and named the wrong thing: the tick belongs to the scheduler, which is generic and lives outside it. The agent's only decision is whether an incoming event deserves an interruption — it sorts, it never acts — so it is now event-triage, matching the functional naming of the two memory lints. - agents/tic/ -> agents/event-triage/, module tic/ -> event_triage/, TicManager -> EventTriageManager, TicConfig -> EventTriageConfig - agent id and chat source: "tic" -> "event-triage" - config keys: tic.* -> event_triage.*, and the config.yml section tic: -> event_triage: (greenfield: previously set values fall back to defaults) - i18n en/it/fr: Event triage / Triage eventi / Tri des evenements; dropped the stale "TIC sessions" mention from the debug-pages description - docs/system-agents.md, docs/index.md, docs/settings.md, CLAUDE.md, SKALD.md	2026-07-28 21:59:37 +01:00
dguiducci	434e27d7c2	system agents: generalise the scheduler and add the two memory lints ... Memory is kept as a maintained wiki, and a wiki nobody prunes rots. This adds the scheduled maintenance pass, and generalises the machinery TIC had grown so that a background agent is a trait impl rather than a loop of its own. Two lint agents, not one. The private pass runs per user over `user-memory/` and reports to them; the shared pass runs once over `shared-memory/`, where the interesting defect is different — a note failing the table rule, i.e. private business written where every member can read it. It names the note and the category without repeating the content, since restating it spreads the very thing being flagged. Both share `agents/common/memory-lint.md`. Both are read-only, and that is enforced twice: the prompt says report-never- repair, and `shared-memory/*` writes are already `@fs_write require`, so an agent that tried to fix something would raise an approval card from an unattended pass, which is auto-denied. Read-only is the only design that works here, not merely the safe one. One scheduler for cadences three orders of magnitude apart. TIC runs every few minutes, a lint weekly — the case that tempts a second loop. It stays one because the wake-up decides nothing: `base_tick` picks only how often to look, and whether an agent runs for a user is `is_due` against persisted state. Due-ness moves out of the run log into a new owner table, `system_agent_state`. The two answer different questions: the run log skips idle ticks so it stays a history rather than a heartbeat, while scheduling needs every attempt. Reading due-ness off the log would re-run an idle agent on every tick and never bring a weekly one due once its last productive run aged out. Persisting it is also what makes a long interval survive a restart — an in-memory deadline is fine at TIC's scale, but a weekly agent on a box rebooted every few days would have it re-armed before it ever fired. The shared store belongs to nobody, so `AgentScope::Instance` runs that pass as the first unlocked admin. An ownerless run would write its trace into system.db, which the runs endpoint shows to nobody by design, and its notify() would have no recipient; attributing it to a user keeps the whole per-user surface working unchanged. Settings move to where the run log is. `ConfigSet` gains `owner`, so placement is data on the set rather than a page that knows set names; the System agents page grows one tab per agent holding its description, its settings (admin only) and its runs — "why did this do nothing last night?" is half a schedule question and half a log question. The form is shared with the Config page, and writes still go through PUT /api/config/{key}. Fixes an authorization gap found on the way: neither /api/config handler took the caller into account, so any authenticated session could read and write instance-wide config. The sidebar hiding the page is presentation, not access control. Both are now admin-gated.	2026-07-28 21:24:16 +01:00
dguiducci	165af19774	tic: run per-user under a system-agent scheduler, with a run log ... Reframe TIC from an ownerless global loop into a per-user system agent. The events it reads live in each user's own encrypted mcp_events, the connectors that produced them run in that user's container, and the notifications go to that user's hub — so the previous design (built against the ownerless Conversation bundle, writing into system.db and notifying a hub with no subscribers) was inert by construction. Core changes - TicManager owns no timer and no user list. It now exposes run_for(user_id, pool, sessions, hub): one tick for one user, over deps unpacked from that user's UserContext. Removed from the Conversation bundle; Skald::tic_manager() is gone. - New spawn_system_agents in wiring.rs: one instance-wide loop, spawned post-construction with a Weak<Skald> (like spawn_user_lifecycle). Each pass walks the directory and runs TIC for one user at a time — sequential, because a pass is N container round-trips and N LLM calls nobody is waiting on. A ConfigKeyUpdated on the interval key cuts the current wait short; enabled is re-read per pass. - A user whose database is still locked is skipped (normal, not an error): the pool is the unlock token, so a user who hasn't logged in since restart has no readable events and nowhere to record a skip. - The configured tic.security_group is re-checked per user through run_context::reconcile_group_for_user — a restricted member never gets a tool set their role wouldn't grant; unconfigured starts from role_default_run_context, never None (None = catch-all = wider). - New system_agent_runs owner table (no user_id column — the file is the owner): start/finish split so a crash leaves a visible 'running' row, swept to 'failed' by the next start; safe because the scheduler is sequential and single-instance. An idle tick writes nothing. - counting_notify wraps the notify tool so the run log can report notifications emitted without the tool knowing it's counted. - The session's event channel is drained by a spawned task instead of a dropped receiver — the translator awaits its sends and would wedge at capacity. EventLog::{Persist,Discard} on McpManager::new - mcp_events is an owner table and its only reader (TIC) is per-user, so an event is something that happened to someone. The per-user runtime gets Persist; the ownerless global runtime gets Discard (its pool is system.db, rows would be unattributable and unread). API + UI - GET /api/system-agents/runs: the caller's own run history, scoped through require_context with no admin override (same promise as the rest of the private pool). - web/components/system-agents.js replaces tic-sessions.js. The old #tic debug page inferred runs from leftover ephemeral sessions; the new #system-agents page (sidebar group 'extensions', visible to everyone — the data is the caller's own) reads the real run log. - i18n: tic.* keys replaced with system_agents.* in en/it/fr. Docs - New docs/system-agents.md (user-facing: what TIC does, why it runs per person, why a run can be missing). Updated docs/settings.md and docs/index.md. - agents/tic/AGENT.md reframed per-user: events are that person's, memory is user-memory/ (private) — never shared-memory/. - CLAUDE.md records the system-agents design and the EventLog seam.	2026-07-27 11:39:13 +01:00
dguiducci	305bdbdd2b	connectors: announce global-server and reinstall refreshes on the bus ... Five call-sites reached into the live-runtime refresh helpers from HTTP handlers, the same shape as the container remounts. Only three of them belonged on the bus, and finding out which was the point. global_enable and global_delete now emit McpGlobalServersChanged, and the marketplace reinstall emits ConnectorReinstalled. All three are pure reconciliation: the first only makes a connector appear; the second is already enforced by stop_server, with the snapshot refresh just tidying each user's filter; the third pushes metadata and code into what is already running. The reinstall gains something from being off the response path, since it re-copies files and restarts servers inside every live user's container. global_set_access and user_connectors_set keep calling refresh_global_mcp_access directly. Their writes *replace* a grant set, so anyone dropped from the list is being revoked and that refresh is what enforces it — on a best-effort broadcast a revoked user would keep the connector until their next login. Both carry a DELIBERATELY SYNCHRONOUS comment, since they are otherwise indistinguishable from the announced call-sites and are exactly what a later cleanup would sweep up. No behaviour change for the two synchronous paths; the three announced ones now return without waiting for the refresh.	2026-07-26 22:22:29 +01:00
dguiducci	0ba140186f	auth: make deactivation and group revocation actually revoke ... Two variants of the same defect: an admin took away access and the running system kept granting it. Deactivating or deleting a user only stopped the *next* login. `login` checks the active flag, but `require_auth` maps token -> id without re-reading the row, so an already-open session kept working over a pool whose key was still in RAM. There was no way to stop one user either: the per-user cron, hub and MCP loops all observed the *instance* shutdown token. They now take a per-user child token stored on UserContext, and Skald::revoke_user_runtime tears a single user down in a load-bearing order — revoke every session, evict and cancel the context, then lock the database, so nothing is left querying a pool we are about to close. Revoking a security group had a durable version of the same problem. The group is validated when selected and then persisted on chat_sessions.run_context, which was replayed verbatim on every later load — so a group removed from a role stayed in force on sessions that already had it, across restarts. get_or_create_handler now runs the stored value through run_context::reconcile_group_for_user, making it advisory: every load re-checks it, whether or not anyone announced the change. The degrade target is the role's default group, never None: a context with no group resolves to the catch-all `default`, whose rules are the fallback tier under every other group, so clearing widens rather than narrows. The reconcile touches only security_group, so a project session's server-built project_root and system_prompt survive a permissions edit, and it leaves the stored group alone when the role cannot be resolved — guessing on a transient error could only widen. role_default_run_context moves into the core seam so the group a session starts on and the group it falls back to cannot drift apart. Both fixes run synchronously in their handlers. Only the container half of deactivation rides the bus, as the new UserActiveChanged event: a lossy 64-slot broadcast whose contract is "settles at the next login" is the wrong transport for taking access away. Tests: revoke_user drops all of one user's sessions and nobody else's, and is a no-op when nothing is live; the reconcile degrades a revoked group to the role default, keeps an allowed one, preserves project fields in both directions, never touches an admin, and stays put when the role is unresolvable. Not exercised at runtime: no Docker/live-server run, so the end-to-end paths (deactivating a logged-in user, editing a role with sessions open) are covered by unit tests only.	2026-07-26 22:17:30 +01:00
dguiducci	c50a0d84da	containers: drive user provisioning and remounts from the system bus ... The endpoints that changed a user or a membership row also reached into ContainerManager themselves: users_mgmt called ensure()/remove(), and both shared_folders and projects called refresh_user_mounts through a local remount() helper. Every future endpoint that grants membership would have had to remember to do the same. Announce instead. SystemEventBus gains UserCreated / UserDeleted / UserMountsChanged, emitted after the DB write, and one subscriber — wiring::spawn_user_lifecycle — does the Docker work: sequentially (which serialises concurrent operations on the same container), best-effort by contract (the row is already committed, so a hiccup settles at the user's next login or at boot reconciliation), and holding only a Weak<Skald>. It is spawned after construction, like set_skald, because it reacts through Skald's own accessors. Also fixes a real gap the event makes impossible to repeat: the web setup wizard created the first admin without provisioning a container. It runs against a live server, where reconcile_all() has already happened, so that admin had no sandbox until the next restart. It now emits UserCreated like any other creator; the console shell needs no equivalent, since it runs before the server and boot reconciliation covers it. Two behaviour changes: POST /api/users and POST /api/projects no longer wait on Docker before responding. Provisioning was already best-effort, and a new project's folder is still created synchronously, so the explorer — which reads host-side — shows it at once; only execute_cmd reachability lands a moment later.	2026-07-26 21:57:25 +01:00
dguiducci	2ec394c17c	Projects: shareable, registry-backed, container-mounted; drop ticket board ... Rework projects from single-user leftovers into shareable endeavours. - DB: move `projects` from the owner bucket to the registry (system.db, not encrypted); add `owner_user_id` + `slug` (drop free `path`); new `project_members(project_id, user_id, can_write)` mirroring `shared_folder_members`. Drop `project_tickets` entirely. Only user↔agent conversations stay encrypted (per-user DB) — each member keeps a private project chat. Registry home dissolves the cross-DB-FK problem. - Filesystem/container: on disk `{WD}/projects/{owner_userid}/{slug}`, agent/container path `projects/{owner_username}/{slug}`. Two-segment routing in UserFs (ProjectMount + host_base_and_tail arm) and a second loop in build_user_fs; read-only members get a :ro mount. Reuse the shared-folder remount machinery (refresh_user_shared_folders -> refresh_user_mounts). - Remove the ticket system: ProjectTicketManager, UserContext.tickets, its wiring, and the project_tickets references in scheduled_jobs/cron. - API: repoint handlers to the registry pool + membership scoping. Sharing is self-service — owner or any write-member may add/remove members and set read/write; only the owner deletes; the owner cannot be removed. New POST/DELETE /api/projects/{id}/members[/{user_id}]. Seed `@fs_any allow projects/*`; build_runtime_run_context sets working_directory to the agent path and drops the host-path allow_fs_writes. - Frontend: create form without the free path field, owner/read-write badges; the detail page becomes header + description + sharing panel + Open chat + a file-explorer placeholder (the future primary surface). i18n en/it/fr.	2026-07-20 22:21:10 +01:00
dguiducci	587958ffe0	llm request tracking, user context cleanup, minor fixes	2026-07-11 00:23:14 +01:00
dguiducci	178a38357e	feat(users): UserManager with per-user SQLCipher, and extract skald-core crate ... Two changes developed together in one session; they share the same module structure (db/mod.rs, the core lib root) and only compile together, so they land as one commit. ## UserManager + per-user encryption (§9/§11) New `users::UserManager`: owns the system.db pool plus a map `userid -> SqlitePool` of unlocked databases. The pool *is* the unlock token — its connect options carry the DEK as SQLCipher's raw key, so an open pool means the key is in RAM until restart and dropping it re-locks (§9). Knows nothing about cookies. New `crypto` module: envelope encryption. A random 256-bit DEK encrypts `{userid}.db`; `users.database_password` holds it sealed with AES-256-GCM under `Argon2id(password, salt)`. The AEAD tag is the password verifier — one derivation both authenticates and yields the key, so encrypted users store no second hash. Cleartext users store the Argon2id output directly, compared in constant time. Argon2 runs in spawn_blocking behind a 2-permit semaphore (256 MiB per derivation). - SQLCipher via `libsqlite3-sys` `bundled-sqlcipher-vendored-openssl`, pinned <0.38 so it unifies with the one sqlx-sqlite links (a newer copy would apply the feature to a SQLite sqlx never uses). OpenSSL is vendored and static, so the binary stays self-contained. - Schema split into `create_registry_tables` (instance-wide, no user key) and `create_owner_tables` (one owner's content, identical in every file). No FK in the owner bucket may reach the registry — enforced by a standalone test. Dropped `chat_history.model_db_id` (write-only, and the only registry-crossing key); moved `projects`/`project_tickets` into the owner bucket. - Provisioning invariant: the file is written before the row, deleted after it, so a crash leaves an orphan file, never a user without a database. `open_db` never creates: a missing file is an error, not a silent empty database. Not consumed yet: no login, call sites still use the shared system.db pool. ## Extract crates/skald-core The headless core moves out of `src/` into its own crate; `skald` (server) and the coming `skald-setup` are shells around it. Two dependencies on the shell were inverted rather than dragged along, so the core names neither Tauri nor any concrete plugin: - `Plugin::tools(self: Arc<Self>)` — plugins contribute tools through this hook (sibling of `http_router`), so the core no longer downcasts to `MobileConnectorPlugin`. - `tools::restart::set_restart_handler` — the desktop shell installs its teardown-and-respawn; the core defaults to the supervisor exit code. The core loses its `desktop` feature. - `boot`'s stdout formatter moves to the binary (`src/boot_format.rs`); the core only emits tracing events. All 79 core tests pass; the binary boots and serves in a clean directory, and the mobile-connector tools still register through the new hook.	2026-07-10 16:48:51 +01:00