20 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Daniele	1b709a880f	feat(agents): learn and reuse the user's writing style ... The chat agents now treat a stated preference about how something should be written — or a correction to a draft they produced — as durable, and record it in a `## Writing style` section of `user-memory/user.md`: preferred wording, how emails open and close, what changes between the formal and the informal register, and contacts written to differently from everyone else. The section is capped at 10 lines, because it shares `user.md`'s own 40-line budget and per-recipient detail is the half that grows without bound; past that, the whole section moves into its own note and leaves a pointer behind. A rule that turns out to be wrong is corrected in place rather than joined by a second bullet contradicting it. Shipped as the shared fragment `common/writing-style.md`, included by the three `type: chat` agents. Only `assistant` injects `user.md` automatically, so the fragment closes by telling the other two to read it before drafting.	2026-08-19 11:03:42 +01:00
Daniele	0042f3dbcb	fix(image-generate): save generated images into the caller's workspace ... image_generate wrote the file into the server's own data/images/ and handed that host path to the model. It is a path in nobody's vocabulary: not the caller's home, not their container. Telegram's send_attachment therefore resolved it under the user's home and answered "file not found", and read_file, execute_cmd and the viewer could not reach it either. The web URL was the only surface that worked, which is why the failure only ever showed on Telegram -- and why the model there, having no working way to hand the file over, started inventing send_photo and send_media. Placement moves to the tool, the one place holding a ToolContext: - The manager returns bytes (generate_bytes) and no longer knows where an image goes. It has no UserFs and no session, so it never could have. - run_with saves through uploads::save_to_home into uploads/{session}/. The returned path is agent vocabulary, so every consumer resolves it, and that is the one directory the media inliner is authorized to read from -- a vision model can be shown the image it just made. execute_async, the context-free path, now fails loudly rather than writing somewhere nobody can read; same shape as execute_cmd. - The extension is sniffed rather than assumed png: it is what decides whether Telegram sends the picture inline or as an anonymous document, and providers return jpeg and webp too. The file is named after the prompt, so it reads as something in the explorer and in Telegram. The result still carries a url, since the chat renders Markdown images and  beats naming a file the user then has to open. It points at /api/file?path=..., which resolves through the caller's own UserFs. The old /api/images/{id} route is removed: it had no writer left once placement moved, and it addressed one instance-wide directory behind require_auth alone, with no notion of who owned the image -- the same shape as the /data static mount removed before it. That leaves data_root unused, so the manager no longer knows about the server's filesystem at all. Docs: the Telegram page explains send_attachment as the channel's equivalent of show_file_to_user; the ComfyUI page says where a generated image lands and which of the two handles to use where. Also introduces CHANGELOG.md and the standing rule for it in CLAUDE.md.	2026-08-19 10:29:14 +01:00
Daniele	402c9ffe50	feat(file-viewer): syntax highlighting for code files and chat code blocks ... Vendor highlight.js (core + python, javascript, typescript, json, yaml, bash) and highlight the file viewer's text kind (computed once per load) plus fenced blocks in renderMarkdown. Colors come from new --syn-* CSS variables aliased to the existing palette, so dark mode follows.	2026-08-11 10:41:50 +01:00
Daniele	4d1b1e63be	fix(async-tasks): wake the parent conversation by id, not by source ... DurableSink resumed the parent through ChatHub::resume(&source), which resolves whatever session the source currently points at. Since one source can now carry several conversations (secondary tabs, a reset since the task started), that pointer is no longer the conversation the result was delivered into: the recovery ran on the wrong one, found nothing pending, and returned silently — the delivered task_completed sat unread until the user's next message drove a normal turn. The sink already knows the parent session id, so resume it directly through resume_for_session, keeping the same in-flight guard. The source lookup and the now-unused pool field go with it. Adds a crate-level regression test: a completed turn, a StoreSink delivery, then a recovery — the result must drive a new round.	2026-08-10 22:13:32 +01:00
Daniele	ae0552d864	fix(container): survive a Docker daemon restart ... A user container was created with no restart policy, so anything that stops the daemon stopped it for good — an `apt upgrade` pulling a new docker-ce SIGTERMs every container (exit 143) and only those carrying a policy come back. Skald's own process survives that, and `ensure()` runs only at boot, at login and off the lifecycle bus, so nothing noticed: every `docker exec` path then failed identically until someone logged in again. The per-user MCP servers respawn-looped on `container ... is not running`, and a connector's dependency install failed with the same line. Create with `--restart unless-stopped`, and reconcile an existing container's policy in place with `docker update`. `unless-stopped` rather than `always` because `stop_all()` stops these deliberately at shutdown: the flag Docker sets there is exactly the one this policy honours, so a daemon restart while Skald is down leaves them alone and the next boot's `ensure` starts them. The in-place reconcile is deliberately not a sixth `reusable()` axis. The policy is the one property Docker can change on a live container, so making it a recreate would throw away a running container — and every `docker exec` under it — to set a flag.	2026-08-10 20:36:19 +01:00
Daniele	905fc54775	ci: build from a persistent tree instead of the runner workspace ... The previous commit tried to fix the mtime invalidation with `git restore-mtime`. It does not work on this box, in the worst way: the packaged version (2022.12) drives `git whatchanged`, which git 2.53 refuses to run without --i-still-use-this — and the tool swallows that failure and exits 0 having updated nothing. Verified on the runner: "0 commits evaluated, 675 files missing, 0 files updated", while the job happily went on to rebuild everything. Fix the cause instead of the symptom. Both building workflows now sync a tree that survives between runs and build there. `git checkout` only rewrites files whose content actually changed, so mtimes are correct as a consequence rather than as a reconstruction — and no external tool is involved. Gitea serves this repo from the same machine the runner runs on, so the sync reads the bare repo directly: no network, no token. Two properties this buys that restore-mtime did not: - The absolute source path is pinned. The runner derives its workspace path from the job definition, so editing a workflow moved it and invalidated every workspace crate by itself — the previous commit paid that cost without knowing it. - It cannot fail towards staleness. Checking out an older commit stamps those files newer, which costs an extra rebuild; restore-mtime moved mtimes backwards, which could have let cargo reuse artifacts built from newer code. Each workflow gets its own tree, for the same reason they already have their own CARGO_TARGET_DIR: they track different branches, and one shared tree would rewrite half the files on every switch.	2026-08-10 18:27:21 +01:00
Daniele	e0d75a8dc8	ci: stop rebuilding the whole workspace on every run ... Cargo decides freshness by mtime, and the Gitea runner deletes the job workspace after each run. So `actions/checkout` stamped every source file with "now" and all 20 workspace crates recompiled regardless of what the commit touched: measured on a JS-only commit, 20 of 722 rlibs rebuilt — the ~700 third-party deps stayed cached, our own code never did. That, not the size of skald-core, was the 4 minutes per architecture. Restore mtimes from git history after checkout (needs the full history, hence fetch-depth: 0 — cheap here, ~170 commits against a Gitea instance on the same machine). Also: - nightly: CARGO_INCREMENTAL=1. Release builds have incremental off by default, the worst case for a 51k-line crate. The nightly trades a marginally less optimised binary for the rebuild time; release does not. - nightly: concurrency with cancel-in-progress. The runner has capacity 1 and the nightly publishes to a fixed filename, so a queued build was 8 minutes spent on a tarball the next one overwrites. - release: its own CARGO_TARGET_DIR. CARGO_INCREMENTAL is part of cargo's profile fingerprint, so one shared cache between a workflow that sets it and one that does not would have each invalidate the other's workspace crates — reintroducing the very rebuild this removes. - packaging steps derive --target-dir from $CARGO_TARGET_DIR instead of repeating the path, so the two cannot drift.	2026-08-10 18:21:06 +01:00
Daniele	f6f94e579d	fix(file-viewer): keep rendering a PDF after a file-watcher reload ... An open PDF went blank the moment the watcher reported the file had changed, and stayed blank for the rest of the session — every later version of the file too. <pdf-view>._teardown() released the previous document with PDFDocumentProxy.destroy(), a method pdf.js no longer has: a document is torn down through its loading task. The absent method threw a TypeError, and _teardown() is the *first* statement of _open(): the page list had already been emptied, so nothing after the throw ran — no new document was loaded, and the emptied .pdfv-pages had nothing to refill it. Since _doc was never cleared either, every subsequent src hit the same throw, which is why the viewer never recovered. _open() is async and its caller (updated()) does not await it, so the TypeError surfaced only as an unhandled rejection. Tear the document down through doc.loadingTask.destroy() instead, and swallow its failure: releasing the previous document must never be able to stop the next one from loading. The same call in _open()'s stale-document path had the identical bug. Verified in headless Chromium against the real component: swapping the blob URL the way FileViewerBase._load does now reloads the document (5 pages -> 7 -> 5, correct text layer, no exceptions), including three reloads fired back-to-back so the stale-document path is exercised.	2026-08-10 17:53:00 +01:00
Daniele	5b79a5fb93	fix(ast-outline): give markdown headings a section range instead of a single line ... The markdown outline emitted `START-END` with END always equal to the heading's own line, so every section showed a degenerate `n-n` range — unlike every other format, where a definition's range covers its whole body. A heading now spans from its line to the line before the next heading of the same or lower level (sibling/ancestor), or to EOF, restoring the read_file contract. Also indents by heading level (matching how methods nest under a class) and detects ATX headings properly (requires a space after the `#` run, caps at 6).	2026-08-10 15:18:52 +01:00
Daniele	1515492938	feat(file-viewer): browse a versioned file's git history ... A clock button in the file viewer header lists the versions of a file whose project keeps a git history; picking one shows the file as of that commit, read-only, with a banner back to the current version. A past version is never served from the working tree: the whole repository is materialized at that revision (git archive streamed through tar into a size-bounded, immutable-by-rev cache) and every fetch — content, compiled LaTeX, markdown images, downloads — resolves inside that tree, so dependencies are contemporaneous with the file: a .tex compiles against its \input's and images of that moment. Backend: new git_versions module (repo discovery bounded by the workspace mount, host-git log/rev-parse/archive, extraction cache with oldest-first prune) + GET /api/file/versions and a rev param on GET /api/file (rev is the ETag; never X-Writable). Frontend: history mode in FileViewerBase shared by the desktop and mobile viewers — popover, banner, watcher paused while browsing, rev propagated to every /api/file URL it builds.	2026-08-10 14:27:48 +01:00
Daniele	cd641ab89e	feat(project-coordinator): offer to keep a history of a project ... A project folder accumulates work with no way to see what changed or undo a wrong turn. The coordinator now offers to keep one, once, in plain words, and initializes it only after an explicit yes — the mechanism is git in the sandbox but the jargon stays out of the conversation, since the person being offered this is not necessarily someone who knows what a commit is. That first yes is standing consent to snapshot at later milestones, so the agent does not re-ask each time. Recorded in the project's SKALD.md so a future session knows the history exists rather than proposing it again, and documented in docs/projects.md, which is what the assistant reads to explain the feature to a user.	2026-08-10 13:20:26 +01:00
Daniele	2dad4824c9	fix(mcp): rebuild the prompt prefix when the connector set changes ... The `## MCP servers` table lives inside the frozen system prefix, which PrefixCache holds for twenty idle minutes. Refreshing a user's global-access snapshot fixed what `mcp.tools()` offers but left the table describing the world before the change, so an admin could enable a connector, ask for it in an open conversation, and be told in good faith that it does not exist — with the tools sitting right there. Same gap on a reinstall, whose new llm_short_description reached the runtime and not the prompt. Both refreshes now call `invalidate_prefixes()` on the live contexts they were already iterating, the seam the skill tools use. Order matters and runs against the intuition: `render_mcp_list` renders the runtime's in-RAM state, not the DB, so the invalidation goes last — after the snapshot refresh and after the servers restart. Rebuild earlier and the prefix is repopulated from the very descriptions being replaced, with nothing left to invalidate it a second time. In the reinstall that means waiting out a dependency install; those users were already reading a stale table, and an early rebuild would only freeze the stale one in place. Also warn when a feed's connector.json and index disagree on the integer version. The manifest silently wins, so if the index is the lower of the two the strict `feed > installed` comparison is false forever: the connector never offers an Update and nothing anywhere says why.	2026-08-10 13:20:26 +01:00
Daniele	59549d2b3b	fix(mcp): install and expose a global connector's deps where they are needed ... Two halves of the same failure, found debugging a marketplace connector that logged "connected — 6 tool(s)" while every call died on a missing module. The verify ran as a bare `sh -c` and inherited nothing, so a python connector was rejected by its own verify for a dependency installed one directory away — `global_enable` installs before it verifies, so the deps were provably there at the moment the check denied them, and the row ended up disabled. Only connectors that bother to declare a verify could hit it. `verify_env` now builds the verify's environment in one place and derives PYTHONPATH from the workdir, which is already the connector dir in both targets; `or_insert`, so a value the form declares still wins. The global branch of the reinstall refresh restarted the server without ever installing its deps: `ensure_installed_host` was reachable from `global_enable` alone, so a marketplace Update that adds a requirements.txt landed the file and brought the connector back exactly as broken. It now runs once per connector folder before the restart loop, best-effort. The per-user branch had always reinstalled, which is why nothing with scope=user ever showed the bug. Known gap, deliberate: POST /api/mcp/test shares run_verify but not the install, so testing a python connector never enabled on the box still fails on missing deps. Making a "try it" button write to disk for minutes is the worse trade.	2026-08-10 13:11:02 +01:00
Daniele	5fb5854ff2	fix(auth): stop the re-login dialog hijacking the login screen ... On a cold load with no session, both shells mount every component before their boot auth check resolves, so a dozen gated /api calls 401 in parallel and the fetch watch raised the re-login dialog over the login screen the boot check was about to show (.relogin-backdrop is z-10000, .login-page z-9999). The user typed their password into the modal, which only closes itself on success — revealing the login page still up with the app hidden, so they were asked a second time and only a manual reload got them in. The dialog is for a session that dies under an open tab, so gate it on one having ever been established. Recognising that is passive, in the same fetch wrapper: mobile.html probes /api/auth/me from a classic inline script that runs before this module exists, so an explicit marker per shell would never fire there and the dialog would be dead on mobile. Any 2xx from a gated endpoint proves a session; only the routes guard.rs::is_public lets through unauthenticated are excluded. Knock-on: with the report now a no-op on a cold load, the chat's reconnect loop no longer stopped on it. Retry only in the native shell, which authenticates on its own — everywhere else something is already asking for a password.	2026-08-10 12:23:45 +01:00
Daniele	5980bdb5b9	feat(users): unlock and start unencrypted users at boot ... A login is what makes an *encrypted* database readable; for an unencrypted one it gated nothing but the runtime — the file has no key and is already readable by this process. The cost was user-visible and read as a bug: after every restart the Telegram bot answered "your account is locked, log in via the web app", cron fired nothing and no background agent ran, until a human opened the SPA. `Skald::new` now calls `UserManager::unlock_all_unencrypted`, which registers the pools exactly as a login would and refuses an encrypted or inactive user. Unlocking alone only makes the data readable, so `wiring::spawn_unlocked_user_runtimes` then builds a `UserContext` for each — cron, the notify queue, the hub and the per-user MCP runtime all hang off it. That build is a background supervisor task rather than part of `new()`: it starts every member's MCP servers inside their container, and the HTTP listener must not wait behind it. The same two steps run per user off the lifecycle bus (`UserCreated`, `UserActiveChanged{active:true}`, after the container `ensure`), so a member created at runtime does not wait for the next restart. Two boundaries stay where they were. Authentication is untouched: `SessionStore` sits above `UserManager`, so no HTTP request authenticates as anyone because of this. And the auto-unlock is deliberately not on a lazy path such as `Skald::user_context` — `revoke_user_runtime` locks a pool synchronously and expects nothing to re-open it, so the writers of that map stay boot, login and the bus. `open_db` and the two unencrypted openers now share `register_unlocked` and `open_unencrypted_file`; `open_unencrypted` (the supervision path) still does not register its pool.	2026-08-10 12:16:33 +01:00
Daniele	55dcb48299	fix(telegram): resolve send_attachment paths in the user's workspace ... `send_attachment` handed its `file_path` argument straight to `InputFile::file`, which resolves against the **server process's** working directory. Every path the model can actually have — relative to the user's home, or absolute inside their container — failed the `path.exists()` check, and the one class that didn't (a name that happens to exist next to the binary) would have sent the wrong file. The routing already exists for the fs-tools, so expose it rather than repeat it: `UserFilesApi` (core-api) reads a path in the agent's own vocabulary and is obtained from `UserChannelHandle::files()`, so it is scoped to one user by construction. skald-core implements it over `resolve_view_target` — host mount read directly, container-only path through `docker exec` — holding the `SharedFs` cell rather than a snapshot, so a remount lands without a login. The size cap is checked before the read (a new `exec_fs::size` for the container branch): the point of a cap is to keep an oversized file out of RAM, so checking it afterwards would protect nothing. A photo above `sendPhoto`'s narrower 10 MB ceiling goes out as a document instead of as an API error.	2026-08-10 00:08:16 +01:00
Daniele	5765941758	feat(prompt): tell the agent what its sandbox can run ... The agent had no way to know its container ships ffmpeg, ripgrep or tesseract, so it either declined work it could do or spent a round finding out. This adds a command list to the system prompt as a **discovery hint** — explicitly not an inventory. Every decision follows from it being a hint: - The allowlist (~35 entries, `container/commands.rs`) is the curation; a full PATH dump is 800 entries of coreutils noise. The probe exists so the list cannot *lie*, not so it can discover: `command -v` at login means we never announce something a container recreate threw away. - The rendered prose says the list is partial and names `command -v`, so a tool outside the allowlist costs one check rather than a wrong conclusion. An empty probe renders as an explicit "could not be read", never as silence under a heading promising a list. - Order is the allowlist's own, grouped by kind of work — the grouping is the curation, and the reader is a model, not a grep. - Staleness is cheap both ways, so there is no invalidation machinery: a login-time snapshot on `UserContext`, non-fatal, refreshed at next login. The gate is the tool, not the sentinel. Every AGENT.md carries `common/sandbox.md` — the four system agents included — and the section is emitted iff the turn's model is shown `execute_cmd`, derived from `allow_tools` plus the security group's visibility filter for a root turn and from `child_defs` for a sub-agent: always the same definitions the model will see. `has_execute_cmd` therefore joins the PrefixCache key, since the group is switchable mid-conversation and that switch already rewrites the tool payload in the same provider cache. The fragment holds only the heading and one stable sentence; every conditional claim lives in the renderer, because prose promising `sudo apt-get install` is not the renderer's to retract when the tool is absent. `execute_cmd`'s own description loses `(python + node available)`: its job is steering away from the shell, and a capability advertisement diluted it.	2026-08-09 09:49:50 +01:00
Daniele	c27da4e6ab	feat(skills): rebuild the skill system for the multi-user model ... Per blueprint/skill-project.md: the old single-namespace, hand-maintained index is gone, replaced by a read-only, two-scope tree whose index is a runtime function of its content. - skills/ index generated at runtime (crates/skald-core/src/skills/: inventory, install, validate, watch), injected through the new <!-- SKILLS_LIST --> placeholder in AGENT.md (agents/common/skills.md); meta.json inject_skills flag removed. 11 chat/task agents carry the include, the 4 system agents do not. - Two trees, both read-only in both directions: skills/shared/{id} (the group's) and skills/{username}/{id} (one member's own, on the stable userid). The root is closed too: UserFs::SkillMounts + RouteError (alias probe, plain-denied paths, no home fallback) and a per-user .skills-root/{userid} container mount with the two scope mounts nested inside, plus the fifth self-heal axis (skills_mounted). - Agent verbs: skill_register/skill_delete (Config group, global scope behind the new skill.manage capability), fetch_repo for public repos, list_items(type="skills"); reads are plain read_file on the printed path. Seeded @fs_read skills/* allow. - Freshness: a digest-gated watcher on the two trees emits SystemEvent::SkillsChanged, whose subscriber rebuilds the frozen prompt prefix via Skald::invalidate_prompt_prefix; in-process writers invalidate directly. - The build ships no skills: the three bundled skills (ics2json, mcp-builder, skill-creator) and skills/index.md are removed, skills/ is instance data (gitignored, not packaged, no longer pruned by update.sh). - Docs: skills.md, agents.md, shared-folders.md added; docs/index.md and agents/README.md updated.	2026-08-08 23:05:35 +01:00
Daniele	71e1a26b08	fix(ui): render PDFs with pdf.js instead of an iframe ... On iOS the file viewer showed only the first page of a PDF, with no way to scroll to the rest — the document had to be downloaded and opened in another app. The cause was not ours: WebKit refuses to mount its PDF viewer inside an <iframe>/<object>/<embed> and paints a static first-page thumbnail instead. That hits Safari on iOS and every WKWebView, so the native shell too. The full viewer only exists for a top-level navigation. The desktop browsers do mount a viewer, but each mounts its own — Chrome's toolbar, Safari's page-index sidebar — so the same document also looked different on every machine. Both are answered by drawing the pages ourselves. New <pdf-view> (web/components/shared/pdf-view.js) renders a continuous scroll of canvas pages on the vendored pdf.js, with a zoom control and a page counter, and replaces the iframe for both native .pdf files and server-compiled LaTeX. Three properties are load-bearing: - pdf.js is imported lazily (~450 KB + a 1.2 MB worker), so a session that never opens a PDF never pays for it. - Canvases are created and destroyed as pages scroll. iOS caps the total canvas backing store a page may hold and silently blanks canvases past it, so an eager render would come out empty on exactly the platform this was written for. Off-screen pages keep only a correctly-sized box, which is also what keeps the scrollbar honest. - The text layer (selection, in-page find) is best-effort: it is transparent DOM over the pixels, so its failures are swallowed rather than surfaced. Vendored from pdfjs-dist 6.2.108: pdf.min.mjs, pdf.worker.min.mjs, the standard-font data (needed by PDFs that reference Helvetica/Times without embedding them) and the .textLayer block of pdf_viewer.css. CJK cmaps are deliberately left out. pdf.js 6 needs Safari/iOS 17.4+. Verified in headless Chromium against both a synthetic 12-page PDF using non-embedded Helvetica and a real 14-page paper with embedded fonts and figures: all pages present, last page renders after scrolling, page 1 released off-screen, text layer populated, zoom re-renders at the new scale, no JS errors.	2026-08-08 17:16:15 +01:00
Daniele	3744884070	fix(relay): detect a silently dead agent WebSocket and redial ... The agent's control WS to the relay was purely reactive: it answered the relay's Ping with a Pong and otherwise never wrote anything for long stretches. So when the path broke silently — NAT rebinding, a reverse proxy dropping its state — there were no unacked bytes for the kernel to retransmit, the socket never errored, and the relay's Close (it gives up after 120s of quiet) fell into the same hole. stream.next() then parked forever on a socket to nobody, is_connected() kept answering true, and the reconnect schedule below it — which works fine, it just never got asked — was never reached. Only a process restart cleared it. Relay logs show the cost: three idle-timeout closes of the agent connection with the agent absent for 2h, 9h and >2h afterwards, while every disconnect it *did* notice was back in 2-4 seconds. During one of those windows the iOS client authenticated four times and not a single pipe matched: pipe_invite rides the E2E channel through the agent's WS, so with the agent gone the web view had nothing to tunnel through. Add a per-session liveness probe. Both halves matter: a WS Ping every 20s keeps unacked bytes on the wire so a dead path finally surfaces as a TCP error (and the relay's Pong refreshes its own idle timer), and 75s of inbound silence — two missed relay pings — returns Err, handing the session to the existing backoff schedule. Covered by a test against a relay that completes the v2 handshake and then goes mute, the shape a black-holed path leaves behind. It reads the raw TCP stream rather than ws.next() because tungstenite auto-answers a Ping with a Pong on the next read, which would keep last_seen fresh and defeat the silence being simulated. Without the probe the test hangs instead of redialing.	2026-08-08 15:30:20 +01:00