19 KiB
19 KiB
Skald — Documentation
Documentation Rule (MANDATORY)
Every source code change — made by a human or by an LLM — must be accompanied by an update to the relevant doc file(s). No exception.
This includes:
- Adding or removing a tool → update tools.md
- Changing a table schema → update database.md
- Modifying the approval gate or tool loop → update session.md
- Adding a new agent → update agents.md
- Any change to the WS protocol → update frontend.md
- Changing the project/ticket lifecycle or project chats → update projects.md
- Changing ChatHub dispatch, the per-source inbox/coalescing, or
/stop//clearsemantics → update chat-hub.md - Adding or changing custom slash commands, the command manager, or the composer autocomplete → update commands.md
- Changing the
desktopfeature, Tauri wiring, tray/menu, window policy, orbootstrap_data_dir→ update desktop.md
Key paths (agent: read this first)
| Resource | Default path | Override |
|---|---|---|
| SQLite database | ./database.db |
db.path in config.yml |
| Config file | ./config.yml |
— (copy from default.config.yaml) |
| Secrets folder | ./secrets/ |
— |
| Model cache | ./models/ |
— |
| Log files | ./logs/ |
— |
| Static web assets | ./web/ |
web.static_dir in config.yml |
When looking for the database, always use ./database.db unless config.yml says otherwise.
Project Summary
A local chat server (Axum + Tokio + SQLite) where an LLM handles user queries via tool calls. The app can rewrite and restart its own source code autonomously. Multiple specialized agents collaborate via a recursive sub-agent system. External tools are integrated via MCP (Model Context Protocol). Entry point: run.sh.
Workspace Structure
The project is a Cargo workspace. Extracted crates live in crates/:
| Crate | Path | Notes |
|---|---|---|
skald |
. (root) |
Main application binary |
core-api |
crates/core-api/ |
Shared types and traits: ServerEvent, GlobalEvent, ChatHubApi, ApprovalApi, InboxApi, Tool, Memory, ChatbotClient, Transcribe, TextToSpeech, ImageGenerate, SecretsApi, LocationManager, InterfaceTool, Plugin, PluginContext, ApiProvider, ApiProviderRegistry, RemoteAccess. Also owns all DB record types: LlmProviderRecord, LlmModelRecord, TtsModelRecord, TranscribeModelRecord, ImageGenerateModelRecord. |
llm-client |
crates/llm-client/ |
OpenAI-compatible, Anthropic, Ollama, LmStudio implementations of ChatbotClient. Depends on core-api and re-exports the trait and associated types for backward compatibility. |
mcp-client |
crates/mcp-client/ |
MCP protocol layer: McpServer (stdio), McpHttpServer, McpServerClient trait, config types |
honcho-client |
crates/honcho-client/ |
Honcho v3 REST API client — zero dependencies on the main crate |
plugin-honcho |
crates/plugin-honcho/ |
Honcho memory sink plugin |
plugin-tailscale-remote |
crates/plugin-tailscale-remote/ |
Remote connectivity via Tailscale mesh |
plugin-transcribe-whisper-local |
crates/plugin-transcribe-whisper-local/ |
Local STT via whisper.cpp (Metal-accelerated) |
plugin-telegram-bot |
crates/plugin-telegram-bot/ |
Private Telegram bot interface |
plugin-tts-orpheus-3b |
crates/plugin-tts-orpheus-3b/ |
Local TTS via Orpheus 3B (Python subprocess) |
plugin-tts-kokoro |
crates/plugin-tts-kokoro/ |
Local TTS via Kokoro ONNX (lightweight, multilingual) |
skald-relay-common |
crates/skald-relay-common/ |
Shared crypto + v2 protobuf frame types for the relay and mobile-connector plugin; owns the gen-vectors reference generator. Implements v2 binary transport (presence, live channel). No axum/tokio/Skald deps. |
skald-relay-server |
crates/skald-relay-server/ |
Zero-trust store-and-forward relay + push bridge (iOS/Android remote control). v2 binary transport (protobuf), presence tracking, live channel route-or-fail. Depends on skald-relay-common. Live APNs sender behind the push-live cargo feature (see crates/workspace.md). |
plugin-mobile-connector |
crates/plugin-mobile-connector/ |
Agent end of the v2 relay protocol: bridges the Inbox to mobile apps over WebSocket, E2E encrypted. Handles presence, live inbox pulls, pairing. See plugins/mobile-connector.md. |
To add a new extracted crate: create crates/<name>/, add it to the [workspace].members list in the root Cargo.toml, then add a path dependency in [dependencies].
Module Map
| Source path | Role | Doc |
|---|---|---|
src/main.rs |
Thin entry point: tracing → Config → into_split → plugins → Skald::new → WebFrontend::start → shutdown |
architecture.md |
src/desktop/mod.rs |
Tauri shell — only compiled under --features desktop. Wraps run_backend() in a Tauri event loop with a system-tray icon (Open / Quit), creates the main WebviewWindow from config.yml's server.port, and handles graceful shutdown |
desktop.md |
src/core/skald.rs |
Skald — headless application core; owns all managers; new(cfg, plugins) / shutdown() |
architecture.md |
src/core/config.rs |
CoreConfig + core config types (DbConfig, LlmConfig, TicConfig, CompactionConfig, …) |
architecture.md |
src/frontend/config.rs |
FrontendConfig (ServerConfig, WebConfig, timezone) |
architecture.md |
src/core/session/handler/ |
Core LLM loop, tool dispatch, approval | session.md |
src/core/session/handler/message_builder.rs |
MessageBuilder — pure service for building OpenAI message arrays, testable in isolation |
session.md |
src/core/session/manager.rs |
Session factory | session.md |
src/core/agents.rs |
Agent discovery, prompt loading | agents.md |
src/core/command/mod.rs |
LlmCommandManager — file-based custom slash commands (commands/<name>/) |
commands.md |
src/core/tools/ |
Built-in tool registry | tools.md |
src/core/tools/tool_names.rs |
Centralised tool name constants (CALL_AGENT, RESTART, …) |
tools.md |
src/core/tool_catalog.rs |
ToolCatalog: unified listing façade for built-in + MCP tools (wraps ToolRegistry + McpManager); AllTools response includes mcp_servers: HashMap<String, McpServerMeta> (friendly name + description per MCP server) |
tools.md |
src/core/provider/ |
ProviderRegistry (implements ApiProviderRegistry) — thin wrapper around core-api::provider. All types re-exported for internal use. |
llm-clients.md |
src/core/service_manager.rs |
ServiceManager trait — lightweight umbrella for all model managers |
llm-clients.md |
src/core/chatbot/ |
LLM provider clients | llm-clients.md |
src/core/llm/manager.rs |
LLM selection, health tracking | llm-clients.md |
src/core/chat_event_bus.rs |
In-process broadcast bus for chat turns and compaction events | chat-event-bus.md |
src/core/compactor.rs |
Context compaction — summarises old history to reduce token usage | compaction.md |
src/core/memory/ |
Pluggable long-term memory layer (trait + manager) | memory.md |
src/core/chat_hub/ |
Central chat orchestrator for interactive, user-facing sessions only (web, mobile, project chats — one persistent session per source via the sources table); per-source coalescing inbox; notification pipeline. provision_session(source, agent_id, rc, reset) is the single source→session entry point; clear() is a thin main-agent wrapper over it. Not for background agents (cron/TIC/sub-agents → TaskManager/ChatSessionManager) |
chat-hub.md |
src/core/tic/ |
Background MCP event processor (TicManager) | architecture.md |
src/core/mcp/ |
MCP server management, push notification ingestion | mcp.md |
src/core/cron/ |
Scheduled job scheduler | cron.md |
src/core/plugin/ |
Plugin system (PluginManager) | plugins.md |
src/core/secrets.rs |
SecretsStore — centralised token/key store over SQLite | secrets.md |
src/core/transcribe/ |
TranscribeManager, OpenAiAudioTranscriber, ElevenLabsTranscriber. Traits and record types re-exported from core-api. |
providers/transcribe.md |
src/core/tts/ |
TtsManager (DB-backed + plugin slots), OpenAiTtsSynthesiser, ElevenLabsTtsSynthesiser. Traits and record types re-exported from core-api. |
providers/tts.md |
src/core/image_generate/ |
ImageGenerate trait, ImageGeneratorManager (DB-backed + plugin slots), OpenRouterImageGenerator | providers/image.md |
src/core/run_context/mod.rs |
RunContext domain object: fields security_group, system_prompt, allow_fs_writes, working_directory + applicative methods tool_group_id(), extra_system_prompt(), effective_working_dir(), is_write_allowed(). RunContextManager: permission group CRUD; set_session_run_context; duplicate_group; check_tool_visibility. |
approval/index.md |
src/core/projects/mod.rs |
ProjectManager — CRUD for projects (filesystem-linked, ordered by updated_at). Free fn build_runtime_run_context(project, base) layers project-runtime fields (working_directory = project.path, allow_fs_writes for the project tree + {skald_cwd}/data, project-context system prompt fragments) over an optional base RC — shared by ticket jobs and interactive project chats |
projects.md |
src/core/projects/tickets.rs |
ProjectTicketManager — CRUD + lifecycle for project tickets (start, on_job_completed, reset); start() resolves the base RunContext (ticket override → project static config) and delegates to projects::build_runtime_run_context for the runtime fields |
projects.md |
src/core/inbox.rs |
Inbox: unified façade for pending approvals + clarifications (wraps ApprovalManager, ClarificationManager, ChatHub) |
approval/index.md |
src/core/db/ |
SQLite schema and queries | database.md |
src/core/events.rs |
WS protocol types | frontend.md |
src/frontend/mod.rs |
WebFrontend — wires router_factory, starts plugins, runs Axum |
architecture.md |
src/frontend/server.rs |
WebServer — Axum router, TcpListener, WebServerHandle |
architecture.md |
src/frontend/api/ |
HTTP + WebSocket handlers — State<Arc<Skald>> |
frontend.md |
src/frontend/api/projects.rs |
REST CRUD for projects and tickets — GET/POST /api/projects, GET/PUT/DELETE /api/projects/{id}, tickets sub-routes, start/reset lifecycle. POST /api/projects/{id}/session opens/resumes the project chat (source project-{id}, agent project-coordinator). provisioning_for_source(skald, source) maps a source → (agent, RunContext) and is reused by POST /api/sessions so project resets recreate with the coordinator |
projects.md |
src/config.rs |
Config (YAML aggregate: ServerConfig, WebConfig + re-exports from core::config) + Config::into_split() + bootstrap_data_dir() (cwd relocation under the desktop feature) |
logging-config.md, desktop.md |
crates/plugin-honcho/ |
Honcho memory sink (standalone crate) | honcho.md |
crates/plugin-tailscale-remote/ |
Remote connectivity via Tailscale mesh (standalone crate) | remote.md |
crates/plugin-transcribe-whisper-local/ |
Local STT via whisper.cpp (standalone crate) | whisper-local.md |
crates/plugin-telegram-bot/ |
Private Telegram bot (standalone crate) | plugins/telegram.md |
crates/plugin-tts-orpheus-3b/ |
Orpheus TTS 3B — local TTS via Python subprocess (standalone crate) | providers/tts.md |
crates/plugin-tts-kokoro/ |
Kokoro ONNX — lightweight local TTS, multilingual (standalone crate) | providers/tts.md |
crates/honcho-client/ |
Honcho v3 REST API client (standalone crate) | honcho.md |
web/components/ |
Lit frontend components | frontend.md |
run.sh |
Supervisor loop | self-rewriting.md |
Critical Constants
| Constant | Value | Location |
|---|---|---|
MAX_AGENT_DEPTH |
5 | src/core/session/handler/mod.rs |
DEFAULT_MAX_TOOL_ROUNDS |
20 | src/core/session/handler/mod.rs |
FAILURE_DEGRADED |
3 consecutive failures | src/core/llm/manager.rs |
FAILURE_DOWN |
5 consecutive failures | src/core/llm/manager.rs |
| Cron scheduler tick | 30 s | src/core/cron/mod.rs |
| Cron fire-check window | 90 s | src/core/cron/mod.rs |
| MCP startup timeout | 120 s | src/core/mcp/mod.rs |
| TIC tick interval | 900 s default | config.yml → tic.interval_secs; overridable at runtime via tic.interval_minutes DB key |
| TIC batch size | 50 events default | config.yml → tic.batch_size |
| Notification batch window | 200 ms | src/core/chat_hub/mod.rs |
Navigation
Core Architecture
- architecture.md — component wiring, startup sequence, request lifecycle
- desktop.md — Tauri desktop bundle: system tray, window policy, two-deployment-shape model, build instructions
- build-and-distribution.md — portable static binary: rustls+ring (no OpenSSL/aws-lc), musl build recipe, cargo features, runtime deps
- self-rewriting.md — restart mechanism, safe self-modification workflow
- database.md — SQLite schema, migration pattern
- logging-config.md — log levels, config.yml full reference
Session & LLM Loop
- session.md — ChatSessionHandler, message flow, approval gate integration
- session/run-context.md — RunContext: permissions, system prompt, file authorization, working directory
- llm-clients.md — ChatbotClient trait, LlmManager, ApiProvider, ProviderRegistry, AUTO selection
- compaction.md — context compaction: trigger, summarisation flow, DB schema, config
- memory.md — Memory trait, MemoryManager, integration in the LLM loop
Approval & Permissions
- approval/index.md — ApprovalManager: human-in-the-loop, rules, pending approvals, session bypass; tool visibility filtering; group duplication
- session/run-context.md — RunContext fields and integration (single source of truth)
Tools & Agents
- agents.md — agent discovery, meta.json, call_agent, depth limit
- commands.md — custom
/commandshortcuts: file layout,{{args}}, dual-view, autocomplete - tools.md — Tool trait, ToolRegistry, built-in catalogue
- chat-event-bus.md — ChatEventBus, event types, publication rules, adding consumers
- cron.md — TaskManager, task kinds (cron/sync/async), 7-field cron syntax, job lifecycle, async result delivery
Model Providers
- llm-clients.md — LLM client trait and selection
- providers/tts.md — Text-to-Speech: trait, manager, provider catalogue, tts_models DB table
- providers/transcribe.md — Cloud STT via OpenAI-compatible audio API, transcribe_models DB table
- providers/image.md — Image generation: trait, manager, async task system, LLM tools, REST endpoint
MCP (Model Context Protocol)
- mcp.md — McpManager, transports, naming convention, enable/disable, integration
- mcp/specs/index.md — Specification reference: one file per official MCP spec revision (2024-11-05 Legacy → 2025-06-18 Stable → 2025-11-25 Latest Stable → 2026-07-28 draft), with a comparative feature matrix
- mcp/servers/gmail.md — Gmail read+modify MCP server (custom Python)
- mcp/servers/gcal.md — Google Calendar read-only MCP server (custom Python)
- mcp/servers/gmaps.md — Google Maps transit/directions MCP server (custom Python)
- mcp/servers/whatsapp.md — WhatsApp read+send MCP server (custom Node.js)
Plugin System
- plugins.md — Plugin trait, PluginManager, HTTP router integration
- plugins/honcho.md — Honcho memory plugin: setup, config, filtering, lifecycle
- plugins/mobile-connector.md — Mobile app relay bridge, E2E encryption, Inbox synchronization
- plugins/telegram.md — Telegram bot setup, pairing, whitelist, HITL approval
- plugins/whisper-local.md — Local STT via whisper.cpp, model setup, TranscribeManager integration
- plugins/remote.md — Tailscale mesh remote connectivity
Relay Protocol (Mobile Remote Control)
- relay/index.md — Architecture, actors, threat model, encoding conventions
- relay/crypto.md — Crypto contract: seed, key derivation, ECDH, HKDF, AES-256-GCM, anti-replay
- relay/relay-protocol.md — WebSocket protocol: protobuf transport, auth, pairing, live channel, presence
- relay/framing.md — E2E plaintext framing: version byte + optional zlib compression
- relay/payloads.md — E2E payload schemas: inbox_update, approval_response, clarification_response, …
- relay/describe-and-push.md — Approval rendering: summary + structured blocks, push delivery model
- relay/server.md — Relay server implementation: zero-trust, store-and-forward, APNs/FCM bridge, deploy
- relay/test-vectors.md — Crypto test vectors + reference generator for byte-for-byte interop
Projects & Tickets
- projects.md — Projects subsystem: kanban tickets, lifecycle,
build_runtime_run_context, interactive project chats
Frontend & Notifications
- frontend.md — WebSocket protocol, ServerEvent types, Lit components
- notifications.md — Notification system:
read_notificationtool, synthetic injection flow,data/notifications.mdformat
Infrastructure & Security
- secrets.md — SecretsApi trait, SecretsStore, well-known keys, security notes
- crates/workspace.md — Workspace crate catalogue,
core-apimodule reference, plugin extraction roadmap
Miscellaneous
- skills.md — Skills system: reusable Python capability packages
When to Update This File
- A new source module is added or removed
- A critical constant changes
- A new doc file is added to
docs/