Files

T

2026-07-10 15:02:09 +01:00

13 KiB

Raw Permalink Blame History

Plugin System

Plugins are long-running subsystems compiled into the binary and managed by PluginManager. They receive a PluginContext (a bundle of Arc<dyn Trait> deps) and run independently from the Axum web server.

Plugin Trait

#[async_trait]
pub trait Plugin: Send + Sync {
    fn id(&self)          -> &str;
    fn name(&self)        -> &str;
    fn description(&self) -> &str;
    fn is_running(&self)  -> bool;
    async fn start(&self, ctx: PluginContext)  -> Result<()>;
    async fn reload(&self, enabled: bool, config: Value, ctx: PluginContext) -> Result<()>;
    async fn stop(&self)  -> Result<()>;
    /// Optional: contribute one Axum router nested under `/api/plugin/<id>/`.
    fn http_router(&self) -> Option<axum::Router> { None }
    fn as_any(&self)      -> &dyn Any;
}

start() and reload() must spawn internal tasks and return immediately. stop() must cancel all internal tasks and await their completion.

PluginContext (crates/core-api/src/plugin.rs) carries typed Arc<dyn Trait> deps sourced from Skald — plugins use only what they need. Beyond the registry/provider deps, it includes:

db: Arc<sqlx::SqlitePool> — Skald's shared SQLite pool (create/use plugin-owned tables here).
inbox: Arc<dyn InboxApi> — unified approvals + clarifications façade (list_pending, approve, reject, answer); idempotent by request_id.
chat_hub.events(...) — subscribe to the global GlobalEvent bus (including the four Inbox events; see approval/index.md).

Plugin HTTP routes (`http_router`)

A plugin can mount HTTP routes by overriding http_router() (default None). After start_enabled(), WebFrontend::start collects routers via PluginManager::collect_plugin_routers() and nests each enabled plugin's router under /api/plugin/<id>/, behind Skald's normal auth. The router must close over the plugin's own state (no axum State is passed). Only routes — no nav entries / JS assets / Lit components. The mesh-facing router (router_factory) does not include plugin routes.

PluginManager — `src/core/plugin/mod.rs`

Method	Purpose
`register(plugin)`	Add a plugin before startup (build phase)
`set_skald(Arc<Skald>)`	Wire core after `Skald` is built
`set_router_factory(factory)`	Provide Axum router factory (called by `WebFrontend`)
`set_web_port(port)`	Provide HTTP port for plugins that need it (called by `WebFrontend`)
`start_enabled()`	Start all DB-enabled plugins
`collect_plugin_routers()`	Gather `(id, Router)` of enabled plugins to nest under `/api/plugin/<id>/` (called by `WebFrontend` after `start_enabled`)
`stop_all()`	Graceful shutdown on SIGINT
`toggle(id, enabled)`	Enable/disable and start/stop at runtime
`list()`	`Vec<PluginInfo>` with enabled + running flags

Startup sequence

main.rs:
  plugins = vec![Arc::new(HonchoPlugin::new()), Arc::new(TelegramPlugin::new(...)), …]

Skald::new(core_cfg, plugins):
  PluginManager::new(pool) → register_arc(plugin) for each plugin → Arc::new(plugin_manager)
  → build tool_registry (list_items / toggle_item reference Arc<PluginManager>)
  → Arc::new(Skald { … })
  → plugin_manager.set_skald(Arc::clone(&skald))

WebFrontend::start():
  → plugin_manager.set_router_factory(factory)
  → plugin_manager.set_web_port(port)
  → plugin_manager.start_enabled().await
  → plugin_manager.collect_plugin_routers().await   // nest under /api/plugin/<id>/

Enabled/disabled persistence

Plugin state and configuration are stored exclusively in the plugins SQLite table (id TEXT PK, enabled INTEGER, config TEXT). config.yml has no plugin section — plugins are configured at runtime via the REST API (PUT /api/plugins/{id}) or by asking the agent to use toggle_item (kind=plugin).

Adding a New Plugin

Plugins live in independent workspace crates (see crates/workspace.md):

Create crates/plugin-<name>/ with a Cargo.toml depending on core-api and any needed external crates.
Implement core_api::plugin::Plugin in crates/plugin-<name>/src/lib.rs.
Add the crate to the workspace members list and as a dependency in the main Cargo.toml.
In src/main.rs, add Arc::new(plugin_name::MyPlugin::new(...)) to the plugins vec before Skald::new().
Rebuild — no restart needed for toggle.

LLM Tools

Tool	Description
`list_items` (type=plugins)	All plugins with enabled + running status
`toggle_item` (kind=plugin)	Enable/disable a plugin by id (immediate + persisted)
`configure_plugin`	Update a plugin's config JSON and restart it immediately

Plugin-provided global LLM tools

The Plugin trait has no tools() method: it cannot register global LLM tools by itself. The pattern (used by mobile-connector) is:

The plugin exposes a domain control trait (e.g. RelayAgent) and implements it on the plugin struct.
The plugin crate exports a factory fn xxx_tools(agent: Arc<dyn Trait>) -> Vec<Arc<dyn Tool>>.
In Tools::build (src/core/skald/bundles.rs), while the registry is being assembled, the plugin is fetched via PluginManager::get_plugin_typed::<P>(), cast to the trait, and its tools are registered with ToolRegistry::register_arc(...).

The tools call into the plugin lazily, so they can be registered before the plugin's runloop starts (they return an error while the plugin is stopped). This keeps the core-api Plugin trait minimal while still allowing tool-based control surfaces (plugin.md §11).

Available Plugins

Plugin id	Source	Doc
`honcho`	`crates/plugin-honcho/src/lib.rs`	honcho.md
`remote_connectivity`	`crates/plugin-tailscale-remote/src/lib.rs`	remote.md
`telegram`	`crates/plugin-telegram-bot/src/lib.rs`	plugins/telegram.md
`whisper_local`	`crates/plugin-transcribe-whisper-local/src/lib.rs`	whisper-local.md
`comfyui`	`crates/plugin-comfyui/src/lib.rs`	providers/image.md
`mobile-connector`	`crates/plugin-mobile-connector/src/lib.rs`	mobile-connector.md

Transcribe Providers and TranscribeManager

Speech-to-Text is decoupled from the plugin system via TranscribeManager (src/core/transcribe/mod.rs).

Plugins that provide transcription (e.g. whisper_local) register an Arc<dyn Transcribe> in skald.transcribe_manager at start() and deregister at stop(). Non-plugin providers (e.g. a future OpenRouter client) can register directly at startup without needing a full plugin lifecycle.

// trait — src/core/transcribe/mod.rs
pub trait Transcribe: Send + Sync {
    fn id(&self) -> &str;
    async fn transcribe(&self, audio: Vec<u8>, format: &str) -> Result<String>;
}

Method	Purpose
`register(Arc<dyn Transcribe>)`	Add/replace a provider by id
`unregister(id)`	Remove a provider
`get()`	Returns the first available provider

Selection strategy is currently first registered. Callers (e.g. Telegram) ask skald.transcribe_manager.get().await — they never reference a concrete type.

See whisper-local.md for the only current provider.

Image Generators and ImageGeneratorManager

Image generation is decoupled from the plugin system via ImageGeneratorManager (src/core/image_generate/) and two traits in core-api::image_generate — same split as TranscribeProvider / TranscribeRegistry.

Two kinds of providers coexist:

Kind	Source	Example
DB-backed	`image_generate_models` table, built from `llm_providers` credentials	OpenRouter `x-ai/grok-2-vision`
Plugin-registered	Ephemeral — registered at runtime by plugins	ComfyUI workflows

Plugins register via ctx.image_generate_registry (type Arc<dyn ImageGenerateRegistry>) in PluginContext. No dependency on the main crate is needed — only core-api.

// crates/core-api/src/image_generate.rs
pub trait ImageGenerate: Send + Sync { fn id(&self) -> &str; fn name(&self) -> &str; async fn generate(&self, prompt: &str) -> Result<Vec<u8>>; }
pub trait ImageGenerateRegistry: Send + Sync { async fn register(&self, provider: Arc<dyn ImageGenerate>); async fn unregister(&self, id: &str); }

Method	Purpose
`ctx.image_generate_registry.register(...)`	Add a plugin provider (ephemeral)
`ctx.image_generate_registry.unregister(id)`	Remove a plugin provider
`add_model / update_model / delete_model`	DB-backed CRUD (called by REST API)
`list()`	Returns all active providers (plugin + DB) — used by LLM tool
`get(id)`	Returns a specific provider by id
`generate(provider_id, prompt)`	Generates and saves image, returns `(PathBuf, url)`

The LLM interacts with providers via two tools: image_generate_providers_list and image_generate. See providers/image.md for the full flow.

TTS and TtsManager

Text-to-speech follows the same split pattern as transcribe and image_generate. TtsManager (src/core/tts/) manages both DB-backed and plugin-registered providers. Traits live in core-api::tts.

Kind	Source	Example
DB-backed	`tts_models` table, built from `llm_providers` credentials	OpenAI `tts-1-hd`
Plugin-registered	Ephemeral — registered at runtime by plugins	`OrpheusTtsPlugin`

Plugins register via ctx.tts_registry (type Arc<dyn TtsRegistry>) in PluginContext.

// crates/core-api/src/tts.rs
pub trait TextToSpeech: Send + Sync {
    fn id(&self) -> &str;
    fn name(&self) -> &str;
    fn description(&self) -> Option<&str>;
    fn instructions(&self) -> Option<&str>;  // default voice style
    async fn synthesize(&self, text: &str, instructions: Option<&str>) -> Result<Vec<u8>>;
}
pub trait TtsRegistry: Send + Sync {
    async fn register(&self, provider: Arc<dyn TextToSpeech>);
    async fn unregister(&self, id: &str);
}

Method	Purpose
`ctx.tts_registry.register(...)`	Add a plugin TTS provider (ephemeral)
`ctx.tts_registry.unregister(id)`	Remove a plugin provider

See providers/tts.md for the full manager API and DB schema.

ApiProvider and ApiProviderRegistry

Plugins that supply an ApiProvider (e.g. a cloud TTS + Transcribe integration without subprocess) register via ctx.api_provider_registry:

// crates/core-api/src/provider.rs
#[async_trait]
pub trait ApiProvider: Send + Sync {
    fn type_id(&self)        -> &'static str;
    fn display_name(&self)   -> &'static str;
    fn supported_types(&self)-> &'static [ServiceType];
    async fn list_tts_models(&self, record: &LlmProviderRecord) -> Result<Option<Vec<RemoteTtsModelInfo>>>;
    async fn list_transcribe_models(&self, ..) -> ..;
    async fn list_llm_models(&self, ..)        -> ..;
    fn build_tts(&self, ..)             -> Option<Result<Arc<dyn TextToSpeech>>>;
    fn build_transcriber(&self, ..)     -> Option<Result<Arc<dyn Transcribe>>>;
    fn build_llm(&self, ..)             -> Option<Result<BuiltLlmClient>>;
    fn build_image_generator(&self, ..) -> Option<Result<Arc<dyn ImageGenerate>>>;
    fn ui_meta(&self) -> ProviderUiMeta;
}

pub trait ApiProviderRegistry: Send + Sync {
    fn register_plugin(&self, provider: Arc<dyn ApiProvider>);
    fn unregister_plugin(&self, type_id: &str);
}

ProviderRegistry in src/core/provider/mod.rs implements ApiProviderRegistry. It is exposed as ctx.api_provider_registry in PluginContext.

Plugin-registered providers shadow builtin ones: ProviderRegistry::get(type_id) checks the plugin list first.

When to use ApiProvider vs TtsRegistry/TranscribeRegistry:

Use TtsRegistry / TranscribeRegistry for ephemeral local engines (subprocess, on-device model) that don't rely on the llm_providers credential DB.
Use ApiProvider for cloud services that the user configures via the LLM-providers UI (API key stored in llm_providers, models managed via tts_models / transcribe_models CRUD).

All types used by ApiProvider (LlmProviderRecord, LlmModelRecord, TtsModelRecord, TranscribeModelRecord, ImageGenerateModelRecord, RemoteLlmModelInfo, RemoteTtsModelInfo, RemoteTranscribeModelInfo, LlmStrength) live in core-api::provider or their respective core-api modules. Plugin crates depend only on core-api.

Plugin catalogue

Plugin ID	Crate	Description
`honcho`	`crates/plugin-honcho`	Honcho long-term memory backend
`telegram_bot`	`crates/plugin-telegram-bot`	Private Telegram bot interface
`whisper_local`	`crates/plugin-transcribe-whisper-local`	Local STT via whisper.cpp
`tailscale_remote`	`crates/plugin-tailscale-remote`	Remote access via Tailscale mesh
`comfyui`	`crates/plugin-comfyui`	ComfyUI image generation workflows
`orpheus_tts_3b`	`crates/plugin-tts-orpheus-3b`	Local TTS via Orpheus 3B (Python subprocess)
`kokoro_tts`	`crates/plugin-tts-kokoro`	Local TTS via Kokoro ONNX (lightweight, multilingual)
`elevenlabs`	`crates/plugin-elevenlabs`	ElevenLabs cloud TTS and transcription (ApiProvider)
`mobile-connector`	`crates/plugin-mobile-connector`	Bridges the Inbox to mobile apps over the relay (E2E encrypted) — see mobile-connector.md

13 KiB Raw Permalink Blame History