dguiducci/Skald-Circle
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(users): UserManager with per-user SQLCipher, and extract skald-core crate

Browse Source 
Two changes developed together in one session; they share the same module
structure (db/mod.rs, the core lib root) and only compile together, so they
land as one commit.

## UserManager + per-user encryption (§9/§11)

New `users::UserManager`: owns the system.db pool plus a map
`userid -> SqlitePool` of unlocked databases. The pool *is* the unlock token —
its connect options carry the DEK as SQLCipher's raw key, so an open pool means
the key is in RAM until restart and dropping it re-locks (§9). Knows nothing
about cookies.

New `crypto` module: envelope encryption. A random 256-bit DEK encrypts
`{userid}.db`; `users.database_password` holds it sealed with AES-256-GCM under
`Argon2id(password, salt)`. The AEAD tag is the password verifier — one
derivation both authenticates and yields the key, so encrypted users store no
second hash. Cleartext users store the Argon2id output directly, compared in
constant time. Argon2 runs in spawn_blocking behind a 2-permit semaphore
(256 MiB per derivation).

- SQLCipher via `libsqlite3-sys` `bundled-sqlcipher-vendored-openssl`, pinned
  <0.38 so it unifies with the one sqlx-sqlite links (a newer copy would apply
  the feature to a SQLite sqlx never uses). OpenSSL is vendored and static, so
  the binary stays self-contained.
- Schema split into `create_registry_tables` (instance-wide, no user key) and
  `create_owner_tables` (one owner's content, identical in every file). No FK in
  the owner bucket may reach the registry — enforced by a standalone test.
  Dropped `chat_history.model_db_id` (write-only, and the only registry-crossing
  key); moved `projects`/`project_tickets` into the owner bucket.
- Provisioning invariant: the file is written before the row, deleted after it,
  so a crash leaves an orphan file, never a user without a database. `open_db`
  never creates: a missing file is an error, not a silent empty database.

Not consumed yet: no login, call sites still use the shared system.db pool.

## Extract crates/skald-core

The headless core moves out of `src/` into its own crate; `skald` (server) and
the coming `skald-setup` are shells around it. Two dependencies on the shell
were inverted rather than dragged along, so the core names neither Tauri nor any
concrete plugin:

- `Plugin::tools(self: Arc<Self>)` — plugins contribute tools through this hook
  (sibling of `http_router`), so the core no longer downcasts to
  `MobileConnectorPlugin`.
- `tools::restart::set_restart_handler` — the desktop shell installs its
  teardown-and-respawn; the core defaults to the supervisor exit code. The core
  loses its `desktop` feature.
- `boot`'s stdout formatter moves to the binary (`src/boot_format.rs`); the core
  only emits tracing events.

All 79 core tests pass; the binary boots and serves in a clean directory, and
the mobile-connector tools still register through the new hook.
This commit is contained in:
dguiducci
2026-07-10 16:48:51 +01:00
parent 38494a85a9
commit 178a38357e
173 changed files with 2650 additions and 1106 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/skald-core/src/tts/db.rs
+126
View File
@@ -0,0 +1,126 @@
use anyhow::{Context, Result};
use sqlx::SqlitePool;
use super::TtsModelRecord;
#[derive(sqlx::FromRow)]
struct TtsModelRow {
id: i64,
provider_id: i64,
model_id: String,
voice_id: Option<String>,
name: String,
description: Option<String>,
instructions: Option<String>,
response_format: Option<String>,
priority: i64,
}
pub async fn load_all(pool: &SqlitePool) -> Result<Vec<TtsModelRecord>> {
let rows = sqlx::query_as::<_, TtsModelRow>(
"SELECT id, provider_id, model_id, voice_id, name, description, instructions, response_format, priority
FROM tts_models
WHERE removed_at IS NULL
ORDER BY priority ASC, name ASC",
)
.fetch_all(pool)
.await
.context("tts_models: load_all")?;
Ok(rows.into_iter().map(row_to_record).collect())
}
pub async fn insert(pool: &SqlitePool, r: &TtsModelRecord) -> Result<i64> {
// Revive a soft-deleted row that collides on (provider_id, model_id) or name
// before attempting a plain INSERT, which would fail on the UNIQUE constraints.
let restored = sqlx::query_scalar::<_, i64>(
"UPDATE tts_models
SET provider_id=?1, model_id=?2, voice_id=?3, name=?4,
description=?5, instructions=?6, priority=?7, response_format=?8, removed_at=NULL
WHERE id = (
SELECT id FROM tts_models
WHERE removed_at IS NOT NULL
AND (provider_id=?1 AND model_id=?2 OR name=?4)
LIMIT 1
)
RETURNING id",
)
.bind(r.provider_id)
.bind(&r.model_id)
.bind(&r.voice_id)
.bind(&r.name)
.bind(&r.description)
.bind(&r.instructions)
.bind(r.priority as i64)
.bind(&r.response_format)
.fetch_optional(pool)
.await
.context("tts_models: restore soft-deleted")?;
if let Some(id) = restored {
return Ok(id);
}
sqlx::query_scalar::<_, i64>(
"INSERT INTO tts_models (provider_id, model_id, voice_id, name, description, instructions, priority, response_format)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8)
RETURNING id",
)
.bind(r.provider_id)
.bind(&r.model_id)
.bind(&r.voice_id)
.bind(&r.name)
.bind(&r.description)
.bind(&r.instructions)
.bind(r.priority as i64)
.bind(&r.response_format)
.fetch_one(pool)
.await
.context("tts_models: insert")
}
pub async fn update(pool: &SqlitePool, id: i64, r: &TtsModelRecord) -> Result<()> {
sqlx::query(
"UPDATE tts_models
SET provider_id=?1, model_id=?2, voice_id=?3, name=?4, description=?5, instructions=?6, priority=?7, response_format=?8
WHERE id=?9",
)
.bind(r.provider_id)
.bind(&r.model_id)
.bind(&r.voice_id)
.bind(&r.name)
.bind(&r.description)
.bind(&r.instructions)
.bind(r.priority as i64)
.bind(&r.response_format)
.bind(id)
.execute(pool)
.await
.context("tts_models: update")?;
Ok(())
}
pub async fn soft_delete(pool: &SqlitePool, id: i64) -> Result<()> {
sqlx::query(
"UPDATE tts_models SET removed_at = datetime('now') WHERE id = ?1",
)
.bind(id)
.execute(pool)
.await
.context("tts_models: soft-delete")?;
Ok(())
}
fn row_to_record(r: TtsModelRow) -> TtsModelRecord {
TtsModelRecord {
id: r.id,
provider_id: r.provider_id,
model_id: r.model_id,
voice_id: r.voice_id,
name: r.name,
description: r.description,
instructions: r.instructions,
response_format: r.response_format,
priority: r.priority as i32,
}
}
crates/skald-core/src/tts/manager.rs
+285
View File
@@ -0,0 +1,285 @@
/// TtsManager — DB-aware registry of Text-to-Speech providers.
///
/// Two kinds of providers coexist:
/// - **DB-backed**: rows in `tts_models`, built from `llm_providers` credentials.
/// Managed via `add_model` / `update_model` / `delete_model`. Loaded on startup
/// and after every mutation.
/// - **Plugin-registered**: ephemeral providers registered at runtime by plugins
/// (e.g. a local Kokoro or Piper TTS plugin). Not persisted — they disappear on plugin stop.
///
/// `get()` returns the first plugin provider if any is running, otherwise the
/// first DB-backed provider ordered by `priority ASC`.
use std::sync::Arc;
use anyhow::{Context, Result, anyhow};
use sqlx::SqlitePool;
use tokio::sync::RwLock;
use tokio_util::sync::CancellationToken;
use tracing::{info, warn};
use core_api::system_bus::{SystemEvent, SystemEventBus};
use async_trait::async_trait;
use crate::llm::LlmProviderRecord;
use crate::llm::db as llm_db;
use crate::provider::ProviderRegistry;
use super::{TextToSpeech, TtsModelInfo, TtsModelRecord};
use super::db as tts_db;
pub use core_api::tts::{TtsProvider, TtsRegistry};
// ── Internal state ────────────────────────────────────────────────────────────
struct TtsSlot {
record: TtsModelRecord,
provider: LlmProviderRecord,
synthesiser: Arc<dyn TextToSpeech>,
}
struct ManagerState {
/// DB-backed synthesisers, ordered by priority ASC. Rebuilt on every reload().
db_slots: Vec<TtsSlot>,
/// Plugin-registered providers (ephemeral — not in DB).
plugins: Vec<Arc<dyn TextToSpeech>>,
}
// ── TtsManager ────────────────────────────────────────────────────────────────
pub struct TtsManager {
pool: Arc<SqlitePool>,
registry: Arc<ProviderRegistry>,
state: RwLock<ManagerState>,
}
impl TtsManager {
pub async fn new(
pool: Arc<SqlitePool>,
registry: Arc<ProviderRegistry>,
system_bus: Arc<SystemEventBus>,
shutdown: CancellationToken,
) -> Result<Arc<Self>> {
let mgr = Arc::new(Self {
pool,
registry,
state: RwLock::new(ManagerState {
db_slots: Vec::new(),
plugins: Vec::new(),
}),
});
mgr.reload().await?;
// Reload whenever an ApiProvider is registered or unregistered.
let weak = Arc::downgrade(&mgr);
let mut rx = system_bus.subscribe();
tokio::spawn(async move {
loop {
tokio::select! {
_ = shutdown.cancelled() => {
info!("tts_manager: reload watcher shutdown");
break;
}
event = rx.recv() => match event {
Ok(SystemEvent::ApiProviderRegistered { .. } | SystemEvent::ApiProviderUnregistered { .. }) => {
match weak.upgrade() {
Some(m) => { if let Err(e) = m.reload().await { warn!(error = %e, "tts_manager: reload failed"); } }
None => break,
}
}
Ok(_) => {}
Err(core_api::system_bus::RecvError::Lagged(n)) => warn!(n, "tts_manager: system_bus lagged"),
Err(core_api::system_bus::RecvError::Closed) => break,
}
}
}
});
Ok(mgr)
}
// ── Resolution ────────────────────────────────────────────────────────────
/// Returns the first available synthesiser:
/// plugin-registered providers take precedence over DB-backed ones.
pub async fn get(&self) -> Option<Arc<dyn TextToSpeech>> {
let state = self.state.read().await;
if let Some(p) = state.plugins.first() {
return Some(Arc::clone(p));
}
state.db_slots.first().map(|s| Arc::clone(&s.synthesiser))
}
// ── Plugin registration (ephemeral) ───────────────────────────────────────
/// Register an ephemeral provider. If a provider with the same `id()` is
/// already present it is replaced.
pub async fn register(&self, provider: Arc<dyn TextToSpeech>) {
let mut state = self.state.write().await;
let id = provider.id().to_string();
state.plugins.retain(|p| p.id() != id);
state.plugins.push(provider);
info!(provider = %id, "tts provider registered (ephemeral)");
}
/// Deregister an ephemeral provider by id. No-op if not found.
pub async fn unregister(&self, id: &str) {
let mut state = self.state.write().await;
let before = state.plugins.len();
state.plugins.retain(|p| p.id() != id);
if state.plugins.len() < before {
info!(provider = %id, "tts provider unregistered (ephemeral)");
}
}
// ── Model CRUD (DB-backed) ────────────────────────────────────────────────
/// Fetch the list of TTS models available from a configured provider.
/// Returns an error if the provider doesn't support model listing.
pub async fn list_provider_models(&self, provider_id: i64) -> Result<Vec<crate::tts::RemoteTtsModelInfo>> {
let record = llm_db::load_all_providers(&self.pool).await?
.into_iter().find(|p| p.id == provider_id)
.ok_or_else(|| anyhow!("provider {provider_id} not found"))?;
let provider = self.registry.get(&record.provider)
.ok_or_else(|| anyhow!("unknown provider type '{}' for provider {provider_id}", record.provider))?;
provider.list_tts_models(&record).await?
.ok_or_else(|| anyhow!("provider '{}' does not support TTS model listing", record.name))
}
pub async fn add_model(&self, record: TtsModelRecord) -> Result<i64> {
let id = tts_db::insert(&self.pool, &record).await?;
self.reload().await?;
Ok(id)
}
pub async fn update_model(&self, id: i64, record: TtsModelRecord) -> Result<()> {
tts_db::update(&self.pool, id, &record).await?;
self.reload().await
}
pub async fn delete_model(&self, id: i64) -> Result<()> {
tts_db::soft_delete(&self.pool, id).await?;
self.reload().await
}
pub async fn get_model(&self, id: i64) -> Option<TtsModelRecord> {
self.state.read().await
.db_slots.iter()
.find(|s| s.record.id == id)
.map(|s| s.record.clone())
}
pub async fn list_models_info(&self) -> Vec<TtsModelInfo> {
self.state.read().await.db_slots.iter().map(|s| TtsModelInfo {
id: s.record.id,
provider_id: s.provider.id,
provider_name: s.provider.name.clone(),
model_id: s.record.model_id.clone(),
voice_id: s.record.voice_id.clone(),
name: s.record.name.clone(),
description: s.record.description.clone(),
instructions: s.record.instructions.clone(),
response_format: s.record.response_format.clone(),
priority: s.record.priority,
from_plugin: false,
}).collect()
}
/// Returns all active providers: plugin-registered first, then DB-backed.
pub async fn list_all_info(&self) -> Vec<TtsModelInfo> {
let state = self.state.read().await;
let plugins = state.plugins.iter().map(|p| TtsModelInfo {
id: 0,
provider_id: 0,
provider_name: "Plugin".into(),
model_id: p.id().to_string(),
voice_id: None,
name: p.name().to_string(),
description: p.description().map(str::to_string),
instructions: p.instructions().map(str::to_string),
response_format: None,
priority: 0,
from_plugin: true,
});
let db = state.db_slots.iter().map(|s| TtsModelInfo {
id: s.record.id,
provider_id: s.provider.id,
provider_name: s.provider.name.clone(),
model_id: s.record.model_id.clone(),
voice_id: s.record.voice_id.clone(),
name: s.record.name.clone(),
description: s.record.description.clone(),
instructions: s.record.instructions.clone(),
response_format: s.record.response_format.clone(),
priority: s.record.priority,
from_plugin: false,
});
plugins.chain(db).collect()
}
// ── Private ───────────────────────────────────────────────────────────────
async fn reload(&self) -> Result<()> {
let model_records: Vec<TtsModelRecord> =
tts_db::load_all(&self.pool).await?;
let provider_records: Vec<LlmProviderRecord> =
llm_db::load_all_providers(&self.pool).await?;
let providers: std::collections::HashMap<i64, LlmProviderRecord> =
provider_records.into_iter().map(|p| (p.id, p)).collect();
let mut db_slots = Vec::new();
for model in model_records {
let provider = match providers.get(&model.provider_id) {
Some(p) => p.clone(),
None => {
warn!(
model = %model.name,
provider_id = model.provider_id,
"orphaned tts model — provider not found, skipping",
);
continue;
}
};
let result = self.registry.get(&provider.provider)
.and_then(|p| p.build_tts(&provider, &model))
.unwrap_or_else(|| anyhow::bail!("provider '{}' does not support TTS", provider.provider));
match result {
Ok(synthesiser) => db_slots.push(TtsSlot { record: model, provider, synthesiser }),
Err(e) => warn!(model = %model.name, error = %e, "failed to build tts synthesiser, skipping"),
}
}
let slot_count = db_slots.len();
self.state.write().await.db_slots = db_slots;
info!(db_backed = slot_count, "tts manager reloaded");
Ok(())
}
}
// ── TtsProvider / TtsRegistry impls ──────────────────────────────────────────
#[async_trait]
impl TtsProvider for TtsManager {
async fn get(&self) -> Option<Arc<dyn TextToSpeech>> {
TtsManager::get(self).await
}
}
#[async_trait]
impl TtsRegistry for TtsManager {
async fn register(&self, provider: Arc<dyn TextToSpeech>) {
TtsManager::register(self, provider).await
}
async fn unregister(&self, id: &str) {
TtsManager::unregister(self, id).await
}
}
crates/skald-core/src/tts/mod.rs
+25
View File
@@ -0,0 +1,25 @@
mod db;
pub mod manager;
pub mod openai_tts;
pub use core_api::tts::{TextToSpeech, TtsProvider, TtsRegistry};
pub use core_api::tts::{TtsModelRecord, RemoteTtsModelInfo};
pub use manager::TtsManager;
/// Public model metadata for API responses.
#[derive(Debug, Clone, serde::Serialize)]
pub struct TtsModelInfo {
pub id: i64,
pub provider_id: i64,
pub provider_name: String,
pub model_id: String,
pub voice_id: Option<String>,
pub name: String,
pub description: Option<String>,
pub instructions: Option<String>,
/// Requested audio `response_format` (`None` ⇒ provider default `mp3`).
pub response_format: Option<String>,
pub priority: i32,
/// `true` for plugin-registered (ephemeral) providers — not editable via the UI.
pub from_plugin: bool,
}
crates/skald-core/src/tts/openai_tts.rs
+161
View File
@@ -0,0 +1,161 @@
/// OpenAiTtsSynthesiser — cloud Text-to-Speech via any OpenAI-compatible
/// audio speech endpoint (OpenAI, …).
///
/// Calls `POST {base_url}/audio/speech` with a JSON body.
/// Returns raw audio bytes in the configured `response_format` (default `mp3`).
use anyhow::{anyhow, Result};
use async_trait::async_trait;
use tracing::{debug, info, warn};
use super::TextToSpeech;
// ── OpenAiTtsSynthesiser ──────────────────────────────────────────────────────
pub struct OpenAiTtsSynthesiser {
/// Stable identifier, e.g. `"openai_tts_alloy"`.
id: String,
base_url: String,
api_key: String,
model: String,
/// Voice/speaker name sent as the `voice` field. `None` ⇒ `alloy`.
/// Provider-specific: OpenAI uses `alloy`/`echo`/`nova`/…; Gemini uses
/// `Kore`/`Puck`/`Zephyr`/… — an unknown name may make the provider error.
voice: Option<String>,
/// Default instructions (voice style, tone, speed). Overridable per call.
instructions: Option<String>,
/// Requested audio format (`response_format`). `None` ⇒ `mp3`.
response_format: Option<String>,
http: reqwest::Client,
}
impl OpenAiTtsSynthesiser {
pub fn new(
id: impl Into<String>,
base_url: impl Into<String>,
api_key: impl Into<String>,
model: impl Into<String>,
voice: Option<String>,
instructions: Option<String>,
response_format: Option<String>,
) -> Self {
Self {
id: id.into(),
base_url: base_url.into(),
api_key: api_key.into(),
model: model.into(),
voice,
instructions,
response_format,
http: reqwest::Client::new(),
}
}
}
#[async_trait]
impl TextToSpeech for OpenAiTtsSynthesiser {
fn id(&self) -> &str { &self.id }
fn name(&self) -> &str { &self.id }
fn instructions(&self) -> Option<&str> { self.instructions.as_deref() }
fn output_format(&self) -> &str { self.response_format.as_deref().unwrap_or("mp3") }
async fn synthesize(&self, text: &str, instructions: Option<&str>) -> Result<Vec<u8>> {
let effective_instructions = instructions.or(self.instructions.as_deref());
// `None` ⇒ provider default. Some models reject `mp3` and require a
// specific value (e.g. Gemini TTS only accepts `pcm`).
let response_format = self.response_format.as_deref().unwrap_or("mp3");
debug!(
chars = text.len(),
model = %self.model,
response_format,
has_instructions = effective_instructions.is_some(),
"openai_tts: synthesising",
);
let url = format!("{}/audio/speech", self.base_url.trim_end_matches('/'));
// Voice from the model config (`tts_models.voice_id`), `alloy` if unset.
// `voice` is required by the OpenAI schema. NOTE: `alloy` is an OpenAI
// voice — providers like Gemini use their own names (`Kore`, `Puck`, …)
// and may reject/500 on an unknown one. Logged below on error.
let voice = self.voice.as_deref().unwrap_or("alloy");
let mut body = serde_json::json!({
"model": self.model,
"input": text,
"voice": voice,
"response_format": response_format,
});
if let Some(instr) = effective_instructions {
body["instructions"] = serde_json::Value::String(instr.to_string());
}
let resp = self.http
.post(&url)
.bearer_auth(&self.api_key)
.header("X-Title", core_api::APP_NAME)
.json(&body)
.send()
.await
.map_err(|e| anyhow!("openai_tts: request failed: {e}"))?;
let status = resp.status();
if !status.is_success() {
// Read the body once, as text. For 5xx the standard `error.message`
// is usually generic ("Internal Server Error") and the real cause sits
// in the raw body — OpenRouter nests the upstream provider error under
// `error.metadata`. Capturing the full body is the only way to see it.
let raw = resp.text().await.unwrap_or_default();
let detail = extract_error_detail(&raw);
warn!(
%status,
model = %self.model,
voice,
response_format,
url = %url,
response_body = %raw.chars().take(2000).collect::<String>(),
"openai_tts: provider returned error",
);
anyhow::bail!("openai_tts: API error {status}: {detail}");
}
let audio = resp
.bytes()
.await
.map_err(|e| anyhow!("openai_tts: failed to read audio bytes: {e}"))?
.to_vec();
info!(
bytes = audio.len(),
model = %self.model,
"openai_tts: synthesis complete",
);
Ok(audio)
}
}
/// Pull the most informative message out of an OpenAI/OpenRouter error body.
/// Prefers OpenRouter's upstream detail (`error.metadata.raw` / `provider_error`),
/// then the standard `error.message`, then the raw body itself. Falls back to a
/// placeholder for an empty body so the caller never reports a bare status code.
fn extract_error_detail(raw: &str) -> String {
if let Ok(v) = serde_json::from_str::<serde_json::Value>(raw) {
let msg = v["error"]["message"].as_str().unwrap_or("").trim();
// OpenRouter wraps the upstream provider's real error here on 5xx.
let upstream = v["error"]["metadata"]["raw"].as_str()
.or_else(|| v["error"]["metadata"]["provider_error"].as_str())
.map(str::trim)
.filter(|s| !s.is_empty());
return match (msg.is_empty(), upstream) {
(false, Some(up)) => format!("{msg} — upstream: {up}"),
(false, None) => msg.to_string(),
(true, Some(up)) => up.to_string(),
(true, None) => raw.trim().to_string(),
};
}
let t = raw.trim();
if t.is_empty() { "<empty response body>".into() } else { t.into() }
}