feat(users): UserManager with per-user SQLCipher, and extract skald-core crate

Two changes developed together in one session; they share the same module
structure (db/mod.rs, the core lib root) and only compile together, so they
land as one commit.

## UserManager + per-user encryption (§9/§11)

New `users::UserManager`: owns the system.db pool plus a map
`userid -> SqlitePool` of unlocked databases. The pool *is* the unlock token —
its connect options carry the DEK as SQLCipher's raw key, so an open pool means
the key is in RAM until restart and dropping it re-locks (§9). Knows nothing
about cookies.

New `crypto` module: envelope encryption. A random 256-bit DEK encrypts
`{userid}.db`; `users.database_password` holds it sealed with AES-256-GCM under
`Argon2id(password, salt)`. The AEAD tag is the password verifier — one
derivation both authenticates and yields the key, so encrypted users store no
second hash. Cleartext users store the Argon2id output directly, compared in
constant time. Argon2 runs in spawn_blocking behind a 2-permit semaphore
(256 MiB per derivation).

- SQLCipher via `libsqlite3-sys` `bundled-sqlcipher-vendored-openssl`, pinned
  <0.38 so it unifies with the one sqlx-sqlite links (a newer copy would apply
  the feature to a SQLite sqlx never uses). OpenSSL is vendored and static, so
  the binary stays self-contained.
- Schema split into `create_registry_tables` (instance-wide, no user key) and
  `create_owner_tables` (one owner's content, identical in every file). No FK in
  the owner bucket may reach the registry — enforced by a standalone test.
  Dropped `chat_history.model_db_id` (write-only, and the only registry-crossing
  key); moved `projects`/`project_tickets` into the owner bucket.
- Provisioning invariant: the file is written before the row, deleted after it,
  so a crash leaves an orphan file, never a user without a database. `open_db`
  never creates: a missing file is an error, not a silent empty database.

Not consumed yet: no login, call sites still use the shared system.db pool.

## Extract crates/skald-core

The headless core moves out of `src/` into its own crate; `skald` (server) and
the coming `skald-setup` are shells around it. Two dependencies on the shell
were inverted rather than dragged along, so the core names neither Tauri nor any
concrete plugin:

- `Plugin::tools(self: Arc<Self>)` — plugins contribute tools through this hook
  (sibling of `http_router`), so the core no longer downcasts to
  `MobileConnectorPlugin`.
- `tools::restart::set_restart_handler` — the desktop shell installs its
  teardown-and-respawn; the core defaults to the supervisor exit code. The core
  loses its `desktop` feature.
- `boot`'s stdout formatter moves to the binary (`src/boot_format.rs`); the core
  only emits tracing events.

All 79 core tests pass; the binary boots and serves in a clean directory, and
the mobile-connector tools still register through the new hook.
This commit is contained in:
2026-07-10 16:48:51 +01:00
parent 38494a85a9
commit 178a38357e
173 changed files with 2650 additions and 1106 deletions
+108
View File
@@ -0,0 +1,108 @@
use anyhow::{Context, Result};
use sqlx::SqlitePool;
use super::TranscribeModelRecord;
#[derive(sqlx::FromRow)]
struct TranscribeModelRow {
id: i64,
provider_id: i64,
model_id: String,
name: String,
language: Option<String>,
priority: i64,
}
pub async fn load_all(pool: &SqlitePool) -> Result<Vec<TranscribeModelRecord>> {
let rows = sqlx::query_as::<_, TranscribeModelRow>(
"SELECT id, provider_id, model_id, name, language, priority
FROM transcribe_models
WHERE removed_at IS NULL
ORDER BY priority ASC, name ASC",
)
.fetch_all(pool)
.await
.context("transcribe_models: load_all")?;
Ok(rows.into_iter().map(row_to_record).collect())
}
pub async fn insert(pool: &SqlitePool, r: &TranscribeModelRecord) -> Result<i64> {
let restored = sqlx::query_scalar::<_, i64>(
"UPDATE transcribe_models
SET provider_id=?1, model_id=?2, name=?3, language=?4, priority=?5, removed_at=NULL
WHERE id = (
SELECT id FROM transcribe_models
WHERE removed_at IS NOT NULL
AND (provider_id=?1 AND model_id=?2 OR name=?3)
LIMIT 1
)
RETURNING id",
)
.bind(r.provider_id)
.bind(&r.model_id)
.bind(&r.name)
.bind(&r.language)
.bind(r.priority as i64)
.fetch_optional(pool)
.await
.context("transcribe_models: restore soft-deleted")?;
if let Some(id) = restored {
return Ok(id);
}
sqlx::query_scalar::<_, i64>(
"INSERT INTO transcribe_models (provider_id, model_id, name, language, priority)
VALUES (?1, ?2, ?3, ?4, ?5)
RETURNING id",
)
.bind(r.provider_id)
.bind(&r.model_id)
.bind(&r.name)
.bind(&r.language)
.bind(r.priority as i64)
.fetch_one(pool)
.await
.context("transcribe_models: insert")
}
pub async fn update(pool: &SqlitePool, id: i64, r: &TranscribeModelRecord) -> Result<()> {
sqlx::query(
"UPDATE transcribe_models
SET provider_id=?1, model_id=?2, name=?3, language=?4, priority=?5
WHERE id=?6",
)
.bind(r.provider_id)
.bind(&r.model_id)
.bind(&r.name)
.bind(&r.language)
.bind(r.priority as i64)
.bind(id)
.execute(pool)
.await
.context("transcribe_models: update")?;
Ok(())
}
pub async fn soft_delete(pool: &SqlitePool, id: i64) -> Result<()> {
sqlx::query(
"UPDATE transcribe_models SET removed_at = datetime('now') WHERE id = ?1",
)
.bind(id)
.execute(pool)
.await
.context("transcribe_models: soft-delete")?;
Ok(())
}
fn row_to_record(r: TranscribeModelRow) -> TranscribeModelRecord {
TranscribeModelRecord {
id: r.id,
provider_id: r.provider_id,
model_id: r.model_id,
name: r.name,
language: r.language,
priority: r.priority as i32,
}
}
+282
View File
@@ -0,0 +1,282 @@
/// TranscribeManager — DB-aware registry of Speech-to-Text providers.
///
/// Two kinds of providers coexist:
/// - **DB-backed**: rows in `transcribe_models`, built from `llm_providers` credentials.
/// Managed via `add_model` / `update_model` / `delete_model`. Loaded on startup
/// and after every mutation (like `LlmManager`).
/// - **Plugin-registered**: ephemeral providers registered at runtime by plugins
/// (e.g. `WhisperLocalPlugin`). Not persisted — they disappear on plugin stop.
///
/// `get()` returns the first plugin provider if any is running, otherwise the
/// first DB-backed provider ordered by `priority ASC`.
use std::sync::Arc;
use anyhow::{Context, Result, anyhow};
use sqlx::SqlitePool;
use tokio::sync::RwLock;
use tokio_util::sync::CancellationToken;
use tracing::{info, warn};
use core_api::system_bus::{SystemEvent, SystemEventBus};
use async_trait::async_trait;
use crate::llm::LlmProviderRecord;
use crate::llm::db as llm_db;
use crate::provider::ProviderRegistry;
use super::{Transcribe, TranscribeModelInfo, TranscribeModelRecord};
use super::db as transcribe_db;
pub use core_api::transcribe::{TranscribeProvider, TranscribeRegistry};
// ── Internal state ────────────────────────────────────────────────────────────
struct TranscribeSlot {
record: TranscribeModelRecord,
provider: LlmProviderRecord,
transcriber: Arc<dyn Transcribe>,
}
struct ManagerState {
/// DB-backed transcribers, ordered by priority ASC. Rebuilt on every reload().
db_slots: Vec<TranscribeSlot>,
/// Plugin-registered providers (ephemeral — not in DB).
/// `WhisperLocalPlugin` registers here via `register()`.
plugins: Vec<Arc<dyn Transcribe>>,
}
// ── TranscribeManager ─────────────────────────────────────────────────────────
pub struct TranscribeManager {
pool: Arc<SqlitePool>,
registry: Arc<ProviderRegistry>,
state: RwLock<ManagerState>,
}
impl TranscribeManager {
pub async fn new(
pool: Arc<SqlitePool>,
registry: Arc<ProviderRegistry>,
system_bus: Arc<SystemEventBus>,
shutdown: CancellationToken,
) -> Result<Arc<Self>> {
let mgr = Arc::new(Self {
pool,
registry,
state: RwLock::new(ManagerState {
db_slots: Vec::new(),
plugins: Vec::new(),
}),
});
mgr.reload().await?;
// Reload whenever an ApiProvider is registered or unregistered.
let weak = Arc::downgrade(&mgr);
let mut rx = system_bus.subscribe();
tokio::spawn(async move {
loop {
tokio::select! {
_ = shutdown.cancelled() => {
info!("transcribe_manager: reload watcher shutdown");
break;
}
event = rx.recv() => match event {
Ok(SystemEvent::ApiProviderRegistered { .. } | SystemEvent::ApiProviderUnregistered { .. }) => {
match weak.upgrade() {
Some(m) => { if let Err(e) = m.reload().await { warn!(error = %e, "transcribe_manager: reload failed"); } }
None => break,
}
}
Ok(_) => {}
Err(core_api::system_bus::RecvError::Lagged(n)) => warn!(n, "transcribe_manager: system_bus lagged"),
Err(core_api::system_bus::RecvError::Closed) => break,
}
}
}
});
Ok(mgr)
}
// ── Resolution ────────────────────────────────────────────────────────────
/// Returns the first available transcriber:
/// plugin-registered providers take precedence over DB-backed ones.
pub async fn get(&self) -> Option<Arc<dyn Transcribe>> {
let state = self.state.read().await;
if let Some(p) = state.plugins.first() {
return Some(Arc::clone(p));
}
state.db_slots.first().map(|s| Arc::clone(&s.transcriber))
}
// ── Plugin registration (ephemeral) ───────────────────────────────────────
/// Register an ephemeral provider. Called by plugins (e.g. WhisperLocalPlugin).
/// If a provider with the same `id()` is already present it is replaced.
pub async fn register(&self, provider: Arc<dyn Transcribe>) {
let mut state = self.state.write().await;
let id = provider.id().to_string();
state.plugins.retain(|p| p.id() != id);
state.plugins.push(provider);
info!(provider = %id, "transcribe provider registered (ephemeral)");
}
/// Deregister an ephemeral provider by id. No-op if not found.
pub async fn unregister(&self, id: &str) {
let mut state = self.state.write().await;
let before = state.plugins.len();
state.plugins.retain(|p| p.id() != id);
if state.plugins.len() < before {
info!(provider = %id, "transcribe provider unregistered (ephemeral)");
}
}
/// Fetch the list of transcription models available from a configured provider.
/// Returns an error if the provider doesn't support model listing.
pub async fn list_provider_models(&self, provider_id: i64) -> Result<Vec<crate::transcribe::RemoteTranscribeModelInfo>> {
let record = llm_db::load_all_providers(&self.pool).await?
.into_iter().find(|p| p.id == provider_id)
.ok_or_else(|| anyhow!("provider {provider_id} not found"))?;
let provider = self.registry.get(&record.provider)
.ok_or_else(|| anyhow!("unknown provider type '{}' for provider {provider_id}", record.provider))?;
provider.list_transcribe_models(&record).await?
.ok_or_else(|| anyhow!("provider '{}' does not support transcription model listing", record.name))
}
// ── Model CRUD (DB-backed) ────────────────────────────────────────────────
pub async fn add_model(&self, record: TranscribeModelRecord) -> Result<i64> {
let id = transcribe_db::insert(&self.pool, &record).await?;
self.reload().await?;
Ok(id)
}
pub async fn update_model(&self, id: i64, record: TranscribeModelRecord) -> Result<()> {
transcribe_db::update(&self.pool, id, &record).await?;
self.reload().await
}
pub async fn delete_model(&self, id: i64) -> Result<()> {
transcribe_db::soft_delete(&self.pool, id).await?;
self.reload().await
}
pub async fn get_model(&self, id: i64) -> Option<TranscribeModelRecord> {
self.state.read().await
.db_slots.iter()
.find(|s| s.record.id == id)
.map(|s| s.record.clone())
}
pub async fn list_models_info(&self) -> Vec<TranscribeModelInfo> {
self.state.read().await.db_slots.iter().map(|s| TranscribeModelInfo {
id: s.record.id,
provider_id: s.provider.id,
provider_name: s.provider.name.clone(),
model_id: s.record.model_id.clone(),
name: s.record.name.clone(),
language: s.record.language.clone(),
priority: s.record.priority,
from_plugin: false,
}).collect()
}
/// Returns all active providers: plugin-registered first (they have precedence
/// in `get()`), then DB-backed ordered by priority. Used by the UI.
pub async fn list_all_info(&self) -> Vec<TranscribeModelInfo> {
let state = self.state.read().await;
let plugins = state.plugins.iter().map(|p| TranscribeModelInfo {
id: 0,
provider_id: 0,
provider_name: "Plugin".into(),
model_id: p.id().to_string(),
name: p.id().to_string(),
language: None,
priority: 0,
from_plugin: true,
});
let db = state.db_slots.iter().map(|s| TranscribeModelInfo {
id: s.record.id,
provider_id: s.provider.id,
provider_name: s.provider.name.clone(),
model_id: s.record.model_id.clone(),
name: s.record.name.clone(),
language: s.record.language.clone(),
priority: s.record.priority,
from_plugin: false,
});
plugins.chain(db).collect()
}
// ── Private ───────────────────────────────────────────────────────────────
async fn reload(&self) -> Result<()> {
let model_records: Vec<TranscribeModelRecord> =
transcribe_db::load_all(&self.pool).await?;
let provider_records: Vec<LlmProviderRecord> =
llm_db::load_all_providers(&self.pool).await?;
let providers: std::collections::HashMap<i64, LlmProviderRecord> =
provider_records.into_iter().map(|p| (p.id, p)).collect();
let mut db_slots = Vec::new();
for model in model_records {
let provider = match providers.get(&model.provider_id) {
Some(p) => p.clone(),
None => {
warn!(
model = %model.name,
provider_id = model.provider_id,
"orphaned transcribe model — provider not found, skipping",
);
continue;
}
};
let result = self.registry.get(&provider.provider)
.and_then(|p| p.build_transcriber(&provider, &model))
.unwrap_or_else(|| anyhow::bail!("provider '{}' does not support transcription", provider.provider));
match result {
Ok(transcriber) => db_slots.push(TranscribeSlot { record: model, provider, transcriber }),
Err(e) => warn!(model = %model.name, error = %e, "failed to build transcriber, skipping"),
}
}
let slot_count = db_slots.len();
// Acquire the write lock once, at the end — no more awaits after this.
// Mirrors LlmManager::reload() to ensure the future stays Send.
// Preserve existing plugin registrations — only replace db_slots.
self.state.write().await.db_slots = db_slots;
info!(db_backed = slot_count, "transcribe manager reloaded");
Ok(())
}
}
// ── TranscribeProvider / TranscribeRegistry impls ────────────────────────────
#[async_trait]
impl TranscribeProvider for TranscribeManager {
async fn get(&self) -> Option<Arc<dyn Transcribe>> {
TranscribeManager::get(self).await
}
}
#[async_trait]
impl TranscribeRegistry for TranscribeManager {
async fn register(&self, provider: Arc<dyn Transcribe>) {
TranscribeManager::register(self, provider).await
}
async fn unregister(&self, id: &str) {
TranscribeManager::unregister(self, id).await
}
}
+21
View File
@@ -0,0 +1,21 @@
mod db;
pub mod manager;
pub mod openai_audio;
pub use core_api::transcribe::{Transcribe, TranscribeProvider, TranscribeRegistry};
pub use core_api::transcribe::{TranscribeModelRecord, RemoteTranscribeModelInfo};
pub use manager::TranscribeManager;
/// Public model metadata for API responses.
#[derive(Debug, Clone, serde::Serialize)]
pub struct TranscribeModelInfo {
pub id: i64,
pub provider_id: i64,
pub provider_name: String,
pub model_id: String,
pub name: String,
pub language: Option<String>,
pub priority: i32,
/// `true` for plugin-registered (ephemeral) providers — not editable via the UI.
pub from_plugin: bool,
}
@@ -0,0 +1,126 @@
/// OpenAiAudioTranscriber — cloud Speech-to-Text via any OpenAI-compatible
/// audio transcription endpoint (OpenAI, OpenRouter, …).
///
/// Calls `POST {base_url}/audio/transcriptions` with a multipart/form-data body.
/// No local model, no GPU, no ffmpeg — the provider handles everything server-side.
use anyhow::{anyhow, Result};
use async_trait::async_trait;
use tracing::{debug, info};
use super::Transcribe;
// ── OpenAiAudioTranscriber ────────────────────────────────────────────────────
pub struct OpenAiAudioTranscriber {
/// Stable identifier, e.g. `"openrouter_whisper"` or `"openai_whisper"`.
id: String,
base_url: String,
api_key: String,
model: String,
/// BCP-47 language hint (e.g. `"it"`, `"en"`). `None` = let the model auto-detect.
language: Option<String>,
http: reqwest::Client,
}
impl OpenAiAudioTranscriber {
pub fn new(
id: impl Into<String>,
base_url: impl Into<String>,
api_key: impl Into<String>,
model: impl Into<String>,
language: Option<String>,
) -> Self {
Self {
id: id.into(),
base_url: base_url.into(),
api_key: api_key.into(),
model: model.into(),
language,
http: reqwest::Client::new(),
}
}
}
#[async_trait]
impl Transcribe for OpenAiAudioTranscriber {
fn id(&self) -> &str { &self.id }
async fn transcribe(&self, audio: Vec<u8>, format: &str) -> Result<String> {
debug!(
bytes = audio.len(),
format,
model = %self.model,
"openai_audio: transcribing",
);
let mime = mime_for_format(format);
let filename = format!("audio.{format}");
let file_part = reqwest::multipart::Part::bytes(audio)
.file_name(filename)
.mime_str(mime)
.map_err(|e| anyhow!("invalid mime type '{mime}': {e}"))?;
let mut form = reqwest::multipart::Form::new()
.text("model", self.model.clone())
.part("file", file_part);
if let Some(lang) = &self.language {
form = form.text("language", lang.clone());
}
let url = format!("{}/audio/transcriptions", self.base_url.trim_end_matches('/'));
let resp = self.http
.post(&url)
.bearer_auth(&self.api_key)
.header("X-Title", core_api::APP_NAME)
.multipart(form)
.send()
.await
.map_err(|e| anyhow!("openai_audio: request failed: {e}"))?;
let status = resp.status();
let body: serde_json::Value = resp
.json()
.await
.map_err(|e| anyhow!("openai_audio: response parse failed: {e}"))?;
if !status.is_success() {
let msg = body["error"]["message"]
.as_str()
.unwrap_or("unknown error");
anyhow::bail!("openai_audio: API error {status}: {msg}");
}
let text = body["text"]
.as_str()
.ok_or_else(|| anyhow!("openai_audio: missing 'text' field in response"))?
.trim()
.to_string();
info!(
chars = text.len(),
model = %self.model,
"openai_audio: transcription complete",
);
Ok(text)
}
}
// ── Helpers ───────────────────────────────────────────────────────────────────
/// Maps a file extension to an appropriate MIME type for the multipart upload.
/// The OpenAI audio API accepts: mp3, mp4, mpeg, mpga, m4a, wav, webm, ogg.
fn mime_for_format(format: &str) -> &'static str {
match format {
"mp3" | "mpeg" | "mpga" => "audio/mpeg",
"mp4" | "m4a" => "audio/mp4",
"wav" => "audio/wav",
"webm" => "audio/webm",
"ogg" => "audio/ogg",
"flac" => "audio/flac",
_ => "application/octet-stream",
}
}