feat(users): UserManager with per-user SQLCipher, and extract skald-core crate

Two changes developed together in one session; they share the same module
structure (db/mod.rs, the core lib root) and only compile together, so they
land as one commit.

## UserManager + per-user encryption (§9/§11)

New `users::UserManager`: owns the system.db pool plus a map
`userid -> SqlitePool` of unlocked databases. The pool *is* the unlock token —
its connect options carry the DEK as SQLCipher's raw key, so an open pool means
the key is in RAM until restart and dropping it re-locks (§9). Knows nothing
about cookies.

New `crypto` module: envelope encryption. A random 256-bit DEK encrypts
`{userid}.db`; `users.database_password` holds it sealed with AES-256-GCM under
`Argon2id(password, salt)`. The AEAD tag is the password verifier — one
derivation both authenticates and yields the key, so encrypted users store no
second hash. Cleartext users store the Argon2id output directly, compared in
constant time. Argon2 runs in spawn_blocking behind a 2-permit semaphore
(256 MiB per derivation).

- SQLCipher via `libsqlite3-sys` `bundled-sqlcipher-vendored-openssl`, pinned
  <0.38 so it unifies with the one sqlx-sqlite links (a newer copy would apply
  the feature to a SQLite sqlx never uses). OpenSSL is vendored and static, so
  the binary stays self-contained.
- Schema split into `create_registry_tables` (instance-wide, no user key) and
  `create_owner_tables` (one owner's content, identical in every file). No FK in
  the owner bucket may reach the registry — enforced by a standalone test.
  Dropped `chat_history.model_db_id` (write-only, and the only registry-crossing
  key); moved `projects`/`project_tickets` into the owner bucket.
- Provisioning invariant: the file is written before the row, deleted after it,
  so a crash leaves an orphan file, never a user without a database. `open_db`
  never creates: a missing file is an error, not a silent empty database.

Not consumed yet: no login, call sites still use the shared system.db pool.

## Extract crates/skald-core

The headless core moves out of `src/` into its own crate; `skald` (server) and
the coming `skald-setup` are shells around it. Two dependencies on the shell
were inverted rather than dragged along, so the core names neither Tauri nor any
concrete plugin:

- `Plugin::tools(self: Arc<Self>)` — plugins contribute tools through this hook
  (sibling of `http_router`), so the core no longer downcasts to
  `MobileConnectorPlugin`.
- `tools::restart::set_restart_handler` — the desktop shell installs its
  teardown-and-respawn; the core defaults to the supervisor exit code. The core
  loses its `desktop` feature.
- `boot`'s stdout formatter moves to the binary (`src/boot_format.rs`); the core
  only emits tracing events.

All 79 core tests pass; the binary boots and serves in a clean directory, and
the mobile-connector tools still register through the new hook.
This commit is contained in:
2026-07-10 16:48:51 +01:00
parent 38494a85a9
commit 178a38357e
173 changed files with 2650 additions and 1106 deletions
+103
View File
@@ -0,0 +1,103 @@
use anyhow::{Context, Result};
use sqlx::SqlitePool;
use super::ImageGenerateModelRecord;
#[derive(sqlx::FromRow)]
struct ImageGenerateModelRow {
id: i64,
provider_id: i64,
model_id: String,
name: String,
priority: i64,
}
pub async fn load_all(pool: &SqlitePool) -> Result<Vec<ImageGenerateModelRecord>> {
let rows = sqlx::query_as::<_, ImageGenerateModelRow>(
"SELECT id, provider_id, model_id, name, priority
FROM image_generate_models
WHERE removed_at IS NULL
ORDER BY priority ASC, name ASC",
)
.fetch_all(pool)
.await
.context("image_generate_models: load_all")?;
Ok(rows.into_iter().map(row_to_record).collect())
}
pub async fn insert(pool: &SqlitePool, r: &ImageGenerateModelRecord) -> Result<i64> {
let restored = sqlx::query_scalar::<_, i64>(
"UPDATE image_generate_models
SET provider_id=?1, model_id=?2, name=?3, priority=?4, removed_at=NULL
WHERE id = (
SELECT id FROM image_generate_models
WHERE removed_at IS NOT NULL
AND (provider_id=?1 AND model_id=?2 OR name=?3)
LIMIT 1
)
RETURNING id",
)
.bind(r.provider_id)
.bind(&r.model_id)
.bind(&r.name)
.bind(r.priority as i64)
.fetch_optional(pool)
.await
.context("image_generate_models: restore soft-deleted")?;
if let Some(id) = restored {
return Ok(id);
}
sqlx::query_scalar::<_, i64>(
"INSERT INTO image_generate_models (provider_id, model_id, name, priority)
VALUES (?1, ?2, ?3, ?4)
RETURNING id",
)
.bind(r.provider_id)
.bind(&r.model_id)
.bind(&r.name)
.bind(r.priority as i64)
.fetch_one(pool)
.await
.context("image_generate_models: insert")
}
pub async fn update(pool: &SqlitePool, id: i64, r: &ImageGenerateModelRecord) -> Result<()> {
sqlx::query(
"UPDATE image_generate_models
SET provider_id=?1, model_id=?2, name=?3, priority=?4
WHERE id=?5",
)
.bind(r.provider_id)
.bind(&r.model_id)
.bind(&r.name)
.bind(r.priority as i64)
.bind(id)
.execute(pool)
.await
.context("image_generate_models: update")?;
Ok(())
}
pub async fn soft_delete(pool: &SqlitePool, id: i64) -> Result<()> {
sqlx::query(
"UPDATE image_generate_models SET removed_at = datetime('now') WHERE id = ?1",
)
.bind(id)
.execute(pool)
.await
.context("image_generate_models: soft-delete")?;
Ok(())
}
fn row_to_record(r: ImageGenerateModelRow) -> ImageGenerateModelRecord {
ImageGenerateModelRecord {
id: r.id,
provider_id: r.provider_id,
model_id: r.model_id,
name: r.name,
priority: r.priority as i32,
}
}
@@ -0,0 +1,296 @@
/// ImageGeneratorManager — DB-aware registry of image generation providers.
///
/// Two kinds of providers coexist:
/// - **DB-backed**: rows in `image_generate_models`, built from `llm_providers` credentials.
/// Managed via `add_model` / `update_model` / `delete_model`. Loaded on startup
/// and after every mutation.
/// - **Plugin-registered**: ephemeral providers registered at runtime by plugins.
/// Not persisted — they disappear on plugin stop.
///
/// `get(id)` resolves by explicit id across both plugin and DB-backed providers.
/// When called without an id, plugin providers take precedence over DB-backed ones.
use std::path::PathBuf;
use std::sync::Arc;
use anyhow::{Context, Result, anyhow};
use async_trait::async_trait;
use rand::RngExt;
use sqlx::SqlitePool;
use tokio::sync::RwLock;
use tracing::{info, warn};
use core_api::image_generate::ImageGenerateRegistry;
use crate::llm::LlmProviderRecord;
use crate::llm::db as llm_db;
use crate::provider::ProviderRegistry;
use crate::tools::Tool;
use super::{ImageGenerate, ImageGenerateInfo, ImageGenerateModelInfo, ImageGenerateModelRecord};
use super::db as image_db;
// ── Internal state ────────────────────────────────────────────────────────────
struct ImageGenerateSlot {
record: ImageGenerateModelRecord,
provider: LlmProviderRecord,
generator: Arc<dyn ImageGenerate>,
}
struct ManagerState {
/// DB-backed generators, ordered by priority ASC. Rebuilt on every reload().
db_slots: Vec<ImageGenerateSlot>,
/// Plugin-registered providers (ephemeral — not in DB).
plugins: Vec<Arc<dyn ImageGenerate>>,
}
// ── ImageGeneratorManager ─────────────────────────────────────────────────────
pub struct ImageGeneratorManager {
pool: Arc<SqlitePool>,
registry: Arc<ProviderRegistry>,
state: RwLock<ManagerState>,
data_root: PathBuf,
}
impl ImageGeneratorManager {
pub async fn new(
pool: Arc<SqlitePool>,
registry: Arc<ProviderRegistry>,
data_root: impl Into<PathBuf>,
) -> Result<Arc<Self>> {
let mgr = Arc::new(Self {
pool,
registry,
state: RwLock::new(ManagerState {
db_slots: Vec::new(),
plugins: Vec::new(),
}),
data_root: data_root.into(),
});
mgr.reload().await?;
Ok(mgr)
}
// ── Plugin registration (ephemeral) ───────────────────────────────────────
pub async fn register(&self, provider: Arc<dyn ImageGenerate>) {
let mut state = self.state.write().await;
let id = provider.id().to_string();
state.plugins.retain(|p| p.id() != id);
state.plugins.push(provider);
info!(provider_id = %id, "image generator registered (plugin)");
}
pub async fn unregister(&self, id: &str) {
let mut state = self.state.write().await;
let before = state.plugins.len();
state.plugins.retain(|p| p.id() != id);
if state.plugins.len() < before {
info!(provider_id = %id, "image generator unregistered (plugin)");
}
}
// ── Model CRUD (DB-backed) ────────────────────────────────────────────────
pub async fn add_model(&self, record: ImageGenerateModelRecord) -> Result<i64> {
let id = image_db::insert(&self.pool, &record).await?;
self.reload().await?;
Ok(id)
}
pub async fn update_model(&self, id: i64, record: ImageGenerateModelRecord) -> Result<()> {
image_db::update(&self.pool, id, &record).await?;
self.reload().await
}
pub async fn delete_model(&self, id: i64) -> Result<()> {
image_db::soft_delete(&self.pool, id).await?;
self.reload().await
}
pub async fn get_model(&self, id: i64) -> Option<ImageGenerateModelRecord> {
self.state.read().await
.db_slots.iter()
.find(|s| s.record.id == id)
.map(|s| s.record.clone())
}
pub async fn list_models_info(&self) -> Vec<ImageGenerateModelInfo> {
self.state.read().await.db_slots.iter().map(|s| ImageGenerateModelInfo {
id: s.record.id,
provider_id: s.provider.id,
provider_name: s.provider.name.clone(),
model_id: s.record.model_id.clone(),
name: s.record.name.clone(),
priority: s.record.priority,
from_plugin: false,
description: None,
}).collect()
}
/// Returns all active providers: plugin-registered first, then DB-backed by priority.
pub async fn list_all_info(&self) -> Vec<ImageGenerateModelInfo> {
let state = self.state.read().await;
let plugins = state.plugins.iter().map(|p| ImageGenerateModelInfo {
id: 0,
provider_id: 0,
provider_name: "Plugin".into(),
model_id: p.id().to_string(),
name: p.name().to_string(),
priority: 0,
from_plugin: true,
description: p.description().map(str::to_string),
});
let db = state.db_slots.iter().map(|s| ImageGenerateModelInfo {
id: s.record.id,
provider_id: s.provider.id,
provider_name: s.provider.name.clone(),
model_id: s.record.model_id.clone(),
name: s.record.name.clone(),
priority: s.record.priority,
from_plugin: false,
description: None,
});
plugins.chain(db).collect()
}
// ── Provider queries ───────────────────────────────────────────────────────
/// Returns all active providers as lightweight info structs (for LLM tool).
pub async fn list(&self) -> Vec<ImageGenerateInfo> {
let state = self.state.read().await;
state.plugins.iter()
.map(|p| ImageGenerateInfo {
id: p.id().to_string(),
name: p.name().to_string(),
description: p.description().map(str::to_string),
extra_params_schema: p.extra_params_schema(),
})
.chain(state.db_slots.iter().map(|s| ImageGenerateInfo {
id: s.record.name.clone(),
name: s.record.name.clone(),
description: None,
extra_params_schema: None,
}))
.collect()
}
/// Looks up a provider by id — plugins first, then DB-backed by name.
pub async fn get(&self, id: &str) -> Option<Arc<dyn ImageGenerate>> {
let state = self.state.read().await;
if let Some(p) = state.plugins.iter().find(|p| p.id() == id) {
return Some(Arc::clone(p));
}
state.db_slots.iter()
.find(|s| s.record.name == id)
.map(|s| Arc::clone(&s.generator))
}
// ── Generation ────────────────────────────────────────────────────────────
pub async fn generate(
&self,
provider_id: &str,
prompt: &str,
extra_params: Option<&serde_json::Value>,
) -> Result<(PathBuf, String)> {
let provider = self.get(provider_id).await
.ok_or_else(|| anyhow!("image provider '{}' not found", provider_id))?;
let images_dir = self.data_root.join("images");
tokio::fs::create_dir_all(&images_dir).await?;
let bytes = provider.generate(prompt, extra_params).await?;
let file_id: String = rand::rng()
.sample_iter(rand::distr::Alphanumeric)
.take(32)
.map(char::from)
.collect();
let path = images_dir.join(format!("{file_id}.png"));
tokio::fs::write(&path, &bytes).await?;
let url = format!("/api/images/{file_id}");
info!(provider_id, path = %path.display(), "image generated");
Ok((path, url))
}
// ── Tool injection ─────────────────────────────────────────────────────────
/// Returns the two image tools when at least one provider is active.
/// Called per-turn by the session handler to conditionally inject tools.
pub async fn tools(self: Arc<Self>) -> Vec<Arc<dyn Tool>> {
let state = self.state.read().await;
if state.plugins.is_empty() && state.db_slots.is_empty() {
return vec![];
}
drop(state);
vec![
Arc::new(crate::tools::image_generate::ImageGenerateProvidersList { mgr: Arc::clone(&self) }) as Arc<dyn Tool>,
Arc::new(crate::tools::image_generate::ImageGenerateTool { mgr: Arc::clone(&self) }) as Arc<dyn Tool>,
]
}
pub fn images_dir(&self) -> PathBuf {
self.data_root.join("images")
}
// ── Private ───────────────────────────────────────────────────────────────
async fn reload(&self) -> Result<()> {
let model_records = image_db::load_all(&self.pool).await?;
let provider_records: Vec<LlmProviderRecord> =
llm_db::load_all_providers(&self.pool).await?;
let providers: std::collections::HashMap<i64, LlmProviderRecord> =
provider_records.into_iter().map(|p| (p.id, p)).collect();
let mut db_slots = Vec::new();
for model in model_records {
let provider = match providers.get(&model.provider_id) {
Some(p) => p.clone(),
None => {
warn!(
model = %model.name,
provider_id = model.provider_id,
"orphaned image model — provider not found, skipping",
);
continue;
}
};
let result = self.registry.get(&provider.provider)
.and_then(|p| p.build_image_generator(&provider, &model))
.unwrap_or_else(|| anyhow::bail!("provider '{}' does not support image generation", provider.provider));
match result {
Ok(generator) => db_slots.push(ImageGenerateSlot { record: model, provider, generator }),
Err(e) => warn!(model = %model.name, error = %e, "failed to build image generator, skipping"),
}
}
let slot_count = db_slots.len();
self.state.write().await.db_slots = db_slots;
info!(db_backed = slot_count, "image generator manager reloaded");
Ok(())
}
}
// ── ImageGenerateRegistry impl ────────────────────────────────────────────────
#[async_trait]
impl ImageGenerateRegistry for ImageGeneratorManager {
async fn register(&self, provider: Arc<dyn ImageGenerate>) {
ImageGeneratorManager::register(self, provider).await;
}
async fn unregister(&self, id: &str) {
ImageGeneratorManager::unregister(self, id).await;
}
}
@@ -0,0 +1,37 @@
mod db;
pub mod manager;
pub mod openrouter_image;
pub use core_api::image_generate::ImageGenerate;
pub use core_api::image_generate::ImageGenerateModelRecord;
pub use manager::ImageGeneratorManager;
/// Public model metadata for API responses.
#[derive(Debug, Clone, serde::Serialize)]
pub struct ImageGenerateModelInfo {
pub id: i64,
pub provider_id: i64,
pub provider_name: String,
pub model_id: String,
pub name: String,
pub priority: i32,
/// `true` for plugin-registered (ephemeral) providers — not editable via the UI.
pub from_plugin: bool,
#[serde(skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
}
// ── Tool-facing types ─────────────────────────────────────────────────────────
/// Lightweight provider listing returned by `image_generate_providers_list`.
#[derive(Debug, Clone, serde::Serialize)]
pub struct ImageGenerateInfo {
pub id: String,
pub name: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
/// JSON Schema for the `extra_params` argument. Present only if the provider
/// accepts provider-specific parameters (e.g. width, height, steps).
#[serde(skip_serializing_if = "Option::is_none")]
pub extra_params_schema: Option<serde_json::Value>,
}
@@ -0,0 +1,94 @@
/// OpenRouter image generation via the chat completions endpoint with `modalities`.
///
/// Calls `POST {base_url}/chat/completions` with:
/// `{"model": ..., "messages": [...], "modalities": ["image", "text"]}`
///
/// The response image is returned as a base64 data URL inside
/// `choices[0].message.images[0].image_url.url`.
use anyhow::{anyhow, Result};
use async_trait::async_trait;
use base64::Engine;
use tracing::{debug, info};
use super::ImageGenerate;
pub struct OpenRouterImageGenerator {
/// Stable display identifier, e.g. `"my_openrouter_grok"`.
id: String,
base_url: String,
api_key: String,
model: String,
http: reqwest::Client,
}
impl OpenRouterImageGenerator {
pub fn new(
id: impl Into<String>,
base_url: impl Into<String>,
api_key: impl Into<String>,
model: impl Into<String>,
) -> Self {
Self {
id: id.into(),
base_url: base_url.into(),
api_key: api_key.into(),
model: model.into(),
http: reqwest::Client::new(),
}
}
}
#[async_trait]
impl ImageGenerate for OpenRouterImageGenerator {
fn id(&self) -> &str { &self.id }
fn name(&self) -> &str { &self.id }
async fn generate(&self, prompt: &str, _extra_params: Option<&serde_json::Value>) -> Result<Vec<u8>> {
debug!(model = %self.model, "openrouter_image: generating");
let url = format!("{}/chat/completions", self.base_url.trim_end_matches('/'));
let body = serde_json::json!({
"model": self.model,
"messages": [{ "role": "user", "content": prompt }],
"modalities": ["image"],
});
let resp = self.http
.post(&url)
.bearer_auth(&self.api_key)
.header("X-Title", core_api::APP_NAME)
.json(&body)
.send()
.await
.map_err(|e| anyhow!("openrouter_image: request failed: {e}"))?;
let status = resp.status();
let json: serde_json::Value = resp
.json()
.await
.map_err(|e| anyhow!("openrouter_image: response parse failed: {e}"))?;
if !status.is_success() {
let msg = json["error"]["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("openrouter_image: API error {status}: {msg}");
}
let data_url = json["choices"][0]["message"]["images"][0]["image_url"]["url"]
.as_str()
.ok_or_else(|| anyhow!("openrouter_image: no image in response — full response: {json}"))?;
let b64 = data_url
.strip_prefix("data:image/png;base64,")
.or_else(|| data_url.strip_prefix("data:image/jpeg;base64,"))
.or_else(|| data_url.strip_prefix("data:image/webp;base64,"))
.unwrap_or(data_url);
let bytes = base64::engine::general_purpose::STANDARD
.decode(b64)
.map_err(|e| anyhow!("openrouter_image: base64 decode failed: {e}"))?;
info!(model = %self.model, bytes = bytes.len(), "openrouter_image: generation complete");
Ok(bytes)
}
}