Commit Graph
8 Commits
Author SHA1 Message Date
dguiducci fefcf95362 mcp: install dep reconciler, connector login/status endpoints, agent prompt updates
Nightly Build / build (push) Successful in 6m26s
2026-07-20 17:39:07 +01:00
dguiducci f85876350e feat(mcp): WhatsApp connector, archivable catalog, MCP connector config endpoint 2026-07-19 10:55:44 +01:00
dguiducci 126886e309 Major rebranding, i18n, dashboard, shared folders, role capabilities
- Rebrand: new app/agent icons, SKALD.md, warm "paper" CSS palette
  (terracotta accent, --radius tokens, WCAG contrast, reduced-motion),
  updated favicon, tray icon, skaldkonur asset
- i18n: backend crate (i18n.rs, locale column, ui_locale config),
  frontend library (web/lib/i18n.js, I18nMixin, t(key)),
  translation files (web/i18n/), every component wired
- Dashboard: <dashboard-page> replaces old home-page content;
  <app-copilot> becomes the landing page (full/dock layout modes)
- Shared folders: API endpoints (shared_folders.rs), frontend page,
  can_write membership, container mount topology, user_fs routing
- Role capabilities: new db table & authorization seam (data not enums),
  roles.attrs JSON for ui_mode / interface select
- Setup: skald-setup prompts for language + password, sets ui_locale
- General: components migrated to CSS variables, Lit conventions cleanup,
  connectors/catalog/marketplace/approval refactoring
2026-07-18 21:38:42 +01:00
dguiducci 760dae06e5 feat(providers): Moonshot AI provider + extract shared OpenAI-compat helpers
- Add MoonshotProvider and MoonshotCodeProvider (OpenAI-compatible)
- Extract `fetch_openai_models()` and `build_openai_llm()` shared functions,
  deduplicating model-listing and client construction across OpenAI,
  OpenRouter, DeepSeek, LM Studio, and Z.AI providers
- Add `lists_models` field to `ProviderUiMeta` — drives frontend model
  picker dynamically instead of hardcoded `type_id` list
- Port DeepSeek, LM Studio, OpenRouter model listing to `fetch_openai_models`
- Set `lists_models: true` on Z.AI provider (missed in prior pass)

feat(mcp): named key placeholders {SECRET:name}/{ENV:name} in connector URLs

- `apply_key_placeholder` now accepts an `env` map alongside `api_key`
- Named tokens resolve from the connector's described `env[]` fields,
  with `{SECRET:name}` falling back to `api_key` for backward compat
- Replace `substitute_secret_tokens` with `substitute_named_tokens`
- Unresolved tokens are left in place (visible misconfig) rather than
  silently producing a wrong URL

fix(ui): connector detail hides generic API key when schema has secret

fix(ui): model picker uses `lists_models` from provider types endpoint
2026-07-18 15:41:53 +01:00
dguiducci e6c4e202a4 feat(mcp): OAuth per-user connectors (§15) — providers, PKCE copy-paste flow, env credential delivery
- oauth_providers registry table (per-provider client creds) + db/oauth_providers.rs
- mcp/oauth.rs: authorization-code + PKCE S256, RAM-only TTL'd flow store, copy-paste consent
- mcp/install.rs + verify.rs: connector file install + manifest verification
- activate persists a pending row (needs_oauth); /mcp/oauth/start + /complete exchange code for refresh token
- credential delivery via env var on docker exec (google_authorized_user JSON), never on disk
- mcp_catalog/mcp_user_servers: additive OAuth columns (ensure_column), catalog_name/oauth_provider/deliver_json bare TEXT snapshots
- frontend: connector-detail.js (OAuth login panel), shared/connector-common.js, connectors.js admin Sign-in providers modal
- API: /mcp/providers (admin OAuth creds), /mcp/oauth/start|complete
- .gitignore: add /homes/ (instance data), /connectors/, /reset.sh; drop stale /secrets/
2026-07-17 21:47:51 +01:00
dguiducci bcd8f7b5c0 feat(mcp): connector marketplace + split the Connectors surface (§7/§14/§15)
Fills a gap the blueprint names: the admin had to hand-author every
`mcp_catalog` entry. A remote feed of vetted connectors now proposes them
and the admin installs — the feed is *consultative*, so §14's risk axis is
untouched and the trust anchor stays on the box.

Marketplace client (`src/frontend/api/marketplace.rs`):
- Fetches the feed server-side (it sends no CORS headers) and caches it;
  icons are proxied for the same reason.
- Verifies every declared SHA-256 before writing, fail-closed and
  all-or-nothing. Feed-supplied paths are refused if they escape
  `./scripts/<id>/`. Importing an `mcp_local` entry still demands the
  admin-only `mcp.register_local_script`.
- Translates the feed's vocabulary into Skald's: `user`→`per_user`,
  `mcp_local`→`local_script`. Scope is read, never inferred from transport
  (a remote connector can be per-user — that is what `mcp.register_remote`
  is for), and an unreadable `type` fails closed to the answer needing more
  authority. The feed's `llm_short_description` maps to `description`, the
  column `render_mcp_list` puts in front of the LLM for `activate_tools()`.
- Feed URL is config (`marketplace.url`), not a constant: an on-premise
  product must not hard-require reaching one vendor's host.

Two silent failures found while wiring it:
- `transport_of` maps anything unknown to Stdio, so the feed's
  `streamable-http` would have tried to spawn a command. Normalised on import.
- Some servers want their key as a query param, not a bearer header, and say
  so with a `{key}` placeholder. Substituted at connect time in
  `global_row_spec`/`user_row_spec` — never at rest, so the key stays in its
  own column and the stored URL stays a template.

Pages, split by the question each answers:
- Connectors — what runs (`UserMcpView` = global ∪ per-user) and what I can
  add. Same page for everyone; the admin just has more verbs. One Available
  list with the verb per row: `per_user`→Activate, `global`→Enable globally.
  Enabling a global is the admin's counterpart to activating a per-user one,
  so the catalog picker dropdown is gone — the entry comes from the row.
- Connector Catalog (admin) — what this box offers. One `Add connector`
  with two sources: marketplace first (vetted, hashed), manual second
  (unvetted by nature) — the order mirrors the trust model.
- Marketplace (admin) — reached from the catalog, not the sidebar: it is a
  destination of an action, not a place.

`available()` no longer returns `McpGlobalServerRow`: that row carries
`api_key` and this view now reaches every logged-in user. A slim `GlobalView`
crosses instead, and an admin sees every global (with `can_use` marking their
own) so one enabled for someone else stays manageable.

Also fixes `connectors-page` having no CSS rule at all — every sibling page
has one, so it never got `flex: 1` and left an empty column beside it.
2026-07-16 18:52:59 +01:00
dguiducci 6d299472e3 feat(mcp): Connectors — catalog + global vs per-user runtimes (§7/§14/§15)
Re-architects MCP from one owner table + agent-written registration into an
admin-curated catalog with two runtimes unioned per session, surfaced in the UI
as "Connectors" (mcp/schema stays neutral, §0.1).

Two runtimes behind one seam (§7):
- Global runtime: shared, stateless connectors (web-search, Tavily…) on the
  HOST, connected at boot from mcp_global_servers, access-filtered per user via
  mcp_global_access.
- Per-user runtime: a user's activated connectors run INSIDE their container,
  started at first login from mcp_user_servers and living until restart (§9);
  docker exec -i children die via kill_on_drop when the UserContext drops.
- McpProvider trait (mcp/provider.rs): the session round-loop never learns which
  runtime owns a server. McpManager implements it directly (inert ownerless
  bundle); UserMcpView implements global ∪ user with an accessible_global
  snapshot. Both share McpManager::connect_all; McpServerSpec +
  global_row_spec/user_row_spec turn a DB row into a connectable spec.
- mcp-client: McpServerConfig.launch_in runs a stdio command inside a container
  via docker exec -i (set at runtime, never parsed from config).

Authorization is a capability on the role, not `if role==admin` (§0.1/§14):
role_capabilities table + db/role_capabilities.rs — register_remote and
register_local_from_catalog are self-service (seeded on every new role), while
register_local_script and manage_catalog are admin-only. admin holds every
capability by construction. This removes the agent-facing register_mcp/delete_mcp
tools and the mcp kinds of list_items/toggle_item, closing the §14 RCE vector.

Schema:
- Registry: mcp_catalog (vetted templates — schema only, no live creds),
  mcp_global_servers + mcp_global_access, role_capabilities.
- Owner: mcp_user_servers (per-user activations; api_key encrypted at rest,
  catalog_name a bare TEXT snapshot, never an owner→registry FK).
- Drops the old owner table mcp_servers.

API + UI: src/frontend/api/mcp.rs (admin catalog/global/access + user
available/activate/activated, all capability-gated via require_cap);
web/components/connectors.js (<connectors-page>) renders the user view always
and the admin view for role_id === 'admin'.

Deferred: interactive per-user auth (OAuth callback / QR / SSH elicitation, §15)
— only none/api_key wired; no boot seed of catalog presets; per-(user, session)
MCP grant model still open.
2026-07-16 16:44:12 +01:00
dguiducci 178a38357e feat(users): UserManager with per-user SQLCipher, and extract skald-core crate
Two changes developed together in one session; they share the same module
structure (db/mod.rs, the core lib root) and only compile together, so they
land as one commit.

## UserManager + per-user encryption (§9/§11)

New `users::UserManager`: owns the system.db pool plus a map
`userid -> SqlitePool` of unlocked databases. The pool *is* the unlock token —
its connect options carry the DEK as SQLCipher's raw key, so an open pool means
the key is in RAM until restart and dropping it re-locks (§9). Knows nothing
about cookies.

New `crypto` module: envelope encryption. A random 256-bit DEK encrypts
`{userid}.db`; `users.database_password` holds it sealed with AES-256-GCM under
`Argon2id(password, salt)`. The AEAD tag is the password verifier — one
derivation both authenticates and yields the key, so encrypted users store no
second hash. Cleartext users store the Argon2id output directly, compared in
constant time. Argon2 runs in spawn_blocking behind a 2-permit semaphore
(256 MiB per derivation).

- SQLCipher via `libsqlite3-sys` `bundled-sqlcipher-vendored-openssl`, pinned
  <0.38 so it unifies with the one sqlx-sqlite links (a newer copy would apply
  the feature to a SQLite sqlx never uses). OpenSSL is vendored and static, so
  the binary stays self-contained.
- Schema split into `create_registry_tables` (instance-wide, no user key) and
  `create_owner_tables` (one owner's content, identical in every file). No FK in
  the owner bucket may reach the registry — enforced by a standalone test.
  Dropped `chat_history.model_db_id` (write-only, and the only registry-crossing
  key); moved `projects`/`project_tickets` into the owner bucket.
- Provisioning invariant: the file is written before the row, deleted after it,
  so a crash leaves an orphan file, never a user without a database. `open_db`
  never creates: a missing file is an error, not a silent empty database.

Not consumed yet: no login, call sites still use the shared system.db pool.

## Extract crates/skald-core

The headless core moves out of `src/` into its own crate; `skald` (server) and
the coming `skald-setup` are shells around it. Two dependencies on the shell
were inverted rather than dragged along, so the core names neither Tauri nor any
concrete plugin:

- `Plugin::tools(self: Arc<Self>)` — plugins contribute tools through this hook
  (sibling of `http_router`), so the core no longer downcasts to
  `MobileConnectorPlugin`.
- `tools::restart::set_restart_handler` — the desktop shell installs its
  teardown-and-respawn; the core defaults to the supervisor exit code. The core
  loses its `desktop` feature.
- `boot`'s stdout formatter moves to the binary (`src/boot_format.rs`); the core
  only emits tracing events.

All 79 core tests pass; the binary boots and serves in a clean directory, and
the mobile-connector tools still register through the new hook.
2026-07-10 16:48:51 +01:00