Compare commits

5 Commits
Author SHA1 Message Date
Daniele 7e3fa3caad fix(honcho): read Honcho 3.0.x response schema — memory reads were silently empty
Nightly Build / build (push) Successful in 3m48s
Against a self-hosted Honcho 3.0.11 every read path came back empty while
the server was healthy and full of derived facts: the plugin parsed a
`conclusions`/`summary` shape the API no longer emits.

- honcho-client: typed models for the real schema — `PeerContext`
  (`representation` markdown + `peer_card`), `SessionContext` (`summary` as
  an object, `peer_representation`), wrapped `PeerCard` (a bare array on PUT
  is a 422, which also broke `honcho_profile` writes).
- plugin: the turn-time injection and `honcho_context` read the
  representation; `honcho_search` and the page's /search now use
  `conclusions/query` (observer/observed scoping inside `filters`) — a real
  ranked semantic search with fact ids, which `peer_context?search_query`
  never provided; /overview returns card + representation + conclusions.
- compose: pin the Honcho image by digest (3.0.11) — ghcr publishes no v3
  semver tags, and an untracked `:latest` pull is what drifted the schema.
- tests: fixture tests from payloads captured on the live server, plus an
  env-gated live smoke test (HONCHO_E2E_URL/_WS/_PEER, `cargo test
  -p honcho-client -- --ignored`) — run it before any future Honcho bump.
2026-09-09 19:15:11 +01:00
Daniele 9c24b02e42 Fixng default config
Nightly Build / build (push) Successful in 1m10s
2026-09-08 16:41:11 +01:00
Daniele 027d815b66 feat(viewer): preview word documents (.docx/.doc/.odt/.rtf) as PDF
Nightly Build / build (push) Canceled after 10m54s
The file viewer converts word-processor documents to PDF server-side via
LibreOffice (skald_core::docx::DocxConverter), mirroring the LaTeX pipeline
but content-hash cached: the format is self-contained, so there is no
dependency graph and the file watcher needs no expansion. Container-only
documents are shuttled out and converted on the host. With no LibreOffice
installed the viewer says so and falls back to download-only. Downloads
still save the original document, not the preview PDF.
2026-09-08 16:30:13 +01:00
Daniele 4ea932ef54 feat(llm): Z.AI GLM-5.3 and GLM-5.3-Flash
Nightly Build / build (push) Successful in 10s
Both are added to the Z.AI static model list with their 1M-token context
and 128K max output. GLM-5.3-Flash is natively multimodal, so it gets the
vision and video capabilities — through an `override` rule, because the
provider's `defaults: { vision: false }` already set the flag and a fill
rule would have skipped it silently.

Neither model can stop thinking (`thinking.type` only accepts "enabled"),
so they get their own reasoning rule with low/high/max and no `disabled`,
placed before the `glm-5*` family rule that would otherwise swallow them.
2026-09-01 21:07:13 +01:00
Daniele 65e0f24326 fix(web): providers page reported a missing API key for every provider
Nightly Build / build (push) Successful in 4m35s
The card tested `p.api_key` on a DTO that has never carried it, so the badge
was falsy for every provider and always read "API key missing".

The list and the new detail DTO now expose `has_api_key: bool` — the key
value itself never reaches the browser, where the edit form used to prefill
it in plain text. Since the form can no longer send the stored key back, an
empty `api_key` on update means "keep the one on file" instead of erasing it,
which is what the field's placeholder already promised.
2026-09-01 21:01:45 +01:00
38 changed files with 1272 additions and 176 deletions
+44
View File
@@ -8,6 +8,50 @@ release PR may merge — and a section is closed at the commit that bumps it.
## [Unreleased]
### Added
- The file viewer opens **word-processor documents** (`.docx`, `.doc`, `.odt`, `.rtf`):
when LibreOffice is installed on the server they are converted to PDF and shown as the
document, live-reloading when the file changes, exactly like a compiled `.tex`. A
document kept only inside the user's container is converted too — the server pulls a
copy out and converts that. With no LibreOffice the page says so and offers the
download, as before. The download button still saves the original document, not the
preview PDF.
- Z.AI's new models are selectable on the **Models** page: **GLM-5.3** and **GLM-5.3-Flash**,
both with a 1M-token context. GLM-5.3-Flash is natively multimodal, so images and videos
attached to a message are sent to it directly instead of as a file path. Both always think —
Z.AI does not allow turning it off — and the reasoning control offers *low / high / max*
(default *max*) instead of an on/off switch.
### Fixed
- **Honcho long-term memory read nothing at all** against a self-hosted Honcho v3
server: the `honcho_context` and `honcho_search` tools answered "no context", the
automatic memory injection into chats silently never happened, and the *Long-term
memory* page showed an empty overview — while the server was healthy and full of
derived facts. The plugin parsed an outdated response shape; it now reads the
`representation` / `peer_card` fields Honcho 3.0.x actually returns, and search is a
real ranked semantic search whose fact ids can be deleted via `honcho_conclude`.
- The `honcho_profile` tool could not **write** the peer card (the API rejects a bare
array — it wants a `{"peer_card": …}` wrapper) and read back a raw JSON envelope;
writes now succeed and reads show the facts, or a clean "no card set yet".
- The **Providers** page said *API key missing* on every provider, including the ones
with a perfectly good key. It now reports the real state. Editing a provider no longer
shows the saved key in the form either — leave the field blank and the existing key is
kept, type a new one to replace it.
### Changed
- The self-hosted Honcho compose setup (`honcho/docker-compose.yml`) pins the server
image **by digest (3.0.11)** instead of tracking `:latest` — ghcr publishes no v3
version tags, and an untracked pull is what silently changed the API schema under the
plugin. Upgrading Honcho is now a deliberate, verified step.
### Security
- An LLM provider's API key is never sent to the browser any more: the provider list and
the edit form receive only whether a key is stored, not its value.
## [0.3.0] - 2026-08-24
### Added
Generated
+1
View File
@@ -1636,6 +1636,7 @@ version = "0.1.0"
dependencies = [
"anyhow",
"reqwest 0.13.4",
"rustls",
"serde",
"serde_json",
"tokio",
+3
View File
@@ -13,3 +13,6 @@ tracing = "0.1"
# The crate-level doc example (`#[tokio::main]`) compiles under `cargo test`.
tokio = { version = "1", features = ["macros", "rt"] }
anyhow = "1"
# The live smoke test builds a reqwest client without the host app around, so
# it must install the rustls crypto provider itself (the app does it in main).
rustls = { version = "0.23", features = ["ring"] }
+73
View File
@@ -171,3 +171,76 @@ fn urlencoding(s: &str) -> String {
}
out
}
#[cfg(test)]
mod tests {
//! Live smoke test against a real Honcho server — the regression net for
//! the schema drift that made every read return empty against 3.0.11.
//! Gated on env vars, run explicitly:
//!
//! ```sh
//! HONCHO_E2E_URL=http://host:8000 HONCHO_E2E_WS=<workspace> HONCHO_E2E_PEER=<peer> \
//! cargo test -p honcho-client -- --ignored --nocapture
//! ```
use super::HonchoClient;
use crate::models::*;
fn live() -> Option<(HonchoClient, String, String)> {
let url = std::env::var("HONCHO_E2E_URL").ok()?;
let ws = std::env::var("HONCHO_E2E_WS").ok()?;
let peer = std::env::var("HONCHO_E2E_PEER").ok()?;
Some((HonchoClient::with_base_url(url, ""), ws, peer))
}
#[tokio::test]
#[ignore = "needs a live Honcho: set HONCHO_E2E_URL/_WS/_PEER"]
async fn live_read_path_smoke() {
// Standalone test process: no host app installed a crypto provider.
let _ = rustls::crypto::ring::default_provider().install_default();
let Some((client, ws, peer)) = live() else {
eprintln!("env vars not set — skipping");
return;
};
// peer_context: the representation string is where the facts live.
let ctx = client
.peer_context(&ws, &peer, &PeerRepresentationGet::default())
.await
.expect("peer_context");
println!("representation: {:?}", ctx.representation.as_deref().map(|r| &r[..r.len().min(120)]));
println!("peer_card: {:?}", ctx.peer_card);
// card endpoint: wrapped, nullable.
let card = client.get_peer_card(&ws, &peer, None).await.expect("get_peer_card");
println!("card endpoint: {:?}", card.peer_card);
// semantic search over conclusions, scoped via filters.
let hits = client
.query_conclusions(&ws, &ConclusionQuery {
query: "test".into(),
top_k: Some(5),
distance: None,
filters: Some(serde_json::json!({ "observer_id": peer, "observed_id": peer })),
})
.await
.expect("query_conclusions");
println!("conclusions hits: {}", hits.len());
for c in &hits {
assert!(!c.id.is_empty() && !c.content.is_empty());
}
// conclusions list with the same scoping (what /overview shows).
let page = client
.list_conclusions(
&ws,
&PageParams { size: Some(50), ..Default::default() },
&ConclusionGet {
filters: Some(serde_json::json!({ "observer_id": peer, "observed_id": peer })),
},
)
.await
.expect("list_conclusions");
println!("conclusions total: {}", page.total);
}
}
+143 -1
View File
@@ -185,13 +185,16 @@ pub struct MessageUpdate {
// Conclusion
// ──────────────────────────────────────────
#[derive(Debug, Clone, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Conclusion {
pub id: String,
pub content: String,
pub observer_id: String,
pub observed_id: String,
pub session_id: Option<String>,
/// explicit | deductive | inductive | contradiction
#[serde(default)]
pub level: Option<String>,
pub created_at: String,
}
@@ -270,6 +273,66 @@ pub struct PeerRepresentationGet {
pub max_conclusions: Option<u32>,
}
// ──────────────────────────────────────────
// Context / card responses (Honcho v3.0.x — verified against 3.0.11)
//
// These endpoints do NOT return a `conclusions` array: the derived facts are
// delivered as a single pre-rendered markdown `representation` string, and the
// curated card is a plain list of fact strings wrapped in an object. Parse
// these typed shapes — reading `conclusions`/`summary` off them yields nothing.
// ──────────────────────────────────────────
/// `GET /workspaces/{ws}/peers/{peer}/context`
#[derive(Debug, Clone, Deserialize)]
pub struct PeerContext {
pub peer_id: String,
#[serde(default)]
pub target_id: Option<String>,
/// Curated subset of the target peer's representation, as seen by the
/// observer — a pre-rendered markdown document (e.g. an
/// `## Explicit Observations` section with one dated line per fact).
#[serde(default)]
pub representation: Option<String>,
#[serde(default)]
pub peer_card: Option<Vec<String>>,
}
/// `GET /workspaces/{ws}/sessions/{session}/context`
#[derive(Debug, Clone, Deserialize)]
pub struct SessionContext {
pub id: String,
#[serde(default)]
pub messages: Vec<Message>,
#[serde(default)]
pub summary: Option<Summary>,
/// Representation of the session's peer, when a perspective is available.
#[serde(default)]
pub peer_representation: Option<String>,
#[serde(default)]
pub peer_card: Option<Vec<String>>,
}
/// Nested in [`SessionContext`]. Only `content` is consumed; the remaining
/// fields (`message_id`, `summary_type`, `created_at`) are ignored.
#[derive(Debug, Clone, Deserialize)]
pub struct Summary {
pub content: String,
}
/// `GET /workspaces/{ws}/peers/{peer}/card` — note the wrapper object: the
/// card never travels as a bare array.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PeerCard {
pub peer_card: Option<Vec<String>>,
}
/// Body of `PUT /workspaces/{ws}/peers/{peer}/card` — same wrapper: sending a
/// bare array is a 422.
#[derive(Debug, Clone, Serialize)]
pub struct PeerCardSet {
pub peer_card: Vec<String>,
}
// ──────────────────────────────────────────
// Search
// ──────────────────────────────────────────
@@ -306,3 +369,82 @@ pub struct PageParams {
pub size: Option<u64>,
pub reverse: Option<bool>,
}
#[cfg(test)]
mod tests {
use super::*;
/// Real `GET /peers/{id}/context` payload captured from a self-hosted
/// Honcho 3.0.11. Guards the schema the plugin parses: facts live in the
/// `representation` markdown string, the card is `peer_card`, and there is
/// NO `conclusions` array (the bug this test would have caught).
const PEER_CONTEXT_3_0_11: &str = r###"{
"peer_id": "506cd15e-ae2a-47f1-9553-85ffe33e3e5b",
"target_id": "506cd15e-ae2a-47f1-9553-85ffe33e3e5b",
"representation": "## Explicit Observations\n\n[2026-09-09 16:55:39] Daniele ha una passione per i vulcani e ha dormito al bordo del cratere del Nyiragongo in RD Congo nel 2015.\n[2026-09-09 16:57:12] 506cd15e-ae2a-47f1-9553-85ffe33e3e5b speaks Italian (greeted with \"Ciao\")\n",
"peer_card": null
}"###;
#[test]
fn peer_context_parses_representation_and_card() {
let ctx: PeerContext = serde_json::from_str(PEER_CONTEXT_3_0_11).unwrap();
assert_eq!(ctx.peer_id, "506cd15e-ae2a-47f1-9553-85ffe33e3e5b");
let rep = ctx.representation.unwrap();
assert!(rep.contains("Nyiragongo"));
assert!(ctx.peer_card.is_none());
}
#[test]
fn peer_context_tolerates_empty_peer() {
let ctx: PeerContext = serde_json::from_str(
r#"{"peer_id":"p","target_id":"p","representation":null,"peer_card":null}"#,
)
.unwrap();
assert!(ctx.representation.is_none());
assert!(ctx.peer_card.is_none());
}
/// Real `GET /sessions/{id}/context` payload: `summary` is an OBJECT
/// (not a string) and the session-scoped facts are in
/// `peer_representation`.
#[test]
fn session_context_summary_is_an_object() {
let ctx: SessionContext = serde_json::from_str(
r###"{
"id": "skaldcircle-u-1",
"messages": [],
"summary": {"content": "They talked about commuting.", "message_id": "m1", "summary_type": "short", "created_at": "2026-09-09T16:00:00Z"},
"peer_representation": "## Explicit Observations\n\n[2026-09-09] fact",
"peer_card": ["likes volcanoes"]
}"###,
)
.unwrap();
assert_eq!(ctx.summary.unwrap().content, "They talked about commuting.");
assert!(ctx.peer_representation.unwrap().contains("fact"));
assert_eq!(ctx.peer_card.unwrap(), vec!["likes volcanoes".to_string()]);
}
/// Real `GET /peers/{id}/card` payload: the card is wrapped in an object
/// and null until curated.
#[test]
fn peer_card_is_wrapped_and_nullable() {
let card: PeerCard = serde_json::from_str(r#"{"peer_card":null}"#).unwrap();
assert!(card.peer_card.is_none());
let card: PeerCard =
serde_json::from_str(r#"{"peer_card":["a","b"]}"#).unwrap();
assert_eq!(card.peer_card.unwrap(), vec!["a".to_string(), "b".to_string()]);
}
/// Real conclusion entries (from `conclusions/query`), including `level`.
#[test]
fn conclusion_tolerates_level_and_null_session() {
let c: Conclusion = serde_json::from_str(
r#"{"id":"uM0BSpIuDMNp9wL97c6M-","content":"Daniele ha una passione per i vulcani","observer_id":"p","observed_id":"p","session_id":null,"level":"explicit","created_at":"2026-09-09T16:55:39.863412Z"}"#,
)
.unwrap();
assert_eq!(c.level.as_deref(), Some("explicit"));
assert!(c.session_id.is_none());
// Round-trip: the router serializes conclusions into its JSON responses.
assert!(serde_json::to_string(&c).unwrap().contains("uM0BSpIuDMNp9wL97c6M-"));
}
}
+10 -6
View File
@@ -92,13 +92,15 @@ impl HonchoClient {
// ── Context ───────────────────────────────────────────────────────────
/// Get context for a peer (conclusions + card, ready for injection into prompts).
/// Get context for a peer: the pre-rendered markdown `representation` of
/// everything derived about them, plus their curated card. (No
/// `conclusions` array exists in this response — see [`PeerContext`].)
pub async fn peer_context(
&self,
workspace_id: &str,
peer_id: &str,
opts: &PeerRepresentationGet,
) -> Result<serde_json::Value> {
) -> Result<PeerContext> {
let mut q: Vec<(&str, String)> = vec![];
if let Some(ref v) = opts.target {
q.push(("target", v.clone()));
@@ -132,7 +134,7 @@ impl HonchoClient {
workspace_id: &str,
peer_id: &str,
target: Option<&str>,
) -> Result<serde_json::Value> {
) -> Result<PeerCard> {
let mut q: Vec<(&str, String)> = vec![];
if let Some(t) = target {
q.push(("target", t.to_owned()));
@@ -144,13 +146,15 @@ impl HonchoClient {
.await
}
/// Overwrite the peer card. The API wraps the list in an object
/// (`{"peer_card": [...]}`) — a bare array is rejected with 422.
pub async fn set_peer_card(
&self,
workspace_id: &str,
peer_id: &str,
target: Option<&str>,
card: serde_json::Value,
) -> Result<serde_json::Value> {
card: Vec<String>,
) -> Result<PeerCard> {
let mut q: Vec<(&str, String)> = vec![];
if let Some(t) = target {
q.push(("target", t.to_owned()));
@@ -158,7 +162,7 @@ impl HonchoClient {
self.put_with_query(
&format!("/v3/workspaces/{workspace_id}/peers/{peer_id}/card"),
&q,
&card,
&PeerCardSet { peer_card: card },
)
.await
}
+5 -2
View File
@@ -161,14 +161,17 @@ impl HonchoClient {
// ── Context / Summaries ───────────────────────────────────────────────
/// Retrieve context for a session (messages + peer conclusions, token-budgeted).
/// Retrieve context for a session (messages + summary + the session peer's
/// representation, token-budgeted). The summary is a `Summary` object and
/// the facts live in `peer_representation` — there is no `conclusions`
/// array (see [`SessionContext`]).
pub async fn session_context(
&self,
workspace_id: &str,
session_id: &str,
tokens: Option<u32>,
search_query: Option<&str>,
) -> Result<serde_json::Value> {
) -> Result<SessionContext> {
let mut q: Vec<(&str, String)> = vec![];
if let Some(t) = tokens {
q.push(("tokens", t.to_string()));
+172 -60
View File
@@ -70,8 +70,8 @@ use core_api::tool::{
use core_api::user_plugin_config::PluginUserConfigApi;
use honcho_client::HonchoClient;
use honcho_client::models::{
ConclusionCreate, MessageCreate, PeerCreate, PeerRepresentationGet,
SessionCreate, SessionPeerConfig, WorkspaceCreate,
Conclusion, ConclusionCreate, ConclusionQuery, MessageCreate, PeerContext, PeerCreate,
PeerRepresentationGet, SessionContext, SessionCreate, SessionPeerConfig, WorkspaceCreate,
};
const PLUGIN_ID: &str = "honcho";
@@ -225,8 +225,8 @@ impl Memory for HonchoMemory {
},
).await {
Ok(ctx) => {
trace!(session_id, raw_json = %ctx, "honcho: peer_context raw response");
let f = format_context(ctx);
trace!(session_id, response = ?ctx, "honcho: peer_context raw response");
let f = format_peer_context(&ctx);
debug!(
"honcho: peer_context (global) for session {session_id} ({} chars)",
f.as_deref().map_or(0, |s| s.len())
@@ -253,8 +253,8 @@ impl Memory for HonchoMemory {
Some(user_message),
).await {
Ok(ctx) => {
trace!(session_id, raw_json = %ctx, "honcho: session_context raw response");
let f = format_context(ctx);
trace!(session_id, response = ?ctx, "honcho: session_context raw response");
let f = format_session_context(&ctx);
debug!(
"honcho: session_context for session {session_id} ({} chars)",
f.as_deref().map_or(0, |s| s.len())
@@ -474,19 +474,31 @@ impl Tool for HonchoProfileTool {
gated_execution(Arc::clone(&self.user_config), ctx.user_id.clone(), move |peer| async move {
match card_update {
Some(facts) => {
let facts: Vec<String> = facts
.iter()
.filter_map(|f| f.as_str().map(str::to_string))
.collect();
let n = facts.len();
client
.set_peer_card(&workspace_id, &peer, None, json!(facts))
.set_peer_card(&workspace_id, &peer, None, facts)
.await
.map_err(|e| anyhow::anyhow!("honcho_profile: {e}"))?;
Ok(format!("Peer card updated ({} facts).", facts.len()))
Ok(format!("Peer card updated ({n} facts)."))
}
None => {
let card = client
.get_peer_card(&workspace_id, &peer, None)
.await
.map_err(|e| anyhow::anyhow!("honcho_profile: {e}"))?;
Ok(serde_json::to_string_pretty(&card)
.unwrap_or_else(|_| card.to_string()))
// The response wraps the list: {"peer_card": [...] | null}.
match card.peer_card.filter(|c| !c.is_empty()) {
Some(facts) => Ok(format!(
"Peer card ({} facts):\n- {}",
facts.len(),
facts.join("\n- ")
)),
None => Ok("No peer card set yet.".to_string()),
}
}
}
})
@@ -532,56 +544,53 @@ impl Tool for HonchoSearchTool {
fn run_with<'a>(&'a self, ctx: &ToolContext, args: Value) -> Box<dyn ToolExecution + 'a> {
let client = Arc::clone(&self.client);
let workspace_id = self.workspace_id.clone();
// Honcho's `conclusions/query` endpoint requires observer/observed
// filters; the proven path (shared with the read-path) is `peer_context`
// with a `search_query`, which ranks the user's conclusions by relevance.
// `conclusions/query` is the semantic search over the user's derived
// facts: ranked results WITH their ids (needed by `honcho_conclude`'s
// delete). The observer/observed scoping goes inside `filters` — that
// is the whole trick, the endpoint is the right one. (`peer_context`
// with a search_query is not: it returns the whole representation
// once it fits the token budget, and no ids.)
gated_execution(Arc::clone(&self.user_config), ctx.user_id.clone(), move |peer| async move {
let query = args["query"]
.as_str()
.ok_or_else(|| anyhow::anyhow!("honcho_search: missing 'query' argument"))?
.to_string();
let ctx = client
.peer_context(
let conclusions = client
.query_conclusions(
&workspace_id,
&peer,
&PeerRepresentationGet {
search_query: Some(query),
search_top_k: Some(10),
..Default::default()
&ConclusionQuery {
query,
top_k: Some(10),
distance: None,
filters: Some(json!({ "observer_id": peer, "observed_id": peer })),
},
)
.await
.map_err(|e| anyhow::anyhow!("honcho_search: {e}"))?;
Ok(format_conclusions(&ctx)
Ok(format_conclusions(&conclusions)
.unwrap_or_else(|| "No relevant context found.".to_string()))
})
}
}
/// Formats the `conclusions` array of a Honcho `peer_context` response as a
/// ranked bullet list, prefixing each fact with its `id` when present so the
/// model can target it via `honcho_conclude`. Returns `None` when empty.
fn format_conclusions(ctx: &Value) -> Option<String> {
let conclusions = ctx.get("conclusions")?.as_array()?;
/// Formats a list of conclusions as a ranked bullet list, prefixing each fact
/// with its `id` so the model can target it via `honcho_conclude`. Returns
/// `None` when empty.
fn format_conclusions(conclusions: &[Conclusion]) -> Option<String> {
let lines: Vec<String> = conclusions
.iter()
.filter_map(|c| {
let content = c.get("content").and_then(|v| v.as_str())?;
match c.get("id").and_then(|v| v.as_str()) {
Some(id) => Some(format!("- [{id}] {content}")),
None => Some(format!("- {content}")),
}
})
.map(|c| format!("- [{}] {}", c.id, c.content))
.collect();
(!lines.is_empty()).then(|| lines.join("\n"))
}
// ── HonchoContextTool ─────────────────────────────────────────────────────────
/// Retrieves a full context snapshot for the calling user (conclusions, card,
/// summary) from Honcho's `peer_context` endpoint. No LLM synthesis.
/// Retrieves a full context snapshot for the calling user (the markdown
/// representation of everything derived about them, plus their peer card)
/// from Honcho's `peer_context` endpoint. No LLM synthesis.
struct HonchoContextTool {
client: Arc<HonchoClient>,
workspace_id: String,
@@ -626,7 +635,7 @@ impl Tool for HonchoContextTool {
.await
.map_err(|e| anyhow::anyhow!("honcho_context: {e}"))?;
Ok(format_context(ctx).unwrap_or_else(|| "No context available yet.".to_string()))
Ok(format_peer_context(&ctx).unwrap_or_else(|| "No context available yet.".to_string()))
})
}
}
@@ -713,36 +722,51 @@ impl Tool for HonchoConcludeTool {
}
}
/// Extracts a human-readable string from the raw Honcho `session_context` /
/// `peer_context` JSON response.
/// Formats a Honcho `peer_context` response for injection into the system
/// prompt / as the `honcho_context` tool result.
///
/// Returns `None` if there is nothing *new* to inject — i.e. when the response
/// contains only raw messages (which are already present in the LLM's own
/// conversation history) or is otherwise empty.
/// Honcho 3.0.x delivers everything a peer knows as a single pre-rendered
/// markdown `representation` string (sections like `## Explicit Observations`
/// with one dated line per fact), plus the curated `peer_card`. There is no
/// `conclusions` array and no `summary` string in this response — parsing
/// those keys silently yields nothing (the bug that made every read come back
/// empty against a healthy server).
///
/// Only synthesised knowledge is injected:
/// - `conclusions` — facts about the user derived by Honcho's background processing
/// - `summary` — a narrative summary produced by Honcho
///
/// Raw `messages` are intentionally ignored: they are redundant with the local
/// `chat_history` already sent to the LLM and would waste context tokens.
fn format_context(ctx: Value) -> Option<String> {
/// Raw `messages` are not part of this response at all; the representation is
/// already the synthesised knowledge, so it is injected verbatim.
fn format_peer_context(ctx: &PeerContext) -> Option<String> {
let mut parts: Vec<String> = Vec::new();
if let Some(conclusions) = ctx.get("conclusions").and_then(|v| v.as_array()) {
let facts: Vec<&str> = conclusions
.iter()
.filter_map(|c| c.get("content").and_then(|v| v.as_str()))
.collect();
if !facts.is_empty() {
parts.push(format!("Known facts about the user:\n- {}", facts.join("\n- ")));
}
if let Some(card) = ctx.peer_card.as_ref().filter(|c| !c.is_empty()) {
parts.push(format!("Peer card (curated key facts):\n- {}", card.join("\n- ")));
}
if let Some(summary) = ctx.get("summary").and_then(|v| v.as_str()) {
if !summary.trim().is_empty() {
parts.push(format!("Conversation summary:\n{summary}"));
}
if let Some(rep) = ctx.representation.as_deref().map(str::trim).filter(|r| !r.is_empty()) {
parts.push(format!("Known facts about the user:\n{rep}"));
}
if parts.is_empty() {
return None;
}
Some(format!(
"--- Honcho memory context ---\n{}\n--- end of memory context ---",
parts.join("\n\n")
))
}
/// Formats a Honcho `session_context` response: the running `summary` (an
/// object — only its `content` is used) plus the session-scoped
/// `peer_representation`. Returns `None` when neither is present.
fn format_session_context(ctx: &SessionContext) -> Option<String> {
let mut parts: Vec<String> = Vec::new();
if let Some(summary) = ctx.summary.as_ref().map(|s| s.content.trim()).filter(|s| !s.is_empty()) {
parts.push(format!("Conversation summary:\n{summary}"));
}
if let Some(rep) = ctx.peer_representation.as_deref().map(str::trim).filter(|r| !r.is_empty()) {
parts.push(format!("Session observations:\n{rep}"));
}
if parts.is_empty() {
@@ -1147,3 +1171,91 @@ async fn get_or_create_session(
let mut map = session_map.write().await;
Ok(map.entry(key).or_insert(session.id).clone())
}
#[cfg(test)]
mod tests {
use super::*;
/// Real `peer_context` payload shape captured from a self-hosted Honcho
/// 3.0.11: the derived facts live in the `representation` markdown string
/// and the card is null. The pre-fix code looked for `conclusions`/
/// `summary` keys and reported "No context available yet" against exactly
/// this response.
fn real_peer_context() -> PeerContext {
serde_json::from_value(json!({
"peer_id": "506cd15e-ae2a-47f1-9553-85ffe33e3e5b",
"target_id": "506cd15e-ae2a-47f1-9553-85ffe33e3e5b",
"representation": "## Explicit Observations\n\n[2026-09-09 16:20:06] 506cd15e works in an office at Battersea Power Station in London, Zone 1.\n[2026-09-09 16:55:39] Daniele ha una passione per i vulcani e ha dormito al bordo del cratere del Nyiragongo in RD Congo nel 2015.\n",
"peer_card": null
}))
.unwrap()
}
#[test]
fn peer_context_formats_the_representation() {
let out = format_peer_context(&real_peer_context()).unwrap();
assert!(out.contains("Nyiragongo"));
assert!(out.contains("Battersea"));
assert!(out.contains("--- Honcho memory context ---"));
assert!(!out.contains("Peer card"));
}
#[test]
fn peer_context_includes_the_card_when_present() {
let mut ctx = real_peer_context();
ctx.peer_card = Some(vec!["Software engineer".to_string()]);
let out = format_peer_context(&ctx).unwrap();
assert!(out.contains("Peer card (curated key facts):\n- Software engineer"));
}
#[test]
fn peer_context_empty_means_none() {
let ctx: PeerContext = serde_json::from_value(json!({
"peer_id": "p", "target_id": "p", "representation": null, "peer_card": null
}))
.unwrap();
assert!(format_peer_context(&ctx).is_none());
let ctx: PeerContext = serde_json::from_value(json!({
"peer_id": "p", "target_id": "p", "representation": " \n ", "peer_card": []
}))
.unwrap();
assert!(format_peer_context(&ctx).is_none());
}
#[test]
fn session_context_reads_summary_object_and_representation() {
// Real 3.0.11 shape: `summary` is an OBJECT, not a string.
let ctx: SessionContext = serde_json::from_value(json!({
"id": "ws-user-1",
"messages": [],
"summary": {"content": "They planned a commute comparison.", "message_id": "m", "summary_type": "short", "created_at": "2026-09-09T16:00:00Z"},
"peer_representation": "## Explicit Observations\n\n[2026-09-09] fact",
"peer_card": null
}))
.unwrap();
let out = format_session_context(&ctx).unwrap();
assert!(out.contains("Conversation summary:\nThey planned a commute comparison."));
assert!(out.contains("Session observations:\n## Explicit Observations"));
}
#[test]
fn session_context_empty_means_none() {
let ctx: SessionContext = serde_json::from_value(json!({
"id": "s", "messages": [], "summary": null, "peer_representation": null, "peer_card": null
}))
.unwrap();
assert!(format_session_context(&ctx).is_none());
}
#[test]
fn conclusions_format_with_ids_for_deletion() {
let conclusions: Vec<Conclusion> = serde_json::from_value(json!([
{"id":"abc","content":"likes volcanoes","observer_id":"p","observed_id":"p","session_id":null,"level":"explicit","created_at":"2026-09-09T16:55:39Z"}
]))
.unwrap();
let out = format_conclusions(&conclusions).unwrap();
assert_eq!(out, "- [abc] likes volcanoes");
assert!(format_conclusions(&[]).is_none());
}
}
+46 -31
View File
@@ -8,7 +8,7 @@
//! - `GET /status` — user-facing service health (reachability + the
//! caller's own processing queue).
//! - `GET /overview` — the caller's full memory snapshot (peer card +
//! conclusions + summary). Cheap GETs, no LLM.
//! representation digest + derived conclusions with ids). Cheap, no LLM.
//! - `POST /search` — semantic search over the caller's derived facts.
//! - `POST /ask` — Dialectic: Honcho's server-side LLM answers a
//! natural-language question from the caller's memory.
@@ -63,7 +63,10 @@ use core_api::user_channel::UserChannelApi;
use core_api::user_plugin_config::PluginUserConfigApi;
use honcho_client::HonchoClient;
use honcho_client::error::HonchoError;
use honcho_client::models::{DialecticOptions, PageParams, PeerRepresentationGet, WorkspaceGet};
use honcho_client::models::{
ConclusionGet, ConclusionQuery, DialecticOptions, PageParams, PeerRepresentationGet,
WorkspaceGet,
};
// Namespaced i18n keys for the router's user-facing strings (backend tables in
// `../i18n/*.json`), resolved to the caller's language via `web.i18n`.
@@ -334,8 +337,14 @@ async fn user_status(
// ── GET /overview ─────────────────────────────────────────────────────────────
/// The caller's full memory snapshot — the direct answer to "what does Honcho
/// know about me?": peer card (curated key facts) + conclusions (derived facts,
/// with their ids) + summary. Two cheap GETs, no LLM synthesis.
/// know about me?": the peer card (curated key facts), the pre-rendered
/// markdown `representation` (what the assistant actually receives), and the
/// individual conclusions with their ids (for targeted deletion). Cheap, no
/// LLM synthesis.
///
/// Schema note (Honcho 3.0.x): `peer_context` carries `representation` +
/// `peer_card`, NOT a `conclusions` array — the ids come from a separate
/// `conclusions/list` scoped to the caller's peer.
async fn user_overview(
State(cell): State<WebCell>,
Extension(caller): Extension<Caller>,
@@ -349,10 +358,12 @@ async fn user_overview(
Err(r) => return r,
};
// A 404 on either call is "no peer/card yet", not a failure — the peer is
// A 404 on the context call is "no peer yet", not a failure — the peer is
// created lazily by the write path on the user's first forwarded turn.
// The card response wraps the list: `{"peer_card": [...] | null}` — unwrap
// it so the page receives the bare value.
let card = match web.client.get_peer_card(&web.workspace_id, &peer, None).await {
Ok(v) => v,
Ok(c) => json!(c.peer_card),
Err(e @ HonchoError::Http { status: 404, .. }) => {
debug!("honcho /overview: no card for peer '{peer}' yet: {e}");
Value::Null
@@ -360,7 +371,7 @@ async fn user_overview(
Err(e) => return honcho_error(&web, &caller, &e).await,
};
let ctx = match web.client.peer_context(
let representation = match web.client.peer_context(
&web.workspace_id,
&peer,
&PeerRepresentationGet {
@@ -368,27 +379,39 @@ async fn user_overview(
..Default::default()
},
).await {
Ok(v) => v,
Ok(ctx) => ctx.representation,
Err(e @ HonchoError::Http { status: 404, .. }) => {
debug!("honcho /overview: no context for peer '{peer}' yet: {e}");
Value::Null
None
}
Err(e) => return honcho_error(&web, &caller, &e).await,
};
let conclusions = match web.client.list_conclusions(
&web.workspace_id,
&PageParams { size: Some(OVERVIEW_MAX_CONCLUSIONS as u64), ..Default::default() },
&ConclusionGet {
filters: Some(json!({ "observer_id": peer, "observed_id": peer })),
},
).await {
Ok(page) => page.items,
Err(e) => return honcho_error(&web, &caller, &e).await,
};
Json(json!({
"card": card,
"conclusions": ctx.get("conclusions").cloned().unwrap_or(Value::Array(vec![])),
"summary": ctx.get("summary").and_then(Value::as_str),
"card": card,
"representation": representation,
"conclusions": conclusions,
})).into_response()
}
// ── POST /search ──────────────────────────────────────────────────────────────
/// Semantic search over the caller's derived facts: `peer_context` with a
/// `search_query`, ranked raw excerpts with their ids — no LLM synthesis. The
/// same proven path as the `honcho_search` tool (the direct
/// `conclusions/query` endpoint needs observer/observed filters and is not it).
/// Semantic search over the caller's derived facts: `conclusions/query`,
/// ranked raw excerpts with their ids — no LLM synthesis. The same path as the
/// `honcho_search` tool; the observer/observed scoping lives in `filters`
/// (and `peer_context` would not do: it returns the whole representation with
/// no ids once it fits the budget).
async fn user_search(
State(cell): State<WebCell>,
Extension(caller): Extension<Caller>,
@@ -407,24 +430,16 @@ async fn user_search(
Err(r) => return r,
};
match web.client.peer_context(
match web.client.query_conclusions(
&web.workspace_id,
&peer,
&PeerRepresentationGet {
search_query: Some(query),
search_top_k: Some(SEARCH_TOP_K),
..Default::default()
&ConclusionQuery {
query,
top_k: Some(SEARCH_TOP_K),
distance: None,
filters: Some(json!({ "observer_id": peer, "observed_id": peer })),
},
).await {
Ok(ctx) => Json(json!({
"conclusions": ctx.get("conclusions").cloned().unwrap_or(Value::Array(vec![])),
})).into_response(),
// No peer in Honcho yet ⇒ nothing was ever derived: distinguish it from
// "nothing matches" so the debug page can say which of the two it is.
Err(e @ HonchoError::Http { status: 404, .. }) => {
debug!("honcho /search: no context for peer '{peer}' yet: {e}");
Json(json!({ "conclusions": [], "empty": true })).into_response()
}
Ok(conclusions) => Json(json!({ "conclusions": conclusions })).into_response(),
Err(e) => honcho_error(&web, &caller, &e).await,
}
}
+6 -6
View File
@@ -43,7 +43,7 @@ export default {
// "What does it remember?" panel (shown once opted in)
[`${P}.panel.title`]: 'What Honcho remembers about you',
[`${P}.panel.guide_title`]: 'How to use this page',
[`${P}.panel.guide_overview`]: 'Overview shows everything Honcho has derived about you so far: your card (key facts), the facts it concluded, and a summary. Nothing to type.',
[`${P}.panel.guide_overview`]: 'Overview shows everything Honcho has derived about you so far: your card (key facts), the individual facts, and the full digest it hands to the assistant. Nothing to type.',
[`${P}.panel.guide_search`]: 'Search finds the stored facts most relevant to the words you type. Fast and exact — no AI rewrite, you see the raw facts.',
[`${P}.panel.guide_ask`]: 'Ask sends your question to Honchos AI, which reads your memory and writes an answer in its own words. Slower, but it can connect the dots.',
[`${P}.panel.status_title`]: 'Service status',
@@ -54,7 +54,7 @@ export default {
[`${P}.panel.overview_title`]: 'Overview',
[`${P}.panel.card_title`]: 'Your card',
[`${P}.panel.facts_title`]: 'Facts',
[`${P}.panel.summary_title`]: 'Summary',
[`${P}.panel.representation_title`]: 'Representation (what the assistant receives)',
[`${P}.panel.no_memory`]: 'Honcho has no memory about you yet — it builds up as you chat.',
[`${P}.panel.query_title`]: 'Search or ask',
[`${P}.panel.query_hint`]: 'Words to find facts, or a full question for the AI.',
@@ -98,7 +98,7 @@ export default {
// Pannello "cosa ricorda di te?" (visibile dopo il consenso)
[`${P}.panel.title`]: 'Cosa ricorda Honcho di te',
[`${P}.panel.guide_title`]: 'Come usare questa pagina',
[`${P}.panel.guide_overview`]: 'La panoramica mostra tutto ciò che Honcho ha ricavato su di te finora: la tua scheda (fatti chiave), i fatti dedotti e un riassunto. Non serve scrivere nulla.',
[`${P}.panel.guide_overview`]: 'La panoramica mostra tutto ciò che Honcho ha ricavato su di te finora: la tua scheda (fatti chiave), i singoli fatti e il riassunto completo che consegna allassistente. Non serve scrivere nulla.',
[`${P}.panel.guide_search`]: 'Cerca trova i fatti memorizzati più rilevanti per le parole che scrivi. Veloce ed esatto — niente riscritture dellAI, vedi i fatti grezzi.',
[`${P}.panel.guide_ask`]: 'Chiedi invia la tua domanda allAI di Honcho, che legge la tua memoria e scrive una risposta con parole sue. Più lento, ma sa collegare i puntini.',
[`${P}.panel.status_title`]: 'Stato del servizio',
@@ -109,7 +109,7 @@ export default {
[`${P}.panel.overview_title`]: 'Panoramica',
[`${P}.panel.card_title`]: 'La tua scheda',
[`${P}.panel.facts_title`]: 'Fatti',
[`${P}.panel.summary_title`]: 'Riassunto',
[`${P}.panel.representation_title`]: 'Rappresentazione (quella che riceve lassistente)',
[`${P}.panel.no_memory`]: 'Honcho non ha ancora nessun ricordo di te — si costruisce chiacchierando.',
[`${P}.panel.query_title`]: 'Cerca o chiedi',
[`${P}.panel.query_hint`]: 'Parole per trovare fatti, oppure una domanda completa per lAI.',
@@ -153,7 +153,7 @@ export default {
// Panneau « que retient-il de vous ? » (visible après le consentement)
[`${P}.panel.title`]: 'Ce que Honcho retient de vous',
[`${P}.panel.guide_title`]: 'Comment utiliser cette page',
[`${P}.panel.guide_overview`]: 'Laperçu montre tout ce que Honcho a déduit de vous jusquici : votre fiche (faits clés), les faits conclus et un résumé. Rien à saisir.',
[`${P}.panel.guide_overview`]: 'Laperçu montre tout ce que Honcho a déduit de vous jusquici : votre fiche (faits clés), les faits individuels et la synthèse complète remise à lassistant. Rien à saisir.',
[`${P}.panel.guide_search`]: 'Rechercher trouve les faits stockés les plus pertinents pour les mots saisis. Rapide et exact — pas de réécriture par lIA, vous voyez les faits bruts.',
[`${P}.panel.guide_ask`]: 'Demander envoie votre question à lIA de Honcho, qui lit votre mémoire et rédige une réponse avec ses mots. Plus lent, mais elle relie les points.',
[`${P}.panel.status_title`]: 'État du service',
@@ -164,7 +164,7 @@ export default {
[`${P}.panel.overview_title`]: 'Aperçu',
[`${P}.panel.card_title`]: 'Votre fiche',
[`${P}.panel.facts_title`]: 'Faits',
[`${P}.panel.summary_title`]: 'Résumé',
[`${P}.panel.representation_title`]: 'Représentation (celle que reçoit lassistant)',
[`${P}.panel.no_memory`]: 'Honcho na pas encore de mémoire vous concernant — elle se construit au fil des conversations.',
[`${P}.panel.query_title`]: 'Rechercher ou demander',
[`${P}.panel.query_hint`]: 'Des mots pour trouver des faits, ou une question complète pour lIA.',
+20 -33
View File
@@ -9,11 +9,11 @@
// 2. Once opted in (saved flag, not the draft toggle): the "what does Honcho
// remember about me?" debug panel, backed by this plugin's own opt-in-gated
// endpoints — `GET ${api}/status` (service health + the caller's own
// processing queue), `GET ${api}/overview` (card + facts + summary, no
// input), and one text field with two actions: `POST ${api}/search` (raw
// ranked facts) and `POST ${api}/ask` (Honcho's server-side LLM answers).
// The built-in mini-guide explains the difference, because "words → facts"
// vs "question → AI answer" is not obvious.
// processing queue), `GET ${api}/overview` (card + facts + representation,
// no input), and one text field with two actions: `POST ${api}/search`
// (raw ranked facts) and `POST ${api}/ask` (Honcho's server-side LLM
// answers). The built-in mini-guide explains the difference, because
// "words → facts" vs "question → AI answer" is not obvious.
//
// Errors from these endpoints arrive already localized *and specific* (the
// backend forwards the real Honcho transport/HTTP detail) — they are surfaced
@@ -37,12 +37,12 @@ export default class HonchoMemoryPage extends HonchoBase {
// Debug panel (only used once the *saved* opt-in flag is on).
_svc: { state: true }, // null | { ok, latency_ms?, queue? } | { ok:false, error }
_svcBusy: { state: true },
_ov: { state: true }, // null | { card, conclusions, summary }
_ov: { state: true }, // null | { card, representation, conclusions }
_ovBusy: { state: true },
_ovErr: { state: true }, // string | null
_q: { state: true }, // query input value
_qBusy: { state: true }, // null | 'search' | 'ask'
_qRes: { state: true }, // null | { kind:'search', conclusions, empty } | { kind:'ask', answer }
_qRes: { state: true }, // null | { kind:'search', conclusions } | { kind:'ask', answer }
_qErr: { state: true }, // string | null
};
}
@@ -145,7 +145,7 @@ export default class HonchoMemoryPage extends HonchoBase {
body: JSON.stringify({ query: q }),
});
this._qRes = kind === 'search'
? { kind, conclusions: r?.conclusions ?? [], empty: !!r?.empty }
? { kind, conclusions: r?.conclusions ?? [] }
: { kind, answer: r?.answer ?? '' };
} catch (e) {
this._qErr = e.message;
@@ -276,19 +276,10 @@ export default class HonchoMemoryPage extends HonchoBase {
<div class="mt-2">${body}</div>`;
}
// Normalize the peer card into renderable pieces: an array (or an object with
// an array under a known key) becomes items; anything else is shown as JSON.
// The backend already unwraps Honcho's `{"peer_card": …}` envelope: the card
// arrives as a bare array of fact strings, or null when none was curated.
_cardItems(card) {
if (card == null) return null;
if (Array.isArray(card)) return card.length ? card : null;
if (typeof card === 'object') {
for (const k of ['card', 'facts', 'items']) {
if (Array.isArray(card[k]) && card[k].length) return card[k];
}
return { raw: JSON.stringify(card, null, 2) };
}
if (typeof card === 'string') return card.trim() ? [card] : null;
return null;
return Array.isArray(card) && card.length ? card : null;
}
_renderOverview() {
@@ -305,24 +296,22 @@ export default class HonchoMemoryPage extends HonchoBase {
} else if (this._ov) {
const conclusions = this._ov.conclusions ?? [];
const card = this._cardItems(this._ov.card);
const summary = (this._ov.summary ?? '').trim();
if (!card && !conclusions.length && !summary) {
const representation = (this._ov.representation ?? '').trim();
if (!card && !conclusions.length && !representation) {
body = html`<div class="text-body-secondary" style="font-size:.8rem">${t(`${P}.panel.no_memory`)}</div>`;
} else {
body = html`
${card ? html`
<div style="font-size:.75rem; font-weight:600" class="text-body-secondary">${t(`${P}.panel.card_title`)}</div>
${Array.isArray(card)
? html`<ul class="mb-2" style="font-size:.82rem">${card.map((c, i) => html`<li key=${i}>${typeof c === 'string' ? c : JSON.stringify(c)}</li>`)}</ul>`
: html`<pre class="mb-2" style="font-size:.72rem; white-space:pre-wrap">${card.raw}</pre>`}
<ul class="mb-2" style="font-size:.82rem">${card.map((c, i) => html`<li key=${i}>${c}</li>`)}</ul>
` : nothing}
${conclusions.length ? html`
<div style="font-size:.75rem; font-weight:600" class="text-body-secondary">${t(`${P}.panel.facts_title`)}</div>
<ul class="mb-2" style="font-size:.82rem">${conclusions.map(this._factLi)}</ul>
` : nothing}
${summary ? html`
<div style="font-size:.75rem; font-weight:600" class="text-body-secondary">${t(`${P}.panel.summary_title`)}</div>
<div style="font-size:.82rem; white-space:pre-wrap">${summary}</div>
${representation ? html`
<div style="font-size:.75rem; font-weight:600" class="text-body-secondary">${t(`${P}.panel.representation_title`)}</div>
<div style="font-size:.82rem; white-space:pre-wrap">${representation}</div>
` : nothing}`;
}
} else {
@@ -347,11 +336,9 @@ export default class HonchoMemoryPage extends HonchoBase {
if (this._qErr) {
result = html`<div class="alert alert-danger py-2" style="font-size:.8rem">${this._qErr}</div>`;
} else if (this._qRes?.kind === 'search') {
result = this._qRes.empty
? html`<div class="alert alert-info py-2" style="font-size:.8rem">${t(`${P}.panel.no_memory`)}</div>`
: this._qRes.conclusions.length
? html`<ul style="font-size:.82rem">${this._qRes.conclusions.map(this._factLi)}</ul>`
: html`<div class="text-body-secondary" style="font-size:.8rem">${t(`${P}.panel.search_empty`)}</div>`;
result = this._qRes.conclusions.length
? html`<ul style="font-size:.82rem">${this._qRes.conclusions.map(this._factLi)}</ul>`
: html`<div class="text-body-secondary" style="font-size:.8rem">${t(`${P}.panel.search_empty`)}</div>`;
} else if (this._qRes?.kind === 'ask') {
result = html`
<div style="font-size:.75rem; font-weight:600" class="text-body-secondary">${t(`${P}.panel.answer_title`)}</div>
+433
View File
@@ -0,0 +1,433 @@
//! `DocxConverter` — converts word-processor documents (`.docx`, `.doc`,
//! `.odt`, `.rtf`) to PDF using LibreOffice in headless mode
//! (`soffice --convert-to pdf`).
//!
//! Used by the file viewer (`GET /api/file?…&compile-docx=true`) to render
//! word documents as PDFs on demand — the word-family twin of
//! [`crate::latex::LatexCompiler`].
//!
//! ## Caching (content-addressed)
//!
//! Unlike a `.tex` source, a word document is **self-contained**: images,
//! styles and fonts travel inside the file itself, so there is no dependency
//! graph to track and the `.fls`-sidecar machinery of the LaTeX cache would
//! buy nothing. The cache key is a short SHA-256 of the document bytes: any
//! edit changes the hash and invalidates naturally, and two paths holding the
//! same document share one cached PDF.
//!
//! One artefact lives under `<tmp>/skald-docx/`:
//!
//! | Artefact | Key | Purpose |
//! |----------------------|----------------------------|-------------------|
//! | `<content-hash>.pdf` | SHA-256 of the file bytes | The converted PDF |
//!
//! ## Container-shuttled inputs
//!
//! [`DocxConverter::convert_bytes`] exists for documents that live **only
//! inside a user's container** (`/tmp/…`): the caller pulls the bytes out
//! (`container::exec_fs::read`) and the converter works on a host-side
//! scratch copy. This is correct precisely because the format is
//! self-contained — a bare copy loses nothing. (LaTeX deliberately does not
//! get this treatment: a shuttled `.tex` would silently lose its relative
//! `\input` / `\includegraphics` dependencies.)
//!
//! ## LibreOffice quirks this lives with
//!
//! - `soffice` locks its user-profile directory, so concurrent conversions —
//! or a stale lock left by a killed run — make later invocations fail.
//! Every conversion therefore gets a **private profile**
//! (`-env:UserInstallation`) inside its per-run scratch directory, which is
//! removed afterwards.
//! - A failed conversion does not always exit non-zero: a missing output
//! file is treated as a failure too, with the captured output as detail.
//! - The scratch copy's **name** is how soffice picks its import filter, so
//! the shuttled input keeps the caller's extension (`input.docx`,
//! `input.odt`, …).
//!
//! ## Failure modes
//! - `ToolMissing` — no LibreOffice on the host (neither `soffice` /
//! `libreoffice` on PATH nor the macOS app bundle).
//! - `Timeout` — conversion exceeded [`CONVERT_TIMEOUT_SECS`].
//! - `Failed { output }` — non-zero exit or missing output file; carries the
//! captured stdout/stderr so the viewer can surface it.
//! - `Io` — underlying I/O error (reading the source, writing the cache…).
use std::path::{Path, PathBuf};
use std::time::Duration;
use sha2::{Digest, Sha256};
use tokio::process::Command;
/// Hard ceiling for a single conversion. A cold `soffice` start with a fresh
/// profile takes a few seconds; large documents add a few more — 60 s leaves
/// generous headroom while still bounding a hung run.
const CONVERT_TIMEOUT_SECS: u64 = 60;
/// Subdirectory of the OS temp dir holding cached PDFs and per-run scratch
/// directories.
const CACHE_DIR_NAME: &str = "skald-docx";
/// The word-processor extensions this converter accepts — the single source
/// of truth the HTTP layer (`api/files.rs::is_word_doc`) shares, so the
/// query flag and the converter can never disagree on the family.
pub const WORD_EXTS: &[&str] = &["docx", "doc", "odt", "rtf"];
/// A successfully converted PDF.
pub struct ConvertedPdf {
pub bytes: Vec<u8>,
/// `true` when served from cache without invoking `soffice`. Informational
/// only; kept on the struct so the API stays stable (mirrors
/// `latex::CompiledPdf`).
#[allow(dead_code)]
pub from_cache: bool,
}
/// Why a conversion request did not yield a PDF.
#[derive(Debug)]
pub enum ConvertError {
/// No LibreOffice binary is reachable on the host.
ToolMissing,
/// `soffice` ran but failed (non-zero exit, or no output file). Carries
/// the captured process output.
Failed { output: String },
/// Conversion did not finish within [`CONVERT_TIMEOUT_SECS`].
Timeout,
/// Underlying I/O error (reading the source, writing the cache, etc.).
Io(std::io::Error),
}
impl std::fmt::Display for ConvertError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::ToolMissing => write!(f, "LibreOffice is not available on the server"),
Self::Failed { output } => write!(f, "conversion failed:\n{output}"),
Self::Timeout => write!(f, "conversion aborted (timeout {CONVERT_TIMEOUT_SECS}s)"),
Self::Io(e) => write!(f, "I/O error: {e}"),
}
}
}
impl std::error::Error for ConvertError {}
impl From<std::io::Error> for ConvertError {
fn from(e: std::io::Error) -> Self { Self::Io(e) }
}
/// Stateless-ish facade around `soffice`. Owns only the cache root path; safe
/// to share via `Arc` (constructed once and stored on `Skald`).
#[derive(Clone)]
pub struct DocxConverter {
cache_dir: PathBuf,
}
impl DocxConverter {
pub fn new() -> Self {
Self { cache_dir: std::env::temp_dir().join(CACHE_DIR_NAME) }
}
/// Convert a word document at `path` (a host file) to PDF, serving from
/// the content-addressed cache when possible.
pub async fn convert_path(&self, path: &Path) -> Result<ConvertedPdf, ConvertError> {
let bytes = tokio::fs::read(path).await?;
let key = content_hash(&bytes);
if let Some(hit) = self.cached(&key).await {
return Ok(hit);
}
let scratch = self.cache_dir.join(format!("run-{}", unique_suffix()));
let pdf_bytes = self.run_soffice(&scratch, path).await?;
self.store(&key, &pdf_bytes).await;
tracing::info!(file = ?path, "word document converted (cache miss)");
Ok(ConvertedPdf { bytes: pdf_bytes, from_cache: false })
}
/// Convert a word document that exists only as bytes — a file shuttled
/// out of a user's container (see the module docs). `ext` (the caller's
/// file extension) selects the import filter through the scratch copy's
/// file name.
pub async fn convert_bytes(&self, bytes: &[u8], ext: &str) -> Result<ConvertedPdf, ConvertError> {
let key = content_hash(bytes);
if let Some(hit) = self.cached(&key).await {
return Ok(hit);
}
// Probe before touching the disk: with no converter installed the
// request fails without leaving a scratch copy behind.
let soffice = find_soffice().await.ok_or(ConvertError::ToolMissing)?;
let scratch = self.cache_dir.join(format!("run-{}", unique_suffix()));
tokio::fs::create_dir_all(&scratch).await?;
let input = scratch.join(format!("input.{}", sanitize_ext(ext)));
if let Err(e) = tokio::fs::write(&input, bytes).await {
let _ = cleanup_dir(&scratch).await;
return Err(ConvertError::Io(e));
}
let pdf_bytes = self.run_soffice_with(&soffice, &scratch, &input).await?;
self.store(&key, &pdf_bytes).await;
tracing::info!(ext, "word document converted from shuttled bytes (cache miss)");
Ok(ConvertedPdf { bytes: pdf_bytes, from_cache: false })
}
/// Look up a cached PDF by content key.
async fn cached(&self, key: &str) -> Option<ConvertedPdf> {
let path = self.cache_dir.join(format!("{key}.pdf"));
match tokio::fs::read(&path).await {
Ok(bytes) => {
tracing::debug!(cached_pdf = ?path, "word-doc cache hit");
Some(ConvertedPdf { bytes, from_cache: true })
}
Err(_) => None,
}
}
/// Persist a converted PDF under its content key. A write failure is
/// non-fatal: the next request simply converts again.
async fn store(&self, key: &str, bytes: &[u8]) {
let path = self.cache_dir.join(format!("{key}.pdf"));
if let Err(e) = tokio::fs::write(&path, bytes).await {
tracing::warn!(cached_pdf = ?path, error = %e, "word-doc cache write failed");
}
}
/// [`run_soffice_with`] with the binary probed first. Used by the
/// host-path entry point, which has nothing to prepare.
async fn run_soffice(&self, scratch: &Path, input: &Path) -> Result<Vec<u8>, ConvertError> {
let soffice = find_soffice().await.ok_or(ConvertError::ToolMissing)?;
self.run_soffice_with(&soffice, scratch, input).await
}
/// Run one conversion of `input` with output to `scratch` (a per-run
/// unique directory, removed before returning regardless of outcome) and
/// return the produced PDF bytes.
///
/// `soffice` gets a **private user profile** inside the scratch dir:
/// the profile is locked while in use, so a shared one would make
/// concurrent conversions fail — and a stale lock from a killed run
/// would make every later one fail.
async fn run_soffice_with(
&self,
soffice: &Path,
scratch: &Path,
input: &Path,
) -> Result<Vec<u8>, ConvertError> {
tokio::fs::create_dir_all(scratch).await?;
let profile = scratch.join("profile");
let mut cmd = Command::new(soffice);
cmd.args(["--headless", "--norestore", "--nolockcheck", "--nologo"]);
// `profile` is always absolute (cache_dir lives under temp_dir), so
// `file://` + path yields a valid `file:///…` URL on unix hosts.
cmd.arg(format!("-env:UserInstallation=file://{}", profile.display()));
cmd.args(["--convert-to", "pdf", "--outdir"]);
cmd.arg(scratch);
cmd.arg(input);
cmd.stdout(std::process::Stdio::piped());
cmd.stderr(std::process::Stdio::piped());
// If our future is dropped (e.g. on shutdown) ensure the process dies.
cmd.kill_on_drop(true);
let output = match tokio::time::timeout(
Duration::from_secs(CONVERT_TIMEOUT_SECS),
cmd.output(),
).await {
Ok(Ok(o)) => o,
Ok(Err(e)) => {
let _ = cleanup_dir(scratch).await;
return Err(ConvertError::Io(e));
}
Err(_) => {
// Timeout: the future is dropped here; `kill_on_drop`
// terminates `soffice`.
let _ = cleanup_dir(scratch).await;
return Err(ConvertError::Timeout);
}
};
let stem = input
.file_stem()
.and_then(|s| s.to_str())
.unwrap_or("output")
.to_string();
let pdf_path = scratch.join(format!("{stem}.pdf"));
// soffice can exit 0 without producing anything (unreadable input,
// unknown filter): the output file is the real success signal.
let pdf_bytes = match tokio::fs::read(&pdf_path).await {
Ok(b) => b,
Err(_) if !output.status.success() => {
let _ = cleanup_dir(scratch).await;
return Err(ConvertError::Failed { output: process_output(&output) });
}
Err(e) => {
let _ = cleanup_dir(scratch).await;
return Err(ConvertError::Failed {
output: format!(
"soffice exited successfully but produced no PDF ({e})\n{}",
process_output(&output)
),
});
}
};
let _ = cleanup_dir(scratch).await;
Ok(pdf_bytes)
}
}
impl Default for DocxConverter {
fn default() -> Self { Self::new() }
}
// ── Helpers ─────────────────────────────────────────────────────────────────
//
// `content_hash` / `unique_suffix` / `find_on_path` / `cleanup_dir` mirror the
// private helpers of the same names in `latex/compiler.rs`. Kept as local
// copies so neither module reaches into the other; if a third converter ever
// appears, extraction into a shared module becomes the obvious move.
/// First 5 bytes (10 hex chars) of SHA-256 — enough to avoid collisions in
/// practice while keeping cache filenames short.
fn content_hash(bytes: &[u8]) -> String {
let mut hasher = Sha256::new();
hasher.update(bytes);
let digest = hasher.finalize();
digest.iter().take(5).map(|b| format!("{b:02x}")).collect()
}
/// Per-run unique suffix (PID + nanosecond timestamp) to namespace the
/// scratch directory and avoid races between concurrent conversions.
fn unique_suffix() -> String {
use std::time::{SystemTime, UNIX_EPOCH};
let pid = std::process::id();
let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0);
format!("{pid}-{nanos:x}")
}
/// Locate a LibreOffice binary: `soffice` / `libreoffice` on PATH, then the
/// standard macOS app-bundle location (an installed LibreOffice that was
/// never linked onto PATH).
async fn find_soffice() -> Option<PathBuf> {
for name in ["soffice", "libreoffice"] {
if let Some(p) = find_on_path(name).await {
return Some(p);
}
}
let app_bundle = PathBuf::from("/Applications/LibreOffice.app/Contents/MacOS/soffice");
if tokio::fs::metadata(&app_bundle).await.map(|m| m.is_file()).unwrap_or(false) {
return Some(app_bundle);
}
None
}
/// Return the absolute path of `bin` if it is found on `PATH` and is a regular
/// file. We avoid pulling in the `which` crate for a single lookup.
async fn find_on_path(bin: &str) -> Option<PathBuf> {
let path_var = std::env::var_os("PATH")?;
for dir in std::env::split_paths(&path_var) {
let candidate = dir.join(bin);
if tokio::fs::metadata(&candidate).await
.map(|m| m.is_file() || m.file_type().is_symlink())
.unwrap_or(false)
{
return Some(candidate);
}
}
None
}
/// The scratch copy's extension drives soffice's import-filter choice, so it
/// must survive the trip. Anything outside the known word family (or weird
/// bytes) becomes `docx` — which is also what content-sniffing would guess.
fn sanitize_ext(ext: &str) -> String {
let e = ext.to_ascii_lowercase();
if WORD_EXTS.contains(&e.as_str()) { e } else { "docx".to_string() }
}
/// Flatten a process's captured stdout+stderr into one displayable string,
/// capped so a noisy run cannot bloat the HTTP error body.
fn process_output(output: &std::process::Output) -> String {
let mut text = String::new();
text.push_str(&String::from_utf8_lossy(&output.stdout));
if !text.is_empty() && !text.ends_with('\n') {
text.push('\n');
}
text.push_str(&String::from_utf8_lossy(&output.stderr));
let text = text.trim();
if text.is_empty() {
return "(no output from soffice)".to_string();
}
text.chars().take(4000).collect()
}
/// Recursively remove a scratch directory. Errors are logged and swallowed:
/// leftover dirs only consume a little disk under the OS temp folder.
async fn cleanup_dir(dir: &Path) -> std::io::Result<()> {
if tokio::fs::try_exists(dir).await.unwrap_or(false) {
tokio::fs::remove_dir_all(dir).await?;
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn hash_is_10_lowercase_hex_chars() {
let h = content_hash(b"hello world");
assert_eq!(h.len(), 10);
assert!(h.chars().all(|c| c.is_ascii_hexdigit() && !c.is_ascii_uppercase()));
}
#[test]
fn hash_is_deterministic() {
assert_eq!(content_hash(b"abc"), content_hash(b"abc"));
assert_ne!(content_hash(b"abc"), content_hash(b"abd"));
}
#[test]
fn sanitize_ext_keeps_the_word_family() {
for ext in WORD_EXTS {
assert_eq!(&sanitize_ext(ext), ext);
}
assert_eq!(sanitize_ext("DOCX"), "docx");
}
#[test]
fn sanitize_ext_defaults_unknowns_to_docx() {
assert_eq!(sanitize_ext("pptx"), "docx");
assert_eq!(sanitize_ext("../../etc/passwd"), "docx");
assert_eq!(sanitize_ext(""), "docx");
}
#[tokio::test]
async fn cache_round_trip() {
let dir = std::env::temp_dir().join(format!("skald-docx-test-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let converter = DocxConverter { cache_dir: dir.clone() };
assert!(converter.cached("deadbeef00").await.is_none());
converter.store("deadbeef00", b"%PDF-fake").await;
let hit = converter.cached("deadbeef00").await.unwrap();
assert_eq!(hit.bytes, b"%PDF-fake");
assert!(hit.from_cache);
let _ = std::fs::remove_dir_all(&dir);
}
/// With no LibreOffice on the host the converter must report ToolMissing —
/// on a box *with* LibreOffice this test is skipped rather than failed,
/// since it would otherwise run a real conversion.
#[tokio::test]
async fn missing_tool_reports_tool_missing() {
if find_soffice().await.is_some() {
eprintln!("LibreOffice present — skipping ToolMissing test");
return;
}
let dir = std::env::temp_dir().join(format!("skald-docx-test-missing-{}", std::process::id()));
let converter = DocxConverter { cache_dir: dir };
let result = converter.convert_bytes(b"not a real docx", "docx").await;
assert!(matches!(result, Err(ConvertError::ToolMissing)));
}
}
+1
View File
@@ -22,6 +22,7 @@ pub mod crypto;
pub mod elicitation;
pub mod cron;
pub mod db;
pub mod docx;
pub mod events;
pub mod git_versions;
pub mod image_generate;
+1
View File
@@ -314,6 +314,7 @@ impl LlmManager {
provider: p.provider.clone(),
base_url: p.base_url.clone(),
description: p.description.clone(),
has_api_key: p.api_key.as_deref().is_some_and(|k| !k.trim().is_empty()),
supported_types,
}
}).collect()
+4 -1
View File
@@ -75,7 +75,8 @@ pub fn dtl_mode_from_format(fmt: &str) -> DtlMode {
// ── Provider ──────────────────────────────────────────────────────────────────
/// Public provider metadata (no api_key).
/// Public provider metadata. The api_key itself never leaves the server: the UI
/// only needs to know **whether** one is stored, so this carries a boolean.
#[derive(Debug, Clone, serde::Serialize)]
pub struct LlmProviderInfo {
pub id: i64,
@@ -84,6 +85,8 @@ pub struct LlmProviderInfo {
pub provider: String,
pub base_url: Option<String>,
pub description: Option<String>,
/// True when a non-empty api_key is stored for this provider.
pub has_api_key: bool,
/// Service types this provider supports (from ProviderRegistry at runtime).
pub supported_types: Vec<ServiceType>,
}
+2
View File
@@ -29,6 +29,7 @@ use crate::elicitation::ElicitationManager;
use crate::image_generate::ImageGeneratorManager;
use crate::inbox::Inbox;
use crate::git_versions::GitVersions;
use crate::docx::DocxConverter;
use crate::latex::LatexCompiler;
use crate::llm::LlmManager;
use crate::location::LocationManager;
@@ -406,6 +407,7 @@ impl Skald {
// Infra
pub fn latex_compiler(&self) -> &LatexCompiler { &self.infra.latex_compiler }
pub fn docx_converter(&self) -> &DocxConverter { &self.infra.docx_converter }
pub fn git_versions(&self) -> &GitVersions { &self.infra.git_versions }
pub fn location_manager(&self) -> &Arc<LocationManager> { &self.infra.location_manager }
pub fn remote(&self) -> &Arc<RwLock<Option<Arc<dyn RemoteAccess>>>> { &self.infra.remote }
+3
View File
@@ -25,6 +25,7 @@ use crate::elicitation::ElicitationManager;
use crate::image_generate::ImageGeneratorManager;
use crate::inbox::Inbox;
use crate::git_versions::GitVersions;
use crate::docx::DocxConverter;
use crate::latex::LatexCompiler;
use crate::llm::LlmManager;
use crate::location::LocationManager;
@@ -440,6 +441,7 @@ impl Conversation {
pub(super) struct Infra {
pub(super) latex_compiler: LatexCompiler,
pub(super) docx_converter: DocxConverter,
pub(super) git_versions: GitVersions,
pub(super) location_manager: Arc<LocationManager>,
pub(super) remote: Arc<RwLock<Option<Arc<dyn RemoteAccess>>>>,
@@ -449,6 +451,7 @@ impl Infra {
pub(super) fn build() -> Self {
Infra {
latex_compiler: LatexCompiler::new(),
docx_converter: DocxConverter::new(),
git_versions: GitVersions::new(),
location_manager: Arc::new(LocationManager::new()),
remote: Arc::new(RwLock::new(None)),
+8 -4
View File
@@ -28,7 +28,7 @@ use crate::tools::tool_names::SHOW_FILE_TO_USER;
/// It then emits a `ServerEvent::OpenFile` carrying the **canonical agent path**, so
/// the file-viewer page fetches the same file back through `/api/file`. The
/// frontend renders every kind in the viewer (HTML live in an origin-isolated
/// iframe; LaTeX compiled to PDF server-side).
/// iframe; LaTeX compiled and word documents converted to PDF server-side).
///
/// `session_id` is the conversation this instance belongs to: clients filter
/// events per conversation, so an untagged `OpenFile` would reach nobody.
@@ -46,8 +46,10 @@ pub fn make_tool(
"name": SHOW_FILE_TO_USER,
"description": "Show a file to the user by opening it in their interface. \
Supports Markdown, source code, plain text, raster images \
(PNG/JPG/GIF/WebP/…), SVG, PDF, and LaTeX (.tex — compiled \
to PDF automatically on the server). HTML files open in a \
(PNG/JPG/GIF/WebP/…), SVG, PDF, LaTeX (.tex — compiled \
to PDF automatically on the server), and word-processor \
documents (.docx/.doc/.odt/.rtf — converted to PDF \
automatically on the server). HTML files open in a \
new browser tab. Use this to surface a file you created or \
found so the user can look at it directly. One file per call. \
The file must already exist on disk — or as a memory note \
@@ -58,7 +60,9 @@ pub fn make_tool(
whenever any of its dependencies (\\input fragments, .sty/.cls, \
images) change. A raw `.pdf` is served statically — never \
recompiled and its dependencies are not watched — so the user \
would keep seeing a stale render.",
would keep seeing a stale render. The same rule applies to \
word documents: pass the original `.docx`/`.odt`/…, never a \
PDF exported from it.",
"parameters": {
"type": "object",
"properties": {
-1
View File
@@ -28,7 +28,6 @@ marketplace:
# The database lives at ./database/system.db — fixed, not configurable.
# ── LLM clients ────────────────────────────────────────────────────────────────
# LLM clients (providers, models, API keys, strength) are configured
# via the web app and stored in the database — not in this file.
+1 -1
View File
@@ -25,7 +25,7 @@ Two views, **one storage**: for the mounted subtree the fs-tools run **host-side
**The security boundary is the container, not the mounted subtree — the mount is the *fast* path, not the only one.** An agent already reaches every corner of its container through `execute_cmd`, which runs there with passwordless `sudo`; fs-tools that stopped at the mounts were not protecting anything, they were offering a poorer view of the same sandbox, and the model answered that by shelling out (the observed failure: `read_file /tmp/cv.txt`*"path escapes your workspace"* → the agent re-read it with `cat`). So `resolve_target` routes a physical path to one of two backings. An **absolute** path is container vocabulary — it is what `execute_cmd` prints — so it is reverse-mapped through `UserFs::container_to_agent` first: landing on a mount takes the host path (**`/root/x` *is* `~/x`**, which the tools used to reject outright, since `PathBuf::join` with an absolute tail silently discards the base and the result then failed the prefix check); landing nowhere means it exists only in the container, and `container::exec_fs` acts there over `docker exec` (paths passed **positionally** as `$1`, so a path containing `$(…)` is data, not syntax). Membership is not bypassed: `/root/shared/{X}` for a non-member still resolves to the same error as `shared/{X}`.
**One implementation per tool, not two.** Every single-file fs-tool already funnels through the same shape — resolve, then run a sync `execute` over one absolute host path — so the container branch is a **shuttle** (`fs::Shuttle`, behind `fs::run_physical`): pull the file out of the container, run the *unchanged* tool on the copy, push it back if the content changed (compared by bytes, not mtime, whose one-second resolution would miss a fast edit). Nothing about a tool's messages, diffs or pure transforms is duplicated. A missing remote file is deliberately **not** pre-created — `write_file` reports "Created" vs "Overwrote" from whether the path existed, and a placeholder would make every creation lie. Three tools opt out of the shuttle because a single file is the wrong unit: `list_files` lists in place via `exec_fs::list` (`find -printf`; `line_count` is omitted, since counting lines would turn a listing into a `docker exec` per file), `read_file` reads container paths as text (a shuttled copy is gone by the time the projection would inline a `MediaRef`, so media stays a mount-only feature), and `grep_files` **refuses** container paths with a pointer to `execute_cmd` + `rg` — its regex flavour, glob, windowing and offset would all have to be re-derived from ripgrep's flags, and a grep that answers *almost* the same is worse than one that says where to go. The viewer follows the same routing through `resolve_view_target` (`GET /api/file` and `show_file_to_user` open container paths; served without an ETag, so the editor stays read-only there).
**One implementation per tool, not two.** Every single-file fs-tool already funnels through the same shape — resolve, then run a sync `execute` over one absolute host path — so the container branch is a **shuttle** (`fs::Shuttle`, behind `fs::run_physical`): pull the file out of the container, run the *unchanged* tool on the copy, push it back if the content changed (compared by bytes, not mtime, whose one-second resolution would miss a fast edit). Nothing about a tool's messages, diffs or pure transforms is duplicated. A missing remote file is deliberately **not** pre-created — `write_file` reports "Created" vs "Overwrote" from whether the path existed, and a placeholder would make every creation lie. Three tools opt out of the shuttle because a single file is the wrong unit: `list_files` lists in place via `exec_fs::list` (`find -printf`; `line_count` is omitted, since counting lines would turn a listing into a `docker exec` per file), `read_file` reads container paths as text (a shuttled copy is gone by the time the projection would inline a `MediaRef`, so media stays a mount-only feature), and `grep_files` **refuses** container paths with a pointer to `execute_cmd` + `rg` — its regex flavour, glob, windowing and offset would all have to be re-derived from ripgrep's flags, and a grep that answers *almost* the same is worse than one that says where to go. The viewer follows the same routing through `resolve_view_target` (`GET /api/file` and `show_file_to_user` open container paths; served without an ETag, so the editor stays read-only there). One read-only shuttle lives here too: `GET /api/file?compile-docx=true` on a **container-only** word document pulls the bytes out with `exec_fs::read` and converts the host-side copy via `DocxConverter::convert_bytes` — correct because the format is self-contained. LaTeX deliberately gets no such branch: a shuttled `.tex` would silently lose its relative `\input`/`\includegraphics` dependencies.
**The memory roots are signposted inside the container, not merely absent.** `user-memory/`/`shared-memory/` are virtual, so nothing of them existed on disk — and the nothing was worse than it sounds: `cat user-memory/x.md` returned a bare ENOENT (which reads as *the note is missing*, not *wrong door*), while `mkdir -p user-memory && echo … > user-memory/x.md` **succeeded**, writing a real file into the home that no reader ever visits and that the next `ls` then confirms as if it had worked. Each root is therefore a **read-only bind mount** (`{WD}/.memory-signpost/{root}``{container_home}/{root}:ro`, gitignored, rewritten from consts on every `ensure`) holding a README that names the tools. Read-only *as a mount*, not as a mode: the container user has passwordless `sudo`, so a `chmod` would be a suggestion, whereas `:ro` holds — remounting needs `CAP_SYS_ADMIN` (verified: write, `sudo` write, `sudo chmod`, `sudo mount -o remount,rw` and `sudo rm` all fail). A README rather than an empty dir because `Permission denied` is an error, not an instruction — models answer it by reaching for `sudo`; the README puts the correction in the directory the failing command just named. These mounts are deliberately **not** in `UserFs`: they back no agent path and the host-side fs-tools must never resolve into them. They are the **fourth self-heal axis** in `reusable()` (`signposts_mounted`) rather than an `IMAGE_TAG` bump, since the image is unchanged and a bump would make every box rebuild it to fix a mount. The matching half is in `classify_memory`, which now strips the home spellings (`./`, `~/`, `/root/`) before matching the root — without it `~/user-memory/x.md` missed the match, fell through to the disk router, and became exactly the invisible physical file the signpost exists to prevent.
+9 -1
View File
@@ -53,7 +53,7 @@ Two independent things have to be true, and both were violated at some point:
| `sidebar.js` | `<app-sidebar>` | Nav sidebar; role-driven (`ui_mode`); inbox badge is **live** — the chat WS forwards the inbox lifecycle events (`approval_requested/resolved`, `clarification_*`, `elicitation_*`) regardless of `source`, `chat-session.js` re-dispatches them as the `inbox-changed` window event, and the sidebar (+ `agent-inbox.js`) refreshes on it; a 60 s poll remains as fallback |
| `topbar.js` | `<app-topbar>` | Top nav bar; per-user avatar color hashed from the username |
| `dashboard-page.js` | `<dashboard-page>` | `#dashboard` — status hero, LLM stats charts, pending inbox, quick guide |
| `shared/file-viewer-base.js` | `FileViewerBase` (base) | Shared file-viewer engine (fetch, kind detection, markdown/PDF/SVG/LaTeX, watcher, `_renderBody`); driven by `_show`/`_hide`. Extended by desktop + mobile |
| `shared/file-viewer-base.js` | `FileViewerBase` (base) | Shared file-viewer engine (fetch, kind detection, markdown/PDF/SVG/LaTeX/word-docs, watcher, `_renderBody`); driven by `_show`/`_hide`. Extended by desktop + mobile |
| `file-viewer-page.js` | `<file-viewer-page>` | Desktop file viewer: `FileViewerBase` + hash routing via `window.openFile(path)``#file_viewer?path=...` |
| `shared/file-viewer-mobile.js` | `<mobile-file-viewer-page>` | Mobile file viewer: `FileViewerBase` + prop-driven (`visible`/`path`), full-screen with back button |
| `agents.js` | `<agents-page>` | Agent discovery and config |
@@ -81,3 +81,11 @@ Two independent things have to be true, and both were violated at some point:
| `models-tts.js` | `<models-tts-section>` | Text-to-speech model CRUD |
| `mobile-app.js` | `<mobile-app>` | Mobile app shell |
| `shared/settings-page.js` | `<settings-page>` | Mobile settings: per-user avatar, locale picker (`I18nMixin`), profile/preferences |
## Server-rendered kinds in the file viewer (LaTeX, word documents)
Two kinds are not served as-is but rendered to PDF server-side on demand: `.tex`/`.latex` (kind `latex`, `?compile-latex=true`, `latexmk`) and `.docx`/`.doc`/`.odt`/`.rtf` (kind `docx`, `?compile-docx=true`, LibreOffice — `skald_core::docx::DocxConverter`, content-hash cache, no dependency graph). Both render through the same `<pdf-view>` and degrade gracefully when the host tool is missing (`501`) or the run fails (`422`): the viewer fetches the flagged URL, keeps the error body, and falls back. Three asymmetries between the two, each deliberate:
- **Fallback content.** A failed LaTeX compile still shows the *source* (readable); a word document is a zip, so its fallback is the binary download state with the reason in a foldable block on top — there is no source to show.
- **Download.** A `.tex` downloads the *compiled PDF* (the source is useless to most people); a word document downloads the **original file** — it is itself the editable artifact someone asking "send me the document" wants, and the PDF is only the preview mechanism.
- **Watching.** A `.tex` subscription expands server-side to its `.fls` dependency set (`file_watch.rs`); a word document is self-contained, so the plain per-file watcher already covers it and a change re-converts via the content-keyed cache — `file_watch.rs` needed no branch. Container-only word documents still convert (the API shuttles the bytes out, see `filesystem-and-containers.md`) but, like any container-only path, they are not watchable.
+14
View File
@@ -10,6 +10,20 @@
LLM client abstraction (OpenAI-compat, Anthropic, Ollama…). OpenAI-compatible provider *types* are runtime data, not code: `providers/declared.rs` loads `providers.yaml` at boot (see Config in [../CLAUDE.md](../CLAUDE.md)); only non-OpenAI-compatible or bespoke providers (anthropic, ollama, openai, openrouter) stay native. **Retriability** (`Model::is_retriable`, `agent-loop`) keys on the real HTTP status carried by `ModelError { status }`, **not** a substring of the message — a model id/token count containing "404"/"401" cannot mis-classify; 401/403/404/422 don't retry, 400/429/5xx/network do. **Request logging** is the `logging.rs::LoggingModel` decorator, attached by the *caller's* `ModelSelector` (`loop_adapters/selector.rs::SkaldSelector::with_log`) — never by `LlmManager`, which builds one shared client per model and cannot know whose traffic it serves. The decorator's `RequestLogTarget` carries the owner: metadata → `llm_requests` in the registry (`user_id`, the column the UI filters on), payload bodies/headers → `llm_request_payloads` in that user's own encrypted DB, keyed by `request_id`; session + frame come from the request's own `conversation`/`frame`, so kernel rounds, sub-agent frames and compaction summaries are all attributed with no extra plumbing (`ModelRequest::log` is unused here)
## `providers.yaml` — two traps in the model metadata
**`enrich` rules stop at the first glob that matches, and the default `mode: fill` skips a field that already has a value.** Both bite when adding a model to an existing family. Ordering: `glm-5.3-flash` matches `glm-5*` too, so a rule for it placed *after* the family rule never runs — the specific glob goes first, always. And `fill` means "the endpoint listing wins", which for a `static:` list is not the same as "nothing is set": `models.defaults` (today only `vision`) stamps every entry before `enrich` sees it, so a provider carrying `defaults: { vision: false }` — Z.AI does — needs **`mode: override`** to turn vision on for one model. A `fill` rule there parses, loads, logs nothing, and leaves the flag off; the only symptom is that images silently keep taking the textual `<system-extra>` path (see [Multimodal attachments](#multimodal-attachments)) on a model that can read them. `vision: true` also pushes the `vision` capability, but only when the rule actually applied.
**A `reasoning.modes` `values` list is the whole contract with the provider — it is not a superset to trim in the UI.** Whatever it lists is what can be sent, so a model that cannot stop thinking (`thinking.type` accepting only `"enabled"`: GLM-5.3 and GLM-5.3-Flash) simply omits `disabled` from its rule, rather than inheriting the family's `[disabled, enabled]` toggle and sending a value the API rejects. With `request: { kind: thinking }`, any value other than `disabled`/`enabled` is emitted as `{"thinking":{"type":"enabled"},"reasoning_effort":v}`, which is exactly the shape those models want.
## The provider API surface — the key is a boolean, never a value
**No provider endpoint ever returns a stored `api_key`, and the trap is that omitting it silently reads as "no key".** `LlmProviderInfo` (list) and `ProviderDetail` (`src/frontend/api/llm.rs`, the detail DTO — deliberately *not* `LlmProviderRecord`, which does carry the secret) both expose **`has_api_key: bool`** instead. That is the whole contract: the UI needs to know *whether* a key is on file, never what it is, and the browser is where a leaked key would end up in a devtools tab or a screenshot.
It shipped broken in exactly the way this shape invites: `list_providers_info` never carried `api_key` (correctly), while the card tested `Boolean(p.api_key)` — always `undefined` — so every provider was badged "API key missing" even with a working key. A missing field is falsy, not an error; nothing logs, nothing fails to build. If you add a provider surface, read `has_api_key`, and if you add a field to either DTO, keep the secret out by construction rather than by remembering to strip it.
The consequence on the write path is load-bearing: since the edit form can no longer prefill the key, **an empty `api_key` in the `PUT` payload means "keep the stored one"**`update_provider` re-reads the record and carries the old value over, because a blind `UPDATE … SET api_key=NULL` would wipe a working provider on any unrelated edit (a renamed description). The i18n placeholder (`providers.modal.api_key_ph`) already promised this behaviour before the backend implemented it. Side effect to know about: there is no longer a way to *clear* a key from the form — deleting the provider is the escape hatch.
## Token streaming & reasoning display
The chat streams tokens live, as a **parallel best-effort side-channel** that never alters the turn's authoritative flow: the final `Done` (or `Thinking`) event still carries the complete content and the frontend treats it as truth.
+2
View File
@@ -11,3 +11,5 @@
**Plugin HTTP routes & web pages.** Every plugin's `http_router()` mounts at boot under `/api/plugin/<id>/`**enabled or not**: two shared gates wrap each router (`require_auth`, then `guard::plugin_enabled_gate`, which re-checks the DB flag per request and answers 404 while disabled), so enable/disable serves/stops routes immediately with no restart, and plugin responses carry `Cache-Control: no-cache`. The router contract: cheap and safe to build pre-start, handlers tolerant of the not-running state (resolve runtime state per request through a shared cell, as mobile-connector does). A plugin may also contribute **frontend pages** via `Plugin::web_pages()` (`PluginPage { page_id, title, icon, entry, admin_only, priority }`): `GET /api/plugins/pages` returns the caller's visible pages (admin: all; others: non-`admin_only` pages of granted, enabled plugins) with `entry_url` resolved, and the sidebar renders them as menu entries routed `#plugin/<plugin_id>/<page_id>`. A single `<plugin-page-host>` (`web/components/plugin-page-host.js`) dynamic-imports the fragment ES module the plugin serves from its own router, registers its default-exported HTMLElement class, and mounts it with the `plugin-id` attribute — the fragment talks to its backend only through `/api/plugin/<id>/…` and runs with full session privileges (plugins are trusted: they ship in the binary). The frontend knows nothing about plugin page contents or behavior.
**A plugin page with its own backend (Honcho is the reference).** Most per-user plugin pages (Telegram's) need no backend of their own — the core `/api/plugins/…` endpoints carry their blob. Honcho's opt-in page grew a **debug panel** over the external memory server, and its router (`crates/plugin-honcho/src/router.rs`) is the pattern for that case, with three load-bearing rules. (1) **The peer id is the multi-user boundary**: the Honcho workspace is shared by every user, so every introspection handler receives its peer *already resolved* from the authenticated `Caller``require_peer` returns it or a 403 — and a client-supplied peer/workspace can never reach the external server. (2) **The opt-in gate is server-side and fail-closed** (`require_peer` re-reads the flag, same as the tools and the write path) — the panel hiding client-side when the flag is off is cosmetics, not the control. (3) **Errors are specific, never "service unavailable"**: transport failures and the external server's HTTP status+body are localized and forwarded (truncated), because the page exists to debug the integration — and a 404 from the external server is translated per-endpoint as "no memory yet", which is a state, not a failure. The shared `WebCell` carries the live client + workspace + user-config store alongside the router's other deps, filled by `start`/`stop`.
**Honcho's response schema is version-specific and bit us once.** Verified against self-hosted **3.0.11** (the compose pins the image by digest — ghcr publishes no v3 semver tags, and a silent `:latest` bump is exactly how the drift arrived): `peers/{id}/context` returns `{representation: markdown string, peer_card: string[]|null}`**no `conclusions` array** — and `sessions/{id}/context` returns `summary` as an *object* (`{content,…}`) plus `peer_representation`. The card endpoints wrap the list (`{"peer_card": …}`) in both directions — a bare array on PUT is a 422. Semantic search with ranked ids is `POST conclusions/query` with the observer/observed scoping inside a `filters` object — `peer_context?search_query=…` is *not* a substitute (it returns the whole representation once it fits the budget, with no ids). All of this is typed in `honcho-client::models` and pinned by fixture tests from real payloads, plus a live smoke test (`HONCHO_E2E_*` env vars, `cargo test -p honcho-client -- --ignored`). If a future Honcho upgrade changes shapes again, that test is the five-second check.
+5 -4
View File
@@ -14,13 +14,14 @@ The header shows the file's path exactly as your tools spell it (`shared/recipes
| SVG | rendered in an isolated frame — scripts inside it never run |
| PDF | drawn by the app itself, so it looks and scrolls the same in every browser and on the phone |
| LaTeX (`.tex`) | **compiled to PDF on the server** and shown as the document |
| Word documents (`.docx`, `.doc`, `.odt`, `.rtf`) | **converted to PDF on the server** (needs LibreOffice installed there) and shown as the document |
| HTML | rendered live in an isolated frame; the toggle in the header switches to the source |
| Anything else | not displayed — the file can still be downloaded |
Two consequences worth knowing:
- **Always give `show_file_to_user` the `.tex`, never a `.pdf` you built from it.** The `.tex` is recompiled and the view follows its dependencies — `\input` fragments, styles, images — so it stays current. A raw `.pdf` is served as bytes: it is never recompiled and the user ends up looking at a stale render.
- **A compile that fails is not a dead end.** The viewer shows the source instead, with the actual error block foldable at the top. That error is worth reading if they ask why "the document is not showing" — it usually names a line.
- **Always give `show_file_to_user` the `.tex`, never a `.pdf` you built from it.** The `.tex` is recompiled and the view follows its dependencies — `\input` fragments, styles, images — so it stays current. A raw `.pdf` is served as bytes: it is never recompiled and the user ends up looking at a stale render. The same applies to word documents: give the `.docx`, not a PDF exported from it — the viewer converts it, and re-converts it when the file changes.
- **A compile that fails is not a dead end.** The viewer shows the source instead, with the actual error block foldable at the top. That error is worth reading if they ask why "the document is not showing" — it usually names a line. (For a word document there is no readable source to show, so a failed conversion explains itself in the same foldable block over the download state.)
## It is live
@@ -52,7 +53,7 @@ Files inside a folder that is under version control — in practice, project fol
## Download
The download button saves the file with its real name. For a `.tex` it downloads the **compiled PDF**, not the source — that is usually what someone asking to "send me the document" wants; if they want the source itself, they want the `.tex`, and it is worth checking which.
The download button saves the file with its real name. For a `.tex` it downloads the **compiled PDF**, not the source — that is usually what someone asking to "send me the document" wants; if they want the source itself, they want the `.tex`, and it is worth checking which. A word document instead downloads as **the original file** (the `.docx`, `.odt`…): unlike a `.tex` source it is the editable document itself, and the PDF on screen is only the preview.
## What the viewer tells you
@@ -66,7 +67,7 @@ If the eye is off, none of that arrives, and you genuinely do not know what they
- *"It says the file changed while I was editing."* — something else wrote to it. Three buttons on the banner; copy-then-reload loses nothing.
- *"The PDF is wrong / old."* — if there is a `.tex` beside it, they are looking at a stale build. Open the `.tex` instead: it recompiles.
- *"Where is the old version?"* — the clock, if the file is in a project folder. Otherwise there is no history to show, and the honest answer is that this file is not versioned.
- *"It won't show the file."* — a kind the viewer cannot render (an archive, an office document, an unknown binary) shows the download instead. That is the whole story; there is no plugin to install.
- *"It won't show the file."* — a kind the viewer cannot render (an archive, an unknown binary) shows the download instead, and that is the whole story. An office document showing only the download means the server has **no LibreOffice installed**: the admin installing it turns the preview on — nothing to change in the app, and the foldable block on the page says exactly this.
- *"Show me that file."* — `show_file_to_user`, one file per call, on any path in their own workspace including a memory note. It must already exist.
## Related
+1 -1
View File
@@ -33,7 +33,7 @@ Long-term memory is **off for every user until they turn it on themselves**. Onc
Once a user has opted in, that same page shows a **debug panel** over their own memory — useful to check the integration is working, and to see (and question) what has been derived:
- **Service status** — whether the Honcho server is reachable (with the response time), how the user's own memory processing is doing (units in progress / pending / completed), and *the specific error* when something is wrong (server down, key rejected, server-side error), not just "unavailable".
- **Overview** — everything Honcho has derived so far: the user's "card" (curated key facts), the individual facts (each with its id), and a summary. Loads automatically; nothing to type.
- **Overview** — everything Honcho has derived so far: the user's "card" (curated key facts — empty until some are set, e.g. via the `honcho_profile` tool), the individual facts (each with its id), and the full **representation** digest Honcho hands to the assistant on every turn. Loads automatically; nothing to type.
- **Search or ask** — one text field, two actions, with a mini-guide on the page:
- **Search** finds the stored facts most relevant to the words typed — fast, exact, raw facts.
- **Ask** sends the question to Honcho's AI, which reads the user's memory and writes an answer in its own words — slower, but it connects the dots.
+12 -6
View File
@@ -1,8 +1,8 @@
# Honcho — self-hosted Docker package
This folder contains a ready-to-run Docker Compose setup for [Honcho](https://honcho.dev),
the memory server used by the personal-agent's Honcho plugin
([`src/plugin/honcho/`](../src/plugin/honcho/)).
the memory server used by the Honcho plugin
([`crates/plugin-honcho/`](../crates/plugin-honcho/)).
---
@@ -10,11 +10,16 @@ the memory server used by the personal-agent's Honcho plugin
| Service | Image | Port | Role |
| --- | --- | --- | --- |
| `api` | `ghcr.io/plastic-labs/honcho:latest` | **8000** | REST API (the endpoint personal-agent talks to) |
| `api` | `ghcr.io/plastic-labs/honcho@sha256:59f0…8c6b` (**3.0.11**, digest-pinned) | **8000** | REST API (the endpoint the app talks to) |
| `deriver` | same image | — | Background worker: extracts conclusions, summaries, peer representations |
| `db` | `pgvector/pgvector:pg17` | 5432 (internal) | PostgreSQL + pgvector (vector search) |
| `redis` | `redis:7-alpine` | 6379 (internal) | Cache for session context |
> The Honcho image is **pinned by digest** because ghcr publishes no v3 semver
> tags (only `latest`), and the plugin parses a version-specific API schema —
> a silent `:latest` bump already changed response shapes once. Upgrade
> deliberately: pick the new digest, verify the plugin against it, then edit.
Data is stored in named Docker volumes (`honcho_db`, `honcho_redis`) and survives container restarts.
---
@@ -140,7 +145,8 @@ docker compose restart api
# Stop and wipe all data (destructive!)
docker compose down -v
# Upgrade to a newer Honcho image
# Upgrade Honcho: bump the pinned digest in docker-compose.yml (deliberately —
# the plugin's API parsing is verified against the pinned version), then
docker compose pull
docker compose up -d
```
@@ -181,5 +187,5 @@ docker compose up -d --build
- [Honcho GitHub](https://github.com/plastic-labs/honcho)
- [Honcho docs](https://docs.honcho.dev)
- [Self-hosting guide (official)](https://docs.honcho.dev/v3/contributing/self-hosting)
- [personal-agent Honcho plugin docs](../docs/honcho.md)
- [personal-agent Memory architecture](../docs/memory.md)
- [Honcho plugin docs](../docs/plugins/honcho.md)
- [Memory architecture](../docs/memory.md)
+8 -3
View File
@@ -13,6 +13,11 @@
#
# The API will be available at http://localhost:8000
# Interactive docs: http://localhost:8000/docs
#
# The Honcho image is PINNED BY DIGEST (ghcr publishes no v3 semver tags, only
# `latest`): this digest is Honcho 3.0.11. The plugin parses a version-specific
# API schema (`representation`/`peer_card`, wrapped card, `conclusions/query`
# filters) — bump the digest deliberately and re-verify, never via `:latest`.
services:
@@ -20,7 +25,7 @@ services:
# Applies Alembic migrations before the API starts.
# Exits with code 0 when done; Docker Compose marks it "completed".
migrate:
image: ghcr.io/plastic-labs/honcho:latest
image: ghcr.io/plastic-labs/honcho@sha256:59f099ad85713105608c9b239a4f1386fbebd8647323d3a04646bb3c975d8c6b # Honcho 3.0.11, pinned — see header
env_file: .env
environment:
DB_CONNECTION_URI: postgresql+psycopg://${POSTGRES_USER:-honcho}:${POSTGRES_PASSWORD:-honcho}@db:5432/${POSTGRES_DB:-honcho}
@@ -33,7 +38,7 @@ services:
# ── API ────────────────────────────────────────────────────────────────────
api:
image: ghcr.io/plastic-labs/honcho:latest
image: ghcr.io/plastic-labs/honcho@sha256:59f099ad85713105608c9b239a4f1386fbebd8647323d3a04646bb3c975d8c6b # Honcho 3.0.11, pinned — see header
restart: unless-stopped
ports:
- "${HONCHO_PORT:-8000}:8000"
@@ -65,7 +70,7 @@ services:
# Without a working LLM key this service will fail to process messages;
# the API itself will still work but no long-term memory will be built.
deriver:
image: ghcr.io/plastic-labs/honcho:latest
image: ghcr.io/plastic-labs/honcho@sha256:59f099ad85713105608c9b239a4f1386fbebd8647323d3a04646bb3c975d8c6b # Honcho 3.0.11, pinned — see header
restart: unless-stopped
command: ["/app/.venv/bin/python", "-m", "src.deriver"]
env_file: .env
+13 -1
View File
@@ -145,11 +145,17 @@ providers:
- { key: api_key, label: "API Key", required: true, secret: true }
models:
# Z.AI exposes no GET /models endpoint; this mirrors the console menu.
static: [glm-5.2, glm-5.1, glm-5, glm-5-turbo, glm-4.7, glm-4.6, glm-4.5, glm-4-32b-0414-128k]
static: [glm-5.3, glm-5.3-flash, glm-5.2, glm-5.1, glm-5, glm-5-turbo, glm-4.7, glm-4.6, glm-4.5, glm-4-32b-0414-128k]
defaults: { vision: false }
base_capabilities: [function_calling]
enrich:
- { match: "*128k*", context_length: 131072 }
# GLM-5.3-Flash is the first natively multimodal GLM (image + video in).
# `mode: override` is load-bearing: `defaults.vision` already stamped
# every static model with `Some(false)`, and a fill rule skips a field
# that is already set — the flag would silently stay off.
- { match: "glm-5.3-flash*", mode: override, context_length: 1048576, max_completion_tokens: 131072, vision: true, add_capabilities: [video] }
- { match: "glm-5.3*", context_length: 1048576, max_completion_tokens: 131072 }
- { match: "glm-5*", context_length: 1048576 }
- { match: "glm-4.7*", context_length: 200000 }
- { match: "glm-4.6*", context_length: 200000 }
@@ -157,6 +163,12 @@ providers:
reasoning:
request: { kind: thinking }
modes:
# GLM-5.3 (and -Flash) cannot stop thinking: `thinking.type` only
# accepts "enabled", so the budget is picked with reasoning_effort
# alone and `disabled` is deliberately absent from the values.
- when: { models: ["glm-5.3*"] }
values: [low, high, max]
default: max
# GLM-5.2+ adds a graded effort on top of the thinking toggle.
- when: { models: ["glm-5.2*"] }
values: [disabled, minimal, low, medium, high, xhigh, max]
+110
View File
@@ -15,6 +15,7 @@ use skald_core::db::memory_docs;
use skald_core::git_versions::{self, GitVersions};
use skald_core::session::handler::media;
use skald_core::skald::Skald;
use skald_core::docx::ConvertError;
use skald_core::latex::CompileError;
use skald_core::tools::fs as fs_tools;
use super::ApiError;
@@ -479,6 +480,12 @@ pub struct FileQuery {
/// source. Other file types ignore this flag.
#[serde(rename = "compile-latex", default)]
pub compile_latex: bool,
/// When `true` and `path` points at a word-processor document
/// (`.docx` / `.doc` / `.odt` / `.rtf`), convert it to PDF via
/// LibreOffice and return the PDF bytes instead of the raw file.
/// Other file types ignore this flag.
#[serde(rename = "compile-docx", default)]
pub compile_docx: bool,
/// When `true`, mark the response as a download (`Content-Disposition:
/// attachment`) so the browser saves the file instead of rendering it
/// inline. For a compiled `.tex` the attachment name is `<stem>.pdf`.
@@ -505,6 +512,13 @@ pub struct FileQuery {
/// with the textual `latexmk` log in the body, so the caller can fall back to
/// showing the raw source.
///
/// With `?compile-docx=true` a word-processor document (`.docx` / `.doc` /
/// `.odt` / `.rtf`) is converted to PDF (see
/// [`skald_core::docx::DocxConverter`]). A document that lives **only inside
/// the caller's container** is shuttled out to a host scratch copy first —
/// correct because the format is self-contained, unlike a `.tex` with its
/// relative `\input`s (which is why LaTeX gets no container branch).
///
/// A path under a virtual memory root (`user-memory/…`, `shared-memory/…`) is
/// served from the `memory_docs` table — the caller's own pool for the private
/// root, the system pool for the shared one — exactly like the fs-tools route
@@ -569,6 +583,32 @@ pub async fn get_file(
let abs = match target {
fs_tools::FsTarget::Host(abs) => abs,
fs_tools::FsTarget::Container { container, path } => {
// Word documents are self-contained, so a container-only one can
// still be previewed: shuttle the bytes out and convert the copy
// on the host (the fs-tools' `Shuttle` pattern, read-only half).
// LaTeX deliberately gets no shuttle: a copied `.tex` would lose
// its relative `\input`/`\includegraphics` dependencies.
if q.compile_docx && is_word_doc(&q.path) {
let ext = Path::new(&q.path)
.extension()
.and_then(|e| e.to_str())
.unwrap_or("docx")
.to_string();
return match skald_core::container::exec_fs::read(&container, &path).await {
Ok(bytes) => match state.docx_converter().convert_bytes(&bytes, &ext).await {
Ok(pdf) => {
let mut response = pdf_response(pdf.bytes);
if q.force_download {
set_attachment(&mut response, &pdf_download_name(&q.path));
}
response
}
Err(err) => convert_error_response(err),
},
Err(_) => (StatusCode::NOT_FOUND, format!("File not found: {}", q.path))
.into_response(),
};
}
return match skald_core::container::exec_fs::read(&container, &path).await {
Ok(bytes) => {
let mut response = bytes.into_response();
@@ -600,6 +640,19 @@ pub async fn get_file(
};
}
if q.compile_docx && is_word_doc(&q.path) {
return match state.docx_converter().convert_path(&abs).await {
Ok(pdf) => {
let mut response = pdf_response(pdf.bytes);
if q.force_download {
set_attachment(&mut response, &pdf_download_name(&q.path));
}
response
}
Err(err) => convert_error_response(err),
};
}
match tokio::fs::read(&abs).await {
Ok(bytes) => {
let mut response = bytes.into_response();
@@ -686,6 +739,19 @@ async fn get_file_at_rev(
};
}
if q.compile_docx && is_word_doc(&q.path) {
return match state.docx_converter().convert_path(&file).await {
Ok(pdf) => {
let mut response = pdf_response(pdf.bytes);
if q.force_download {
set_attachment(&mut response, &pdf_download_name(&q.path));
}
response
}
Err(err) => convert_error_response(err),
};
}
match tokio::fs::read(&file).await {
Ok(bytes) => {
let mut response = bytes.into_response();
@@ -851,6 +917,35 @@ fn compile_error_response(err: CompileError) -> Response {
(status, response).into_response()
}
/// Map a [`ConvertError`] to an HTTP status, mirroring
/// [`compile_error_response`]: `ToolMissing` → `501 Not Implemented`,
/// `Timeout` → `504 Gateway Timeout`, `Failed` → `422 Unprocessable Entity`
/// (body = captured `soffice` output), `Io` → `500`. The body is plain text
/// so the viewer can show it directly.
fn convert_error_response(err: ConvertError) -> Response {
let (status, body): (StatusCode, String) = match err {
ConvertError::ToolMissing => (
StatusCode::NOT_IMPLEMENTED,
"LibreOffice is not installed on the server.".to_string(),
),
ConvertError::Timeout => (
StatusCode::GATEWAY_TIMEOUT,
"Document conversion aborted due to timeout.".to_string(),
),
ConvertError::Failed { output } => (StatusCode::UNPROCESSABLE_ENTITY, output),
ConvertError::Io(e) => (
StatusCode::INTERNAL_SERVER_ERROR,
format!("I/O error during conversion: {e}"),
),
};
let mut response = body.into_response();
response.headers_mut().insert(
header::CONTENT_TYPE,
HeaderValue::from_static("text/plain; charset=utf-8"),
);
(status, response).into_response()
}
/// True for `.tex` / `.latex` extensions — i.e. inputs worth compiling.
fn is_latex(path: &str) -> bool {
matches!(
@@ -862,6 +957,17 @@ fn is_latex(path: &str) -> bool {
)
}
/// True for word-processor extensions (`.docx` / `.doc` / `.odt` / `.rtf`) —
/// the family LibreOffice converts to PDF. The extension list itself lives in
/// [`skald_core::docx::WORD_EXTS`] so this check and the converter agree.
fn is_word_doc(path: &str) -> bool {
Path::new(path)
.extension()
.and_then(|e| e.to_str())
.map(|e| skald_core::docx::WORD_EXTS.contains(&e.to_ascii_lowercase().as_str()))
.unwrap_or(false)
}
/// Best-effort `Content-Type` from a file extension. Known binary types get their
/// specific MIME; everything else is served as UTF-8 text (markdown, code, configs,
/// and unknown files the viewer treats as plain text or "binary, no preview").
@@ -882,6 +988,10 @@ fn content_type_for(path: &str) -> &'static str {
"svg" => "image/svg+xml",
"pdf" => "application/pdf",
"tex" | "latex" => "application/x-tex",
"docx" => "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"doc" => "application/msword",
"odt" => "application/vnd.oasis.opendocument.text",
"rtf" => "application/rtf",
"html" | "htm" => "text/html; charset=utf-8",
_ => "text/plain; charset=utf-8",
}
+39 -3
View File
@@ -98,12 +98,43 @@ pub async fn create_provider(
Ok(StatusCode::CREATED)
}
/// One provider, as the edit form sees it. Deliberately **not** `LlmProviderRecord`:
/// the stored api_key never travels to the browser — the form only needs to know
/// whether one exists, so it can offer "leave blank to keep it".
#[derive(Serialize)]
pub struct ProviderDetail {
pub id: i64,
pub name: String,
#[serde(rename = "type")]
pub provider: String,
pub has_api_key: bool,
pub base_url: Option<String>,
pub description: Option<String>,
}
impl From<LlmProviderRecord> for ProviderDetail {
fn from(r: LlmProviderRecord) -> Self {
ProviderDetail {
id: r.id,
name: r.name,
provider: r.provider,
has_api_key: has_key(&r.api_key),
base_url: r.base_url,
description: r.description,
}
}
}
fn has_key(key: &Option<String>) -> bool {
key.as_deref().is_some_and(|k| !k.trim().is_empty())
}
pub async fn get_provider(
State(skald): State<Arc<Skald>>,
axum::extract::Path(id): axum::extract::Path<i64>,
) -> Result<Json<LlmProviderRecord>, ApiError> {
) -> Result<Json<ProviderDetail>, ApiError> {
skald.llm_manager().get_provider(id).await
.map(Json)
.map(|r| Json(ProviderDetail::from(r)))
.ok_or_else(|| ApiError::not_found(format!("provider {id} not found")))
}
@@ -113,7 +144,12 @@ pub async fn update_provider(
Json(payload): Json<ProviderPayload>,
) -> Result<StatusCode, ApiError> {
validate_provider_type(&skald, &payload.provider)?;
let record = LlmProviderRecord::from(payload);
let mut record = LlmProviderRecord::from(payload);
// The form never receives the stored key, so it cannot send it back: an empty
// api_key means "keep the one on file", not "erase it".
if !has_key(&record.api_key) {
record.api_key = skald.llm_manager().get_provider(id).await.and_then(|r| r.api_key);
}
skald.llm_manager().update_provider(id, record).await?;
Ok(StatusCode::NO_CONTENT)
}
+4 -3
View File
@@ -40,9 +40,10 @@ The URL returned by image_generate already points to the correct endpoint — us
Do NOT append \".png\" or any extension to the URL.\n\
\n\
FILES: To let the user look at a file directly, call show_file_to_user(path). Supported: \
Markdown, source code, images (PNG/JPG/GIF/WebP/SVG), PDF, and LaTeX (.tex — auto-compiled \
to PDF server-side). HTML opens in a new browser tab. Prefer this over pasting long file \
contents into chat.";
Markdown, source code, images (PNG/JPG/GIF/WebP/SVG), PDF, LaTeX (.tex — auto-compiled \
to PDF server-side), and word-processor documents (.docx/.doc/.odt/.rtf — converted to \
PDF server-side when LibreOffice is installed). HTML opens in a new browser tab. Prefer \
this over pasting long file contents into chat.";
const HELP_TEXT: &str = "\
**Available commands**\n\n\
+1
View File
@@ -464,6 +464,7 @@ function attachmentIcon(att) {
const n = (att.name || '').toLowerCase();
if (m.startsWith('image/')) return 'bi-file-earmark-image';
if (m === 'application/pdf' || n.endsWith('.pdf')) return 'bi-file-earmark-pdf';
if (/\.(docx?|odt|rtf)$/.test(n)) return 'bi-file-earmark-word';
if (m.startsWith('audio/')) return 'bi-file-earmark-music';
if (m.startsWith('video/')) return 'bi-file-earmark-play';
if (m.startsWith('text/') || /\.(md|txt|csv|json|ya?ml|rs|js|ts|py)$/.test(n)) return 'bi-file-earmark-text';
+5 -4
View File
@@ -98,14 +98,15 @@ export class LlmProvidersPage extends LightElement {
const res = await fetch(`/api/llm/providers/${provider.id}`);
if (!res.ok) throw new Error(`HTTP ${res.status}`);
const record = await res.json();
// The server never sends the stored key back — an empty box means "keep it".
this._form = {
name: record.name,
type: record.type,
api_key: record.api_key ?? '',
api_key: '',
base_url: record.base_url ?? '',
description: record.description ?? '',
};
this._modal = { mode: 'edit', id: record.id };
this._modal = { mode: 'edit', id: record.id, hasKey: Boolean(record.has_api_key) };
} catch (e) {
this._error = e.message;
}
@@ -172,7 +173,7 @@ export class LlmProvidersPage extends LightElement {
const icon = meta.icon;
const label = meta.display_name;
const count = this._modelCounts[String(p.id)];
const hasKey = Boolean(p.api_key);
const hasKey = Boolean(p.has_api_key);
const needsUrl = meta.fields.some(f => f.key === 'base_url');
return html`
@@ -268,7 +269,7 @@ export class LlmProvidersPage extends LightElement {
<label class="form-label fw-semibold" style="font-size:0.82rem">${t('providers.modal.api_key')}</label>
<input type="password" class="form-control form-control-sm" .value=${f.api_key}
autocomplete="new-password"
placeholder=${isEdit ? t('providers.modal.api_key_ph') : ''}
placeholder=${isEdit && this._modal?.hasKey ? t('providers.modal.api_key_ph') : ''}
@input=${(e) => this._setField('api_key', e.target.value)} />
</div>
` : ''}
+1 -1
View File
@@ -329,7 +329,7 @@ export class FileExplorer extends LightElement {
zip: 'bi-file-zip', gz: 'bi-file-zip', tar: 'bi-file-zip',
mp3: 'bi-file-music', wav: 'bi-file-music', ogg: 'bi-file-music',
mp4: 'bi-file-play', mov: 'bi-file-play', webm: 'bi-file-play',
doc: 'bi-file-word', docx: 'bi-file-word',
doc: 'bi-file-word', docx: 'bi-file-word', odt: 'bi-file-word', rtf: 'bi-file-word',
xls: 'bi-file-excel', xlsx: 'bi-file-excel', csv: 'bi-file-excel',
};
return map[ext] ?? 'bi-file-earmark';
+69 -3
View File
@@ -10,8 +10,9 @@ import './pdf-view.js'; // registers <pdf-view>; pdf.js itself is imported laz
/**
* Shared file-viewer engine. Holds all of the fetch / kind-detection /
* markdown-asset-rewriting / LaTeX-compile / live-watch logic plus `_renderBody`,
* driven purely by two methods: `_show(path)` and `_hide()`. It carries no
* markdown-asset-rewriting / LaTeX-compile / word-doc-convert / live-watch
* logic plus `_renderBody`, driven purely by two methods: `_show(path)` and
* `_hide()`. It carries no
* navigation or page chrome of its own subclasses (desktop `<file-viewer-page>`
* and mobile `<mobile-file-viewer-page>`) wire visibility/path to those methods
* and provide their own `render()` header.
@@ -19,6 +20,10 @@ import './pdf-view.js'; // registers <pdf-view>; pdf.js itself is imported laz
const IMG_EXTS = ['png', 'jpg', 'jpeg', 'gif', 'webp', 'bmp', 'ico', 'avif'];
const LATEX_EXTS = ['tex', 'latex'];
// Word-processor documents, converted to PDF server-side via LibreOffice
// (`?compile-docx=true`). Unlike LaTeX there is no readable source to fall
// back to: a failed or unavailable conversion leaves the binary state.
const WORD_EXTS = ['docx', 'doc', 'odt', 'rtf'];
const TEXT_EXTS = [
'txt', 'md', 'markdown', 'rs', 'js', 'mjs', 'cjs', 'ts', 'tsx', 'jsx',
'py', 'json', 'yml', 'yaml', 'toml', 'sh', 'bash', 'zsh', 'fish',
@@ -54,6 +59,7 @@ export function kindFor(path) {
// (srcdoc + sandbox="allow-scripts", no allow-same-origin) — see _renderBody.
if (ext === 'html' || ext === 'htm') return 'html';
if (LATEX_EXTS.includes(ext)) return 'latex';
if (WORD_EXTS.includes(ext)) return 'docx';
if (TEXT_EXTS.includes(ext)) return 'text';
return 'binary';
}
@@ -137,6 +143,7 @@ const VIEW_KINDS = {
svg: 'an SVG image',
html: 'a rendered HTML page',
latex: 'a compiled LaTeX document',
docx: 'an office document converted to PDF',
binary: 'a binary file, whose content is not displayed',
};
@@ -391,7 +398,10 @@ export class FileViewerBase extends LightElement {
/**
* Download the current file. LaTeX sources always download the compiled PDF
* (`compile-latex=true`); every kind is served with `force_download=true` so
* (`compile-latex=true`); word documents instead download the **original**
* file unlike a `.tex` source, a `.docx` is itself the editable document
* people want to keep or send, while the PDF is only the preview mechanism.
* Every kind is served with `force_download=true` so
* the server sets `Content-Disposition: attachment` and the browser saves it
* (with the server-supplied name) instead of rendering inline.
*/
@@ -488,6 +498,8 @@ export class FileViewerBase extends LightElement {
if (oldUrl) URL.revokeObjectURL(oldUrl);
} else if (this._kind === 'latex') {
await this._loadLatex(path);
} else if (this._kind === 'docx') {
await this._loadDocx(path);
} else if (this._kind === 'text' || this._kind === 'html') {
const res = await fetch(url);
if (!res.ok) throw new Error(`HTTP ${res.status}`);
@@ -551,6 +563,39 @@ export class FileViewerBase extends LightElement {
this._content = await res.text();
}
/**
* Load a word-processor document (`.docx` / `.doc` / `.odt` / `.rtf`).
* Asks the server to convert it to PDF via LibreOffice; on any non-OK
* response (501 no LibreOffice, 504 timeout, 422 conversion error) there is
* no readable source to fall back to the body is a zip so the reason is
* kept in `_compileError` and `_renderBody` shows the binary state with the
* error block on top.
*/
async _loadDocx(path) {
const convertUrl = this._fileUrl(path, { 'compile-docx': 'true' });
try {
const res = await fetch(convertUrl);
if (res.ok) {
const blob = await res.blob();
// Swap URLs only after the new blob is ready so the preview never flickers.
const oldUrl = this._blobUrl;
this._blobUrl = URL.createObjectURL(blob);
if (oldUrl) URL.revokeObjectURL(oldUrl);
this._compileError = null;
return;
}
// The error body is a short plain-text reason (converter missing,
// timeout, or the captured soffice output) — shown verbatim, unlike the
// latex log which needs distilling.
let detail = '';
try { detail = (await res.text()).trim(); } catch { /* ignore */ }
this._compileError = detail || `HTTP ${res.status}`;
} catch (e) {
this._compileError = e.message || String(e);
}
this._revokeBlobUrl();
}
// ── History mode (git-versioned files) ─────────────────────────────────────
/**
@@ -901,6 +946,10 @@ export class FileViewerBase extends LightElement {
// a native .pdf is rendered (see the note above).
return html`<pdf-view class="fv-pdf" .src=${this._blobUrl}></pdf-view>`;
}
if (this._kind === 'docx' && this._blobUrl) {
// Successfully converted server-side (LibreOffice) — same render path.
return html`<pdf-view class="fv-pdf" .src=${this._blobUrl}></pdf-view>`;
}
if (this._kind === 'svg' && this._blobUrl) {
// `allow-same-origin` (and nothing else) is required so the iframe can load
// the blob: URL — those are only readable from their creating origin. With
@@ -910,6 +959,23 @@ export class FileViewerBase extends LightElement {
${keyed(this._blobUrl, html`<iframe class="fv-svg" sandbox="allow-same-origin" src=${this._blobUrl} title=${this._path}></iframe>`)}
</div>`;
}
if (this._kind === 'docx') {
// Conversion failed or LibreOffice is not installed — unlike LaTeX
// there is no readable source to show (the file is a zip), so the
// reason sits in a foldable block over the download-only state.
return html`
${this._compileError
? html`<details class="fv-compile-error">
<summary><i class="bi bi-exclamation-triangle text-warning"></i>&nbsp;${t('fv.docx_failed')}</summary>
<pre>${this._compileError}</pre>
</details>`
: nothing}
<div class="fv-state text-muted">
<i class="bi bi-file-earmark-word fs-3 d-block mb-2"></i>
${t('fv.binary_unavailable')}
</div>
`;
}
if (this._kind === 'binary') {
return html`<div class="fv-state text-muted">
<i class="bi bi-file-earmark-binary fs-3 d-block mb-2"></i>
+1
View File
@@ -1049,6 +1049,7 @@ export default {
'fv.mode_source': 'Show source',
'fv.binary_unavailable': 'Preview not available for this file type.',
'fv.latex_failed': 'LaTeX compilation failed — showing source instead',
'fv.docx_failed': 'Document conversion failed',
'fv.tab_view': 'View',
'fv.tab_edit': 'Edit',
'fv.dirty_badge': 'Unsaved changes',
+1
View File
@@ -1036,6 +1036,7 @@ export default {
'fv.mode_source': 'Afficher la source',
'fv.binary_unavailable': 'Aperçu non disponible pour ce type de fichier.',
'fv.latex_failed': 'Échec de la compilation LaTeX — affichage de la source à la place',
'fv.docx_failed': 'Échec de la conversion du document',
'fv.tab_view': 'Afficher',
'fv.tab_edit': 'Modifier',
'fv.dirty_badge': 'Modifications non enregistrées',
+1
View File
@@ -1036,6 +1036,7 @@ export default {
'fv.mode_source': 'Mostra sorgente',
'fv.binary_unavailable': 'Anteprima non disponibile per questo tipo di file.',
'fv.latex_failed': 'Compilazione LaTeX fallita — mostra il sorgente',
'fv.docx_failed': 'Conversione del documento non riuscita',
'fv.tab_view': 'Visualizza',
'fv.tab_edit': 'Modifica',
'fv.dirty_badge': 'Modifiche non salvate',