feat(skills): rebuild the skill system for the multi-user model
Nightly Build / build (push) Successful in 8m6s
Nightly Build / build (push) Successful in 8m6s
Per blueprint/skill-project.md: the old single-namespace, hand-maintained
index is gone, replaced by a read-only, two-scope tree whose index is a
runtime function of its content.
- skills/ index generated at runtime (crates/skald-core/src/skills/:
inventory, install, validate, watch), injected through the new
<!-- SKILLS_LIST --> placeholder in AGENT.md (agents/common/skills.md);
meta.json inject_skills flag removed. 11 chat/task agents carry the
include, the 4 system agents do not.
- Two trees, both read-only in both directions: skills/shared/{id} (the
group's) and skills/{username}/{id} (one member's own, on the stable
userid). The root is closed too: UserFs::SkillMounts + RouteError (alias
probe, plain-denied paths, no home fallback) and a per-user
.skills-root/{userid} container mount with the two scope mounts nested
inside, plus the fifth self-heal axis (skills_mounted).
- Agent verbs: skill_register/skill_delete (Config group, global scope
behind the new skill.manage capability), fetch_repo for public repos,
list_items(type="skills"); reads are plain read_file on the printed
path. Seeded @fs_read skills/* allow.
- Freshness: a digest-gated watcher on the two trees emits
SystemEvent::SkillsChanged, whose subscriber rebuilds the frozen prompt
prefix via Skald::invalidate_prompt_prefix; in-process writers invalidate
directly.
- The build ships no skills: the three bundled skills (ics2json,
mcp-builder, skill-creator) and skills/index.md are removed, skills/ is
instance data (gitignored, not packaged, no longer pruned by update.sh).
- Docs: skills.md, agents.md, shared-folders.md added; docs/index.md and
agents/README.md updated.
This commit is contained in:
Daniele
@@ -0,0 +1,437 @@
|
||||
//! The one door into the two read-only trees (blueprint §7.3/§9).
|
||||
//!
|
||||
//! Everything here exists because a skill is an **immutable, validated
|
||||
//! artefact**: it is copied in whole or not at all, it is never edited in place,
|
||||
//! and modifying one means registering it again. The tree therefore never holds
|
||||
//! a half-written skill, which is what lets the index (`super::list`) read it
|
||||
//! without tolerating intermediate states.
|
||||
//!
|
||||
//! Two mechanics carry that promise and neither is optional:
|
||||
//!
|
||||
//! - **Staging plus a rename**, never a copy in place, so the indexer cannot
|
||||
//! observe a directory being filled.
|
||||
//! - **Three steps on replacement**, because `rename` over a non-empty directory
|
||||
//! fails: move the old one aside, move the new one in, delete the old. Not
|
||||
//! atomic in the strict sense — but the uncovered window contains only a state
|
||||
//! where the id *does not exist*, never one where it exists half-written, and
|
||||
//! that is the property the indexer actually needs.
|
||||
|
||||
use std::path::Path;
|
||||
|
||||
use anyhow::{Context, Result, bail};
|
||||
use core_api::user_fs::UserFs;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use super::validate::{ValidSkill, validate_dir};
|
||||
use super::{SKILL_FILE, Scope};
|
||||
|
||||
/// The provenance ticket a fetched skill carries, written next to its files.
|
||||
///
|
||||
/// The seam between two tools that deliberately know nothing about each other:
|
||||
/// `fetch_repo` (blueprint §7.5, session 4) downloads without knowing what it
|
||||
/// downloaded, `skill_register` installs without knowing where it came from, and
|
||||
/// this file is what crosses between them. `git clone` leaves nothing traceable,
|
||||
/// so without it neither "where is this skill from?" nor "has it changed
|
||||
/// upstream?" has an answer later.
|
||||
///
|
||||
/// It is *pinning*, not authenticity — whoever serves the repository serves the
|
||||
/// commit too — the same honesty the marketplace states about its digests.
|
||||
pub const SOURCE_FILE: &str = ".source.json";
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Provenance {
|
||||
pub url: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub sub_path: Option<String>,
|
||||
/// The commit the files were taken from — the field that makes a later
|
||||
/// upstream change *detectable*.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub commit: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub fetched_at: Option<String>,
|
||||
/// Stamped by [`install`], so the ticket answers "since when is this here?"
|
||||
/// as well as "where from?".
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub installed_at: Option<String>,
|
||||
}
|
||||
|
||||
/// Reads the ticket a source folder carries, if it carries one. A malformed one
|
||||
/// is *ignored*, never fatal: provenance is metadata about the skill, and losing
|
||||
/// it must not stop an otherwise valid installation.
|
||||
pub fn read_provenance(dir: &Path) -> Option<Provenance> {
|
||||
let raw = std::fs::read_to_string(dir.join(SOURCE_FILE)).ok()?;
|
||||
serde_json::from_str(&raw).ok()
|
||||
}
|
||||
|
||||
/// What an installation did, for the tool's answer to the model.
|
||||
pub struct Installed {
|
||||
pub id: String,
|
||||
/// The agent path of the installed folder (`skills/shared/ics-import`).
|
||||
pub agent_dir: String,
|
||||
/// Whether it took the place of a skill with the same id in the same scope.
|
||||
pub replaced: bool,
|
||||
}
|
||||
|
||||
/// The host directory backing one scope of this user's tree.
|
||||
pub fn tree_of(fs: &UserFs, scope: Scope) -> Result<&Path> {
|
||||
let sk = fs.skills.as_ref().ok_or_else(|| {
|
||||
anyhow::anyhow!("skills are not available in this context")
|
||||
})?;
|
||||
Ok(match scope {
|
||||
Scope::Shared => sk.shared_host.as_path(),
|
||||
Scope::Own => sk.own_host.as_path(),
|
||||
})
|
||||
}
|
||||
|
||||
/// The agent-visible scope segment (`shared`, or the caller's username).
|
||||
pub fn segment_of(fs: &UserFs, scope: Scope) -> Result<&str> {
|
||||
let sk = fs.skills.as_ref().ok_or_else(|| {
|
||||
anyhow::anyhow!("skills are not available in this context")
|
||||
})?;
|
||||
Ok(match scope {
|
||||
Scope::Shared => core_api::user_fs::SKILLS_SHARED_SCOPE,
|
||||
Scope::Own => sk.own_username.as_str(),
|
||||
})
|
||||
}
|
||||
|
||||
/// Validates `source` and installs it into `scope`, replacing an existing skill
|
||||
/// of the same id in that scope.
|
||||
///
|
||||
/// The source is copied *before* anything at the destination moves, so
|
||||
/// registering a skill onto itself (`skill_register("global",
|
||||
/// "skills/daniele/foo")` — the promotion path, which is deliberately the same
|
||||
/// call rather than an endpoint of its own) needs no special case.
|
||||
pub fn install(fs: &UserFs, scope: Scope, source: &Path) -> Result<Installed> {
|
||||
let valid = validate_dir(source)?;
|
||||
install_validated(fs, scope, source, &valid)
|
||||
}
|
||||
|
||||
fn install_validated(
|
||||
fs: &UserFs,
|
||||
scope: Scope,
|
||||
source: &Path,
|
||||
valid: &ValidSkill,
|
||||
) -> Result<Installed> {
|
||||
let tree = tree_of(fs, scope)?.to_path_buf();
|
||||
std::fs::create_dir_all(&tree)
|
||||
.with_context(|| format!("cannot open the skills tree at {}", tree.display()))?;
|
||||
|
||||
let target = tree.join(&valid.id);
|
||||
let replaced = target.exists();
|
||||
let tag = uuid::Uuid::new_v4().simple().to_string();
|
||||
// Staging lives **inside the destination tree**: `rename` only works within
|
||||
// one filesystem, and a dot-directory is skipped by the indexer (see
|
||||
// `super::collect`), so a crash mid-copy leaves litter, never a skill.
|
||||
let staging = tree.join(format!(".staging-{tag}"));
|
||||
|
||||
let outcome = (|| -> Result<()> {
|
||||
copy_tree(source, &staging)?;
|
||||
stamp_provenance(source, &staging);
|
||||
if replaced {
|
||||
let parked = tree.join(format!(".old-{tag}"));
|
||||
std::fs::rename(&target, &parked)
|
||||
.with_context(|| format!("cannot replace the installed `{}`", valid.id))?;
|
||||
// From here the id does not exist. If the second rename fails we put
|
||||
// the old one back rather than leave the scope short of a skill it
|
||||
// had a moment ago.
|
||||
if let Err(e) = std::fs::rename(&staging, &target) {
|
||||
let _ = std::fs::rename(&parked, &target);
|
||||
return Err(anyhow::Error::new(e)
|
||||
.context(format!("cannot install `{}`", valid.id)));
|
||||
}
|
||||
let _ = std::fs::remove_dir_all(&parked);
|
||||
} else {
|
||||
std::fs::rename(&staging, &target)
|
||||
.with_context(|| format!("cannot install `{}`", valid.id))?;
|
||||
}
|
||||
Ok(())
|
||||
})();
|
||||
|
||||
if outcome.is_err() {
|
||||
let _ = std::fs::remove_dir_all(&staging);
|
||||
}
|
||||
outcome?;
|
||||
|
||||
Ok(Installed {
|
||||
id: valid.id.clone(),
|
||||
agent_dir: format!(
|
||||
"{}/{}/{}",
|
||||
core_api::user_fs::SKILLS_ROOT,
|
||||
segment_of(fs, scope)?,
|
||||
valid.id
|
||||
),
|
||||
replaced,
|
||||
})
|
||||
}
|
||||
|
||||
/// Carries the provenance ticket across, stamping the install date. Best-effort
|
||||
/// for the same reason [`read_provenance`] is tolerant: this is a label on the
|
||||
/// artefact, not part of it.
|
||||
fn stamp_provenance(source: &Path, staging: &Path) {
|
||||
let Some(mut p) = read_provenance(source) else { return };
|
||||
p.installed_at = Some(chrono::Utc::now().to_rfc3339());
|
||||
if let Ok(json) = serde_json::to_string_pretty(&p) {
|
||||
let _ = std::fs::write(staging.join(SOURCE_FILE), json);
|
||||
}
|
||||
}
|
||||
|
||||
/// Copies a validated tree, refusing links again on the way.
|
||||
///
|
||||
/// The re-check is not redundant paranoia about the walk in `validate`: the
|
||||
/// source folder is writable by the caller and by their container, so between
|
||||
/// the two passes it can change. The cheap answer is to never follow a link at
|
||||
/// either point.
|
||||
///
|
||||
/// Shared with `fetch_repo`, whose staging area is writable by the caller's
|
||||
/// container for exactly as long and so needs exactly the same guarantee.
|
||||
pub(crate) fn copy_tree(from: &Path, to: &Path) -> Result<()> {
|
||||
std::fs::create_dir_all(to)
|
||||
.with_context(|| format!("cannot create {}", to.display()))?;
|
||||
for entry in std::fs::read_dir(from)
|
||||
.with_context(|| format!("cannot read {}", from.display()))?
|
||||
{
|
||||
let entry = entry?;
|
||||
let src = entry.path();
|
||||
let dst = to.join(entry.file_name());
|
||||
let meta = std::fs::symlink_metadata(&src)?;
|
||||
if meta.file_type().is_symlink() {
|
||||
bail!("`{}` is a symbolic link", entry.file_name().to_string_lossy());
|
||||
}
|
||||
if meta.is_dir() {
|
||||
copy_tree(&src, &dst)?;
|
||||
} else {
|
||||
std::fs::copy(&src, &dst)
|
||||
.with_context(|| format!("cannot copy {}", src.display()))?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Removes an installed skill. No recycle bin, deliberately: the source it was
|
||||
/// registered from almost always still exists, and a bin would be a second tree
|
||||
/// to index, mount and explain.
|
||||
pub fn remove(fs: &UserFs, scope: Scope, id: &str) -> Result<()> {
|
||||
let tree = tree_of(fs, scope)?;
|
||||
// The id names a directory *inside* the tree and nothing else — a `/` or a
|
||||
// `..` here would be a path, and paths are not ids.
|
||||
if id.is_empty() || id.contains('/') || id.contains('\\') || id.starts_with('.') {
|
||||
bail!("`{id}` is not a skill id (it is the folder name, e.g. `ics-import`)");
|
||||
}
|
||||
let dir = tree.join(id);
|
||||
if !dir.is_dir() {
|
||||
bail!(
|
||||
"no skill `{id}` in {}/{}/",
|
||||
core_api::user_fs::SKILLS_ROOT,
|
||||
segment_of(fs, scope)?
|
||||
);
|
||||
}
|
||||
std::fs::remove_dir_all(&dir).with_context(|| format!("cannot remove `{id}`"))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// ── The approval card ─────────────────────────────────────────────────────────
|
||||
|
||||
/// What the human is shown before a registration goes through: `(old, new)` for
|
||||
/// the diff card the write tools already use.
|
||||
///
|
||||
/// This is the review moment blueprint §9.1 calls the point of the whole design
|
||||
/// — for the group's scope it is the **only** time a person reads a text that
|
||||
/// will enter everybody's prompt — so `new` is the candidate's `SKILL.md` in
|
||||
/// full, not a summary of it. When the id already exists, `old` is the installed
|
||||
/// body, which turns the card into a diff of what actually changes.
|
||||
///
|
||||
/// `None` when the source cannot be read or does not validate: the card then
|
||||
/// falls back to the generic approval event, and the refusal comes from the tool
|
||||
/// itself with its own message.
|
||||
pub fn preview(fs: &UserFs, scope: Scope, source: &Path) -> Option<(String, Option<String>, String)> {
|
||||
let valid = validate_dir(source).ok()?;
|
||||
let tree = tree_of(fs, scope).ok()?;
|
||||
let target = tree.join(&valid.id);
|
||||
let old = std::fs::read_to_string(target.join(SKILL_FILE)).ok();
|
||||
|
||||
let agent_dir = format!(
|
||||
"{}/{}/{}",
|
||||
core_api::user_fs::SKILLS_ROOT,
|
||||
segment_of(fs, scope).ok()?,
|
||||
valid.id
|
||||
);
|
||||
let header = card_header(&valid, scope, old.is_some(), &agent_dir);
|
||||
let body = std::fs::read_to_string(source.join(SKILL_FILE)).ok()?;
|
||||
|
||||
Some((
|
||||
agent_dir,
|
||||
old.map(|o| format!("{}\n{o}", card_header(&valid, scope, true, ""))),
|
||||
format!("{header}\n{body}"),
|
||||
))
|
||||
}
|
||||
|
||||
fn card_header(valid: &ValidSkill, scope: Scope, replacing: bool, _dir: &str) -> String {
|
||||
let what = if replacing { "REPLACES the installed skill" } else { "new skill" };
|
||||
let audience = match scope {
|
||||
Scope::Shared => "the whole group — every member reads it as instructions",
|
||||
Scope::Own => "you only",
|
||||
};
|
||||
let deps = valid.deps().unwrap_or_else(|| "none".into());
|
||||
format!(
|
||||
"<!-- {what}: `{}` · visible to {audience}\n \
|
||||
{} files, {} KB · scripts: {} · dependency manifest: {deps}\n \
|
||||
files: {} -->\n",
|
||||
valid.id,
|
||||
valid.files.len(),
|
||||
valid.size_bytes.div_ceil(1024),
|
||||
if valid.has_scripts() { "yes" } else { "no" },
|
||||
valid
|
||||
.files
|
||||
.iter()
|
||||
.map(|f| f.to_string_lossy().into_owned())
|
||||
.collect::<Vec<_>>()
|
||||
.join(", "),
|
||||
)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::skills::tests_support::Tree;
|
||||
|
||||
fn front(name: &str, description: &str) -> String {
|
||||
format!("---\nname: {name}\ndescription: {description}\n---\n\nBody of {name}.\n")
|
||||
}
|
||||
|
||||
/// The installed folder is named by the frontmatter, and the index picks it
|
||||
/// up straight away.
|
||||
#[test]
|
||||
fn a_draft_folder_installs_under_its_declared_name() {
|
||||
let t = Tree::new("install-name", "daniele");
|
||||
let src = t.root.join("homes/u1/draft-2");
|
||||
std::fs::create_dir_all(&src).unwrap();
|
||||
std::fs::write(src.join(SKILL_FILE), front("ics-import", "Import an ICS feed.")).unwrap();
|
||||
|
||||
let got = install(&t.fs, Scope::Own, &src).unwrap();
|
||||
assert_eq!(got.id, "ics-import");
|
||||
assert_eq!(got.agent_dir, "skills/daniele/ics-import");
|
||||
assert!(!got.replaced);
|
||||
|
||||
let listed = crate::skills::list(&t.fs);
|
||||
assert_eq!(listed.len(), 1);
|
||||
assert_eq!(listed[0].skill_file(), "skills/daniele/ics-import/SKILL.md");
|
||||
}
|
||||
|
||||
/// Re-registering the same id replaces it, and nothing of the old copy
|
||||
/// survives — the artefact is whole or absent, never merged.
|
||||
#[test]
|
||||
fn re_registering_replaces_without_leaving_the_old_files() {
|
||||
let t = Tree::new("install-replace", "daniele");
|
||||
let src = t.root.join("homes/u1/v1");
|
||||
std::fs::create_dir_all(&src).unwrap();
|
||||
std::fs::write(src.join(SKILL_FILE), front("x", "First.")).unwrap();
|
||||
std::fs::write(src.join("old-helper.py"), "1").unwrap();
|
||||
install(&t.fs, Scope::Own, &src).unwrap();
|
||||
|
||||
let src2 = t.root.join("homes/u1/v2");
|
||||
std::fs::create_dir_all(&src2).unwrap();
|
||||
std::fs::write(src2.join(SKILL_FILE), front("x", "Second.")).unwrap();
|
||||
let got = install(&t.fs, Scope::Own, &src2).unwrap();
|
||||
|
||||
assert!(got.replaced);
|
||||
let installed = t.root.join("skills-users/u1/x");
|
||||
assert!(!installed.join("old-helper.py").exists());
|
||||
assert_eq!(crate::skills::list(&t.fs)[0].description, "Second.");
|
||||
// No staging or parked leftovers.
|
||||
let stray: Vec<_> = std::fs::read_dir(t.root.join("skills-users/u1"))
|
||||
.unwrap()
|
||||
.filter_map(|e| e.ok())
|
||||
.filter(|e| e.file_name().to_string_lossy().starts_with('.'))
|
||||
.collect();
|
||||
assert!(stray.is_empty(), "leftovers: {stray:?}");
|
||||
}
|
||||
|
||||
/// Promotion is the same call with a different scope, and its source is the
|
||||
/// already-installed copy — which the copy-first ordering makes safe.
|
||||
#[test]
|
||||
fn promoting_ones_own_skill_to_the_group_is_the_same_call() {
|
||||
let t = Tree::new("install-promote", "daniele");
|
||||
let src = t.root.join("homes/u1/draft");
|
||||
std::fs::create_dir_all(&src).unwrap();
|
||||
std::fs::write(src.join(SKILL_FILE), front("x", "Mine.")).unwrap();
|
||||
install(&t.fs, Scope::Own, &src).unwrap();
|
||||
|
||||
let mine = t.root.join("skills-users/u1/x");
|
||||
install(&t.fs, Scope::Shared, &mine).unwrap();
|
||||
|
||||
let ids: Vec<(String, String)> = crate::skills::list(&t.fs)
|
||||
.into_iter()
|
||||
.map(|s| (s.scope, s.id))
|
||||
.collect();
|
||||
assert_eq!(
|
||||
ids,
|
||||
vec![("shared".into(), "x".into()), ("daniele".into(), "x".into())]
|
||||
);
|
||||
}
|
||||
|
||||
/// A refused source leaves the tree exactly as it was — the validation runs
|
||||
/// before a single byte is copied.
|
||||
#[test]
|
||||
fn an_invalid_source_touches_nothing() {
|
||||
let t = Tree::new("install-invalid", "daniele");
|
||||
let src = t.root.join("homes/u1/broken");
|
||||
std::fs::create_dir_all(&src).unwrap();
|
||||
std::fs::write(src.join("notes.txt"), "no frontmatter here").unwrap();
|
||||
|
||||
assert!(install(&t.fs, Scope::Own, &src).is_err());
|
||||
assert!(crate::skills::list(&t.fs).is_empty());
|
||||
assert_eq!(std::fs::read_dir(t.root.join("skills-users/u1")).unwrap().count(), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_provenance_ticket_crosses_into_the_installed_copy() {
|
||||
let t = Tree::new("install-prov", "daniele");
|
||||
let src = t.root.join("homes/u1/fetched");
|
||||
std::fs::create_dir_all(&src).unwrap();
|
||||
std::fs::write(src.join(SKILL_FILE), front("x", "y")).unwrap();
|
||||
std::fs::write(
|
||||
src.join(SOURCE_FILE),
|
||||
r#"{"url":"https://example.invalid/r","commit":"a1b2c3d"}"#,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
install(&t.fs, Scope::Own, &src).unwrap();
|
||||
let p = read_provenance(&t.root.join("skills-users/u1/x")).unwrap();
|
||||
assert_eq!(p.commit.as_deref(), Some("a1b2c3d"));
|
||||
assert!(p.installed_at.is_some(), "install date not stamped");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delete_removes_the_folder_and_refuses_a_path() {
|
||||
let t = Tree::new("install-delete", "daniele");
|
||||
let src = t.root.join("homes/u1/d");
|
||||
std::fs::create_dir_all(&src).unwrap();
|
||||
std::fs::write(src.join(SKILL_FILE), front("x", "y")).unwrap();
|
||||
install(&t.fs, Scope::Own, &src).unwrap();
|
||||
|
||||
assert!(remove(&t.fs, Scope::Own, "../../etc").is_err());
|
||||
assert!(remove(&t.fs, Scope::Own, "nope").is_err());
|
||||
remove(&t.fs, Scope::Own, "x").unwrap();
|
||||
assert!(crate::skills::list(&t.fs).is_empty());
|
||||
}
|
||||
|
||||
/// The card shows the body that will enter the prompt, and on a replacement
|
||||
/// it shows the one being replaced — so the human reads a diff, not a name.
|
||||
#[test]
|
||||
fn the_card_carries_the_whole_skill_body() {
|
||||
let t = Tree::new("install-card", "daniele");
|
||||
let src = t.root.join("homes/u1/c");
|
||||
std::fs::create_dir_all(&src).unwrap();
|
||||
std::fs::write(src.join(SKILL_FILE), front("x", "y")).unwrap();
|
||||
|
||||
let (path, old, new) = preview(&t.fs, Scope::Shared, &src).unwrap();
|
||||
assert_eq!(path, "skills/shared/x");
|
||||
assert!(old.is_none());
|
||||
assert!(new.contains("Body of x."), "{new}");
|
||||
assert!(new.contains("every member reads it as instructions"), "{new}");
|
||||
|
||||
install(&t.fs, Scope::Shared, &src).unwrap();
|
||||
let (_, old, _) = preview(&t.fs, Scope::Shared, &src).unwrap();
|
||||
assert!(old.unwrap().contains("Body of x."));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,183 @@
|
||||
//! The administrative view of the two trees — what `list_items(type="skills")`
|
||||
//! returns (blueprint §7.7).
|
||||
//!
|
||||
//! **Not the index, and the distinction is worth keeping sharp.** The index is
|
||||
//! for *deciding*: path plus a cut description, the minimum needed to tell
|
||||
//! whether a skill bears on the request, injected into every prompt and paid for
|
||||
//! in tokens on every request of every user. This is for *administering*: the
|
||||
//! full description, size, health and provenance, as JSON, only when asked. One
|
||||
//! is always there and thin; the other is on demand and complete.
|
||||
//!
|
||||
//! Two consequences fall out of that split:
|
||||
//!
|
||||
//! - The **description is not truncated here.** Truncate in both places and the
|
||||
//! full text becomes unreadable anywhere, while this tool exists precisely to
|
||||
//! be the place it can be read. The real ceiling belongs at registration
|
||||
//! ([`super::validate::DESCRIPTION_MAX`]), where the author is present and the
|
||||
//! refusal is useful.
|
||||
//! - **A broken skill appears here**, with the reason. The index skips it —
|
||||
//! correctly, since it cannot be trusted to describe itself — but then nothing
|
||||
//! would say *why* a folder placed on the box never showed up.
|
||||
|
||||
use std::path::Path;
|
||||
|
||||
use core_api::user_fs::{SKILLS_ROOT, SKILLS_SHARED_SCOPE, UserFs};
|
||||
use serde_json::{Value, json};
|
||||
|
||||
use super::install::read_provenance;
|
||||
use super::validate::validate_dir;
|
||||
use super::{DESCRIPTION_LIMIT, SKILL_FILE, parse_front_matter};
|
||||
|
||||
/// Every skill folder visible to this user — valid or not — as JSON rows, in the
|
||||
/// index's order (group's tree first, then their own, each by id).
|
||||
pub fn report(fs: &UserFs) -> Vec<Value> {
|
||||
let Some(sk) = &fs.skills else { return Vec::new() };
|
||||
|
||||
let mut rows: Vec<(String, String, Value)> = Vec::new();
|
||||
for (scope, host) in [
|
||||
(SKILLS_SHARED_SCOPE, sk.shared_host.as_path()),
|
||||
(sk.own_username.as_str(), sk.own_host.as_path()),
|
||||
] {
|
||||
let Ok(entries) = std::fs::read_dir(host) else { continue };
|
||||
let mut ids: Vec<String> = entries
|
||||
.flatten()
|
||||
.filter(|e| e.path().is_dir())
|
||||
.filter_map(|e| e.file_name().to_str().map(str::to_string))
|
||||
// Dot-directories are plumbing (a staging leftover), never a skill.
|
||||
.filter(|id| !id.starts_with('.'))
|
||||
.collect();
|
||||
ids.sort();
|
||||
for id in ids {
|
||||
let row = inspect(scope, &host.join(&id), &id);
|
||||
rows.push((id, scope.to_string(), row));
|
||||
}
|
||||
}
|
||||
|
||||
// A collision is a property of the *pair*, so it can only be marked once
|
||||
// both trees have been read — the same reason the index marks both lines.
|
||||
let colliding: std::collections::HashSet<String> = rows
|
||||
.iter()
|
||||
.filter(|(id, scope, _)| rows.iter().any(|(o, os, _)| o == id && os != scope))
|
||||
.map(|(id, _, _)| id.clone())
|
||||
.collect();
|
||||
|
||||
rows.into_iter()
|
||||
.map(|(id, _, mut row)| {
|
||||
row["collision"] = Value::Bool(colliding.contains(&id));
|
||||
row
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// One folder, described as fully as it allows itself to be.
|
||||
fn inspect(scope: &str, dir: &Path, id: &str) -> Value {
|
||||
let path = format!("{SKILLS_ROOT}/{scope}/{id}");
|
||||
let mut row = json!({
|
||||
"id": id,
|
||||
"scope": scope,
|
||||
"path": path,
|
||||
"valid": false,
|
||||
"problem": Value::Null,
|
||||
});
|
||||
|
||||
// Validity here means **what the index does**: does its frontmatter parse?
|
||||
// The structural rules (`validate_dir`) are a stricter set — they gate what
|
||||
// may be *installed* — so a folder that fails them but parses is still shown
|
||||
// in the prompt and must be reported as such, with the extra problem named.
|
||||
let body = match std::fs::read_to_string(dir.join(SKILL_FILE)) {
|
||||
Ok(b) => b,
|
||||
Err(e) => {
|
||||
row["problem"] = json!(format!("no readable {SKILL_FILE}: {e}"));
|
||||
return row;
|
||||
}
|
||||
};
|
||||
let front = match parse_front_matter(&body) {
|
||||
Ok(f) => f,
|
||||
Err(problem) => {
|
||||
row["problem"] = json!(format!("invalid frontmatter: {problem}"));
|
||||
return row;
|
||||
}
|
||||
};
|
||||
|
||||
row["valid"] = Value::Bool(true);
|
||||
row["description"] = json!(front.description);
|
||||
row["truncated_in_index"] =
|
||||
Value::Bool(front.description.chars().count() > DESCRIPTION_LIMIT);
|
||||
if front.name != id {
|
||||
row["problem"] = json!(format!(
|
||||
"the frontmatter `name` is `{}` but the folder is `{id}`; the folder wins",
|
||||
front.name
|
||||
));
|
||||
}
|
||||
|
||||
match validate_dir(dir) {
|
||||
Ok(v) => {
|
||||
row["files"] = json!(v.files.len());
|
||||
row["size_bytes"] = json!(v.size_bytes);
|
||||
row["has_scripts"] = Value::Bool(v.has_scripts());
|
||||
row["deps"] = v.deps().map(Value::String).unwrap_or(Value::Null);
|
||||
}
|
||||
// Reachable only for a folder placed by hand: everything installed
|
||||
// through `skill_register` passed this very check.
|
||||
Err(e) => row["problem"] = json!(e.to_string()),
|
||||
}
|
||||
|
||||
if let Some(p) = read_provenance(dir) {
|
||||
row["source_url"] = json!(p.url);
|
||||
row["commit"] = p.commit.map(Value::String).unwrap_or(Value::Null);
|
||||
row["installed_at"] = p.installed_at.map(Value::String).unwrap_or(Value::Null);
|
||||
}
|
||||
row
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::skills::tests_support::{Tree, valid};
|
||||
|
||||
/// The full description survives here even when the index cut it — this is
|
||||
/// the one place it can be read whole.
|
||||
#[test]
|
||||
fn the_description_is_reported_untruncated_and_flagged() {
|
||||
let long = "d".repeat(DESCRIPTION_LIMIT + 40);
|
||||
let t = Tree::new("inv-desc", "daniele");
|
||||
t.write("shared", "x", &valid("x", &long));
|
||||
|
||||
let rows = report(&t.fs);
|
||||
assert_eq!(rows[0]["description"], json!(long));
|
||||
assert_eq!(rows[0]["truncated_in_index"], json!(true));
|
||||
assert_eq!(rows[0]["path"], json!("skills/shared/x"));
|
||||
}
|
||||
|
||||
/// A folder the index skipped still shows up, with the reason — otherwise
|
||||
/// nothing anywhere answers "why did my skill never appear?".
|
||||
#[test]
|
||||
fn a_broken_skill_is_reported_with_its_problem() {
|
||||
let t = Tree::new("inv-broken", "daniele");
|
||||
t.write("shared", "good", &valid("good", "Works."));
|
||||
t.write("shared", "broken", "no frontmatter at all\n");
|
||||
|
||||
let rows = report(&t.fs);
|
||||
assert_eq!(rows.len(), 2);
|
||||
let broken = rows.iter().find(|r| r["id"] == json!("broken")).unwrap();
|
||||
assert_eq!(broken["valid"], json!(false));
|
||||
assert!(broken["problem"].as_str().unwrap().contains("frontmatter"));
|
||||
// …and the index really did skip it, so the two views agree on the facts
|
||||
// while disagreeing on what they show.
|
||||
assert_eq!(crate::skills::list(&t.fs).len(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_colliding_id_is_marked_on_both_rows() {
|
||||
let t = Tree::new("inv-collide", "daniele");
|
||||
t.write("shared", "x", &valid("x", "Group's."));
|
||||
t.write("mine", "x", &valid("x", "Mine."));
|
||||
t.write("mine", "y", &valid("y", "Untouched."));
|
||||
|
||||
let rows = report(&t.fs);
|
||||
for r in &rows {
|
||||
let expect = r["id"] == json!("x");
|
||||
assert_eq!(r["collision"], json!(expect), "{r}");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,656 @@
|
||||
//! The skills index — **the only thing about a skill that reaches the prompt**.
|
||||
//!
|
||||
//! A skill is a folder with a `SKILL.md` in it, living in one of the two trees
|
||||
//! `UserFs` mounts read-only (`skills/shared/<id>` and `skills/<username>/<id>`,
|
||||
//! see [`core_api::user_fs::SkillMounts`]). Nothing here is a manager in the usual
|
||||
//! sense: this module is a set of **pure functions over those two paths**, in the
|
||||
//! shape of `LlmCommandManager` and for the same reason — the list must be a
|
||||
//! function of the content, never a file someone maintains by hand, or it diverges
|
||||
//! at the first skill added.
|
||||
//!
|
||||
//! What reaches the model is deliberately thin (blueprint §5, progressive
|
||||
//! disclosure): the **path** of each `SKILL.md` plus a truncated `description`.
|
||||
//! The body is never injected — the model reads it with `read_file` when the
|
||||
//! description matches. Carrying the full path rather than an id plus a
|
||||
//! composition rule is what makes a dedicated read tool unnecessary: it costs a
|
||||
//! few tokens per line and removes the one step the model can get wrong on its own.
|
||||
//!
|
||||
//! Three properties are load-bearing, and each is here because the alternative
|
||||
//! fails in a specific way:
|
||||
//!
|
||||
//! - **A stable order** (scope, then id). The index sits inside the string every
|
||||
//! provider uses as its cache key, so a non-deterministic order would cost a
|
||||
//! miss on every rebuild.
|
||||
//! - **A deterministic cut at the budget**, closed by an explicit omission line.
|
||||
//! Without the determinism the stable order stops buying a stable cache key;
|
||||
//! without the line the model reads a truncated list *believing it complete* and
|
||||
//! concludes in good faith that a skill does not exist.
|
||||
//! - **A broken skill is skipped, never fatal.** The index is built while
|
||||
//! assembling a system prompt; one malformed frontmatter must not take a
|
||||
//! conversation down with it.
|
||||
|
||||
use std::path::Path;
|
||||
use std::sync::{Arc, OnceLock};
|
||||
|
||||
use core_api::user_fs::{SKILLS_ROOT, SKILLS_SHARED_SCOPE, UserFs};
|
||||
use tracing::warn;
|
||||
|
||||
pub mod install;
|
||||
pub mod inventory;
|
||||
pub mod validate;
|
||||
pub mod watch;
|
||||
|
||||
/// The file that makes a directory a skill.
|
||||
pub const SKILL_FILE: &str = "SKILL.md";
|
||||
|
||||
/// Which of a user's two trees an operation addresses.
|
||||
///
|
||||
/// The tools take `"mine"` / `"global"` rather than a username, and that is
|
||||
/// deliberate (blueprint §4.1): a tool argument naming the caller invites
|
||||
/// passing somebody *else's* name, which the server would then have to ignore.
|
||||
/// Human-readable path, stable argument for the machine.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum Scope {
|
||||
/// `skills/{username}/…` — the caller's own.
|
||||
Own,
|
||||
/// `skills/shared/…` — the group's, gated by the `skill.manage` capability.
|
||||
Shared,
|
||||
}
|
||||
|
||||
impl Scope {
|
||||
pub fn parse(raw: &str) -> anyhow::Result<Self> {
|
||||
match raw {
|
||||
"mine" => Ok(Scope::Own),
|
||||
"global" => Ok(Scope::Shared),
|
||||
other => anyhow::bail!(
|
||||
"unknown scope `{other}`: use \"mine\" (your own skills) or \"global\" \
|
||||
(the whole group's)"
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
/// The spelling the tools take and the model reads back.
|
||||
pub fn as_arg(self) -> &'static str {
|
||||
match self {
|
||||
Scope::Own => "mine",
|
||||
Scope::Shared => "global",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ── Prompt freshness (blueprint §6) ──────────────────────────────────────────
|
||||
|
||||
/// Whose system prompts a skills change has made stale.
|
||||
///
|
||||
/// Distinct from [`Scope`], which says *where a write went*: a write to the
|
||||
/// group's tree makes every live member's prompt stale, a write to one member's
|
||||
/// own tree makes only theirs. Session 5's `SkillsChanged` event carries the
|
||||
/// same shape, for the same reason.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum PromptScope {
|
||||
/// The group's tree changed: everyone's index moved.
|
||||
Everyone,
|
||||
/// One member's own tree changed.
|
||||
User(String),
|
||||
}
|
||||
|
||||
/// How a skills write reaches conversations that are already running.
|
||||
///
|
||||
/// The index sits inside the frozen system prefix, which is rebuilt only after
|
||||
/// twenty idle minutes ([`crate::loop_adapters::prefix_cache`]). That is right
|
||||
/// for a file edited underneath a running conversation and wrong here: an admin
|
||||
/// who installs a skill and then asks the assistant to use it must not be told
|
||||
/// for twenty minutes that it does not exist.
|
||||
///
|
||||
/// **Not on the system bus.** `SkillsChanged` (session 5) is for a change made
|
||||
/// *outside* the process, by someone editing files on the box; a write made by a
|
||||
/// tool is already inside the process and can say so directly, which is both
|
||||
/// immediate and impossible to lose. The bus stays for the case it was designed
|
||||
/// for.
|
||||
#[async_trait::async_trait]
|
||||
pub trait PromptPrefixes: Send + Sync {
|
||||
async fn invalidate(&self, scope: PromptScope);
|
||||
}
|
||||
|
||||
/// The cell the skill tools hold, filled once the instance exists.
|
||||
///
|
||||
/// The tools are built during composition, before `Skald` does; the reactor they
|
||||
/// need can therefore only be installed afterwards — the same post-construction
|
||||
/// shape as the plugin manager's `set_skald` and the user-lifecycle reconciler.
|
||||
/// An empty cell is a silent no-op rather than an error: the only way to reach
|
||||
/// it is a `Skald` that failed to finish building, in which case there are no
|
||||
/// live conversations to keep fresh either.
|
||||
#[derive(Default)]
|
||||
pub struct PromptPrefixCell(OnceLock<Arc<dyn PromptPrefixes>>);
|
||||
|
||||
impl PromptPrefixCell {
|
||||
pub fn install(&self, sink: Arc<dyn PromptPrefixes>) {
|
||||
let _ = self.0.set(sink);
|
||||
}
|
||||
|
||||
pub async fn invalidate(&self, scope: PromptScope) {
|
||||
if let Some(sink) = self.0.get() {
|
||||
sink.invalidate(scope).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// How much of a `description` the index carries. The full text lives on disk and
|
||||
/// is surfaced by the enumeration tool; this cap applies **only** to the index,
|
||||
/// which is the one place tokens are paid on every request of every user.
|
||||
///
|
||||
/// Hermes cuts at 60. Sixty is too few here: the `description` *is* the use
|
||||
/// condition ("when should I reach for this?"), and cutting mid-sentence removes
|
||||
/// exactly the part that decides the trigger.
|
||||
pub const DESCRIPTION_LIMIT: usize = 200;
|
||||
|
||||
/// Ceiling on the whole rendered index, in bytes. A badly written skill must not
|
||||
/// be able to eat the prompt of everybody in the house.
|
||||
pub const INDEX_BUDGET: usize = 8 * 1024;
|
||||
|
||||
/// Room kept aside for the omission line, so appending it can never push the
|
||||
/// render past [`INDEX_BUDGET`]. A fixed reserve rather than a computed one keeps
|
||||
/// the cut a pure function of the ordered list.
|
||||
const OMISSION_RESERVE: usize = 96;
|
||||
|
||||
/// One installed skill, as the index sees it: where it is and when to reach for
|
||||
/// it. The body never enters this type — it is read from disk, by the model.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Skill {
|
||||
/// The directory name, which **is** the id.
|
||||
pub id: String,
|
||||
/// The agent-visible scope segment: `shared`, or the owner's username.
|
||||
pub scope: String,
|
||||
/// The frontmatter `description`, verbatim and untruncated.
|
||||
pub description: String,
|
||||
}
|
||||
|
||||
impl Skill {
|
||||
/// The skill's folder, in agent vocabulary (`skills/shared/ics-import`).
|
||||
pub fn agent_dir(&self) -> String {
|
||||
format!("{SKILLS_ROOT}/{}/{}", self.scope, self.id)
|
||||
}
|
||||
|
||||
/// The path the index prints — the `SKILL.md` itself, ready for `read_file`.
|
||||
pub fn skill_file(&self) -> String {
|
||||
format!("{}/{SKILL_FILE}", self.agent_dir())
|
||||
}
|
||||
}
|
||||
|
||||
/// Every skill visible to this user, in the index's stable order: the group's
|
||||
/// tree first, then their own, each sorted by id.
|
||||
///
|
||||
/// Per-user by construction — the own tree comes from `fs.skills`, which is built
|
||||
/// for one member — so another member's private skills cannot appear here. A
|
||||
/// `UserFs` without the skills tree (an inert placeholder, a unit test) simply has
|
||||
/// none.
|
||||
pub fn list(fs: &UserFs) -> Vec<Skill> {
|
||||
let Some(sk) = &fs.skills else { return Vec::new() };
|
||||
let mut out = Vec::new();
|
||||
collect(SKILLS_SHARED_SCOPE, &sk.shared_host, &mut out);
|
||||
collect(&sk.own_username, &sk.own_host, &mut out);
|
||||
out
|
||||
}
|
||||
|
||||
/// Reads one scope's tree, appending its valid skills in id order. A tree that
|
||||
/// does not exist yet is not an error: it is the state of every fresh instance.
|
||||
fn collect(scope: &str, host: &Path, out: &mut Vec<Skill>) {
|
||||
let Ok(entries) = std::fs::read_dir(host) else { return };
|
||||
|
||||
let mut found: Vec<Skill> = Vec::new();
|
||||
for entry in entries.flatten() {
|
||||
let path = entry.path();
|
||||
if !path.is_dir() {
|
||||
continue;
|
||||
}
|
||||
let Some(id) = path.file_name().and_then(|n| n.to_str()) else { continue };
|
||||
// Dot-directories are plumbing (a staging leftover, an editor's cruft),
|
||||
// never a skill: an id starting with a dot cannot be registered.
|
||||
if id.starts_with('.') {
|
||||
continue;
|
||||
}
|
||||
|
||||
let body = match std::fs::read_to_string(path.join(SKILL_FILE)) {
|
||||
Ok(b) => b,
|
||||
Err(e) => {
|
||||
warn!(scope, skill = id, error = %e, "skill skipped: no readable SKILL.md");
|
||||
continue;
|
||||
}
|
||||
};
|
||||
let front = match parse_front_matter(&body) {
|
||||
Ok(f) => f,
|
||||
Err(problem) => {
|
||||
warn!(scope, skill = id, problem, "skill skipped: invalid frontmatter");
|
||||
continue;
|
||||
}
|
||||
};
|
||||
// The id is the directory, always — that is what every path in the index
|
||||
// is built from. A `name` that disagrees is worth saying out loud (the
|
||||
// registration tool makes the two agree by construction, so this can only
|
||||
// be a folder placed by hand) but not worth hiding the skill over.
|
||||
if front.name != id {
|
||||
warn!(
|
||||
scope, skill = id, declared = front.name,
|
||||
"skill frontmatter `name` differs from its directory; the directory wins"
|
||||
);
|
||||
}
|
||||
found.push(Skill { id: id.to_string(), scope: scope.to_string(), description: front.description });
|
||||
}
|
||||
|
||||
found.sort_by(|a, b| a.id.cmp(&b.id));
|
||||
out.extend(found);
|
||||
}
|
||||
|
||||
/// The two mandatory frontmatter fields. Unknown keys (`license`,
|
||||
/// `allowed-tools`, anything a skill written elsewhere carries) are ignored.
|
||||
#[derive(serde::Deserialize)]
|
||||
pub(crate) struct FrontMatter {
|
||||
#[serde(default)]
|
||||
pub(crate) name: String,
|
||||
#[serde(default)]
|
||||
pub(crate) description: String,
|
||||
}
|
||||
|
||||
/// Parses the leading `---` YAML block of a `SKILL.md`. `Err` carries a short
|
||||
/// reason, which the caller logs — this is the one place a hand-edited skill goes
|
||||
/// wrong, so the log line has to say which of the three ways it did.
|
||||
///
|
||||
/// Shared with [`validate`] on purpose: two frontmatter parsers would agree on
|
||||
/// the easy cases and diverge on the first odd one, and the pair that must never
|
||||
/// disagree is exactly *what the registration accepts* and *what the index
|
||||
/// shows* — a skill installed but invisible is the worst of both.
|
||||
pub(crate) fn parse_front_matter(body: &str) -> Result<FrontMatter, &'static str> {
|
||||
let rest = body
|
||||
.strip_prefix("---\n")
|
||||
.or_else(|| body.strip_prefix("---\r\n"))
|
||||
.ok_or("no frontmatter block")?;
|
||||
let end = rest
|
||||
.split_inclusive('\n')
|
||||
.scan(0usize, |at, line| {
|
||||
let start = *at;
|
||||
*at += line.len();
|
||||
Some((start, line))
|
||||
})
|
||||
.find(|(_, line)| matches!(line.trim_end(), "---" | "..."))
|
||||
.map(|(start, _)| start)
|
||||
.ok_or("unterminated frontmatter block")?;
|
||||
|
||||
let front: FrontMatter = serde_yaml::from_str(&rest[..end]).map_err(|_| "not valid YAML")?;
|
||||
if front.name.trim().is_empty() {
|
||||
return Err("frontmatter has no `name`");
|
||||
}
|
||||
if front.description.trim().is_empty() {
|
||||
return Err("frontmatter has no `description`");
|
||||
}
|
||||
Ok(FrontMatter { name: front.name.trim().to_string(), description: front.description.trim().to_string() })
|
||||
}
|
||||
|
||||
/// The index for one user, ready to replace `__SKILLS_LIST__`.
|
||||
pub fn render_index(fs: &UserFs) -> String {
|
||||
render(&list(fs))
|
||||
}
|
||||
|
||||
/// A stable digest of a rendered index. The invalidation rule of blueprint §6 is
|
||||
/// keyed on **this string changing**, not on a file inside a skill changing:
|
||||
/// editing a script or a reference document leaves the index byte-identical, so it
|
||||
/// costs nobody a cache miss. Only adding, removing or re-describing a skill does.
|
||||
pub fn digest(rendered: &str) -> String {
|
||||
use sha2::{Digest, Sha256};
|
||||
format!("{:x}", Sha256::digest(rendered.as_bytes()))
|
||||
}
|
||||
|
||||
/// A digest of one scope **tree's** visible content — the sorted
|
||||
/// (id, description) pairs, which is everything the index ever prints from it.
|
||||
///
|
||||
/// This is the file-watcher's gate (blueprint §8.2), and the rule is §6's, per
|
||||
/// tree instead of per render: editing a script or a reference document leaves
|
||||
/// it alone; adding, removing or re-describing a skill moves it. A collision
|
||||
/// marker needs no hashing of its own — it is a function of the two id sets,
|
||||
/// and those are hashed. An empty or missing tree digests as [`digest`]`("")`.
|
||||
pub fn tree_digest(host: &Path) -> String {
|
||||
use sha2::{Digest, Sha256};
|
||||
let label = host.file_name().and_then(|n| n.to_str()).unwrap_or("?");
|
||||
let mut found = Vec::new();
|
||||
collect(label, host, &mut found);
|
||||
let mut hasher = Sha256::new();
|
||||
for s in &found {
|
||||
hasher.update(s.id.as_bytes());
|
||||
hasher.update([0]);
|
||||
hasher.update(s.description.as_bytes());
|
||||
hasher.update([0]);
|
||||
}
|
||||
format!("{:x}", hasher.finalize())
|
||||
}
|
||||
|
||||
/// The imperative preamble. Deliberately pushy — the failure mode of every skill
|
||||
/// system is the model *under*-triggering, and a neutral "the following skills are
|
||||
/// available" produces a model that scrolls past them.
|
||||
const HEADER: &str = "\
|
||||
## Skills (mandatory)
|
||||
|
||||
Before replying, scan the skills below. If a skill matches or is even partially \
|
||||
relevant to your task, you MUST read its `SKILL.md` with `read_file` and follow \
|
||||
its instructions. Err on the side of reading it — it is always better to have \
|
||||
context you don't need than to miss critical steps, pitfalls or established \
|
||||
workflows. Skills encode how a task should be done here, so read one even for a \
|
||||
task you already know how to do.
|
||||
|
||||
<available_skills>
|
||||
";
|
||||
|
||||
/// Closing rules. The `workdir` sentence is here, said once, because there is no
|
||||
/// launch tool to hide it in: a skill's scripts are run with the general-purpose
|
||||
/// `execute_cmd`, and a model that runs one from the home gets a bare ENOENT.
|
||||
const FOOTER: &str = "\
|
||||
</available_skills>
|
||||
|
||||
Only proceed without reading a skill if genuinely none are relevant.
|
||||
|
||||
Run a skill's scripts with `execute_cmd`, setting `workdir` to the skill's own \
|
||||
folder. The whole `skills/` tree is read-only: anything a skill needs to write \
|
||||
(caches, state, dependencies) goes in your home or `/tmp`.";
|
||||
|
||||
/// Renders the index from an ordered skill list — pure, so the budget cut and the
|
||||
/// collision marking are testable without a filesystem.
|
||||
///
|
||||
/// **Empty in, empty out**, and that is a contract rather than an optimisation:
|
||||
/// every word of prose lives in here, so an instance with no skills spends nothing
|
||||
/// and leaves no orphan sentence from which the model could infer that something
|
||||
/// exists. (The MCP list is the counter-example — its prose sits *around* the
|
||||
/// placeholder, so an empty list left a promise of a table behind, and the model
|
||||
/// answered by inventing a discovery tool.)
|
||||
pub fn render(skills: &[Skill]) -> String {
|
||||
if skills.is_empty() {
|
||||
return String::new();
|
||||
}
|
||||
|
||||
// An id present in both trees is marked on **both** lines: neither wins in
|
||||
// silence. The personal one winning would be a quiet divergence from the
|
||||
// group's set, the group's one winning would ignore the member's own work.
|
||||
// With full paths in the index the disambiguation is already free — the two
|
||||
// lines differ — so fail-loud costs only the marker.
|
||||
let colliding: std::collections::HashSet<&str> = skills
|
||||
.iter()
|
||||
.filter(|s| skills.iter().any(|o| o.id == s.id && o.scope != s.scope))
|
||||
.map(|s| s.id.as_str())
|
||||
.collect();
|
||||
|
||||
let paths: Vec<String> = skills.iter().map(Skill::skill_file).collect();
|
||||
let width = paths.iter().map(String::len).max().unwrap_or(0);
|
||||
|
||||
let rows: Vec<String> = skills
|
||||
.iter()
|
||||
.zip(&paths)
|
||||
.map(|(s, path)| {
|
||||
let mut desc = truncate(&flatten(&s.description), DESCRIPTION_LIMIT);
|
||||
if colliding.contains(s.id.as_str()) {
|
||||
desc.push_str(" [name collision]");
|
||||
}
|
||||
format!(" {path:<width$} {desc}\n")
|
||||
})
|
||||
.collect();
|
||||
|
||||
let mut out = String::with_capacity(HEADER.len() + FOOTER.len() + 128);
|
||||
out.push_str(HEADER);
|
||||
|
||||
let mut room = INDEX_BUDGET
|
||||
.saturating_sub(HEADER.len() + FOOTER.len() + OMISSION_RESERVE);
|
||||
let mut omitted = 0;
|
||||
for (i, row) in rows.iter().enumerate() {
|
||||
// Stop at the **first** row that does not fit, rather than skipping it and
|
||||
// trying the next: the cut has to be a suffix of the stable order, or two
|
||||
// renders of the same set could keep different skills.
|
||||
if row.len() > room {
|
||||
omitted = rows.len() - i;
|
||||
break;
|
||||
}
|
||||
room -= row.len();
|
||||
out.push_str(row);
|
||||
}
|
||||
if omitted > 0 {
|
||||
warn!(omitted, total = rows.len(), "skills index over budget: tail omitted");
|
||||
out.push_str(&format!(" [{omitted} more skills omitted — index budget reached]\n"));
|
||||
}
|
||||
|
||||
out.push_str(FOOTER);
|
||||
out
|
||||
}
|
||||
|
||||
/// A description as one line: a multi-line one would break the column layout, and
|
||||
/// the index is read as a table.
|
||||
fn flatten(s: &str) -> String {
|
||||
s.split_whitespace().collect::<Vec<_>>().join(" ")
|
||||
}
|
||||
|
||||
/// Truncates to `limit` **characters** (never bytes — this text is user-authored
|
||||
/// and routinely accented), marking the cut so the model knows it read a prefix.
|
||||
fn truncate(s: &str, limit: usize) -> String {
|
||||
if s.chars().count() <= limit {
|
||||
return s.to_string();
|
||||
}
|
||||
let mut out: String = s.chars().take(limit).collect();
|
||||
out.push('…');
|
||||
out
|
||||
}
|
||||
|
||||
/// A temporary `{WD}` with the two scope trees and a `UserFs` over it — shared
|
||||
/// by the index, install and inventory tests, which all need the same fixture
|
||||
/// and would otherwise each grow their own slightly different one.
|
||||
#[cfg(test)]
|
||||
pub(crate) mod tests_support {
|
||||
use super::*;
|
||||
use core_api::user_fs::SkillMounts;
|
||||
use std::path::PathBuf;
|
||||
|
||||
pub(crate) struct Tree {
|
||||
pub(crate) root: PathBuf,
|
||||
pub(crate) fs: UserFs,
|
||||
}
|
||||
|
||||
impl Tree {
|
||||
pub(crate) fn new(tag: &str, username: &str) -> Self {
|
||||
let root = std::env::temp_dir().join(format!(
|
||||
"skald-skills-{tag}-{}-{:?}",
|
||||
std::process::id(),
|
||||
std::thread::current().id()
|
||||
));
|
||||
let _ = std::fs::remove_dir_all(&root);
|
||||
let shared = root.join("skills");
|
||||
let own = root.join("skills-users").join("u1");
|
||||
std::fs::create_dir_all(&shared).unwrap();
|
||||
std::fs::create_dir_all(&own).unwrap();
|
||||
let fs = UserFs::new(
|
||||
"u1",
|
||||
root.join("homes").join("u1"),
|
||||
"skald-u1",
|
||||
PathBuf::from("/root"),
|
||||
vec![],
|
||||
vec![],
|
||||
None,
|
||||
)
|
||||
.with_skills(SkillMounts {
|
||||
root_host: root.join(".skills-root").join("u1"),
|
||||
shared_host: shared,
|
||||
own_host: own,
|
||||
own_username: username.into(),
|
||||
});
|
||||
Self { root, fs }
|
||||
}
|
||||
|
||||
/// Drops a skill straight into a scope tree, the way a hand-placed folder
|
||||
/// on the box arrives — bypassing the registration tool, which these
|
||||
/// tests are deliberately not exercising.
|
||||
pub(crate) fn write(&self, scope: &str, id: &str, body: &str) {
|
||||
let base = match scope {
|
||||
"shared" => self.root.join("skills"),
|
||||
_ => self.root.join("skills-users").join("u1"),
|
||||
};
|
||||
let dir = base.join(id);
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
std::fs::write(dir.join(SKILL_FILE), body).unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for Tree {
|
||||
fn drop(&mut self) {
|
||||
let _ = std::fs::remove_dir_all(&self.root);
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn valid(name: &str, description: &str) -> String {
|
||||
format!("---\nname: {name}\ndescription: {description}\n---\n\nThe body.\n")
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::tests_support::{Tree, valid};
|
||||
use super::*;
|
||||
use std::path::PathBuf;
|
||||
|
||||
fn skill(scope: &str, id: &str, description: &str) -> Skill {
|
||||
Skill { id: id.into(), scope: scope.into(), description: description.into() }
|
||||
}
|
||||
|
||||
/// Both trees are enumerated, the group's first, each in id order — and the
|
||||
/// order is the whole reason: it is inside the provider's cache key.
|
||||
#[test]
|
||||
fn both_scopes_are_listed_in_a_stable_order() {
|
||||
let t = Tree::new("order", "daniele");
|
||||
t.write("shared", "pdf-forms", &valid("pdf-forms", "Fill a PDF form."));
|
||||
t.write("shared", "ics-import", &valid("ics-import", "Import an ICS feed."));
|
||||
t.write("mine", "spesa", &valid("spesa", "Reconcile the statement."));
|
||||
|
||||
let got: Vec<(String, String)> =
|
||||
list(&t.fs).into_iter().map(|s| (s.scope, s.id)).collect();
|
||||
assert_eq!(
|
||||
got,
|
||||
vec![
|
||||
("shared".into(), "ics-import".into()),
|
||||
("shared".into(), "pdf-forms".into()),
|
||||
("daniele".into(), "spesa".into()),
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
/// A skill nobody can parse is skipped; the ones around it are not. The index
|
||||
/// is built while assembling a prompt, so a broken folder must cost its own
|
||||
/// line and nothing else.
|
||||
#[test]
|
||||
fn a_malformed_skill_is_skipped_not_fatal() {
|
||||
let t = Tree::new("malformed", "daniele");
|
||||
t.write("shared", "good", &valid("good", "Works."));
|
||||
t.write("shared", "no-frontmatter", "Just a body, no YAML at all.\n");
|
||||
t.write("shared", "unterminated", "---\nname: x\ndescription: y\n");
|
||||
t.write("shared", "not-yaml", "---\nname: [unclosed\n---\n");
|
||||
t.write("shared", "no-description", "---\nname: x\n---\n");
|
||||
std::fs::create_dir_all(t.root.join("skills").join("no-skill-md")).unwrap();
|
||||
|
||||
let ids: Vec<String> = list(&t.fs).into_iter().map(|s| s.id).collect();
|
||||
assert_eq!(ids, vec!["good".to_string()]);
|
||||
}
|
||||
|
||||
/// The directory is the id, whatever the frontmatter says — every path in the
|
||||
/// index is built from it.
|
||||
#[test]
|
||||
fn the_directory_is_the_id() {
|
||||
let t = Tree::new("id", "daniele");
|
||||
t.write("shared", "ics-import", &valid("something-else", "Import an ICS feed."));
|
||||
let got = list(&t.fs);
|
||||
assert_eq!(got[0].id, "ics-import");
|
||||
assert_eq!(got[0].skill_file(), "skills/shared/ics-import/SKILL.md");
|
||||
}
|
||||
|
||||
/// The heart of the multi-user half: the own tree is keyed on the userid, so a
|
||||
/// private skill of one member cannot reach another member's prompt.
|
||||
#[test]
|
||||
fn a_private_skill_belongs_to_one_member_only() {
|
||||
let a = Tree::new("private-a", "anna");
|
||||
a.write("mine", "budget", &valid("budget", "Anna's own."));
|
||||
let b = Tree::new("private-b", "bruno");
|
||||
|
||||
assert!(render_index(&a.fs).contains("skills/anna/budget/SKILL.md"));
|
||||
assert_eq!(render_index(&b.fs), "");
|
||||
}
|
||||
|
||||
/// Nothing installed ⇒ nothing rendered. Not "an empty section": the whole
|
||||
/// prose lives inside the render precisely so that this case costs zero tokens
|
||||
/// and leaves no sentence the model could read as a promise.
|
||||
#[test]
|
||||
fn no_skills_renders_nothing_at_all() {
|
||||
assert_eq!(render(&[]), "");
|
||||
let bare = UserFs::new("u1", PathBuf::from("/wd/homes/u1"), "c", PathBuf::from("/root"), vec![], vec![], None);
|
||||
assert_eq!(render_index(&bare), "");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_line_carries_the_full_path_and_a_truncated_description() {
|
||||
let long = "x".repeat(DESCRIPTION_LIMIT + 50);
|
||||
let out = render(&[
|
||||
skill("shared", "ics-import", "Download an iCalendar (ICS) feed\nand output JSON."),
|
||||
skill("daniele", "spesa", &long),
|
||||
]);
|
||||
|
||||
// The path is printed in full: no id-plus-composition-rule for the model
|
||||
// to get wrong.
|
||||
assert!(out.contains("skills/shared/ics-import/SKILL.md"), "{out}");
|
||||
assert!(out.contains("skills/daniele/spesa/SKILL.md"), "{out}");
|
||||
// A multi-line description becomes one line.
|
||||
assert!(out.contains("Download an iCalendar (ICS) feed and output JSON."), "{out}");
|
||||
// …and a long one is cut, visibly.
|
||||
assert!(out.contains(&format!("{}…", "x".repeat(DESCRIPTION_LIMIT))), "{out}");
|
||||
assert!(!out.contains(&"x".repeat(DESCRIPTION_LIMIT + 1)), "{out}");
|
||||
// The imperative header and the closing rule are both there.
|
||||
assert!(out.starts_with("## Skills (mandatory)"), "{out}");
|
||||
assert!(out.contains("MUST read its `SKILL.md`"), "{out}");
|
||||
assert!(out.ends_with("goes in your home or `/tmp`."), "{out}");
|
||||
}
|
||||
|
||||
/// The same id in both trees: both lines stay, both are marked. Neither tree
|
||||
/// shadows the other, here or in the path router.
|
||||
#[test]
|
||||
fn a_colliding_id_is_marked_on_both_lines() {
|
||||
let out = render(&[
|
||||
skill("shared", "ics-import", "The group's."),
|
||||
skill("shared", "pdf-forms", "Untouched."),
|
||||
skill("daniele", "ics-import", "My fork."),
|
||||
]);
|
||||
assert_eq!(out.matches("[name collision]").count(), 2, "{out}");
|
||||
for line in out.lines().filter(|l| l.contains("pdf-forms")) {
|
||||
assert!(!line.contains("[name collision]"), "{line}");
|
||||
}
|
||||
}
|
||||
|
||||
/// Over budget the index cuts from the tail of the stable order and says how
|
||||
/// many it dropped — deterministically, because the cut is part of the cache
|
||||
/// key, and out loud, because a silently truncated index makes the model
|
||||
/// conclude in good faith that a skill does not exist.
|
||||
#[test]
|
||||
fn over_budget_the_tail_is_cut_deterministically_and_announced() {
|
||||
let many: Vec<Skill> = (0..400)
|
||||
.map(|i| skill("shared", &format!("skill-{i:03}"), &"d".repeat(DESCRIPTION_LIMIT)))
|
||||
.collect();
|
||||
|
||||
let out = render(&many);
|
||||
assert!(out.len() <= INDEX_BUDGET, "budget blown: {} bytes", out.len());
|
||||
assert!(out.contains("more skills omitted — index budget reached"), "{out}");
|
||||
// A suffix of the order is what went missing: the first is in, the last is not.
|
||||
assert!(out.contains("skills/shared/skill-000/SKILL.md"), "{out}");
|
||||
assert!(!out.contains("skills/shared/skill-399/SKILL.md"), "{out}");
|
||||
// Same set in, same bytes out — otherwise the stable order buys nothing.
|
||||
assert_eq!(out, render(&many));
|
||||
assert_eq!(digest(&out), digest(&render(&many)));
|
||||
}
|
||||
|
||||
/// The digest keys on what the model can see. Editing a script or a reference
|
||||
/// document leaves it alone; re-describing a skill moves it.
|
||||
#[test]
|
||||
fn the_digest_follows_the_index_not_the_files() {
|
||||
let before = render(&[skill("shared", "ics-import", "Import an ICS feed.")]);
|
||||
let same = render(&[skill("shared", "ics-import", "Import an ICS feed.")]);
|
||||
let after = render(&[skill("shared", "ics-import", "Import an ICS feed, then dedupe.")]);
|
||||
assert_eq!(digest(&before), digest(&same));
|
||||
assert_ne!(digest(&before), digest(&after));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,311 @@
|
||||
//! What a folder must be before it is allowed to become a skill.
|
||||
//!
|
||||
//! **One validation site, because there is one door.** The two trees are
|
||||
//! read-only in both directions (blueprint §9), so every byte that ever lands in
|
||||
//! them passes through here — and the index on the other side can therefore
|
||||
//! assume it is reading well-formed skills instead of tolerating half-written
|
||||
//! ones. That is the whole argument for the read-only trees: with three write
|
||||
//! paths (fs-tools, the container shell, an HTTP copy) validation would either
|
||||
//! live in three places or nowhere.
|
||||
//!
|
||||
//! This module is deliberately callable from more than the registration tool:
|
||||
//! the ZIP upload and the marketplace of blueprint §11.2 are the same check with
|
||||
//! a different source of bytes, and two validators would diverge at the first
|
||||
//! edge case.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use anyhow::{Result, bail};
|
||||
|
||||
use super::{SKILL_FILE, parse_front_matter};
|
||||
|
||||
/// Longest `name` accepted, matching the `^[a-z0-9][a-z0-9-]{0,63}$` shape: the
|
||||
/// name becomes a directory name and then a path segment in every prompt.
|
||||
pub const NAME_MAX: usize = 64;
|
||||
|
||||
/// Ceiling on the `description`, applied **here** rather than in the index.
|
||||
///
|
||||
/// One limit, at the one place it can fail usefully: the author is present, sees
|
||||
/// the refusal and can shorten the text. The index's 200-character cut
|
||||
/// ([`super::DESCRIPTION_LIMIT`]) is a different rule for a different reason —
|
||||
/// tokens paid on every request of every user — and truncating there is not a
|
||||
/// rejection, since the full text stays readable through the enumeration tool.
|
||||
pub const DESCRIPTION_MAX: usize = 1000;
|
||||
|
||||
/// Ceiling on how many files one skill may carry.
|
||||
pub const MAX_FILES: usize = 500;
|
||||
|
||||
/// Ceiling on a skill's total size. Generous for instructions plus scripts and
|
||||
/// reference documents; far below anything that would be a *dataset*, which is
|
||||
/// not what this tree is for.
|
||||
pub const MAX_TOTAL_BYTES: u64 = 8 * 1024 * 1024;
|
||||
|
||||
/// A source folder that passed every check — the only thing [`super::install`]
|
||||
/// accepts, so an unvalidated path cannot reach the tree by construction.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ValidSkill {
|
||||
/// The id, taken from the frontmatter `name` and **not** from the source
|
||||
/// folder's name. The working copy may be called `draft-2`; the installed
|
||||
/// artefact is called what it declares itself to be, and from then on
|
||||
/// id = directory name by construction.
|
||||
pub id: String,
|
||||
pub description: String,
|
||||
/// Every regular file, relative to the source root, in a stable order —
|
||||
/// what the approval card lists.
|
||||
pub files: Vec<PathBuf>,
|
||||
pub size_bytes: u64,
|
||||
}
|
||||
|
||||
impl ValidSkill {
|
||||
/// Whether the skill carries anything executable, for the enumeration tool.
|
||||
pub fn has_scripts(&self) -> bool {
|
||||
self.files.iter().any(|f| {
|
||||
matches!(
|
||||
f.extension().and_then(|e| e.to_str()),
|
||||
Some("py" | "js" | "mjs" | "cjs" | "ts" | "sh" | "bash")
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
/// Which ecosystem's dependency manifest it ships, if any. Reported rather
|
||||
/// than acted on: v1 installs nothing (blueprint §10), and a skill that
|
||||
/// needs a package says so in its body.
|
||||
pub fn deps(&self) -> Option<String> {
|
||||
let has = |n: &str| self.files.iter().any(|f| f == Path::new(n));
|
||||
match (has("requirements.txt"), has("package.json")) {
|
||||
(true, true) => Some("python+node".into()),
|
||||
(true, false) => Some("python".into()),
|
||||
(false, true) => Some("node".into()),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Validates a candidate skill folder on the host filesystem.
|
||||
///
|
||||
/// Every refusal names what to fix: this error text is read by a model that will
|
||||
/// try again, and "invalid skill" would only produce a guess.
|
||||
pub fn validate_dir(dir: &Path) -> Result<ValidSkill> {
|
||||
if !dir.is_dir() {
|
||||
bail!(
|
||||
"not a folder: {}. A skill is a folder containing a `{SKILL_FILE}`.",
|
||||
dir.display()
|
||||
);
|
||||
}
|
||||
|
||||
let skill_md = dir.join(SKILL_FILE);
|
||||
if !skill_md.is_file() {
|
||||
bail!(
|
||||
"no `{SKILL_FILE}` in that folder. A skill is a folder whose `{SKILL_FILE}` \
|
||||
opens with a YAML frontmatter block declaring `name` and `description`."
|
||||
);
|
||||
}
|
||||
|
||||
let body = std::fs::read_to_string(&skill_md)
|
||||
.map_err(|e| anyhow::anyhow!("cannot read {SKILL_FILE}: {e}"))?;
|
||||
let front = parse_front_matter(&body).map_err(|problem| {
|
||||
anyhow::anyhow!(
|
||||
"invalid `{SKILL_FILE}` frontmatter ({problem}). It must start with a `---` line, \
|
||||
then `name:` and `description:`, then a closing `---`."
|
||||
)
|
||||
})?;
|
||||
|
||||
check_name(&front.name)?;
|
||||
if front.description.chars().count() > DESCRIPTION_MAX {
|
||||
bail!(
|
||||
"the `description` is {} characters; the limit is {DESCRIPTION_MAX}. It is the \
|
||||
*use condition* — when to reach for this skill — not a manual; the body of \
|
||||
`{SKILL_FILE}` is where the detail goes.",
|
||||
front.description.chars().count()
|
||||
);
|
||||
}
|
||||
|
||||
let mut walk = Walk::default();
|
||||
walk.visit(dir, Path::new(""))?;
|
||||
|
||||
Ok(ValidSkill {
|
||||
id: front.name,
|
||||
description: front.description,
|
||||
files: walk.files,
|
||||
size_bytes: walk.bytes,
|
||||
})
|
||||
}
|
||||
|
||||
/// The id charset: `^[a-z0-9][a-z0-9-]{0,63}$`.
|
||||
///
|
||||
/// Checked by hand rather than by regex because the failure has to *teach* — the
|
||||
/// caller is a model that will retry, and "does not match a pattern" is not a
|
||||
/// correction.
|
||||
fn check_name(name: &str) -> Result<()> {
|
||||
if name.len() > NAME_MAX {
|
||||
bail!("the frontmatter `name` is longer than {NAME_MAX} characters: `{name}`");
|
||||
}
|
||||
let ok_first = name.chars().next().is_some_and(|c| c.is_ascii_lowercase() || c.is_ascii_digit());
|
||||
let ok_rest = name.chars().all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-');
|
||||
if !ok_first || !ok_rest {
|
||||
bail!(
|
||||
"the frontmatter `name` must be lowercase letters, digits and hyphens, starting \
|
||||
with a letter or digit (it becomes the folder name and the path the assistant \
|
||||
reads): `{name}`"
|
||||
);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Recursive walk that enforces the structural rules while it counts.
|
||||
#[derive(Default)]
|
||||
struct Walk {
|
||||
files: Vec<PathBuf>,
|
||||
bytes: u64,
|
||||
}
|
||||
|
||||
impl Walk {
|
||||
fn visit(&mut self, dir: &Path, rel: &Path) -> Result<()> {
|
||||
let mut entries: Vec<_> = std::fs::read_dir(dir)
|
||||
.map_err(|e| anyhow::anyhow!("cannot read {}: {e}", dir.display()))?
|
||||
.filter_map(|e| e.ok())
|
||||
.collect();
|
||||
// Stable order: the file list ends up on an approval card, and a card
|
||||
// that reshuffles between two renders of the same folder reads as a
|
||||
// different change.
|
||||
entries.sort_by_key(|e| e.file_name());
|
||||
|
||||
for entry in entries {
|
||||
let path = entry.path();
|
||||
let name = entry.file_name();
|
||||
let child_rel = rel.join(&name);
|
||||
|
||||
// `symlink_metadata` does NOT follow the link, which is the point:
|
||||
// a link is refused for what it is, before anything asks where it
|
||||
// points. A skill is copied into a tree every member reads, and a
|
||||
// link out of that tree would make the installed artefact a window
|
||||
// onto something the installation never reviewed.
|
||||
let meta = std::fs::symlink_metadata(&path)
|
||||
.map_err(|e| anyhow::anyhow!("cannot stat {}: {e}", child_rel.display()))?;
|
||||
|
||||
if meta.file_type().is_symlink() {
|
||||
bail!(
|
||||
"`{}` is a symbolic link. A skill must be self-contained — copy the real \
|
||||
file in instead.",
|
||||
child_rel.display()
|
||||
);
|
||||
}
|
||||
if meta.is_dir() {
|
||||
self.visit(&path, &child_rel)?;
|
||||
continue;
|
||||
}
|
||||
if !meta.is_file() {
|
||||
bail!("`{}` is not a regular file.", child_rel.display());
|
||||
}
|
||||
|
||||
self.bytes += meta.len();
|
||||
self.files.push(child_rel);
|
||||
if self.files.len() > MAX_FILES {
|
||||
bail!("that folder holds more than {MAX_FILES} files — too much for a skill.");
|
||||
}
|
||||
if self.bytes > MAX_TOTAL_BYTES {
|
||||
bail!(
|
||||
"that folder is over {} MiB — too much for a skill. Skills hold \
|
||||
instructions, scripts and reference documents; bulk data belongs in your \
|
||||
home or a project.",
|
||||
MAX_TOTAL_BYTES / (1024 * 1024)
|
||||
);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
struct Dir(PathBuf);
|
||||
|
||||
impl Dir {
|
||||
fn new(tag: &str) -> Self {
|
||||
let p = std::env::temp_dir().join(format!(
|
||||
"skald-skillval-{tag}-{}-{:?}",
|
||||
std::process::id(),
|
||||
std::thread::current().id()
|
||||
));
|
||||
let _ = std::fs::remove_dir_all(&p);
|
||||
std::fs::create_dir_all(&p).unwrap();
|
||||
Dir(p)
|
||||
}
|
||||
fn write(&self, rel: &str, body: &str) {
|
||||
let p = self.0.join(rel);
|
||||
std::fs::create_dir_all(p.parent().unwrap()).unwrap();
|
||||
std::fs::write(p, body).unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for Dir {
|
||||
fn drop(&mut self) {
|
||||
let _ = std::fs::remove_dir_all(&self.0);
|
||||
}
|
||||
}
|
||||
|
||||
fn front(name: &str, description: &str) -> String {
|
||||
format!("---\nname: {name}\ndescription: {description}\n---\n\nBody.\n")
|
||||
}
|
||||
|
||||
/// The id comes from the frontmatter, never from the folder: the working
|
||||
/// copy is allowed a scratch name, the artefact is not.
|
||||
#[test]
|
||||
fn the_id_comes_from_the_frontmatter_not_the_folder() {
|
||||
let d = Dir::new("id");
|
||||
d.write("SKILL.md", &front("ics-import", "Import an ICS feed."));
|
||||
d.write("scripts/run.py", "print(1)\n");
|
||||
let v = validate_dir(&d.0).unwrap();
|
||||
assert_eq!(v.id, "ics-import");
|
||||
assert_eq!(v.files, vec![PathBuf::from("SKILL.md"), PathBuf::from("scripts/run.py")]);
|
||||
assert!(v.has_scripts());
|
||||
assert_eq!(v.deps(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_folder_without_a_skill_md_is_refused() {
|
||||
let d = Dir::new("nomd");
|
||||
d.write("notes.txt", "hello");
|
||||
let e = validate_dir(&d.0).unwrap_err().to_string();
|
||||
assert!(e.contains("SKILL.md"), "{e}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_name_that_cannot_be_a_folder_is_refused() {
|
||||
for bad in ["Ics Import", "../escape", "-leading", "UPPER"] {
|
||||
let d = Dir::new("badname");
|
||||
d.write("SKILL.md", &front(bad, "x"));
|
||||
assert!(validate_dir(&d.0).is_err(), "accepted `{bad}`");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_overlong_description_is_refused_here_not_truncated() {
|
||||
let d = Dir::new("longdesc");
|
||||
d.write("SKILL.md", &front("x", &"d".repeat(DESCRIPTION_MAX + 1)));
|
||||
let e = validate_dir(&d.0).unwrap_err().to_string();
|
||||
assert!(e.contains(&DESCRIPTION_MAX.to_string()), "{e}");
|
||||
}
|
||||
|
||||
/// A symlink is refused for being one, without asking where it points: the
|
||||
/// installed copy is read as instruction by everyone the scope covers.
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn a_symlink_anywhere_inside_is_refused() {
|
||||
let d = Dir::new("symlink");
|
||||
d.write("SKILL.md", &front("x", "y"));
|
||||
std::os::unix::fs::symlink("/etc/passwd", d.0.join("secrets.txt")).unwrap();
|
||||
let e = validate_dir(&d.0).unwrap_err().to_string();
|
||||
assert!(e.contains("symbolic link"), "{e}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn dependency_manifests_are_reported_not_installed() {
|
||||
let d = Dir::new("deps");
|
||||
d.write("SKILL.md", &front("x", "y"));
|
||||
d.write("requirements.txt", "requests\n");
|
||||
assert_eq!(validate_dir(&d.0).unwrap().deps().as_deref(), Some("python"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,298 @@
|
||||
//! The skills freshness watcher (blueprint §8.2): freshness for edits made **by
|
||||
//! hand on the box**.
|
||||
//!
|
||||
//! Every in-process writer — `skill_register`, `skill_delete`, the future UI —
|
||||
//! already invalidates the prompt prefix directly; this task exists for the one
|
||||
//! writer that is not in the process: the admin in SSH, a `git pull` of skills.
|
||||
//! It is deliberately *not* on the correctness path: a missed event costs a
|
||||
//! stale index for the twenty minutes of the prefix TTL, never a wrong one.
|
||||
//!
|
||||
//! The gate is the digest, and it is the whole design. `notify` is noisy — an
|
||||
//! editor's save is a burst, an install is dozens of events, and every prompt
|
||||
//! build *reads* every `SKILL.md` — so what reaches the bus is never "the fs
|
||||
//! moved" but "the rendered index would differ" ([`super::tree_digest`]). A
|
||||
//! script edit produces events and then silence, which is exactly the property
|
||||
//! §6 asks for: the frozen prefix citing that skill has not aged.
|
||||
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use core_api::system_bus::{SkillScope, SystemEvent, SystemEventBus};
|
||||
use notify::{Config, EventKind, RecommendedWatcher, RecursiveMode, Watcher};
|
||||
use tokio::sync::mpsc;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
use tracing::{info, warn};
|
||||
|
||||
use crate::container::{SKILLS_DIR, SKILLS_USERS_DIR};
|
||||
|
||||
/// The quiet period after the last fs event before the digests are recomputed.
|
||||
/// What matters is the settled state of the tree, not any intermediate one.
|
||||
const DEBOUNCE: Duration = Duration::from_millis(800);
|
||||
|
||||
/// Spawns the watcher on the two trees (`{WD}/skills`, `{WD}/skills-users`),
|
||||
/// emitting `SystemEvent::SkillsChanged` for each scope whose digest moved.
|
||||
pub fn spawn(bus: Arc<SystemEventBus>, shutdown: CancellationToken) -> tokio::task::JoinHandle<()> {
|
||||
let wd = std::env::current_dir().unwrap_or_else(|_| PathBuf::from("."));
|
||||
tokio::spawn(run(bus, shutdown, wd, DEBOUNCE))
|
||||
}
|
||||
|
||||
/// The watcher's body, split from [`spawn`] so the tests can point it at a
|
||||
/// temporary `{WD}` and shorten the debounce.
|
||||
async fn run(bus: Arc<SystemEventBus>, shutdown: CancellationToken, wd: PathBuf, debounce: Duration) {
|
||||
// The fs backend reports **canonical** paths (on macOS `/var` is a symlink
|
||||
// to `/private/var`, and FSEvents answers with the real one), so the roots
|
||||
// must be canonical too or `classify` never matches and every event is
|
||||
// silently dropped.
|
||||
let wd = match wd.canonicalize() {
|
||||
Ok(w) => w,
|
||||
Err(e) => {
|
||||
warn!(path = %wd.display(), error = %e, "skills-watch: cannot canonicalize the working directory, not started");
|
||||
return;
|
||||
}
|
||||
};
|
||||
let shared_dir = wd.join(SKILLS_DIR);
|
||||
let users_dir = wd.join(SKILLS_USERS_DIR);
|
||||
// Created rather than merely watched: these are instance data dirs that
|
||||
// `ContainerManager::ensure` would create anyway, and on a box before its
|
||||
// first user neither exists yet — a watcher that fails to install at boot
|
||||
// would never notice the first tree appearing.
|
||||
for dir in [&shared_dir, &users_dir] {
|
||||
if let Err(e) = std::fs::create_dir_all(dir) {
|
||||
warn!(path = %dir.display(), error = %e, "skills-watch: cannot create the tree, not started");
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
let (tx, mut rx) = mpsc::unbounded_channel::<Vec<PathBuf>>();
|
||||
let mut watcher = match RecommendedWatcher::new(
|
||||
move |res: notify::Result<notify::Event>| {
|
||||
let Ok(event) = res else { return };
|
||||
// A pure read is never a change — and it matters here: every prompt
|
||||
// build reads every `SKILL.md`, so IN_ACCESS / CLOSE_NOWRITE would
|
||||
// re-digest the trees after every single conversation turn.
|
||||
if matches!(event.kind, EventKind::Access(_)) {
|
||||
return;
|
||||
}
|
||||
if event.paths.is_empty() {
|
||||
return;
|
||||
}
|
||||
let _ = tx.send(event.paths);
|
||||
},
|
||||
Config::default(),
|
||||
) {
|
||||
Ok(w) => w,
|
||||
Err(e) => {
|
||||
warn!(error = %e, "skills-watch: watcher create failed, not started");
|
||||
return;
|
||||
}
|
||||
};
|
||||
for dir in [&shared_dir, &users_dir] {
|
||||
if let Err(e) = watcher.watch(dir, RecursiveMode::Recursive) {
|
||||
warn!(path = %dir.display(), error = %e, "skills-watch: watch install failed, not started");
|
||||
return;
|
||||
}
|
||||
}
|
||||
info!("skills-watch: watching the two skills trees");
|
||||
|
||||
// Baselines, taken before anything can be emitted: only a *change* from
|
||||
// here on is worth an announcement. `empty` is the digest of a tree with no
|
||||
// visible skill — a tree first seen in that state (e.g. created empty by
|
||||
// `ensure` at container setup) alters no index and announces nothing.
|
||||
let empty = super::tree_digest(&shared_dir.join("__never__"));
|
||||
let mut shared_digest = super::tree_digest(&shared_dir);
|
||||
let mut user_digests = digests_by_user(&users_dir);
|
||||
|
||||
let mut touched_shared = false;
|
||||
let mut touched_users: HashSet<String> = HashSet::new();
|
||||
let mut quiet: Option<std::pin::Pin<Box<tokio::time::Sleep>>> = None;
|
||||
|
||||
loop {
|
||||
tokio::select! {
|
||||
_ = shutdown.cancelled() => break,
|
||||
paths = rx.recv() => {
|
||||
let Some(paths) = paths else { break }; // watcher dropped
|
||||
for p in &paths {
|
||||
match classify(p, &shared_dir, &users_dir) {
|
||||
Some(SkillScope::Global) => touched_shared = true,
|
||||
Some(SkillScope::User(u)) => { touched_users.insert(u); }
|
||||
None => {}
|
||||
}
|
||||
}
|
||||
// Restart the quiet period on every event: a save burst or an
|
||||
// install settles only when the events stop coming.
|
||||
quiet = Some(Box::pin(tokio::time::sleep(debounce)));
|
||||
}
|
||||
// `pending` when disarmed: the arm below fires only once armed.
|
||||
_ = async { match &mut quiet { Some(s) => s.as_mut().await, None => std::future::pending().await } } => {
|
||||
quiet = None;
|
||||
|
||||
if touched_shared {
|
||||
touched_shared = false;
|
||||
let now = super::tree_digest(&shared_dir);
|
||||
if now != shared_digest {
|
||||
shared_digest = now;
|
||||
info!("skills-watch: the group's tree changed, announcing");
|
||||
bus.send(SystemEvent::SkillsChanged { scope: SkillScope::Global });
|
||||
}
|
||||
}
|
||||
for uid in touched_users.drain() {
|
||||
let now = super::tree_digest(&users_dir.join(&uid));
|
||||
let old = user_digests.insert(uid.clone(), now.clone());
|
||||
let changed = match old {
|
||||
Some(old) => old != now,
|
||||
None => now != empty,
|
||||
};
|
||||
if changed {
|
||||
info!(user = %uid, "skills-watch: a member's tree changed, announcing");
|
||||
bus.send(SystemEvent::SkillsChanged { scope: SkillScope::User(uid) });
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
info!("skills-watch: stopped");
|
||||
}
|
||||
|
||||
/// Maps a changed fs path to the scope tree it touches.
|
||||
///
|
||||
/// `Path::starts_with` is component-wise, which is exactly what saves the
|
||||
/// prefix trap here: `{WD}/skills-users/…` does **not** start with
|
||||
/// `{WD}/skills`. An event on the `skills-users` root itself names no user
|
||||
/// yet and is ignored — a new member's directory reports itself by its own
|
||||
/// path.
|
||||
fn classify(path: &Path, shared_dir: &Path, users_dir: &Path) -> Option<SkillScope> {
|
||||
if path.starts_with(shared_dir) {
|
||||
return Some(SkillScope::Global);
|
||||
}
|
||||
let rel = path.strip_prefix(users_dir).ok()?;
|
||||
let uid = rel.components().next()?.as_os_str().to_str()?;
|
||||
Some(SkillScope::User(uid.to_string()))
|
||||
}
|
||||
|
||||
/// The baseline digests of every member tree present at startup.
|
||||
fn digests_by_user(users_dir: &Path) -> HashMap<String, String> {
|
||||
let Ok(entries) = std::fs::read_dir(users_dir) else { return HashMap::new() };
|
||||
entries
|
||||
.flatten()
|
||||
.filter(|e| e.path().is_dir())
|
||||
.filter_map(|e| e.file_name().into_string().ok())
|
||||
.map(|uid| {
|
||||
let d = super::tree_digest(&users_dir.join(&uid));
|
||||
(uid, d)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::skills::tests_support::{Tree, valid};
|
||||
|
||||
/// The classification trap the whole design leans on: `skills-users` is a
|
||||
/// **sibling** of `skills`, and a component-wise prefix check must not let
|
||||
/// a member's edit fall into the group's scope.
|
||||
#[test]
|
||||
fn classify_never_confuses_the_two_sibling_trees() {
|
||||
let wd = Path::new("/wd");
|
||||
let shared = wd.join(SKILLS_DIR);
|
||||
let users = wd.join(SKILLS_USERS_DIR);
|
||||
|
||||
assert_eq!(
|
||||
classify(Path::new("/wd/skills/ics-import/SKILL.md"), &shared, &users),
|
||||
Some(SkillScope::Global)
|
||||
);
|
||||
assert_eq!(classify(Path::new("/wd/skills"), &shared, &users), Some(SkillScope::Global));
|
||||
assert_eq!(
|
||||
classify(Path::new("/wd/skills-users/u1/spesa/SKILL.md"), &shared, &users),
|
||||
Some(SkillScope::User("u1".into()))
|
||||
);
|
||||
assert_eq!(
|
||||
classify(Path::new("/wd/skills-users/u1"), &shared, &users),
|
||||
Some(SkillScope::User("u1".into()))
|
||||
);
|
||||
// The users root itself names nobody.
|
||||
assert_eq!(classify(Path::new("/wd/skills-users"), &shared, &users), None);
|
||||
// Anything else is not ours at all.
|
||||
assert_eq!(classify(Path::new("/wd/homes/u1/x"), &shared, &users), None);
|
||||
}
|
||||
|
||||
/// The gate itself, over a real tree: an edit the index cannot see passes
|
||||
/// in silence; one it can see announces. (The pure half —
|
||||
/// `the_digest_follows_the_index_not_the_files` — lives in `skills/mod.rs`;
|
||||
/// this is the watcher half §15 asks for.)
|
||||
#[test]
|
||||
fn tree_digest_gates_on_what_the_index_can_see() {
|
||||
let t = Tree::new("watch-digest", "daniele");
|
||||
t.write("shared", "ics-import", &valid("ics-import", "Import an ICS feed."));
|
||||
let dir = t.root.join(SKILLS_DIR);
|
||||
|
||||
let before = super::super::tree_digest(&dir);
|
||||
// A script appears: events fire, the index does not move.
|
||||
std::fs::write(dir.join("ics-import").join("run.py"), "print('hi')\n").unwrap();
|
||||
assert_eq!(super::super::tree_digest(&dir), before);
|
||||
// The body changes under the same description: still invisible.
|
||||
t.write("shared", "ics-import", &valid("ics-import", "Import an ICS feed."));
|
||||
assert_eq!(super::super::tree_digest(&dir), before);
|
||||
// A re-description is what the prefix is made of: the digest moves.
|
||||
t.write("shared", "ics-import", &valid("ics-import", "Import an ICS feed, then dedupe."));
|
||||
assert_ne!(super::super::tree_digest(&dir), before);
|
||||
// An empty or missing tree is the same digest as no tree.
|
||||
assert_eq!(super::super::tree_digest(&dir.join("__never__")), super::super::digest(""));
|
||||
}
|
||||
|
||||
/// End to end: a hand edit on the box reaches the bus — but only when the
|
||||
/// index would notice.
|
||||
#[tokio::test]
|
||||
async fn a_hand_edit_announces_only_what_the_index_feels() {
|
||||
let t = Tree::new("watch-e2e", "daniele");
|
||||
t.write("shared", "ics-import", &valid("ics-import", "Import an ICS feed."));
|
||||
t.write("mine", "spesa", &valid("spesa", "Reconcile the statement."));
|
||||
|
||||
let bus = Arc::new(SystemEventBus::new());
|
||||
let mut rx = bus.subscribe();
|
||||
let shutdown = CancellationToken::new();
|
||||
let task = tokio::spawn(run(
|
||||
Arc::clone(&bus),
|
||||
shutdown.clone(),
|
||||
t.root.clone(),
|
||||
Duration::from_millis(100),
|
||||
));
|
||||
|
||||
// Give the watcher a moment to install before touching the tree.
|
||||
tokio::time::sleep(Duration::from_millis(300)).await;
|
||||
|
||||
// A script edit is fs activity the index cannot see: no announcement,
|
||||
// however long we wait.
|
||||
std::fs::write(t.root.join(SKILLS_DIR).join("ics-import").join("run.py"), "print(1)\n").unwrap();
|
||||
let quiet = tokio::time::timeout(Duration::from_millis(1500), rx.recv()).await;
|
||||
assert!(quiet.is_err(), "a script edit announced something: {quiet:?}");
|
||||
|
||||
// A re-description of a shared skill announces the group's scope.
|
||||
t.write("shared", "ics-import", &valid("ics-import", "Import an ICS feed, then dedupe."));
|
||||
let announced = tokio::time::timeout(Duration::from_secs(10), rx.recv())
|
||||
.await
|
||||
.expect("no SkillsChanged within 10s of a description edit")
|
||||
.expect("bus closed");
|
||||
assert!(
|
||||
matches!(announced, SystemEvent::SkillsChanged { scope: SkillScope::Global }),
|
||||
"expected SkillsChanged(Global), got {announced:?}"
|
||||
);
|
||||
|
||||
// And one of an own skill announces only that member.
|
||||
t.write("mine", "spesa", &valid("spesa", "Reconcile the statement, monthly."));
|
||||
let announced = tokio::time::timeout(Duration::from_secs(10), rx.recv())
|
||||
.await
|
||||
.expect("no SkillsChanged within 10s of an own-skill edit")
|
||||
.expect("bus closed");
|
||||
assert!(
|
||||
matches!(announced, SystemEvent::SkillsChanged { scope: SkillScope::User(ref u) } if u == "u1"),
|
||||
"expected SkillsChanged(User(u1)), got {announced:?}"
|
||||
);
|
||||
|
||||
shutdown.cancel();
|
||||
let _ = tokio::time::timeout(Duration::from_secs(2), task).await;
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user