diff --git a/SKALD.md b/SKALD.md index 4f8dfa9..03bb4bf 100644 --- a/SKALD.md +++ b/SKALD.md @@ -1,6 +1,21 @@ # Skald Connectors Marketplace +### 2026-08-10 — http-fetch + firecrawl: fix `npx` (v5 / 1.1.0) + +Entrambi i connector erano **non funzionanti**: dichiaravano `mcp_config: {command: "npx", args: ["-y", ""]}` e non spedivano nessun file di codice (solo `connector.json` + icone). + +- **`npx -y ` non è esprimibile in skald.** Per un `type: mcp_local` skald tratta `args[0]` come il **nome del file da eseguire**, non come un argomento: all'install calcola `script_path = "/" + args[0]` e azzera `args_json` (`marketplace.rs::install`), poi `global_enable` (`api/mcp.rs`) lo risolve in path assoluto e lancia ` `. Il comando reale diventava `npx /…/connectors/http-fetch/-y` — un path inesistente, con `-y` e il nome del package spariti. Il processo non rispondeva mai a `initialize`, `start_server` falliva. +- **Il fallimento era silenzioso**: `global_enable` restituisce comunque HTTP 200 con un campo `error` nel body, quindi la UI mostrava il connector come abilitato mentre il runtime non aveva nessun server. E `render_mcp_list` (`loop_adapters/system.rs`) costruisce la tabella `## MCP servers` da `mcp.tools()`, cioè lo **stato vivo del runtime**, non dal DB → il connector risultava attivato e concesso all'utente ma **assente dal system context**. ⚠️ Questa combinazione (200 + `error` nel body) rende invisibile in UI qualsiasi connector che non parte: vale la pena farla emergere lato skald. +- **Fix — wrapper di due file** per entrambi: `package.json` che pinna il package upstream (`mcp-fetch-server@1.1.2`, `firecrawl-mcp@3.23.7`) e `index.js` che lo importa per side-effect (il modulo avvia il loop JSON-RPC su stdio all'import). `mcp_config` diventa `{command: "node", args: ["index.js"], transport: "stdio"}`, cioè un vero `local_script`: `ensure_installed_host` fa `npm ci --omit=dev || npm install --omit=dev` nella cartella del connector prima del lancio, esattamente come per whatsapp. Nessun `node_modules` spedito, nessun lockfile (come whatsapp). +- **Rimossi i campi legacy** `launch_command`, `transport` top-level e `dependencies` (`dependencies` è solo per la card, come già visto su gmaps; `transport` va dentro `mcp_config`). +- **firecrawl: rimosso `mcp_config.env: {"FIRECRAWL_API_KEY": "{SECRET:FIRECRAWL_API_KEY}"}`** — inerte, stesso caso di gmaps del 2026-08-10: `apply_key_placeholder` sostituisce i token solo nella URL, mai nei valori di `env`. Funziona perché la form admin manda `env` e quel payload sovrascrive `entry.env_json`. +- **firecrawl**: aggiunto `firecrawl_developer_search` a `tools[]` (27 tool live contro i 26 dichiarati, verificato su 3.23.7); `requires` `["NODE"]` → `["NODE", "API_KEY"]`. +- **Versioni riallineate** manifest↔fragment a `5` / `1.1.0` / `2026-08-10` per entrambi: erano 2/1.0.1 (manifest) vs 4/1.0.3 (fragment), e skald preferisce il manifest — quindi `installed_version` restava 2 e il badge "Update available" non sarebbe mai comparso. +- **Requisito host**: sono connector `scope: global`, girano sull'**host** e non nel container. `mcp-fetch-server` vuole Node ≥18, `firecrawl-mcp` vuole Node ≥**22**. +- Testati end-to-end riproducendo il path di skald (`npm ci || npm install` + `node /index.js`): `initialize`, `tools/list` e una `tools/call` reale, stdout solo JSON-RPC, stderr pulito ✅ +- Indice rigenerato con compile.py ✅ + ### 2026-08-10 — gmaps: fix dipendenze + verify (v6 / 1.1.0) - **Aggiunto `requirements.txt` (`googlemaps>=4.10.0`)** — era l'unico connector python senza. Le dipendenze erano dichiarate nel campo `dependencies` del manifest, che skald usa **solo per la card**: `ensure_installed_host` guarda esclusivamente `requirements.txt` / `package.json`. Risultato: `.pydeps` vuoto e log pieno di `No module named 'googlemaps'`, con il server che rispondeva comunque a `tools/list` (→ `connected — 6 tool(s)` su un connector non funzionante). - **Cablato il `verify`** (`python3 verify.py`, 20s): `verify.py` era shippato ma il manifest non aveva il blocco `verify`, e skald legge `verify_command` solo da lì. Ora un'abilitazione con dipendenze rotte fallisce visibilmente invece di partire in silenzio. diff --git a/connectors/connectors.json b/connectors/connectors.json index c9acaf7..e465d6c 100644 --- a/connectors/connectors.json +++ b/connectors/connectors.json @@ -304,7 +304,8 @@ "icon_large": "firecrawl/icon_lg.png", "user_description": "Web scraping, crawling, search, deep research, and autonomous browsing \u2014 extract content from any website.", "requires": [ - "NODE" + "NODE", + "API_KEY" ], "tags": [ "scraping", @@ -319,9 +320,9 @@ "type": "api_key" }, "folder": "firecrawl", - "version": 4, - "version_string": "1.0.3", - "version_release_date": "2026-07-21", + "version": 5, + "version_string": "1.1.0", + "version_release_date": "2026-08-10", "tools": [ { "name": "firecrawl_scrape", @@ -335,6 +336,10 @@ "name": "firecrawl_search", "display_name": "Web Search" }, + { + "name": "firecrawl_developer_search", + "display_name": "Developer Search" + }, { "name": "firecrawl_search_feedback", "display_name": "Search Feedback" @@ -431,8 +436,8 @@ "files": [ { "path": "connector.json", - "sha256": "6cab1a43e32741eb4f31638841e65b68fb937d2c6313fd18ace38e722bc2eb23", - "size": 5008 + "sha256": "15713684157541a255b3c1eb143a00e715bc833878fd46470bd2d5e396acfb32", + "size": 5006 }, { "path": "icon_lg.png", @@ -443,6 +448,16 @@ "path": "icon_sm.png", "sha256": "3a4be30d4c16ce93a1d132e063dfdffb8bd27b6c97715c9f3b12414439aaeb01", "size": 6858 + }, + { + "path": "index.js", + "sha256": "bbd77881f07ea9fc17fe9545e9ad9bc8fa74b652b77b9d20d42b012e2356d0e9", + "size": 817 + }, + { + "path": "package.json", + "sha256": "d5f96ea065a92e12adc41be3babb00bdd8c86de149d7beb8631184530c5fc617", + "size": 278 } ] }, @@ -470,9 +485,9 @@ "type": "none" }, "folder": "http-fetch", - "version": 4, - "version_string": "1.0.3", - "version_release_date": "2026-07-21", + "version": 5, + "version_string": "1.1.0", + "version_release_date": "2026-08-10", "tools": [ { "name": "fetch_html", @@ -502,8 +517,8 @@ "files": [ { "path": "connector.json", - "sha256": "8087d8ed8af7c6b628c95ff74f5126023a1ed1c99c2b3fc5c0599f665ffaa6f2", - "size": 1862 + "sha256": "7d9b0881325fe559e0d637dd3a2787c0d3d190b89e5f437db1754ac87f565f2c", + "size": 1908 }, { "path": "icon_lg.svg", @@ -514,6 +529,16 @@ "path": "icon_sm.svg", "sha256": "fa32e634744d8cba369f6d8ba45746166124f7d3507cefd9a1f96180c787e1a8", "size": 380 + }, + { + "path": "index.js", + "sha256": "bc092a323631b8de99bac70a0b0d0486fa77313c4d75d1182b8c426f6d529d28", + "size": 703 + }, + { + "path": "package.json", + "sha256": "29a4e0c21b65e7f0a1d69381b48c3fd505520eb9e2e4a4afeb0118135b80348e", + "size": 284 } ] }, diff --git a/connectors/firecrawl/connector.json b/connectors/firecrawl/connector.json index fbfabd2..f9be908 100644 --- a/connectors/firecrawl/connector.json +++ b/connectors/firecrawl/connector.json @@ -2,12 +2,10 @@ "id": "firecrawl", "name": "Firecrawl", "type": "mcp_local", - "launch_command": "npx -y firecrawl-mcp", - "transport": "stdio", "requires": [ - "NODE" + "NODE", + "API_KEY" ], - "dependencies": [], "env": [ { "name": "FIRECRAWL_API_KEY", @@ -19,9 +17,9 @@ } ], "setup_instructions": [ - "Ensure Node.js is installed.", + "Requires Node.js 22+ on the host that runs the connector.", "Get an API key from https://www.firecrawl.dev/app/api-keys.", - "No pip install needed \u2014 Firecrawl MCP runs via npx." + "Dependencies are installed automatically by Skald from package.json." ], "docs": [ { @@ -34,14 +32,11 @@ "type": "api_key" }, "mcp_config": { - "command": "npx", + "command": "node", "args": [ - "-y", - "firecrawl-mcp" + "index.js" ], - "env": { - "FIRECRAWL_API_KEY": "{SECRET:FIRECRAWL_API_KEY}" - } + "transport": "stdio" }, "verify": { "command": "node -e \"const k = process.env.FIRECRAWL_API_KEY; if (!k || k.length < 10) { process.exit(1); } try { fetch('https://api.firecrawl.dev/v1/scrape', {method:'POST', headers:{'Content-Type':'application/json','Authorization':'Bearer '+k}, body:JSON.stringify({url:'https://example.com',formats:['markdown']})}).then(r=>r.json()).then(d=>{if(d.success) {console.log(JSON.stringify({ok:true,message:'Firecrawl API key valid \u2014 scrape probe successful'}))} else {console.log(JSON.stringify({ok:false,message:d.error||'API probe failed'}));process.exit(1)}}).catch(e=>{console.log(JSON.stringify({ok:false,message:e.message}));process.exit(1)}) } catch(e) { console.log(JSON.stringify({ok:false,message:e.message})); process.exit(1) }\"", @@ -60,9 +55,9 @@ "browser", "research" ], - "version": 2, - "version_string": "1.0.1", - "version_release_date": "2026-07-21", + "version": 5, + "version_string": "1.1.0", + "version_release_date": "2026-08-10", "tools": [ { "name": "firecrawl_scrape", @@ -76,6 +71,10 @@ "name": "firecrawl_search", "display_name": "Web Search" }, + { + "name": "firecrawl_developer_search", + "display_name": "Developer Search" + }, { "name": "firecrawl_search_feedback", "display_name": "Search Feedback" diff --git a/connectors/firecrawl/fragment.json b/connectors/firecrawl/fragment.json index 8bd5f60..06fde20 100644 --- a/connectors/firecrawl/fragment.json +++ b/connectors/firecrawl/fragment.json @@ -7,7 +7,8 @@ "icon_large": "firecrawl/icon_lg.png", "user_description": "Web scraping, crawling, search, deep research, and autonomous browsing \u2014 extract content from any website.", "requires": [ - "NODE" + "NODE", + "API_KEY" ], "tags": [ "scraping", @@ -22,9 +23,9 @@ "type": "api_key" }, "folder": "firecrawl", - "version": 4, - "version_string": "1.0.3", - "version_release_date": "2026-07-21", + "version": 5, + "version_string": "1.1.0", + "version_release_date": "2026-08-10", "tools": [ { "name": "firecrawl_scrape", @@ -38,6 +39,10 @@ "name": "firecrawl_search", "display_name": "Web Search" }, + { + "name": "firecrawl_developer_search", + "display_name": "Developer Search" + }, { "name": "firecrawl_search_feedback", "display_name": "Search Feedback" diff --git a/connectors/firecrawl/index.js b/connectors/firecrawl/index.js new file mode 100644 index 0000000..3686a50 --- /dev/null +++ b/connectors/firecrawl/index.js @@ -0,0 +1,16 @@ +// Skald connector entry point. +// +// The MCP server itself is the upstream `firecrawl-mcp` package, pinned in +// package.json and installed by Skald beside this file (`npm ci --omit=dev`, +// falling back to `npm install --omit=dev`). +// +// This file exists because Skald launches a `mcp_local` connector as +// ` `, where args[0] is rewritten to the absolute +// path of a *shipped* file — so `npx -y ` cannot be expressed: args[0] +// would be taken as the entry file's name. Importing the package's server module +// for its side effect (it starts the stdio JSON-RPC loop at import time) is the +// whole of the wrapper. +// +// The API key arrives as FIRECRAWL_API_KEY in the process environment, collected +// from the manifest's env[] form. +import "firecrawl-mcp/dist/index.js"; diff --git a/connectors/firecrawl/package.json b/connectors/firecrawl/package.json new file mode 100644 index 0000000..c2f37ba --- /dev/null +++ b/connectors/firecrawl/package.json @@ -0,0 +1,13 @@ +{ + "name": "skald-connector-firecrawl", + "version": "1.1.0", + "private": true, + "type": "module", + "description": "Skald connector wrapper around the firecrawl-mcp MCP server.", + "dependencies": { + "firecrawl-mcp": "3.23.7" + }, + "engines": { + "node": ">=22" + } +} diff --git a/connectors/http-fetch/connector.json b/connectors/http-fetch/connector.json index 45bca68..56aba9a 100644 --- a/connectors/http-fetch/connector.json +++ b/connectors/http-fetch/connector.json @@ -2,37 +2,35 @@ "id": "http-fetch", "name": "HTTP Fetch", "type": "mcp_local", - "launch_command": "npx -y mcp-fetch-server", - "transport": "stdio", + "scope": "global", "requires": [ "NODE" ], - "dependencies": [], "setup_instructions": [ - "Ensure Node.js is installed.", - "No configuration needed \u2014 HTTP Fetch is a free tool with no API key." + "Requires Node.js 18+ on the host that runs the connector.", + "No configuration needed — HTTP Fetch is free and needs no API key.", + "Dependencies are installed automatically by Skald from package.json." ], "docs": [ { "lang": "en", - "description": "Fetch any URL and convert it to HTML, Markdown, plain text, JSON, or YouTube transcripts \u2014 free, no API key required, no rate limits, no cost. Prefer this over Exa or Firecrawl for simple URL fetching. Supports YouTube video transcripts via fetch_youtube_transcript.", - "llm_short_description": "HTTP Fetch \u2014 fetch any URL and convert to HTML, Markdown, plain text, JSON, or readable article format (strips ads). Also extracts YouTube transcripts. Free, no API key." + "description": "Fetch any URL and convert it to HTML, Markdown, plain text, JSON, or YouTube transcripts — free, no API key required, no rate limits, no cost. Prefer this over Exa or Firecrawl for simple URL fetching. Supports YouTube video transcripts via fetch_youtube_transcript.", + "llm_short_description": "HTTP Fetch — fetch any URL and convert to HTML, Markdown, plain text, JSON, or readable article format (strips ads). Also extracts YouTube transcripts. Free, no API key." } ], "auth": { "type": "none" }, "mcp_config": { - "command": "npx", + "command": "node", "args": [ - "-y", - "mcp-fetch-server" - ] + "index.js" + ], + "transport": "stdio" }, - "homepage": "", + "homepage": "https://github.com/zcaceres/fetch-mcp", "icon_small": "icon_sm.svg", "icon_large": "icon_lg.svg", - "scope": "global", "tags": [ "http", "fetch", @@ -42,9 +40,9 @@ "scraping", "youtube" ], - "version": 2, - "version_string": "1.0.1", - "version_release_date": "2026-07-21", + "version": 5, + "version_string": "1.1.0", + "version_release_date": "2026-08-10", "tools": [ { "name": "fetch_html", @@ -71,4 +69,4 @@ "display_name": "Get YouTube Transcript" } ] -} \ No newline at end of file +} diff --git a/connectors/http-fetch/fragment.json b/connectors/http-fetch/fragment.json index 361f2c3..f2e9613 100644 --- a/connectors/http-fetch/fragment.json +++ b/connectors/http-fetch/fragment.json @@ -22,9 +22,9 @@ "type": "none" }, "folder": "http-fetch", - "version": 4, - "version_string": "1.0.3", - "version_release_date": "2026-07-21", + "version": 5, + "version_string": "1.1.0", + "version_release_date": "2026-08-10", "tools": [ { "name": "fetch_html", diff --git a/connectors/http-fetch/index.js b/connectors/http-fetch/index.js new file mode 100644 index 0000000..e0f0f66 --- /dev/null +++ b/connectors/http-fetch/index.js @@ -0,0 +1,13 @@ +// Skald connector entry point. +// +// The MCP server itself is the upstream `mcp-fetch-server` package, pinned in +// package.json and installed by Skald beside this file (`npm ci --omit=dev`, +// falling back to `npm install --omit=dev`). +// +// This file exists because Skald launches a `mcp_local` connector as +// ` `, where args[0] is rewritten to the absolute +// path of a *shipped* file — so `npx -y ` cannot be expressed: args[0] +// would be taken as the entry file's name. Importing the package's server module +// for its side effect (it starts the stdio JSON-RPC loop at import time) is the +// whole of the wrapper. +import "mcp-fetch-server/dist/index.js"; diff --git a/connectors/http-fetch/package.json b/connectors/http-fetch/package.json new file mode 100644 index 0000000..4ec0f61 --- /dev/null +++ b/connectors/http-fetch/package.json @@ -0,0 +1,13 @@ +{ + "name": "skald-connector-http-fetch", + "version": "1.1.0", + "private": true, + "type": "module", + "description": "Skald connector wrapper around the mcp-fetch-server MCP server.", + "dependencies": { + "mcp-fetch-server": "1.1.2" + }, + "engines": { + "node": ">=18" + } +}