WhatsApp delivers a history sync at login, not on reconnect, so an in-memory
store left the connector blind after every process restart (23 in two weeks).
The store is now mirrored to store/, next to auth/ and bind-mounted the same
way.
Not SQLite: node:sqlite needs Node 22 (unflagged only from 24) and the runtime
image ships Debian trixie's nodejs = 20.19.2; better-sqlite3 is native and the
slim image has no toolchain. So an append-only JSONL log for messages plus a
debounced JSON snapshot for chats/contacts, both written tmp+rename so a crash
cannot truncate them. Compaction on load and every 500 appends keeps the log
from creeping upward.
Also fixes a pre-existing duplication bug: pushMessage appended unconditionally,
re-adding every message a history re-sync redelivered. It now returns false on a
known id and the message is skipped in both the transcript and the log.
MAX_MSGS_PER_CHAT 200 -> 500. logout deletes store/ with auth/, so re-linking a
different phone cannot inherit the previous account's history.
Data at rest: message text is now written to the user's bind-mounted home.
Nothing but session keys was persisted before.
Verified on skald-runtime:v4 with a seeded store: 751 lines with 50 duplicates,
a 700-message chat and a torn trailing line -> 701 loaded, compacted to 501, cap
applied, second run 501 -> 501 unchanged.
Diagnosed from the live server (two users, two accounts, one shared log).
- history sync was gated off entirely on Baileys 6.7.x: the library derives
shouldSyncHistoryMessage from syncFullHistory when unset, and the connector
shipped syncFullHistory: false. Now both are passed explicitly, so behaviour
no longer depends on the resolved Baileys version. browser -> ['Mac OS', ...]
because PLATFORM_MAP only grants a desktop-grade sync to Mac OS / Windows.
- reconnects leaked their socket, leaving two writers over auth/: no end(), no
removeAllListeners(). ~15 reconnects/day produced 1422 Bad MAC lines, all on
the account's own LID device 0. Added teardownSock(), per-socket generation
guards, and a single-slot reconnect timer with exponential backoff.
- undecryptable messages (stubType CIPHERTEXT) were stored with empty text and
rendered as blank transcript lines; now labelled and counted. Text-less
protocol frames are dropped.
- fetchLatestBaileysVersion() ran on every reconnect; now cached for 6h.
- libsignal's direct console.error spam collapsed into one counted line, log
lines tagged with the linked account, console.log redirected off stdout.
Baileys pinned ^6.7.9 -> 7.0.0-rc14 (the range resolved to 6.7.24 for one user
and 6.17.16 for the other). RC is ESM-only, so index.js is now ESM.
Verified on skald-runtime:v4 (Node 20.19.2): initialize, tools/list, status,
list_chats, version fetch, QR, and logout -> teardown -> single reconnect.
Remaining gap: the store is still in-memory only, so synced history is lost on
the next process restart.
Activating Exa always failed with "Exa API key is invalid or
unauthorized (HTTP 403)" — the key was never actually tested.
Two bugs stacked:
1. mcp.exa.ai is behind Cloudflare, which bans urllib's default
Python-urllib/3.x agent with 403 / "error code: 1010" before Exa
sees the request. verify.py sent no User-Agent and mapped any 403
to "API key is invalid". Reproduced on the server with no key set
at all — same "invalid key" message.
2. The probed endpoint cannot validate a key anyway: JSON-RPC
initialize against the MCP endpoint returns 200 no matter what
?exaApiKey= carries (checked with a real key, a bogus key, and no
key). Fixing only the headers would have flipped the bug to
accepting every key, including garbage.
With a key, the probe is now a minimal POST to api.exa.ai/search with
the key in the x-api-key header — the only call that exercises the
credential (200 valid, 401/403 + Exa JSON error invalid, 402 out of
credits, 429 valid but throttled). With no key it probes MCP
initialize and reports reachability only, never validity. Both
requests send a User-Agent; the MCP one also sends
Accept: application/json, text/event-stream (else HTTP 406).
An opaque 401/403 with no Exa JSON error is now reported as "blocked
before reaching the API — the key was not tested", instead of blaming
the credential.
Also re-aligns manifest/fragment versions to 5 / 1.0.4 (were 2/1.0.1
vs 4/1.0.3; skald reads installed_version from the manifest, so the
update badge would never have appeared) and raises verify.timeout_secs
15 -> 20.
refresh_connector_after_reinstall updates only the description on the
mcp_global_servers row before restarting from it, so an instance that already
had http-fetch/firecrawl enabled keeps the stale `npx …/-y` launch after an
Update. Record the deploy → Update → re-save sequence that actually applies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both connectors declared `mcp_config: {command: "npx", args: ["-y", "<pkg>"]}`
and shipped no code at all. Skald treats a mcp_local `args[0]` as the *name of
the file to run*, not as an argument: install stores `script_path = "<id>/-y"`
and global_enable launches `npx /…/connectors/<id>/-y`, a path that does not
exist. The server never answered `initialize`, so start_server failed — and
since global_enable still returns 200 with an `error` in the body, the UI showed
the connector as enabled while `render_mcp_list` (built from the live runtime)
left it out of the agent's system context entirely.
Ship a two-file wrapper instead: a package.json pinning the upstream package
(mcp-fetch-server@1.1.2, firecrawl-mcp@3.23.7) and an index.js importing it for
its side effect, so `node index.js` is a real local_script and
ensure_installed_host installs the deps beside it. No node_modules, no lockfile.
Also: drop the legacy `launch_command` / top-level `transport` / `dependencies`
fields; drop firecrawl's inert `mcp_config.env` placeholder (same case as gmaps
— tokens are only substituted in the URL); add firecrawl_developer_search to
firecrawl's tools[] and API_KEY to its requires; realign manifest and fragment
versions to 5 / 1.1.0 (they were 2 vs 4, so no Update badge could ever appear).
Verified end-to-end against the shipped files by reproducing Skald's own path:
`npm ci || npm install --omit=dev` then `node <abs>/index.js` — initialize,
tools/list and a real tools/call, JSON-RPC only on stdout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The connector shipped no requirements.txt, so ensure_installed_host()
installed nothing and every tool call failed with "No module named
'googlemaps'" while the server still answered tools/list — skald logged
"connected — 6 tool(s)" on a connector that could not work.
- add requirements.txt (googlemaps>=4.10.0); deps were declared in the
manifest's `dependencies` field, which skald only uses for the card
- wire the shipped verify.py through a `verify` manifest block, so a
broken install fails visibly at enable time instead of silently
- verify.py puts .pydeps on sys.path: skald sets PYTHONPATH only for the
server process, not for the `sh -c "python3 verify.py"` verify step
- drop the inert mcp_config.env {SECRET:…} placeholder — those tokens are
substituted in the URL only, never in env values
- realign manifest/fragment versions (were 2/1.0.1 vs 5/1.0.4); skald
prefers the manifest, so "update available" could never appear
- requires ENV -> API_KEY; drop deprecated secrets/ path from error text
- regenerate the index
Daniele