dguiducci/skald-connectors
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

whatsapp: persist chats and messages across restarts (v8 / 2.2.0)

Browse Source 
WhatsApp delivers a history sync at login, not on reconnect, so an in-memory
store left the connector blind after every process restart (23 in two weeks).
The store is now mirrored to store/, next to auth/ and bind-mounted the same
way.

Not SQLite: node:sqlite needs Node 22 (unflagged only from 24) and the runtime
image ships Debian trixie's nodejs = 20.19.2; better-sqlite3 is native and the
slim image has no toolchain. So an append-only JSONL log for messages plus a
debounced JSON snapshot for chats/contacts, both written tmp+rename so a crash
cannot truncate them. Compaction on load and every 500 appends keeps the log
from creeping upward.

Also fixes a pre-existing duplication bug: pushMessage appended unconditionally,
re-adding every message a history re-sync redelivered. It now returns false on a
known id and the message is skipped in both the transcript and the log.

MAX_MSGS_PER_CHAT 200 -> 500. logout deletes store/ with auth/, so re-linking a
different phone cannot inherit the previous account's history.

Data at rest: message text is now written to the user's bind-mounted home.
Nothing but session keys was persisted before.

Verified on skald-runtime:v4 with a seeded store: 751 lines with 50 duplicates,
a 700-message chat and a torn trailing line -> 701 loaded, compacted to 501, cap
applied, second run 501 -> 501 unchanged.
This commit is contained in:
Daniele
2026-08-23 18:22:50 +01:00
parent a2bbedad40
commit 0c04ba2e4c
5 changed files with 145 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+10
View File
@@ -9,6 +9,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## 2026-08-23
### Added
- **whatsapp: chats and messages now survive a restart (v8 / 2.2.0)** — closes the gap left open by v7. WhatsApp delivers a history sync at *login*, not on reconnect, so with a purely in-memory store every process restart left the connector blind until new messages happened to arrive — and the process had restarted 23 times in two weeks. The store is now mirrored to `store/`, next to `auth/` and bind-mounted the same way, so it outlives both a restart and a container recreate.
- **No SQLite, deliberately.** `node:sqlite` needs Node 22 (and is only unflagged from Node 24); the runtime image ships Debian trixie's `nodejs` = 20.19.2. `better-sqlite3` is a native module the slim image has no toolchain to build. So: `store/messages.jsonl`, an append-only log (one `appendFileSync` per message, O(1)), plus `store/meta.json`, a debounced snapshot of chats/contacts (5s, since they churn in bursts during a history sync). Both written via write-tmp-then-`rename`, which is atomic — a crash mid-write cannot truncate the store.
- The log is compacted on load and every 500 appends: the capped in-memory Maps are re-serialised, so the file cannot creep upward across restarts.
- **Fixes a pre-existing duplication bug in the process.** `pushMessage` appended unconditionally, so every history re-sync re-added messages already held. It now returns false on a known message id (a ≤500-element scan per message) and `ingestMessage` skips both the transcript and the log — without it, persistence would have multiplied the duplicates once per restart instead of merely showing them.
- `MAX_MSGS_PER_CHAT` 200 → 500, worth more now that it is not thrown away at every restart. `logout` deletes `store/` along with `auth/`, so re-linking a different phone cannot inherit the previous account's history.
- ⚠️ Data at rest: message text is now written to disk in the user's bind-mounted home. Nothing was persisted before this change beyond the session keys.
- Verified on `skald-runtime:v4` with a seeded store — 751 log lines containing 50 exact duplicates, a 700-message chat, and a deliberately torn trailing line → loaded as 701 messages (duplicates collapsed, torn line skipped, no crash), compacted to 501 lines, the 700-message chat trimmed to its most recent 500, `meta.json` round-tripped, and a second run reloading 501 → 501 unchanged ✅
### Fixed
- **whatsapp: history sync silently disabled, and Signal sessions corrupted by reconnects (v7 / 2.1.0)** — diagnosed from the live server, where two users (two separate accounts, separate `auth/` dirs) share one log file. Four distinct defects, plus a dependency upgrade.