Skald-Circle/docs/relay/pipe.md

Pipe — Relayed Byte-Stream over Skald Relay

Implementation reference. A generic, content-blind, end-to-end-encrypted byte-stream channel between two members of a namespace, relayed (TURN-style) through the Skald relay. It sits ON TOP of the existing transport: signaling rides the existing E2E Message channel (no new RelayFrame); the data plane is one new relay endpoint (/v1/pipe). The relay splices opaque ciphertext and never reads it.

Status (v1, implemented). Scope = client↔agent (the shared E2E key already exists, so the ephemeral handshake is authenticated by the channel that carries it). Suite = x25519-sealed. Compression = none (negotiation present for forward-compat). client↔client is deferred (needs a signed roster/manifest + a self-authenticating suite — see §7).

Read after: index.md → relay-protocol.md → crypto.md → framing.md.

1. Why

The message channel (relay-protocol.md) is for discrete E2E payloads (approvals, clarifications, health sync): ≤60 msg/min, ≤512 KiB/frame, store-and-forward. It serves stream-shaped, high-volume flows poorly — log tailing, file transfer, audio, remote shell, real-time sensors. The pipe is the reusable streaming primitive for those. It is TURN's relayed mode: a control plane brokers a rendezvous; a separate connection carries a raw encrypted byte stream the relay blindly splices, so TCP/WS gives reliability/ordering/flow-control for free (no reinvented windowing).

        Control plane (existing E2E Message channel)         Data plane (new WSS /v1/pipe)
A ──pipe_invite (live)──▶ R ──▶ B                    A ──▶ R ◀── B   (each dials out; NAT-friendly)
A ◀──pipe_accept (live)── R ◀── B                    R verifies auth, matches by connection_id,
   (ephemeral X25519 exchanged → per-pipe key, PFS)     then splices opaque ciphertext frames
   B offline ⇒ A gets PeerOffline ⇒ abort            A ⇄ B: AES-256-GCM stream, relay sees ciphertext

2. Control plane — signaling

Pipe signaling rides the existing Message{live=true} E2E frame. It is not a new RelayFrame; the relay stays content-blind. To distinguish it from JSON app payloads on the same channel, the decrypted plaintext uses a reserved framing header (crypto.md §1):

FRAMING_VERSION_PIPE (0x02) ‖ COMP_NONE (0x00) ‖ <MsgPack PipeSignal>

The receiver peeks the first byte (crypto::is_pipe_signal): 0x02 ⇒ route to the pipe layer; 0x01 ⇒ the existing JSON app path, unchanged. live=true is required — a stale "please connect" is useless; if the peer is offline the initiator gets PeerOffline (relay-protocol.md §6.4) and aborts.

Wire format = MsgPack (rmp-serde, named maps). PipeSignal is externally tagged ({ "Invite": {…} }) so a blob is self-describing. Byte fields are length-validated on decode.

Message	Fields
Invite	connection_id (32B), suite, handshake (opaque; initiator ephemeral X25519 pub for x25519-sealed), stream_type (app-defined), compress (advertised list), headers (arbitrary String→String)
Accept	connection_id, suite, handshake (responder ephemeral pub), compress (selected codec)
Reject	connection_id, reason

• connection_id: 32 random bytes, single-use, short-lived. The rendezvous key, known only to A and B (sent E2E). Not a security boundary on its own — the data-plane signature (§3) is.
• suite is a discriminator and handshake is opaque: adding a Noise suite (§7) is a new variant with the same wire shape. Signaling is symmetric (initiator/responder by role, never agent-vs-client) so client↔client is not blocked.
• headers: app metadata for the stream (filename/size for a transfer, filters for a log tail).

By pipe_accept both sides have the peer's ephemeral pubkey and derive the per-pipe key (§4).

3. Data plane — WSS /v1/pipe

A second WebSocket, separate from the control WS, binary frames carrying raw bytes (no protobuf). Chosen over HTTP CONNECT / raw TCP for reachability: 443/TLS, traverses CDN / L7 LB / mobile carriers, camouflaged as a normal WS. The socket is the tunnel (one connection per pipe); the control WS stays separate and alive.

3.1 Auth handshake (relay-mediated, MsgPack)

Mirrors the main WS "relay speaks first":

A → WSS /v1/pipe
R → PipeChallenge { nonce: 32B }                       (relay speaks first)
A → PipeAuth {
      connection_id, pubkey (ed25519, 32B),
      dest = SHA256(peer_ed25519_pub) (32B),            (declares intended counterparty)
      namespace_id (raw 32B),
      signature = sign_ed25519(priv, PIPE_AUTH_DOMAIN ‖ 0x00 ‖ nonce ‖ connection_id) (64B)
    }
R verifies, in order:
  1. signature valid under pubkey (verify_strict)                  → else close
  2. pubkey is the agent of namespace_id, OR an authorized client  → else close
  3. (on the second side) cross-refs match (§3.2)                  → else close both

The reply is a signature, not an echo — it proves control of pubkey, exactly like the main WS auth. connection_id is not trusted as identity.

3.2 Matching & splice (relay state machine)

challenge → pipe_auth → pending → matched → streaming → teardown

• pending: first authenticated side for connection_id is parked (TTL); the namespace pipe count is incremented.
• matched: second side authenticates → relay verifies the cross-refs SHA256(A.pubkey)==B.dest AND SHA256(B.pubkey)==A.dest AND same namespace, then hands the second side's socket halves to the first.
• streaming: the first side owns a bidirectional forward loop of binary-frame payloads. The relay reads nothing else; WS-level pings are answered on the originating leg; data is rate-limited per direction.
• teardown: either side closing/erroring tears down both (no orphans). (v1 closes both; FIN half-close propagation is a future refinement.)

3.3 Relay limits (NORMATIVE; env-overridable)

The relay becomes a stateful connection proxy (TURN resource model): fd+buffers per pipe, idle reaping, pending TTL, per-namespace concurrency cap, backpressure (no unbounded buffering).

Limit	Env var	Default	Why
Pending half-open TTL	RELAY_PIPE_PENDING_TTL_SECS	30 s	A dialed, B never showed → reap.
Idle pipe timeout	RELAY_PIPE_IDLE_TIMEOUT_SECS	120 s	Reclaim dead pipes.
Max concurrent pipes / namespace	RELAY_PIPE_MAX_PER_NS	8	Bound proxy resource use.
Max data-plane frame	RELAY_PIPE_MAX_FRAME_BYTES	1 MiB	Bulk transfer; separate from the message-channel quota.
Bandwidth cap (per connection, per direction)	RELAY_PIPE_MAX_BPS	0 (unlimited)	Token bucket; stops the pipe being a free unmetered tunnel.

4. Secure channel — reused AES-256-GCM, ephemeral DH (PFS)

The A↔B stream reuses the existing crypto primitives (crypto.md), not Noise/TLS — the same AES-256-GCM / X25519 / HKDF stack already interop-tested against the iOS client:

• Per-pipe key: each side samples a fresh ephemeral X25519, exchanges the pubkey in the signaling, and computes pipe_key = HKDF(ECDH(eph), salt=PIPE_KDF_SALT, info=PIPE_KDF_INFO). Ephemeral DH ⇒ Perfect Forward Secrecy per pipe (closes the gap in index.md §4.3 for this channel).
• Authentication: the ephemeral pubkeys travel inside the E2E-sealed signaling, so for client↔agent they are authenticated by the existing channel — no signatures needed in the handshake (that is the x25519-sealed suite). client↔client (no pre-shared key) needs a self-authenticating suite (§7).
• Frame crypto: each chunk is AES-256-GCM(pipe_key, nonce, aad). The 12-byte nonce is DIR (4B) ‖ counter (8B) with a per-direction counter (DIR_PIPE_INITIATOR / DIR_PIPE_RESPONDER), not transmitted (reconstructed by the receiver — strict in-order WS/TCP delivery). aad = connection_id (binds frames to the rendezvous). Counters start at 1.

The relay never holds pipe_key; mismatched keys fail the GCM tag (confidentiality holds even if the relay mis-splices).

5. Compression

Negotiated in the handshake (compress advertise→select), per direction, none | zlib. v1 ships none only — the negotiation field exists for forward-compat. Stateful streaming zlib is deferred (it is a shared-dictionary deflate context, and on a generic bus it reintroduces the CRIME/BREACH class for stream_types mixing attacker-controlled plaintext with secrets).

6. Library API (skald-relay-client)

RelayClient::open_pipe(peer, stream_type, headers) -> PipeConnection      // initiator
RelayClient::incoming_pipes() -> broadcast::Receiver<IncomingPipe>        // responder feed
RelayClient::accept_pipe(&IncomingPipe) -> PipeConnection                 // responder
RelayClient::reject_pipe(&IncomingPipe, reason)
PipeConnection::{ send(&[u8]), recv() -> Option<Vec<u8>>, close() }       // sealed/opened transparently
PipeConnection::split() -> (PipeSender, PipeReceiver)                     // independent halves
PipeSender::send(&[u8])          // seals + queues; blocks only when the send buffer is full
PipeReceiver::recv() -> Option<Vec<u8>>

Inbound pipe invites surface on a separate channel (incoming_pipes), not as a RelayEvent variant — so adding the pipe is purely additive and the plugin-mobile-connector consumer compiles unchanged. The relay client owns the pipe control plane end-to-end (it intercepts only the pipe_* signaling kinds; every other payload stays pass-through).

6.1 Full-duplex & client-side backpressure

A PipeConnection is full-duplex: on connect the data-plane socket is split into a sink + stream and a background writer task takes the sink. send seals the chunk and hands it to the writer (returning before the flush); recv reads the stream directly. So a slow flush on one direction never stalls the other — and split() -> (PipeSender, PipeReceiver) lets each direction run in its own task with no shared &mut. The per-direction counter nonce stays ordered because send is single-writer (&mut self) and the writer drains FIFO.

Backpressure is an in-memory byte budget (SEND_BUFFER_BYTES, ~10 MiB) held as a semaphore: send reserves the sealed frame's bytes and the writer releases them after the frame is flushed. While the socket drains normally send returns immediately; if the peer/relay stops draining, the budget empties and send blocks until space frees up (bounding memory). This sits under the relay's own per-pipe limits (§3.3). WS-level pings stay responsive: the read half forwards Pongs to the writer on a separate control channel that the budget never throttles. Dropping both halves (or close()) tears the pipe down — the writer closes the socket once its channels are gone.

6.2 Agent-side consumers (by stream_type)

• http-local-proxy — plugin-mobile-connector (src/proxy.rs) accepts these pipes and reverse-proxies each, byte-for-byte, to the local web server at 127.0.0.1:<web_port>, letting a native WebView render the Skald web UI over the relay (no NAT hole / Tailscale). It splits each pipe and runs the two directions in separate tasks (remote→local writes to the web server; local→remote reads it and sends back, backpressured by the send buffer), so a stalled direction can't block the other. Destination is pinned (not client-chosen); access is already gated by §3.1 (agent or authorized client). See ../plugins/mobile-connector.md.

7. client↔client (deferred)

The data plane is already client↔client-capable: the relay authenticates by namespace membership + cross-dest, not by agent-vs-client. Two things are missing above it, both additive:

1. Key/identity distribution — clients don't know each other's keys. Plan: the agent signs a manifest (versioned roster of authorized members' pubkeys) the relay caches and serves; clients verify the agent's ed25519 signature.
2. Self-authenticating handshake — without a pre-shared client↔client key the ephemeral exchange can't be sealed in an existing channel. Plan: a new suite (e.g. noise-nn+ed25519) — same signaling wire shape, new handshake interpretation. The relay does not change.

8. Verification (implemented)

• relay-common (crypto.rs, pipe.rs tests): MsgPack round-trips; pipe_auth sign/verify binds nonce + connection_id (and rejects an AUTH_DOMAIN signature — domain separation); pipe-key symmetry over ephemeral DH; pipe-signal framing peek.
• relay-server (tests/pipe.rs): two raw WS peers → pending→matched→streaming→teardown; rejects bad signature, cross-dest mismatch, non-member; teardown closes the peer.
• relay-client (src/pipe.rs net tests): two PipeConnections stream bytes (incl. a 200 KiB blob) both ways through the real relay; a wrong key fails AEAD open (relay never had plaintext); split both ends and stream 1 MiB each way simultaneously (full-duplex — neither direction blocks the other). Signaling routing (src/state.rs): invite → incoming_pipes, accept/reject → the waiter.
• Non-regression: plugin-mobile-connector builds unchanged; existing protocol.rs / integration.rs suites still pass.

9. Out of scope (deferred)

• Stateful zlib compression (negotiation reserved).
• HTTP/2 extended CONNECT multiplexing (many pipes over one socket).
• client↔client (§7) and the specific app stream_type consumers.