Shoal exists because my local-first apps (mnemonic, habit-tracker, scalidraw) needed to move between devices without an account or a cloud I have to trust. The server stores an append-only, per-user log of encrypted operations, assigns each one a sequence number, and serves everything after a cursor. Every merge decision is made on the client. A server that only orders ciphertext has nowhere for app-specific logic to accumulate, so it has not changed as apps were added.

architecture

The server is an axum router with four routes: POST and GET /v1/ops, POST /v1/compact, and GET /v1/poke for server-sent events, plus a health check. Auth verifies an ed25519 signature over three X-Shoal-* headers with a 300 s clock-skew window. Storage is SQLite through rusqlite in WAL mode, two tables, with migrations keyed on PRAGMA user_version. About 1,900 lines of Rust.

The client has five modules: keys (BIP39 to keys, AEAD, request signatures), a hybrid logical clock, the sync engine (outbox drain, pull and merge, poke loop, backoff, compaction), a Dexie storage adapter over three tables inside the app’s own IndexedDB database, and errors.

the path of one write

1. The app calls record("habit/<id>", state) with the full record. The client stamps a clock value, generates a UUID op id, and enqueues plaintext into the outbox table. No network and no crypto yet. Oversized records are refused here so an undeliverable op cannot block the FIFO outbox.
2. On push, each payload is encrypted with XChaCha20-Poly1305, nonce prepended, with the collection and record id as associated data, then batched under both an op-count and a byte budget.
3. The server verifies the signature, decodes the payload to raw bytes so malformed data returns 400 instead of persisting, and appends inside one transaction that also enforces caps and assigns seq. A duplicate op id returns the existing seq, which makes retries idempotent.
4. The server broadcasts a poke carrying only {head: N} to that user’s SSE subscribers. A poke is a hint, never a data channel.
5. The other device pulls ?since=cursor, folds each remote clock into its own, skips its own echoes (an equal stamp is its own write), decrypts, and hands the winner to the app.

The server sees the collection name, record id, clock (which includes a device-stable node id), timestamps, payload sizes, and the public key. That list is written down in the protocol doc under “what the server can see”. It cannot read contents, forge ops, or move an op between records, because the associated data binds each ciphertext to its record. It can withhold ops or lie about head. There is no hash chain, and the docs say so.

technical decisions

• An op log of full record states, not a CRDT. Automerge and Yjs solve collaborative text. A playlist name or a habit completion needs a winner. Full-state ops make merge order-insensitive and make compaction trivially sound.
• Last-writer-wins on a hybrid logical clock. The clock is encoded as 12 hex digits of milliseconds, 4 of counter, 8 of node id, so lexicographic comparison is causal comparison and a merge is a string compare. Tables can opt into append-only instead, which is set union on unique ids.
• Keys from 12 words. BIP39 seed, then HKDF-SHA256 with shoal/v1/sign for the ed25519 seed and shoal/v1/enc for the encryption key. The consequences are documented: no forward secrecy, and key loss is data loss.
• No accounts and no cookies. Every request signs the method, the path with query, a timestamp, and the SHA-256 of the body. There is no ambient authority, which is why CORS defaults open. Users are created implicitly by the first valid signed push.
• SQLite, with head and op_count stored. Compaction leaves gaps, so MAX(seq) would overstate and COUNT(*) would be an O(ops) scan per push.
• HTTPS and SSE only. EventSource cannot set headers, so the client frames SSE by hand from fetch and signs that request like any other.

hard problems

• Compaction is scoped to one collection and capped at the client’s own cursor. It keeps the highest-clock op per record and leaves head untouched, so older cursors stay valid and seq is never reused. Tombstones survive it, so a device that went offline before a delete still learns about the delete.
• An op that fails to decrypt records its clock and fires onUndecryptable instead of wedging the whole sync.
• Backoff reads stream lifetime as a health signal: a poke stream that stayed up resets to the floor, one that dies at once keeps escalating.
• A 413 halves the batch and retries instead of stalling the outbox.
• Concurrent sync() calls join the in-flight round and schedule exactly one more, so a poke landing mid-sync is never lost.
• Schema migrations run in place inside a transaction, including a v0 to v1 rewrite of payloads from base64 text to blobs that rolls back leaving the file untouched.

numbers