Slow
Links

A WebSocket in Node reports bufferedAmount: bytes the library still holds. It looks like the perfect signal: stop sending when it grows. The experiment says otherwise.

The transport's backpressure protects the sender's memory, but freshness depends on buffers the sender cannot see: the operating system's socket buffer, autotuned to hundreds of kilobytes, and every proxy on the way. A normal streamed reply (50 tokens per second, 23 bytes each) never fills them; a burst or a tunnel does. Only the receiving application knows what it has applied.

The rule: a provider has at most K = 2 state messages in flight per link. The consumer sends ack (three bytes in the binary codec) after it applied one. While two are in flight, new commits do not produce messages; their ops wait, merged per hold, and go out in the next message once an ack arrives.

The simulation above shows the same effect in miniature. Drag the link below the commit rate: the upper lane's queue grows without bound and the phone falls further behind every second; the lower lane never has more than two messages waiting, and the phone is at most a message behind.

While messages are in flight, each hold member keeps the ops of new commits in a pending queue and folds them: a new op merges into the last pending op on the same path when that is safe, so a backlog keeps the net change, not the history. When an acknowledgement frees a slot, the queue is encoded once into one message.

The merge partner is found through a leaf index: a map from path to the last pending append or primitive set on that path. Leaf ops on different paths commute, so a new op may merge into an earlier one, not only into its neighbour; a status field changing between two tokens no longer stops the reply from folding. Any structural op clears the index, because after a splice entries[7] may name another entry. Past 10,000 pending ops the queue keeps only "replace with the current value", which then costs nothing while the consumer stays slow.

Why no size rule. The first idea was to send a snapshot once the merged ops outgrow the member. The reviewers measured it on real p2 data and it failed three ways: "snapshot bytes plus sent op bytes" is no bound (an interned path lets a value grow faster than its ops: 5,404 B of snapshot after 832 B of ops), it drifts 1,254 times too high under churn, and an exact size needs a size on every node of the value. Folding alone came within about 10% of an exact oracle, with no measuring at all.

One queue for every slow consumer. The same queue serves Chord's shared watch handle: a listener still busy with the last frame gets the commits made meanwhile as one call with the newest value and the folded ops, instead of today's "after 100 frames, replace everything". With a listener taking one second, a conversation view's watch went from 402 calls and 17.9 MB of ops to 312 calls and 257 KB. Durable's event stream keeps a queue of its own, because events are not state: folding would drop them.

Animation of the fold: How folding works.

Calls are small and the provider acknowledges nothing until they finish, so a byte or message window on state says nothing about how many handlers run. A tab with a bug:

for (const note of notes) ref.add(note, ctx);   // 100,000 calls, none awaited

The provider's hello announces a call window N (256 by default) per origin (chapter VI). A consumer keeps at most N of its own calls unanswered per link; further calls wait in the link's call queue, in order. Nothing is refused (K16); a peer that sends more than N is a protocol error. The flood waits in the tab that caused it. CPU-heavy handlers block the event loop whatever the window: that stays the handler author's responsibility.

The server relays every tab's holds to a session's worker over one link. Three ways to apply the acknowledged window across that hop were considered:

How (c) works:

• Origins. The relay gives each downstream link a number, never reused; every message upstream carries it (origin), and the worker keys holds, calls, windows and acks by (link, origin, id). Ids are not renumbered.
• What the relay keeps. Per relayed hold its upstream link and its original hold message (to route calls and hold again after a worker restart); per tab a FIFO of which worker sent each message in flight, so each in-order ack goes back to the right worker. Nothing is parsed, folded, dropped or reset.
• Calls. N is per origin at the worker; the relay forwards calls at once, each tab bounded by its own window.
• A tab leaves. Its link closes; the relay sends closed { origin } to each worker it relayed to, which releases that origin's holds and aborts its calls.
• Cost. A tab served by W workers may have 2W messages in flight (plus the relay's own 2). Two bytes of origin per message on the worker link.