Fixing a data loss bug in Zero

Zero is a local-first sync engine by Rocicorp. We use it in Dara to sync between the client and our API, and in prod we ran into 2 bugs around JWT expiry that were silently dropping writes.

Race condition

When a JWT expires, the server sends an AuthInvalidated error and then closes the websocket with code 3000. Both events call #disconnect() on the client. The error handler moves the state to NeedsAuth, and the close handler calls connecting() with a CleanClose reason.

connecting() in connection-manager.ts only guards against transitions from Closed and Disconnected:

if (this.#state.name === ConnectionStatus.Closed) {
  return;
}
if (this.#state.name === ConnectionStatus.Disconnected && !isHiddenDisconnect) {
  return;
}

NeedsAuth and Error are terminal states, and connecting() doesn't check for them. So if the close event fires after the error, connecting() overrides NeedsAuth and the client ends up in a 60-second retry loop with an expired token.

I opened #5500 with a fix + test case. Rocicorp shipped a cleaner version in #5504 that guards connecting() against all terminal states.

Zero-cache holds on to the old token

With the race fixed, NeedsAuth fired correctly and the client refreshed its token. But mutations kept failing with JWTExpired on the API side.

Zero-cache saves the auth token when the connection opens, in a private field:

// DEPRECATED: remove #token
// and forward auth and cookie headers that were
// sent with the push.
readonly #token: string | undefined;

Every push to the upstream API uses that token from then on. So when the client refreshes its JWT and sends a mutation, zero-cache forwards the old connection token and ignores the new one.

And when the API returns 401, zero-cache marks the mutation as processed. It doesn't retry and doesn't surface an error, so the write is lost.

Here's what it looked like in prod with a 5-minute token lifetime:

token issued:  00:44:44 (5 min lifetime)
token expires: 00:49:44

00:49:49 - zero-cache forwards token, expiresIn: -5s  → auth ok (60s tolerance)
00:50:24 - zero-cache forwards token, expiresIn: -40s → auth ok
00:50:52 - zero-cache forwards token, expiresIn: -68s → 401 FAIL

The client refreshed several times in that window, but zero-cache kept sending the 6-minute-old connection token.

Fixing it

I added an optional auth field to the PushBody schema, so the client can send its current token with each push. Zero-cache then uses the push auth if it's there and falls back to the cached connection token:

const authToken = msg[1].auth ?? this.#token;

It's backwards compatible - clients without the field get the old behavior. I bumped PROTOCOL_VERSION to 47 since the schema change affects the protocol hash. There was some discussion about whether the new field would break old servers, because connection.ts rejects unknown fields in schema validation. We kept the approach and documented that servers need to update before clients.

That's #5503, reviewed and merged by Matt Wonlaw. Rocicorp followed up with #5530, which does the same for changeDesiredQueries, so query auth now gets refreshed on each request too.