The road to Postgres: how we killed the 10-user hang

GlassChat Engineering · 2026-07-12

For a while, GlassChat had an embarrassing ceiling: put ten enthusiastic people in one group chat and the whole server could stall. Not crash — stall. Messages queued, typing indicators froze, and everything came back a few seconds later as if nothing happened.

The root cause

Every message, reaction, and read receipt was persisted by rewriting one giant JSON file — synchronously, on the same thread that serves every other user. One write meant: read the whole store, change one record, write the whole store back. At small scale you never notice. At "one lively group chat" scale, the event loop spends its life doing file I/O.

Worse, two near-simultaneous writes raced each other: both read the same snapshot, both wrote it back, and one of them silently lost. A reaction here, an unread badge there — small losses, structurally guaranteed.

Step one: a repository layer

Before touching the database we put an interface between the app and its storage: every entity got a repository with small, precise operations — append this message, patch this one message, bump this chat's unread counts. The JSON files stayed, but every write went through a per-file queue with atomic writes, and the race conditions died overnight.

That layer is the whole trick. Once every call site spoke "repository" instead of "file", swapping the storage engine underneath became a flag flip instead of a rewrite.

Step two: PostgreSQL

Each repository got a second implementation backed by PostgreSQL: a message append is one INSERT, a reaction is one row UPDATE guarded by a row lock, history is an indexed range scan. The load characteristics change completely — a thousand concurrent writers is a normal Tuesday for Postgres.

The migration nobody noticed

The cutover ran in stages, and the old store never stopped working:

  1. Backfill — every existing record copied into Postgres, byte-for-byte.
  2. Dual-write — every new write lands in both stores; the old one stays authoritative while we watch.
  3. Flip reads — Postgres becomes the source of truth; the old store keeps receiving every write as a live fallback.

Rollback at any stage is a one-line config change, with zero data loss, because nothing was ever turned off.

The part we care about most

GlassChat's end-to-end encrypted messages are ciphertext the server cannot read — that's the point. So the migration treats E2EE content as opaque bytes: never parsed, never decrypted (impossible anyway), never re-encrypted, copied verbatim, and guarded by tests that fail the build if a code path ever tries to rewrite ciphertext.

Faster infrastructure is nice. Faster infrastructure that provably can't peek at your messages is the actual requirement.