How production systems work, and how they fail.

Queues, caches, databases, and model serving: what each one does under load, where it breaks, and how to tell before it does.

Start here

InferenceJan 2026
5 min
KV cache explained
Generating each new token would otherwise recompute attention over the whole sequence. The KV cache stores that work so each step is incremental. It is also the reason serving is bounded by memory rather than compute.
Distributed systemsFeb 2018
5 min
Redis as a queue: when it works and when it loses jobs
A Redis list is a workable job queue until a worker crashes, Redis restarts, or memory fills. Each of those loses jobs under the default settings, and each has a specific fix.
Distributed systemsMar 2020
5 min
Backpressure in software: what it is and how to apply it
When a producer is faster than its consumer, something has to give. Backpressure is the set of mechanisms by which the consumer makes the producer slow down, instead of the system falling over.
Distributed systemsApr 2019
5 min
Idempotency: designing operations that survive retries
A network call can fail after the server has done the work. The client does not know, so it retries. Idempotency is what makes that retry safe, and it has to be designed in, not added later.