Engineering problems / Amazon
Stop a cache stampede.
BY THE END, YOU CANChoose a stampede defence and name the new failure it introduces.
01 / Try the idea
One popular key has just expired. Twenty clients arrive over the 300ms it takes to rebuild it. Count the queries that reach the database.
Worst wait 300ms · 20 clients blocked
Every client that arrives before the first rebuild finishes misses, and asks the database the same question. Twenty identical queries; nineteen of them are waste.
Showing the first ten of 20 clients. A fixed teaching model with one key and one recompute time, not a benchmark.
02 / Check your understanding
Read the idea in plain text
Twenty clients arrive over the 300ms it takes to rebuild one expired key. With no protection, all twenty miss and all twenty query the database — nineteen of those queries return the same answer as the first. With coalescing, one client rebuilds and the other nineteen wait, so the database sees one query and everybody pays up to 300ms of latency. With an early background refresh, the value never expires under traffic: zero queries, zero waiting. But if the client holding the coalescing lock dies, the other nineteen wait for the lock to expire — turning a 300ms spike into a five-second hang.
This is a small teaching model. Use it to reason about a design, then check the details against your own system.
Stop a cache stampede.
MockStruct / Quick Learn · 31 October 2026
https://mockstruct.com/learn/cache-stampede
Your one-minute recap
- A stampede is a concurrency bug in your cache, not a capacity problem in your database.
- Coalescing collapses N misses into one rebuild; everyone else waits on it.
- Every defence moves the failure: now the lock holder dying is your outage.
Primary source: https://builder.aws.com/content/3K9TeXp6strMiJ4vhakpRD7BQWO/caching-challenges-and-strategies
Built from Amazon: Caching challenges and strategies (Jas Chhabra, 5 December 2019). Original MockStruct teaching model; no affiliation. Report a correction.