Engineering problems / Amazon

Stop a cache stampede.

~5 minutesKnow what a cache TTL isFree to explore

BY THE END, YOU CANChoose a stampede defence and name the new failure it introduces.

01 / Try the idea

One popular key has just expired. Twenty clients arrive over the 300ms it takes to rebuild it. Count the queries that reach the database.

client 1
arrives at 0ms, waits 300ms, queries the database
client 2
arrives at 15ms, waits 300ms, queries the database
client 3
arrives at 30ms, waits 300ms, queries the database
client 4
arrives at 45ms, waits 300ms, queries the database
client 5
arrives at 60ms, waits 300ms, queries the database
client 6
arrives at 75ms, waits 300ms, queries the database
client 7
arrives at 90ms, waits 300ms, queries the database
client 8
arrives at 105ms, waits 300ms, queries the database
client 9
arrives at 120ms, waits 300ms, queries the database
client 10
arrives at 135ms, waits 300ms, queries the database
0 ms585 ms
queries the databasewaitingserved from cache
20 of 20 reach the database

Worst wait 300ms · 20 clients blocked

Every client that arrives before the first rebuild finishes misses, and asks the database the same question. Twenty identical queries; nineteen of them are waste.

Showing the first ten of 20 clients. A fixed teaching model with one key and one recompute time, not a benchmark.

02 / Check your understanding

You add a mutex so only one request rebuilds the value. What new failure have you bought?

Read the idea in plain text

Twenty clients arrive over the 300ms it takes to rebuild one expired key. With no protection, all twenty miss and all twenty query the database — nineteen of those queries return the same answer as the first. With coalescing, one client rebuilds and the other nineteen wait, so the database sees one query and everybody pays up to 300ms of latency. With an early background refresh, the value never expires under traffic: zero queries, zero waiting. But if the client holding the coalescing lock dies, the other nineteen wait for the lock to expire — turning a 300ms spike into a five-second hang.

This is a small teaching model. Use it to reason about a design, then check the details against your own system.

Stop a cache stampede.

MockStruct / Quick Learn · 31 October 2026

https://mockstruct.com/learn/cache-stampede

Your one-minute recap

  1. A stampede is a concurrency bug in your cache, not a capacity problem in your database.
  2. Coalescing collapses N misses into one rebuild; everyone else waits on it.
  3. Every defence moves the failure: now the lock holder dying is your outage.

Primary source: https://builder.aws.com/content/3K9TeXp6strMiJ4vhakpRD7BQWO/caching-challenges-and-strategies

03 / Connect the dots

See the engineering behind the idea.

Read the full explainer →
Next lesson →

Built from Amazon: Caching challenges and strategies (Jas Chhabra, 5 December 2019). Original MockStruct teaching model; no affiliation. Report a correction.