Caching at Scale
Cache tiers, read and write cache patterns, invalidation strategies, negative caching, cache stampede protection, and cache observability.
7 questions
JuniorTheoryVery commonName the main caching patterns and how each handles reads and writes.
Name the main caching patterns and how each handles reads and writes.
cache-aside: the app checks the cache, on a miss reads the DB and fills the cache itself (most common). read-through: the cache loads from the DB on a miss transparently. write-through: write to the cache then synchronously to the DB — consistent but slower. write-back: write to the cache, flush to the DB asynchronously — fastest but loses data on crash.
Common mistakes
- ✗Saying
cache-asideandread-throughare the same — in cache-aside the app populates the cache, in read-through the cache does. - ✗Calling
write-backsafe for critical data; an async flush means a crash can lose writes the DB never saw. - ✗Assuming
write-throughspeeds up writes — it adds the cache write on the synchronous path, so writes get slower.
Follow-up questions
- →When would you choose write-back despite the risk of losing data on a crash?
- →How does cache-aside handle a write — and what staleness does that cause?
MiddleTheoryVery commonHow do you keep cached data fresh, and what do TTL-only, event-based, and tag-based invalidation trade off?
How do you keep cached data fresh, and what do TTL-only, event-based, and tag-based invalidation trade off?
TTL-only lets entries expire, so reads can be stale up to the TTL but writes stay simple. Event-based deletes or updates the key the instant data changes. Tag-based groups keys under a tag and drops the whole tag at once. Invalidating L1 and L2 together is hard.
Common mistakes
- ✗Relying on TTL-only for data that must be fresh, then being surprised reads are stale until the TTL elapses
- ✗Invalidating only L2 and forgetting that each instance's L1 copy keeps serving the old value
- ✗Updating the key on write but never handling the delete, so stale entries linger after a record is removed
Follow-up questions
- →How would you propagate an invalidation to every instance's L1 cache?
- →When is allowing staleness up to the TTL an acceptable trade-off?
JuniorTheoryCommonWhat are the cache tiers from browser to database, and what does each trade off?
What are the cache tiers from browser to database, and what does each trade off?
Browser cache, then CDN/edge, then an L1 in-process cache (per-instance, fastest, but not shared and can go stale across instances), then a shared L2 distributed cache like Redis, then the DB as source of truth. Each tier trades latency for hit-rate and consistency.
Common mistakes
- ✗Calling the L1 in-process cache shared across instances; it is per-instance and can diverge between them.
- ✗Treating the L2 distributed cache as the source of truth; the DB is, and the cache can always be cold.
- ✗Ignoring the latency-versus-consistency trade-off and assuming more cache tiers are always strictly better.
Follow-up questions
- →When would you skip the L1 cache and read straight from
Redis? - →How do you keep two instances' L1 caches from drifting apart?
MiddleTheoryCommonWhat is a cache stampede under a cache-aside pattern, and how do you prevent it?
What is a cache stampede under a cache-aside pattern, and how do you prevent it?
A cache stampede is when a hot key expires and many concurrent requests miss at once, all hammering the DB (a thundering herd). Mitigate with singleflight coalescing (one rebuild, others wait), a rebuild lock, early refresh, and staggered TTLs.
Common mistakes
- ✗Thinking a single longer TTL fixes it — it only delays the stampede to one big synchronized expiry instead of preventing it.
- ✗Confusing a stampede with cache penetration; coalescing rebuilds a present key, negative-caching guards a missing one.
- ✗Assuming per-instance
singleflightis enough at scale — it coalesces within one process, not across many service replicas.
Follow-up questions
- →How does per-instance
singleflightdiffer from a distributed Redis lock for coalescing? - →What is probabilistic early expiration and when does it beat a fixed early refresh?
JuniorTheoryOccasionalWhat is negative caching and when should you use it?
What is negative caching and when should you use it?
Negative caching stores the ABSENCE of data — e.g. a 404 from a catalog lookup — under a SHORT TTL, so repeated requests for a missing key are served from the cache instead of all falling through to the DB (cache penetration). Cache only stable negatives, not transient 5xx.
Common mistakes
- ✗Caching
5xxerrors so a transient outage gets pinned in the cache long after the service recovers - ✗Reusing the normal long TTL for negatives, keeping a
404cached long after the item is created - ✗Skipping negative caching entirely, letting every miss for a hot missing key hammer the DB
Follow-up questions
- →How do you size the negative TTL versus the positive one for the same key?
- →How does negative caching interact with the failure mode cache penetration from random unknown keys?
MiddleTheoryOccasionalWhich metrics do you track to tune a multi-tier cache, and what do they tell you?
Which metrics do you track to tune a multi-tier cache, and what do they tell you?
Track hit rate, miss rate, eviction rate, and lookup latency per tier. A low hit rate means wrong TTLs or wrong keys; a climbing eviction rate means the cache is too small for the working set. Watch Redis memory and maxmemory evictions on L2. You can't tune a cache you don't measure.
Common mistakes
- ✗Tracking only the hit rate and ignoring eviction rate, so a too-small cache goes unnoticed
- ✗Treating a low hit rate as harmless instead of a signal of wrong TTLs or keys
- ✗Never watching
Redismemory andmaxmemoryevictions, so the L2 tier silently thrashes
Follow-up questions
- →What does a high hit rate paired with rising latency usually tell you?
- →How would you alert on a sudden drop in cache hit rate?
SeniorDesignRareDesign a multi-tier cache in Go for a read-heavy product catalog: updates must reflect within seconds, lookups for non-existent SKUs are frequent, and hot products spike the DB when their keys expire.
Design a multi-tier cache in Go for a read-heavy product catalog: updates must reflect within seconds, lookups for non-existent SKUs are frequent, and hot products spike the DB when their keys expire.
Layer an L1 in-process cache plus an L2 Redis cache, both fronting the DB with cache-aside. On a product update, do event-based invalidation that deletes the key in both tiers, backed by short TTLs so drift self-heals. Negative-cache missing SKUs with a short TTL to stop penetration. Coalesce hot-key rebuilds with singleflight plus staggered or early refresh to kill stampedes, and export hit, miss, eviction, and latency metrics.
Common mistakes
- ✗Invalidating only by TTL, so a product update is invisible for the whole TTL instead of dropping the key on the event
- ✗Skipping negative caching, so repeated lookups for missing SKUs all penetrate to the DB
- ✗Letting a hot key's expiry cause a stampede because rebuilds are not coalesced behind one flight
Follow-up questions
- →Where do you publish the update event, and how do you invalidate both tiers reliably?
- →How do you pick TTLs so the L1 and L2 tiers do not serve conflicting versions?
