Caching at Scale
A cache exists to offload the database and external APIs: instead of an expensive query to the source of truth, the service returns an already-computed answer from memory or from Redis. While traffic is small, a cache looks like a free speedup. At scale it becomes a distributed system of its own with its own failures — drift between replicas, stale data, and bursts to the database at the exact moment a key expires.
The central trap of this topic is treating a cache like an in-memory dictionary rather than a tier with its own consistency semantics. A local L1 cache is fast but not shared: one instance updates the data while the others still serve the old value. A write-through-the-cache pattern with an asynchronous flush to the database loses data on a process crash. A key with no TTL and no invalidation serves stale data forever. And the most insidious failure is the cache stampede: a hot key expires, hundreds of concurrent requests miss at once and hit the database simultaneously. This topic breaks caching into layers — from the hierarchy of tiers down to the metrics you tune it by.
Topic Map
- Cache tiers — a hierarchy from the browser and CDN through an in-process L1 to an L2 in Redis and the database as the source of truth, trading latency, hit rate, and cross-instance consistency.
- Caching patterns — cache-aside, read-through, write-through, and write-back and their trade-offs between consistency and write speed.
- Cache invalidation — TTL, event-based, and tag-based invalidation, and why "expiry" is one of the two genuinely hard problems.
- Negative caching — caching the absence of data with a short TTL against penetration, without caching transient errors.
- Cache stampede — a thundering herd when a hot key expires, and how to tame it with coalescing, a lease, early refresh, and staggered TTLs.
- Cache observability — hit/miss/eviction rate, latency, and Redis memory: you can't tune a cache you don't measure.
Common Mistakes and Traps
| Mistake | Consequence |
|---|---|
| Keeping only an in-process L1 cache and treating it as "shared" | Each instance sees its own — drift and stale data between replicas |
| Treating any cache tier as a free speedup | A cache is a distributed system with its own failures and consistency |
| Writing to the cache with an async flush to the database, unwatched | Write-back loses data on a process crash before the flush |
| Living on TTL alone for changing data | The client gets stale data the entire time until the key expires |
| Not invalidating a key when the data changes | Dirty data is served until the TTL expires |
| Not caching the absence of data | Cache penetration — every lookup of a missing key hits the database |
Caching transient 5xx errors as a negative result | A stuck failure is served to the client for the whole TTL |
| Leaving a hot key with a single shared expiry moment | A cache stampede — all misses hit the database at once on expiry |
| Not measuring the cache's hit/miss/eviction rate | You can't see wrong TTLs, evictions, or growing Redis memory |
Interview Relevance
Caching is a mandatory topic at the senior level of a Go interview in the system-design part, and the question is not "do you know Redis" but whether you understand a cache as a tier with its own consistency semantics and failures. The interviewer checks whether you keep in mind that an in-process cache is not shared between replicas, that each write pattern has its own speed-vs-durability trade-off, and that the expiry of a hot key is not a minor detail but a distinct failure mode.
What interviewers usually check:
- Which tiers make up the cache hierarchy (browser, CDN, in-process L1, L2 in Redis, the database) and what goes on each by latency and consistency.
- How cache-aside, read-through, write-through, and write-back differ — especially in consistency with the database and the risk of data loss.
- Which invalidation strategies exist (TTL, event-based, tag-based) and why invalidation is hard.
- Why negative caching with a short TTL is needed and why you must not cache
5xx. - What a cache stampede is and how it is tamed — coalescing/singleflight, a lease, an early or probabilistic refresh, staggered TTLs.
- Which cache metrics to track and what a low hit rate tells you.
A typical wrong answer: "put everything in Redis with a TTL and forget it — the cache will sort itself out." This triggers a discussion that a single TTL solves neither invalidation when data changes, nor penetration on missing keys, nor a stampede when a hot key expires, and that without hit/miss metrics such a cache can be neither tuned nor noticed when it stops helping.