When Not to Cache
A cache buys you a consistency problem and an availability dependency. Sometimes it does not buy anything back.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.
When is adding a cache the wrong answer, and what should be tried first?
An endpoint is slow. Someone in the review says "just cache it", and everyone nods.
Caching is a performance best practice. If a read is slow, put a cache in front of it — the worst case is that it does not help much.
The worst case is not "it does not help much". The worst case is a stale-data bug six weeks later that nobody connects to the cache, in a code path nobody remembers is cached.
- The worst case is not "it does not help much". The worst case is a stale-data bug six weeks later that nobody connects to the cache, in a code path nobody remembers is cached.
- A cache in front of a single indexed primary-key lookup replaces one fast query with one network round trip plus serialization. That can be a net loss before any consistency cost is counted.
- A low hit rate makes it strictly worse: you pay the lookup on every request and still do the original work on most of them.
- The slow endpoint was slow because of an N+1, a missing index, or an unbounded result set. The cache hides it — until an invalidation, a deploy or a cold start exposes the original problem to full traffic at once (Cache Stampede).
- Now you own a second data store on the read path. Whatever availability your service had, it now has that multiplied by the cache's, unless you designed the fallback deliberately.
What is actually happening
- A cache pays off when three things are true at once: the data is read far more than it is written, the *same keys* recur, and someone can name a tolerable staleness. Miss any one and the arithmetic stops working.
- Low hit rate kills it directly. If most requests miss, you have added a round trip to every request in order to occasionally skip work.
- Cheap origin work kills it too. When the query is a primary-key lookup that the database serves from its own buffer pool, the cache is competing with something already fast, over a network.
- Write-heavy data kills it a third way. If a value is invalidated as often as it is read, every populate is wasted work and the cache is a slower path to the same answer.
- Correctness requirements kill it outright. Anything a user can dispute — a balance, an inventory count at the moment of purchase, a rate limit that must be enforced, a permission that must be revocable now — cannot be served from a copy that is allowed to be wrong (Cache Invalidation).
- The alternatives are usually cheaper *and* have no consistency cost: an index, a fixed N+1, a smaller payload, pagination, or a query that returns the ten rows you display instead of the ten thousand you filter in application code.
Try these before you try a cache
Every option here has one property a cache does not: it makes the system faster without making it capable of being wrong. That property is worth a great deal, and it is the reason this list should be exhausted first.
In practice the first two rows account for most "we need a cache" conversations. An endpoint that issues one query per row in a list, or scans a table for want of an index, is slow for a reason that a cache conceals rather than resolves.
| Symptom | Try this first | Why it beats a cache |
|---|---|---|
| One query per item in a list | Batch or eager-load the relation (The N+1 Query Problem) | Removes the work instead of hiding it; no staleness window |
| A single slow query | Read the plan and add the index (Should I Add an Index?) | The database gets faster for every caller, including ones you did not think about |
| A large response | Select fewer columns; paginate (Pagination That Survives a Large Table) | Less serialization, less network, less memory — no second data store |
| Repeated identical work inside one request | Memoize for the request's lifetime | Scoped to a request, so it cannot go stale by construction |
| An expensive aggregate over many rows | A maintained counter or a materialized view | The database keeps it consistent transactionally (Denormalization on Purpose) |
| Read load exceeding one primary | A read replica | Scales reads with a lag you can measure, rather than a staleness you have to invent (Read Replicas From the Application) |
| Load from one abusive client | A rate limit (Rate Limiting) | Addresses the cause; a cache just makes the abuse cheaper to sustain |
The two costs you are agreeing to
A cache adds exactly two things to your system that were not there before, and both are permanent. The first is a consistency problem: some reads can now return data that was true a moment ago and is not true now, and the bound on that is a number you chose. The second is an availability dependency: a second data store on the read path, which can be slow, full, failing over or unreachable.
Neither cost is a reason never to cache. They are the reason a cache should be a response to a measured problem, with a named staleness budget and a written-down answer for the day the cache is unavailable.
Which of these can you answer right now?
when You have a number a product owner would accept.
cost If you cannot get one, this data should not be cached — and finding that out now is the cheap outcome.
when The same keys recur across requests and users.
cost If keys are near-unique per request, the cache is a round trip added to every request for nothing.
when There is a write path you control, or an event you can consume.
cost If the answer is "the TTL", then the TTL is your entire correctness story — say so explicitly (Cache Invalidation).
when The origin can absorb 100% of read traffic, or there is a bounded fallback.
cost If untested, you have converted a slow endpoint into an outage waiting for a failover (Circuit Breakers).
when The index exists, the N+1 is gone, the payload is bounded.
cost A cache over a pathological query is a load-bearing bandage that fails at the worst moment.
when It is descriptive content, not money, stock or entitlement.
cost If disputable, do not cache the value. Cache the immutable context and read the number fresh.
Cache the part that cannot be wrong
The usual argument for caching a correctness-critical page is that most of it is not correctness-critical. That argument is right, and the conclusion is to split the response rather than to cache all of it or none of it.
The product name, description, images and category are editorial content and can be cached for a long time. The stock level and the price a user is about to be charged are authoritative and must be read fresh. One is a hundred-fold reduction in work and the other is a single indexed lookup, so the combined path is close to as fast as the fully-cached one — and it cannot sell something that is not there.
const page = await getOrLoad(`product:page:${id}`, 300, () =>
buildProductPage(id)) // includes stock and price
return page
// Sold out five minutes ago; still shows "3 left".
// The user completes checkout and the order fails at fulfilment.const [content, live] = await Promise.all([
getOrLoad(`product:content:${id}`, jitter(3600), () => loadContent(id)),
db.one('SELECT stock, price_cents FROM products WHERE id = $1', [id]),
])
return { ...content, stock: live.stock, price: live.price_cents }
// Editorial content cached for an hour; the two fields
// a user can dispute come from the database every time.Caching is per-field, not per-endpoint. The expensive part of the page is the editorial content and the joins behind it; the correctness-critical part is one indexed lookup. Splitting them keeps almost all of the saving and removes the class of bug where a user is shown, and charged for, something that is not there.
How to build it
Most important first.
- Measure first. Find out which step is actually expensive before choosing a remedy (Why Is My API Slow?) — the answer is frequently a query plan, not a missing cache (The Slow Query Workflow).
- Exhaust the no-consistency-cost fixes: add the index, fix the N+1, select fewer columns, paginate, batch, or denormalize a counter that the database maintains transactionally (Eager Loading and Batching).
- If you cache anyway, cache the *immutable* part. The product description can be cached for an hour; the stock level cannot be cached at all. Splitting one response into a cacheable half and an authoritative half is usually the right shape.
- State the hit rate you expect and check it after a week. A cache with a hit rate below what makes the round trip worthwhile should be deleted, not tuned.
- Write down what happens when the cache is unavailable, before shipping. If the answer is "the database gets 100% of reads", verify that the database can take it — or add a bounded fallback (Bulkheads).
- Prefer making the origin fast to making it rare. A fast origin has no staleness window, no invalidation path and no cold-start cliff.
What can go wrong
- The cache hides a pathological query, which then reappears at full concurrency on any invalidation, deploy or restart.
- A hit rate low enough that the cache is pure overhead, invisible because nobody put a hit-rate metric on it (A 95% Hit Rate Tells You Almost Nothing).
- Cached data that should never have been cached: a balance, an entitlement, a rate-limit counter, a one-time token.
- Caching in front of a write-heavy row so that almost every populate is immediately invalidated.
- A cache added to fix a load problem that was really a retry storm or a runaway client, so the cache absorbs the symptom and the cause grows (Retry Storms).
- The availability dependency nobody planned: the cache is down, the fallback path was never load-tested, and the database is now the bottleneck (Cascading Failure).
- Caching a value that is concurrently mutated means a stale populate can undo a write from the reader's perspective — the reason correctness-critical values must be read authoritatively (Cache-Aside).
- A cached rate-limit or inventory counter read-modify-written by two instances loses one update, permitting exactly the thing the counter existed to prevent (Atomic Operations).
- A cache introduced to absorb a stampede on an expensive path can create one when it expires, moving the race rather than removing it (Cache Stampede).
- Rate limits and quotas must be enforced against authoritative state. A cached counter with a TTL is a limit an attacker can exceed by exactly the size of the staleness window (Rate Limit Algorithms).
- Authorization decisions cached for convenience become revocation delays. If "remove this user's access" cannot take effect within your stated window, the cache is a security control failure (Role-Based Access Control).
- Never cache one-time values — password-reset tokens, MFA challenges, idempotency outcomes intended to be consumed once. A cached copy defeats the single-use property (Idempotency Storage).
- A cache added to absorb load from an abusive client is a bandage over a missing limit. The limit is the control; the cache just makes the abuse cheaper for the attacker (Rate Limiting).
- "Caching is free performance." It is performance bought with consistency and availability. The bill arrives later and in a different currency than the one you were optimising.
- "Every backend needs Redis." Most do not. Reach for it when you have measured a specific repeated read that meets the three conditions, not because it appears in every architecture diagram.
- "We will add invalidation later." Invalidation is the design, not a follow-up. A cache shipped without one is a system whose staleness is bounded only by a number someone typed once (Cache Invalidation).
- "The cache made the p99 better, so it worked." Check the p99 on a miss and the p99 during a cold start. Those are the numbers that describe the bad day.
- "Removing the cache is risky." Removing a cache is far easier to reason about than adding one: the failure mode is "slower", which is visible immediately, rather than "wrong", which is not.
Operating it
- Hit rate per key prefix is the single number that tells you whether a cache should exist. Below the point where the round trip pays for itself, it should be removed.
- Compare origin load before and after. If database queries per second did not fall meaningfully, the cache is not doing anything (Which Signal Actually Means "The Database Is Slow").
- Track cache lookup latency as a share of total request latency. A cache that is a visible fraction of a request it rarely satisfies is a straightforward loss.
- Count stale-data bug reports as a cost attributable to the cache. Nobody does this, and it is the cost that actually dominates.
- Measure the cache-down scenario deliberately — a game day with the cache disabled — rather than discovering the answer during an incident.
- At small scale most caches are unnecessary: the database is bored and the query is fast. Adding one buys complexity now against a load problem you may never have.
- At 10x, the calculation can flip — but only for data that meets the three conditions. A cache in front of write-heavy or uniquely-keyed data gets worse with scale, not better.
- At 100x, the cache itself becomes a system with hot keys, connection limits and failover behaviour of its own (Hot Keys: When Aggregate Metrics Hide a Saturated Node).
- Sometimes the honest answer at every scale is "nothing changes": an indexed lookup on a well-sized table stays fast, and the correct amount of caching in front of it stays zero.
- Not caching means the origin carries the full read load, and you must keep it fast — which means indexes, query discipline and payload discipline as ongoing work rather than a one-time cache.
- Fixing the query instead of caching it is usually slower to implement and always cheaper to operate.
- Splitting a response into cacheable and authoritative halves costs an extra request or an extra query, and it is the only way to cache a page that contains one field which must be exact.
- Deleting a cache that is not earning its keep is a small, unglamorous change that reduces the number of ways your system can be wrong.
Where this applies
Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.
- GENERALThe three conditions — read-heavy, repeated keys, statable staleness budget — apply to any cache anywhere.
- SCALE-SPECIFICBelow the load where the origin is actually strained, almost every application cache is a net negative: it adds a failure mode and a staleness window to buy headroom you are not using. The threshold is a property of your data and hardware, not a universal number.
- DATABASE-SPECIFICDatabases cache aggressively themselves. A repeated query over a small hot table is typically served from the engine's own buffer pool without touching disk, so "the query is slow" and "the query reads disk" are different claims — check which one is true before adding a layer that duplicates the buffer pool badly.
Where the depth lives
This domain teaches the application-side mechanism and hands the rest off.
- — System Design — read-heavy versus write-heavy workload characterisation, which is the framing that decides this question before any technology is named.