Caching Strategies — The Highest-Leverage Performance Lever

Latency: ~1 nanosecond (L1) to ~10 ns (L3). Size: 32 KB to ~32 MB.

Managed entirely by the CPU hardware — your code doesn't control it directly. When the CPU reads memory, it stores a copy in L1/L2/L3 so future reads of the same address are near-instant. Why does this matter for backend engineers? Cache-friendly data structures (arrays over linked lists, struct-of-arrays over array-of-structs) get massive throughput improvements. A tight loop over a contiguous array hits L1 cache almost every access; chasing pointers across a linked list misses cache constantly.

Latency: ~1 microsecond. Size: MB to GB (limited by process heap). Examples: a Dictionary<string, Product> in C#, Map in Java, lru-cache npm package in Node.

Lives inside your application process — no network call needed. Reading from it is just a hash-map lookup: roughly 1 µs. The downside: it's per-process. If you have 20 app server instances, each has its own copy. A write to Product 42 on server 1 doesn't update the cache on servers 2–20. This makes in-process caching great for truly static data (config values, feature flags, reference tables that change hourly) and risky for frequently-changing data.

Latency: 0.1–1 ms. Size: GB to TB. The workhorse of backend caching.

A separate in-memory service that all your app servers share. Why separate? Because all 20 app servers hit the same Redis instance — a write on server 1 is immediately visible to servers 2–20. Why in-memory? Because RAM is ~10,000× faster than disk for random reads. Redis adds rich data structures (sorted sets for leaderboards, lists for queues, streams for event logs) and optional disk persistence. Memcached is simpler but lacks persistence and data structures.

Latency: 1–50 ms (from nearest edge node). Size: TB–PB. Examples: Cloudflare, Fastly, AWS CloudFront, Akamai.

A globally distributed network of servers that caches your static assets (images, JS, CSS) and increasingly dynamic API responses close to the user's physical location. Why does geography matter? Light travels at ~200 km/ms in fiber. A user in Singapore connecting to a server in Virginia adds ~170 ms of pure travel time. A CDN edge node in Singapore drops that to ~5 ms. Why does this help for dynamic content? Edge functions (Cloudflare Workers, Lambda@Edge) can run logic at the edge and cache personalized or semi-dynamic responses.

Latency: ~0 ms (already on device). Size: Configurable, typically a few hundred MB per origin. Controlled by: HTTP headers.

The fastest cache of all — already on the user's machine, no network needed. Controlled by HTTP response headers: Cache-Control: max-age=86400 tells the browser to use its local copy for 24 hours without asking the server. ETag and Last-Modified enable conditional requests — "only send me the file if it changed." The challenge: browser caches are hard to invalidate immediately. The standard technique is cache-busting — change the filename or add a hash to the URL (app.a3f8c2.js) so the browser treats it as a brand-new file.

When your code asks the cache for a key and the cache has it, that's a cache hit. Data is returned instantly — no database query needed. When the cache doesn't have it (because it never was cached, or it expired, or it was evicted), that's a cache miss. On a miss, your code falls back to the source of truth (usually a database), fetches the data, and typically writes it back to the cache for next time.

Why the distinction matters: hits cost ~1 ms; misses cost ~50 ms + the overhead of writing to cache. Your goal is to maximize the hit rate for your hot data.

A TTL is just a countdown timer attached to a cache entry. When you store a value in Redis with SET product:42 {...} EX 300, you're saying: "keep this for 300 seconds, then automatically delete it." After 300 seconds, the next request for product:42 gets a cache miss, your code fetches fresh data from the DB, and stores it again with a fresh TTL.

TTL is the simplest way to handle cache invalidation — you accept that your data might be up to N seconds stale. The right TTL depends on how often the data changes and how much staleness your users can tolerate. A product price might need a 10-second TTL; a country list might be fine with a 24-hour TTL.

A cache is finite. When it fills up and a new entry needs to be stored, something old must be thrown out — but which old entry? That decision is the eviction policy.

LRU (Least Recently Used): evict whatever was accessed longest ago. Most common; works well because recently-accessed items are likely to be accessed again. LFU (Least Frequently Used): evict whatever was accessed fewest times total. Better for workloads with very hot keys. FIFO (First In, First Out): evict the oldest entry regardless of access. Simple but often suboptimal. Redis defaults to LRU with configurable variants.

Cache invalidation is the act of marking a cached entry as stale or removing it entirely so the next request fetches fresh data. This is the hard part of caching — Phil Karlton's famous "two hard problems" quote is about exactly this.

Why is it hard? Because data can be updated from multiple places (direct DB writes, background jobs, other services), and your cache might not know about those updates. TTL is the lazy approach: wait for the entry to expire. Explicit purge is more precise: delete the entry immediately when you write to the DB. Event-driven invalidation uses a CDC stream or pub/sub to invalidate across services. We'll dig into each in S6.

The hit ratio (or hit rate) is simply: hits ÷ (hits + misses). If you make 1,000 cache requests and 950 of them are hits, your hit ratio is 95%. This is the single most important metric for a cache.

A hit ratio below ~80% means your cache isn't saving much — most requests still hit the database. A hit ratio above 95% means your cache is doing real work. Target 95%+ for hot caches on read-heavy workloads. Low hit ratio diagnoses: TTL too short, cache too small (too much eviction), wrong keys being cached (caching long-tail data instead of hot data), or a cache that just cold-started.

A cache stampede — also called a thundering herd — happens when a popular cache entry expires (or is evicted) and many requests all miss at the same time, causing all of them to simultaneously hit the database to recompute and re-cache the same value.

Imagine product:42 has a 60-second TTL and is requested 500 times per second. At T=60, it expires. In the next 50 milliseconds — before any request has had time to re-populate the cache — 25 requests all miss, all query the database, all compute the same result, and all try to write it back. That's 25 redundant DB queries and 25 redundant writes. On a really popular key at high traffic, this number is in the thousands. Mitigation strategies: probabilistic early expiration, mutex/lock on miss, or background refresh — we'll cover each in S6.

The most common pattern. Your application code is responsible for all cache interactions — it checks the cache first, handles misses by fetching from the DB, and writes back to the cache after a miss. The cache is only populated on demand (lazily).

Read path: Check cache → hit: return data; miss: query DB → write to cache → return data.
Write path: Write to DB → (optionally) delete or update the cache entry.

Best for: General-purpose read-heavy workloads. Works with any data store. Cache failures are non-fatal (app falls back to DB). Watch out for: Stampede on popular key expiry. Cache and DB can diverge if the write path doesn't invalidate cache. Cold start: first request for every key always misses.

Cleaner code, same logic. Like cache-aside, but the caching library itself handles the DB fetch on a miss — your application code only ever talks to the cache. You configure the cache with a "loader" function that knows how to fetch from the DB.

Read path: App asks cache → hit: return; miss: cache calls loader → loader fetches from DB → cache stores + returns.
Write path: Depends on configuration — often paired with write-through (below).

Best for: Applications using a caching framework (e.g., Spring Cache, NCache, AWS ElastiCache with a loader). Keeps business logic clean. Watch out for: First request for each key still misses. The loader adds coupling between cache and DB layers.

Cache stays perfectly fresh. Every write goes to the cache and the database synchronously — both are updated before the write is acknowledged to the caller. The cache never holds stale data; any read immediately after a write will get the new value.

Write path: App writes → update cache + update DB (both synchronously) → return success.
Read path: Cache always has the data (no cold start for written keys).

Best for: Read-heavy workloads where you can tolerate the extra write latency. Financial records, inventory counts, any data where reads must always see the latest value. Watch out for: Every write takes longer (cache write + DB write). If you write many keys that are rarely read, you're wasting cache space (write amplification).

Lowest write latency, highest risk. Writes go to the cache immediately and are acknowledged to the caller at once. The cache then flushes data to the database asynchronously — on a schedule, in batches, or when triggered. The app sees sub-millisecond write acknowledgment.

Write path: App writes → update cache → return success immediately → (background) flush to DB.
Risk: If the cache crashes before the flush completes, data is permanently lost.

Best for: High-frequency writes where occasional data loss is acceptable (analytics event counters, view counts, game state). Gaming backends use this heavily. Watch out for: Data loss on cache failure. Database can be seconds or minutes behind. Requires careful flush scheduling and crash recovery logic.

Proactive, not reactive. Instead of waiting for a TTL to expire and then fetching fresh data on the next miss, the cache proactively refreshes an entry before it expires — typically when remaining TTL drops below a threshold (e.g., refresh when 80% of TTL is consumed). No request ever sees a cold-cache miss for hot data.

Read path: Always a hit (if entry is hot enough to keep refreshing). No miss latency.
Background: Cache detects entries approaching expiry → triggers a background fetch → updates cache entry silently.

Best for: Data that's always hot and expensive to compute (homepage content, featured items, aggregated dashboards). Watch out for: If you refresh too aggressively, you do unnecessary DB work. If you predict wrong (entry stops being hot), you refreshed for nothing. Complexity: requires a background job or thread.

How it works: Every cache entry has a time-to-live. When the timer hits zero, Redis auto-deletes the entry. The next request gets a miss, fetches from the DB, and re-caches with a fresh TTL. You never explicitly invalidate anything — you just wait for entries to age out.

Why it's attractive: Simplest possible implementation — just add EX 300 to your SET command. No need to track what changed or which keys to invalidate. Works automatically even if the write path forgets to invalidate.

The catch: During the TTL window, your cache serves stale data. If a product price changes at T=0 and the TTL is 5 minutes, users see the wrong price for up to 5 minutes. For some data (country names, navigation menus) that's fine. For prices, stock levels, or balances — it depends on your business requirements. A short TTL reduces staleness but increases DB load. A long TTL reduces DB load but increases staleness. There's no free lunch.

How it works: When your application writes to the database, it also immediately deletes (or updates) the corresponding cache entry. The next read gets a miss, fetches fresh data, and re-caches. No stale window — as soon as a write completes, the cache is clean.

Why it's attractive: Near-zero stale window. Reads immediately after a write see fresh data. More cache efficiency — entries stay valid longer between purges, so your hit ratio stays high even with short effective lifetimes.

The catch: You must remember to invalidate on every code path that writes. One missed invalidation in a background job, a migration script, or an admin tool means the cache holds stale data indefinitely until it expires via TTL (which you may have set to hours or infinity). This is the "only two hard things" problem in concrete form. The standard defense: always have a TTL as a backstop even when using explicit purge.

How it works: Instead of the application explicitly deleting cache entries, a change-data-capture (CDC) system reads the database's write-ahead log and publishes change events. A cache-invalidation service subscribes to these events and deletes or updates cache entries automatically.

Why it's attractive: Works even if application code forgets to invalidate. Catches writes from background jobs, migrations, admin tools, and other services that don't go through your main application code. Decoupled — the write path doesn't need to know about the cache.

The catch: Complexity. You need Kafka or a similar event bus, a CDC connector (e.g., Debezium for Postgres), and a cache-invalidation consumer service. Adds operational overhead and a small propagation delay (typically 10–200 ms between DB write and cache invalidation). Overkill for small systems; standard practice at scale in services with microservices writing to shared data.

The single most common cause of stale data in production. Your main application code deletes the cache entry on update — but your nightly batch job, data migration script, and admin tool all write directly to the database and never touch the cache. The cache holds stale data until its TTL expires (which you may have set to hours). Users see old prices, wrong balances, deleted items that reappear.

Defense: Always use a TTL as a backstop, even when relying on explicit purge. If your primary strategy fails, the TTL limits damage to N seconds. Short TTL = less damage, more DB load. Pick based on data sensitivity.

Two concurrent requests read the same cached value (e.g., stock count = 10), both decrement it (-1), and both write back 9. You sold two items but only decremented the count once. The correct value is 8; the cache (and possibly DB) says 9.

This is a TOCTOU (time-of-check / time-of-use) race. Defense: Use atomic operations where possible (Redis DECR, Lua scripts, or database-level UPDATE stock = stock - 1 WHERE stock > 0). Never do read-modify-write at the application level for values that need to be exact.

TTL too long: users see stale data that damages trust (wrong prices, phantom deleted items). TTL too short: the cache expires so fast that the hit ratio drops below 80% and your DB is nearly as loaded as without a cache. Both extremes cost you.

Defense: Start with a TTL based on data change frequency, then tune with metrics. If your price changes once per minute and your TTL is 1 second, you're flushing 59× more often than necessary. Monitor your hit ratio and P99 DB query latency together. A drop in hit ratio without a change in traffic usually means your TTL is too short or your cache is evicting too aggressively.

You update a popular entity (e.g., a major brand on an e-commerce site) and your invalidation logic deletes 100,000 cache keys — every product, every category listing, every search result page that references that brand. All 100,000 keys are now cold. In the next few seconds, all 100,000 keys get stampeded simultaneously, generating 100,000 DB queries.

Defense: Stagger invalidation (batch deletions with small delays). Use probabilistic or token-bucket rate limiting on DB re-population. Consider background refresh for highly-referenced entities rather than immediate delete. For extreme cases, use a short lock window — mark entries as "refreshing" and serve slightly-stale data while the refresh happens rather than serving a miss.

LRU keeps a mental note of when each item was last touched. The item untouched the longest gets kicked out first. The logic: if you haven't needed something in a while, you probably won't need it soon. Redis implements this by sampling a handful of random keys and evicting the oldest-touched among them — a good approximation with near-zero overhead. Use LRU as your starting point for any general-purpose cache.

LFU tracks how many times each item has been accessed. The item accessed least often goes first. Great for workloads where some items are dramatically more popular than others (think: the front page vs. an obscure product page). The downside is bookkeeping cost — you need a frequency counter for every entry, and that memory adds up. A clever variant called W-TinyLFU solves this elegantly: instead of exact counters, it uses a compact "approximate counter" structure that estimates frequencies with a tiny memory footprint.

W-TinyLFU is the modern "best of both worlds" eviction policy — it blends frequency tracking (LFU's strength) with a short grace period for brand-new entries (so a hot new item is not killed before it has a chance to prove itself). It is the default policy in Caffeine (a popular Java cache library) and is available in Redis as allkeys-lfu. The internal design splits the cache into a small "window" area for fresh items and a larger LFU-style main area — that combination handles both bursty popularity and steady-state hot keys at the same time. If you are tuning a high-traffic cache and have the option, W-TinyLFU will typically outperform pure LRU.

Rather than tracking access patterns, TTL-based eviction simply lets time decide. Every entry has a clock ticking down; when it hits zero the entry expires. This is less about eviction strategy and more about a freshness guarantee — you're saying "I accept data up to N seconds stale." Redis combines TTL with LRU: entries expire on schedule, and if memory still fills up, LRU kicks in. TTL is mandatory for any data that changes in the source of truth — product prices, user sessions, feature flags.

FIFO evicts by insertion order: the oldest entry in the cache goes first, regardless of whether it's being used constantly. It's easy to reason about and implement (just a queue), but it ignores actual usage entirely. You can end up evicting an item that's hit a thousand times per minute just because it was inserted first. FIFO is mostly useful in very constrained embedded systems where tracking access time would be too expensive, not in typical web caching scenarios.

Pick a random victim. No bookkeeping, no clock, no counters. This sounds terrible but at scale it's often within a few percent of LRU's hit ratio — because popular items are statistically less likely to be chosen as the random victim. Redis uses randomized sampling in its LRU approximation partly for this reason. Pure random eviction is a useful baseline and performs better than FIFO in most real access distributions. It's rarely the right first choice, but it's a useful reminder that complexity doesn't always buy you proportionate gain.

Picture every possible key arranged around the edge of a clock face — that imaginary ring is the "keyspace circle." Each cache node owns a slice of the circle. To find where a key lives, you hash it to a point on the circle, then walk clockwise to the first node that owns that slice. Why a circle? Because when you add or remove a node, only the keys in one slice need to move — not every key in the cluster. That makes scaling cheap: adding a node reshuffles roughly 1/N of your data, not all of it. This is why consistent hashing became the standard for distributed caches. Redis Cluster uses a variation with 16,384 pre-defined slots — keys hash into one of those slots, and each node owns a fixed batch of slots, which makes administration even simpler than a pure circle.

Divide the keyspace alphabetically or numerically: keys starting with A–F go to node 1, G–M to node 2, and so on. Simple to understand and easy to implement. The problem: access patterns are rarely uniform. If every popular product ID starts with "1", node 1 gets hammered while nodes 2–5 sit idle. This is called a hot shard. Key-range sharding is fine for range queries but needs careful key design to avoid hotspots.

Each primary (master) node can have one or more replica nodes that hold a copy of the same data. Replicas serve two purposes: failover (if the primary dies, a replica is promoted) and read scale (you can route read-only queries to replicas, spreading load). The catch: replicas lag behind the primary by a few milliseconds. If you read from a replica immediately after writing to the primary you might get stale data — this is the coherence problem covered in Section 11.

Memcached does one thing: store strings by key. Its simplicity is its strength — it's multi-threaded and can squeeze more throughput per CPU than Redis on pure key-value workloads. If all you need is "cache this blob of bytes for 60 seconds" and you're at extreme throughput (millions of operations per second), Memcached is worth evaluating. Its weaknesses: no persistence, no replication built in, no pub/sub, no rich data types. You can't do atomic counters, sorted sets, or queues. It's a cache, only a cache, and always a cache.

Redis supports strings, hashes, lists, sorted sets, bitmaps, HyperLogLog, streams, and geospatial indexes. It has optional persistence (RDB snapshots + AOF), cluster mode for horizontal scaling, Pub/Sub for messaging, and Lua scripting. Modern teams default to Redis because the feature set removes the need for multiple specialized services. The mild downside is that Redis is single-threaded on the command-processing path — though Redis 7+ adds multi-threaded I/O and the single-thread bottleneck rarely matters below hundreds of thousands of commands per second.

Images, JavaScript bundles, CSS files — these are the ideal CDN citizens. They change infrequently (or never, once fingerprinted) and can be served from edge with a one-year TTL. The HTTP header Cache-Control: public, max-age=31536000, immutable tells both the browser and CDN to cache aggressively. The immutable directive says "don't bother revalidating this even on a forced refresh — the URL itself will change when the content changes." This is why modern build tools add content hashes to filenames: app.a1b2c3.js instead of app.js. Change the file → change the URL → old cache entries naturally expire.

CDNs can cache JSON API responses too, not just assets. A product listing endpoint that doesn't personalise its response can be cached at the edge for 5–60 seconds. This is powerful: instead of your app servers and Redis handling 50,000 req/s, the CDN handles most of it, and only genuine cache misses (or 5-second intervals) reach your origin. The key constraint is that the response must be the same for all users — once personalisation enters (prices, recommendations, inventory held in cart), you must either exclude those parts from CDN caching or use ESI.

The header Cache-Control: max-age=60, stale-while-revalidate=30 tells the CDN: "serve this for 60 seconds. After 60 seconds, keep serving the stale version for up to 30 more seconds while you fetch a fresh copy in the background." The user sees no latency spike — they get the slightly stale response instantly, and the refresh happens asynchronously. This is one of the most effective patterns for busy endpoints where a few seconds of staleness is acceptable. It eliminates the "cold TTL expiry → every user hits origin simultaneously" problem at the CDN layer.

Rather than waiting for TTLs to expire, CDN providers offer purge APIs. When a product price changes, your backend calls POST /api/purge?url=/products/42 (or tag-based purge: "purge all responses tagged product-42"). This is the push side of cache invalidation applied at the CDN level — you trade the simplicity of TTL-only for real-time freshness when it matters. Cloudflare and Fastly can propagate purges globally in under a second. The operational overhead is writing the purge calls wherever your data changes; it's worth it for high-value or price-sensitive content.

The core idea is simple: imagine 500 concurrent requests all miss the cache for the same key at the same instant. Instead of all 500 racing to the database, you elect one request to do the fetch — the rest wait, and when the first one finishes they all share the same fresh result. This "only one person flies the mission, the rest ride along" pattern is called singleflight.

In Go, the singleflight package does this exactly (the name literally means "one flight" — one fetch goes out, many results come back). In Redis-based systems, you use a distributed lock: the first thread to miss sets a lock key (SETNX article:42:fetching 1 EX 5). Other threads see the lock and wait (or serve slightly stale data). When the first thread finishes and populates the cache, the lock is released and waiters serve from cache. Net effect: N duplicate DB queries collapse into 1.

Rather than letting the TTL hit zero and causing a stampede, start refreshing the entry slightly before it expires. The intuition: as the remaining TTL gets smaller, more and more requests "flip a coin" and decide to refresh in the background. When 5 seconds remain, maybe 1% of requests flip heads and refresh. When 1 second remains, maybe 20% do. By the time the entry would have officially expired, somebody has almost certainly already refreshed it — so no request ever sees a cold miss. The formula from the "Optimal Probabilistic Cache Stampede Prevention" paper is: refresh if currentTime - delta * beta * log(rand()) > expiryTime. In English: the closer you are to expiry, the more likely a random coin flip lands on "refresh now." The practical result: the cache refreshes itself gradually, with at most a handful of DB fetches instead of thousands.

Rather than letting hot keys expire cold, actively pre-populate the cache before expiry. A background job running every 50 seconds (for a 60-second TTL) re-fetches the hot keys and refreshes them, so they never actually expire from the cache perspective. This is simple and reliable for a small, known set of hot keys (homepage, top-10 products, trending articles). It doesn't scale to thousands of keys but is very effective for the most critical ones. Some teams also warm the cache during deploys to prevent stampedes after a restart wipes the in-process cache.

If 1,000 items are all cached at the same moment (say, during a deploy that hydrates the cache from a batch job), they'll all expire at the same moment too. One stampede of 1,000 entries × 1,000 concurrent users = chaos. The fix is trivial: instead of TTL = 3600 for all entries, use TTL = 3600 + random(0, 600). Now expirations spread across a 10-minute window. The total DB load is the same — but instead of 1,000 entries expiring in one second, they expire 1–2 per second over 10 minutes. Your DB barely notices. This is the simplest stampede mitigation and should be default behaviour in any cache wrapper you write.

The simplest solution: for any read that immediately follows a write — specifically "read-your-own-writes" scenarios — route the read to the primary instead of a replica. Your app can tag requests with a "consistency required" flag, or certain operations always route to primary by convention. The downside: you lose the load-balancing benefit of replicas for those reads, and the primary sees more traffic. Acceptable for low-frequency, high-importance operations (profile updates, payment confirmations) but not for every read in a high-read system.

Redis provides a WAIT numreplicas timeout command that blocks until a given number of replicas have acknowledged the latest write, or until the timeout fires. For example: WAIT 1 100 means "wait until at least 1 replica has the data, or 100 ms, whichever comes first." This trades latency for durability and consistency. It's useful for critical writes (financial transactions, address changes before showing confirmation), but adds up to ~10–50 ms per write and should not be used for every operation in a high-throughput path.

In multi-region deployments, each region has its own Redis cluster. When a write happens in US-East, it publishes an invalidation event to a cross-region message bus (Kafka, Redis Streams, or a managed pub/sub). All regional caches subscribe and delete or update the affected keys. This is how Facebook's Memcache paper described their invalidation pipeline (McRouter). It's operationally complex but is the right pattern when you need near-real-time coherence across geographically separated caches. The lag is now bounded by the pub/sub delivery latency, typically 50–200 ms cross-region.

The simplest approach: when a DB query returns no rows, store a sentinel value in Redis — e.g., user:xyz → "NULL" — with a short TTL of 1–5 seconds. The next request for that key hits Redis, gets "NULL", and your app returns a 404 without touching the DB. The TTL must be short because if the item is later created, you don't want to serve a stale "not found" indefinitely. 5 seconds is a common choice: long enough to absorb a burst attack, short enough that legitimately created items show up quickly.

Imagine a tiny, memory-efficient "membership checker" that can quickly tell you "this username definitely does NOT exist in the database" — without doing a database query. That is a Bloom filter: a clever, compact data structure you pre-populate with all existing user IDs at startup. Before any DB query, check the Bloom filter first. If it says "definitely not in the set," skip the DB entirely — guaranteed safe. If it says "might be in the set," fall through and check the DB normally (Bloom filters can occasionally say "maybe" when the answer is actually "no," but they never say "no" when the answer is "yes"). A Bloom filter with 1 million items and a 1% maybe-rate uses about 1 MB of memory. It can't replace full negative caching (it does not handle newly created items without regular updates), but it's an excellent first line of defence against enumeration attacks.

Negative cache entries can stampede too. If a popular non-existent key expires (TTL hits zero), a burst of requests will all miss and all hit the DB simultaneously — even though the DB will return nothing. Apply the same singleflight pattern from Section 10: use a Redis distributed lock or an in-process singleflight group to ensure only one request fetches from the DB for a given missing key. All others wait a few milliseconds and share the "NULL" result. The combination of short TTL + singleflight means you get fast protection against floods while keeping the cache fresh enough that newly-created items appear promptly.

The browser cache is the fastest possible cache because it never leaves the user's machine — no network round-trip at all. It's controlled by the HTTP Cache-Control header you send from your server.

Use it for static assets that almost never change: JavaScript bundles, CSS, images, fonts. A well-configured Cache-Control: max-age=31536000, immutable means the browser won't ask your server for that file for a full year. For API responses, a short max-age=60 lets browsers avoid duplicate requests on page refreshes.

Why it matters: this is the only layer that requires zero infrastructure on your end. It's free capacity.

A CDN sits at the network edge, close to your users geographically. It absorbs requests before they even reach your origin infrastructure.

Best for: large static assets (videos, large images), and short-TTL API responses that are the same for all users (e.g., trending product lists). Immutable assets with content-addressed filenames (e.g., bundle.abc123.js) can be cached indefinitely.

Why it matters: CDN PoPs are distributed globally. A user in London gets sub-10ms response from a London PoP vs 120ms round-trip to your Virginia origin. Performance and cost both improve.

Redis sits inside your infrastructure, between your application code and your database. This is where you cache hot data: user sessions, frequently read records, expensive query results, aggregated metrics.

Unlike the browser and CDN, Redis is programmable — you control exactly what goes in, when it expires, and when it's explicitly invalidated. You can also use Redis for more advanced patterns like distributed locks and pub/sub messaging.

Why it matters: a DB query takes 5–50 ms; a Redis read takes under 1 ms. For hot data hit thousands of times per second, this gap is enormous.

Even inside the database itself, there's a cache: the buffer pool. PostgreSQL calls it shared_buffers; MySQL/InnoDB calls it the InnoDB buffer pool. It caches recently accessed database pages (8 KB chunks of table/index data) in RAM.

This means even if a query "misses" all your application-level caches and actually runs against the database, there's a good chance the rows it needs are already in the DB's own memory — no actual disk I/O required.

Why it matters: disk reads are ~100× slower than RAM reads. The buffer pool ensures that only cold data — pages that haven't been accessed recently — requires actual disk I/O.

Before picking a cache size, find out how much data is actually hot. Guessing leads to either over-provisioning (wasted money) or under-provisioning (cache that barely helps).

In PostgreSQL, pg_stat_statements shows which queries run most often and how much data they touch. In Redis, the MONITOR command streams live commands so you can see key access patterns. Most APM tools (Datadog, New Relic) can show you cache hit rates in real time.

A practical starting point: size your cache to hold the data accessed in the last 24 hours at typical traffic. That usually covers 80–90% of your working set.

A hit ratio below 80% means your cache is barely helping — more than 1 in 5 requests still hits the database. You're paying for cache infrastructure without getting the full benefit.

90% is a reasonable minimum target for a hot application cache. At 90%, only 1 in 10 requests reaches the database — a 10× load reduction. Getting from 90% to 95% typically requires doubling cache size because you're now catching the long tail of less-frequently-accessed data. From 95% to 99% can require 5× more memory. Chase hit ratio improvements with awareness of those costs.

If your working set is 10 GB under normal traffic, provision 13 GB. During a traffic spike, more unique data gets requested, expanding the effective working set. A cache that runs at 95% memory utilization during normal traffic will start evicting hot data during spikes, dropping hit ratios exactly when you need them most.

Redis tracks memory usage in real time (INFO memory). Set an alert at 75–80% utilization to give yourself time to resize before you hit the ceiling.

Even if your cache is perfectly sized today, a product change can silently explode the working set. Adding a new dimension to your cache key is the classic trap. If you cache products:{id} and there are 100K products, your working set is 100K keys. If you add per-user personalization and change the key to products:{id}:{user_id} with 1M users, your working set is now 100 billion keys — impossible to cache.

Before adding a new dimension to a cache key, calculate the cardinality explosion. If the product of all dimensions exceeds available memory, you need a different strategy (e.g., cache only the base data and apply personalization in-app).

Memoization is the simplest form of caching: if a function is pure (same input always gives same output, no side effects), you can cache the result the first time and skip the computation on future calls with the same input. It's most valuable for expensive computations — machine learning inference, complex mathematical transformations, report generation.

The key insight is that memoization works at the function level rather than the data store level. You're caching the result of a computation, not a row from a table. This can be in-process (a dictionary in memory) or distributed (store results in Redis keyed by a hash of the inputs).

Instead of caching individual rows, you cache the full result set of a specific query. The cache key is typically a hash of the query string plus its parameters. This is great for expensive aggregation queries or complex JOINs that are run frequently with the same filters.

The invalidation challenge: if any row in the underlying tables changes, the cached result may be stale. A practical approach is to use a TTL short enough for your use case (e.g., 60 seconds for a dashboard metric that refreshes every minute) rather than trying to invalidate precisely on every write — which would be complex and often slower than just recomputing.

For API endpoints that return the same response for the same inputs (route + query params + relevant headers), you can cache the entire serialized HTTP response body. On a cache hit, you skip the entire request pipeline — no controller logic, no DB query, no serialization.

This is especially powerful for public, anonymous endpoints like product listings, blog posts, or search results. Be careful with personalized or authenticated endpoints — the cache key must include anything that affects the response (user ID, locale, feature flags) or you'll return the wrong data to the wrong user.

Sometimes the most expensive operations are not single queries but the assembly of multiple data sources into a view. A user's activity feed might require joining posts, likes, comments, friend relationships, and personalization scores. Rather than recomputing this assembly on every request, you pre-compute it and store the final result.

Database materialized views do this at the DB level. Application-level computed views store the assembled JSON blob in Redis. Trade-off: write complexity increases (you must invalidate or rebuild when source data changes), but reads become extremely fast.

Sessions, auth tokens, preferences, shopping cart contents, and personalized recommendations are all data tied to a specific user and accessed on almost every request. Storing these in Redis (keyed by user ID or session token) avoids a DB read on every authenticated API call.

The key consideration is cache key design: include enough to uniquely identify the user and context (user ID, tenant ID in multi-tenant systems) but no more. Including unnecessary dimensions explodes the key space and wastes memory. Also set a TTL based on session lifetime — there's no point keeping a cache entry for a user who hasn't been active in 30 days.

Facebook's social graph is a web of objects (users, posts, comments) and associations (friendships, likes, shares). Standard key-value caching doesn't map well to graph traversals — fetching a user's feed requires following dozens of edges. TAO (The Associations and Objects) is a purpose-built, graph-aware caching layer that sits in front of MySQL.

TAO understands the semantics of graph operations, not just key lookups. When a post gets a new like, TAO can invalidate exactly the cache entries that reference that association — rather than doing a broad TTL-based sweep. This graph-aware invalidation is what allows Facebook to serve roughly 1 billion reads per second across its data centers (at ~96% cache hit rate) while keeping MySQL CPU at manageable levels.

The lesson: at extreme scale, a generic cache may not be enough. Sometimes the access pattern of your data (graph, time-series, geospatial) justifies building or adopting a domain-specific caching layer.

Netflix serves video metadata (titles, descriptions, thumbnails, recommendation lists) to 200M+ subscribers globally. Fetching this from a central database for every play request would be impossible — the round-trip latency alone would be hundreds of milliseconds for users far from the origin region.

EVCache is Netflix's solution: a Memcached-based caching system that replicates cache writes across multiple AWS regions automatically. A write to the US-East cache is asynchronously propagated to US-West and EU caches. Reads are always served locally from the nearest region. The trade-off is eventual consistency — for a few hundred milliseconds after a write, different regions may serve slightly different versions of the data. For video metadata (where a title's description changing by one word isn't user-critical), this is completely acceptable.

Cloudflare's Workers KV is designed for a specific problem: you have configuration data, feature flags, or static content that needs to be read from anywhere in the world with sub-millisecond latency. The solution is to push the data to every one of Cloudflare's 300+ Points of Presence globally.

Reads from Workers KV are served from the local PoP — no origin round-trip. This makes reads extremely fast regardless of geography. Writes go to a central store and then propagate to all PoPs within a few seconds. The trade-off is that KV is eventually consistent and optimized for read-heavy workloads where writes are infrequent. It's not suitable for data that changes frequently or requires strong consistency between concurrent writers.

Reddit's architecture illustrates a clean separation between hot and cold data. Subreddit listing pages (the front page, r/gaming, etc.) are hit millions of times per day with the same content. These hot listings are pre-computed and stored in Memcached. User timelines and post histories — accessed less frequently but needed for longer — are stored in Cassandra, which provides durable storage with fast reads. Canonical user and post data lives in PostgreSQL.

The interesting design point is the hot/cold boundary: Reddit doesn't try to cache everything in RAM. Instead, they classify data by access frequency and match it to the right store. Hot = Memcached (RAM). Warm = Cassandra (SSD). Cold = PostgreSQL (disk). This tiering lets Reddit serve massive read traffic without over-provisioning expensive in-memory stores for data that's rarely accessed.

The most dangerous anti-pattern is treating the cache as a write-once store. You put data in, set no expiry, and never think about removing it. Three months later, users are seeing product prices from last quarter because no one ever purged the cache when prices were updated in the database.

Every cache entry needs a clear answer to: "what event causes this entry to become invalid?" If the answer is a time window ("stale after 60 seconds is fine"), use TTL. If the answer is a specific data write ("this entry must be removed when this user's profile is updated"), use explicit invalidation from your write path. If neither applies, ask whether this data should be cached at all.

Imagine your application receives a request for user ID 99999, which doesn't exist. Your app checks the cache — miss. Fetches from DB — not found, returns 404. Now someone sends 10,000 requests per second for non-existent user IDs (intentional or not). Each one is a cache miss, each one hits the DB. Your cache is completely bypassed for the exact traffic pattern that could take you down.

The fix is negative caching: when a lookup returns "not found," store a sentinel value in the cache (e.g., the string "__NOT_FOUND__") with a short TTL (30–60 seconds). Future requests for the same missing key are served from cache with a 404, without touching the database. This turns a potential DoS amplifier into a safely absorbed response.

Setting a 5-second TTL on a cache that takes 50 ms to refill from the database means you're doing that expensive refill 12 times per minute. If 1,000 users request the same data concurrently, you could be hitting the database up to 12,000 times per minute — almost as often as with no cache at all. You've added infrastructure complexity and cost without meaningful benefit.

Before setting a TTL, ask: what is the acceptable staleness for this data? For a trending posts list, 60 seconds of staleness is invisible to users. For a balance in a financial system, 0 seconds (no caching, or write-through only) may be required. TTL should be derived from business tolerance, not set to a default.

If a function returns different results for the same input based on random numbers, the current time, or external state, caching its result is dangerous. You're locking in one random outcome for the duration of the TTL — every user who hits the cache sees that same "random" result, which defeats the purpose of randomness.

Common culprits: A/B test assignment logic that uses a random number per request (should be deterministic based on user ID instead), recommendations with a random shuffle (shuffle after cache retrieval, not before), and time-sensitive checks embedded in cached responses. Before caching a function's output, verify it is deterministic for a given input.

A popular cache entry expires. In the milliseconds before any one request can refill it, 500 concurrent requests all see a cache miss. All 500 hit the database simultaneously. The database — which was previously handling 50 queries/sec comfortably — suddenly receives 500 queries at once. Response times spike. The DB may become overwhelmed and start timing out. The cascading failures spread.

Two practical fixes: (1) Mutex/lock: when a miss occurs, the first request acquires a Redis lock and refills the cache; all other concurrent requests wait for the lock to release and then read from the now-filled cache. (2) TTL jitter: instead of all entries of the same type expiring at exactly the same time (e.g., all cached after the same deploy), add random jitter: TTL = base_ttl + random.randint(0, base_ttl * 0.2). This spreads expirations out in time, preventing synchronized stampedes.

On the opposite extreme from TTL-too-short, a TTL set to hours or days creates a long window where your cache serves incorrect data. A user updates their profile picture; the old picture persists in cache for 24 hours. A product goes out of stock; the "in stock" flag stays cached. An admin disables a user account for abuse; the session remains valid in cache.

Long TTLs are fine for genuinely static data (images, archived content, product descriptions that rarely change). For data that can change in response to user actions or operational events, TTL should be short enough that the staleness window is acceptable, and you should also build an explicit invalidation path that fires when the data actually changes — so you don't have to wait for the TTL to expire.

Hit ratio is hits / (hits + misses). It tells you what fraction of requests are being served from cache versus falling through to the database. A hit ratio of 94% means 94 in every 100 requests are served from cache; 6 hit the database.

Why it matters: a sudden drop in hit ratio is the earliest warning that something is wrong. Common causes: a new feature that generates cache keys with much higher cardinality (more unique keys → more misses), a deploy that cleared the cache (normal, but causes a temporary dip), or a traffic pattern change (e.g., a bot crawling random URLs with unique parameters that all miss). Every cache monitoring setup should have an alert for "hit ratio dropped below 80% for more than 5 minutes."

When Redis runs out of memory, it must remove existing entries to make room for new ones — this is called eviction. A healthy cache has near-zero eviction rate. A cache with a high eviction rate is too small for its working set — it's constantly pushing out data that will be needed again soon, creating a cycle of misses that hammer the database.

Track eviction rate in Redis with INFO stats → evicted_keys. A rising eviction rate while hit ratio falls is a clear signal to increase cache memory. In Redis, set maxmemory-policy allkeys-lru to ensure evictions use LRU (least recently used) rather than random, which gives you better hit ratios for the available memory.

One of the main reasons you use a cache is to reduce latency. If your cache reads are slow, you're paying the infrastructure cost without getting the speed benefit. A healthy Redis read should be under 1 ms on the same network. Under 5 ms is acceptable. Consistent p99 latency above 5 ms indicates a problem.

Common causes of slow cache reads: Redis memory pressure (when Redis is using swap, reads are disk-speed, not RAM-speed), large values (serializing/deserializing a 1 MB JSON blob takes real time — split large objects into smaller keys), network congestion between app servers and Redis, and Redis CPU saturation from heavy Lua scripts or large SCAN operations running concurrently with reads.

Redis performs best with 50–75% memory utilization. Below 50%, you may be over-provisioned (wasting money). Above 80%, you're cutting it close — a traffic spike will push you into evictions. Above 95%, Redis may start swapping to disk (on systems without maxmemory set), which destroys latency.

Set a maxmemory limit in Redis so it uses your configured eviction policy rather than growing unbounded. Alert at 75–80% utilization. When you receive the alert, you have time to evaluate: is this a temporary spike (do nothing), a trend (provision more memory), or a code change that added high-cardinality keys (fix the key design)? You want to make that decision proactively, not reactively when the cache falls over.

A stampede count tracks how often multiple concurrent requests miss the same cache key at the same time. This metric requires some custom instrumentation — Redis itself doesn't expose it directly. A common pattern is to track it in your application: increment a counter whenever a cache miss triggers a database fetch while another request for the same key is already in-flight.

Any non-zero stampede rate is worth investigating. A stampede count that spikes periodically often correlates with your TTL pattern — if all entries of a type were populated at the same time (e.g., after a cache warm-up), they all expire at the same time. Adding TTL jitter (randomizing expiry times by ±10–20%) is usually enough to eliminate the pattern without code changes.

This metric is often overlooked until it causes an incident. If your application can't reach Redis — due to a network partition, Redis restart, or connection pool exhaustion — every request will either fail or fall through to the database. If the database can't handle the full load without the cache, you're now in a cascading failure: Redis is down, DB is overwhelmed, your entire service degrades.

Protection strategy: implement a circuit breaker on your Redis client. After N consecutive connection failures, stop trying to reach Redis for a cooldown period and fall back gracefully (go direct to DB, or return a degraded response). Alert immediately when network errors to Redis exceed 0 — this should never be non-zero in a healthy system.

Redis is the most widely deployed distributed cache in the world — and for good reason. It stores everything in RAM, so reads and writes take under 1 ms. But what makes Redis special is its rich data structures: strings, lists, sorted sets, hashes, bitmaps, and streams. These let you do more than just "store a blob and get it back." You can, for example, increment a counter atomically, pop the oldest item from a queue, or retrieve the top-10 scores from a leaderboard — all without a round-trip to your database. Redis also supports optional persistence (so a cache restart does not always mean a cold cache) and pub/sub messaging. The downside: it is a network hop away from your app, and cluster setup adds operational overhead.

Best for: distributed caches, session stores, leaderboards, rate limiters, job queues.

Memcached is the stripped-down sibling of Redis. It does one thing: store a blob by key and retrieve it by key. There are no lists, no sorted sets, no pub/sub — just pure key-value. Why does it exist if Redis does everything Memcached does plus more? Because stripping everything out makes Memcached slightly faster and easier to scale horizontally — there is no replication state to manage, and the memory overhead per key is lower. At Facebook scale, those margins matter. For most teams, Redis is the right default; Memcached is a viable choice when you have a pure "store and retrieve blob" workload at extremely high throughput and want the simplest possible operational footprint.

Best for: pure string/blob caching at extreme throughput where Redis's extra features are not needed.

Caffeine is a Java in-process cache library that lives inside the same JVM as your application code. Because it uses the same heap memory as your app, there is zero network overhead — a cache lookup is a hash-map lookup in RAM, which takes roughly 0.1 ms or less. Caffeine implements a sophisticated eviction algorithm called TinyLFU that achieves higher hit ratios than plain LRU by tracking access frequency alongside recency. It handles size-based eviction, time-based expiration, and asynchronous loading. The catch: because it lives in your process, every app server has its own copy of the cache — so cache state is not shared between servers. This is fine for very hot, read-heavy, rarely-changing data (like a config object), but wrong for user-session data that different servers might need to read.

Best for: JVM applications, in-process L1 cache in front of Redis, config and reference data.

Hazelcast is a distributed in-memory data grid — a fancy way of saying it is a cache that automatically partitions and replicates data across a cluster of nodes, and the nodes can live inside the same JVM as your app (embedded mode) or as a separate cluster (client-server mode). Unlike Redis, Hazelcast is Java-native and integrates directly with the JDK's Map interface, which makes it feel natural to Java developers. It also supports distributed locks, queues, and topics. Hazelcast is a strong choice when you want in-process caching but need shared state across app servers without running a separate Redis cluster — it merges both layers into one. The trade-off: more complex than Redis to operate and tune, and less community tooling.

Best for: Java microservices needing distributed shared state without a separate cache tier.

These are managed Redis (and Memcached) services from the major cloud providers. The core technology is identical to open-source Redis — you get the same data structures, the same commands, the same performance characteristics. What you are paying for is operational automation: the cloud handles cluster provisioning, automatic failover, patching, backups, and monitoring dashboards. If you are already on AWS or GCP, using ElastiCache or Memorystore typically takes an afternoon to set up versus days for a self-managed Redis cluster. The cost per GB is higher than running Redis on EC2 yourself, but for most teams the operational savings far outweigh the premium. The key thing to know: ElastiCache "Redis OSS" and ElastiCache "Serverless" behave slightly differently under high load — benchmark before committing at scale.

Best for: teams on AWS/GCP who want Redis without the ops burden of self-managing a cluster.

A CDN (Content Delivery Network) is a cache that sits between your servers and your users, distributed across 200+ edge locations around the planet. When a user in Tokyo requests your homepage, the CDN serves it from a node in Tokyo rather than from your origin server in Virginia — shaving off 100–300 ms of round-trip latency just from geography. CDNs cache static assets (images, CSS, JS, fonts) automatically and can also cache API responses if you configure the right Cache-Control headers. Major players: Cloudflare (most popular, free tier), Fastly (highly programmable, used by GitHub and Stripe), Amazon CloudFront (deep AWS integration), Bunny.net (cost-effective for storage-heavy use cases). CDN caching is the single highest-leverage cache for public-facing web apps because it absorbs traffic before it ever reaches your infrastructure.

Best for: static assets, cacheable API responses, any public-facing content where geography adds latency.

The instinct to cache everything is wrong. Caching has real costs: memory, operational complexity, staleness risk. The right approach is to identify your actual "hot working set" — the keys that absorb 80–90% of your traffic — and cache only those. How? Add a read counter to your application layer, or use Redis keyspace notifications to track access frequency. In most systems, 80% of traffic hits less than 20% of the data (the Pareto principle applies to data access patterns). Caching the other 80% of data that absorbs only 20% of traffic wastes memory and pollutes your eviction queue. Profile before caching.

TTL is not a magic number — it encodes a trade-off between data freshness and database load. The formula to reason about it: "How many seconds of staleness is acceptable for this specific piece of data?" For a promotional banner, maybe 300 seconds. For a user's real-time balance, maybe 0 (do not cache at all). The second input is your database's capacity: if your DB can handle 5,000 QPS comfortably and you have 500,000 hot keys, a TTL of 100 seconds means each key is fetched from DB roughly every 100 seconds — so about 5,000 DB reads per second just for cache misses. TTL × traffic density = DB load. Use the math to choose a defensible number, not intuition.

Stampedes (see Section 21) are one of the most common cache-related outages. Two tools prevent them. Singleflight means: when multiple concurrent threads/goroutines/requests all miss the same key, only one is allowed to fetch from the DB; the others wait and share the single result. In Go, this is the singleflight package. In Java, it is a ConcurrentHashMap of pending futures. Jitter means adding ±10–20% random offset to your TTLs at write time so that a batch of entries written at the same time do not all expire simultaneously. These two techniques together reduce stampede probability to near zero, and they are straightforward to implement.

Negative caching means storing a "this key does not exist" marker in the cache rather than just returning a miss. Why? Without it, every request for a non-existent key goes straight to the database. An attacker who queries thousands of non-existent user IDs can trivially DoS your database with misses — this is called a cache penetration attack. The fix: when the database returns "not found," store a small sentinel value (e.g., "__nil__") in Redis with a short TTL (30–60 seconds). Subsequent requests for the same key hit the cache, get the sentinel, and return 404 without touching the database. Use a short TTL on negatives so that when the object is eventually created, the cache heals quickly.

The highest-performing systems stack multiple cache layers: in-process (Caffeine/MemoryCache) as L1 — zero network overhead, sub-millisecond; distributed Redis as L2 — shared across servers, ~1 ms; CDN as L3 — absorbs traffic before it reaches your data center, ~5 ms from edge but saves origin latency entirely. The layering means that even when L1 misses (rare request, or just started up), L2 absorbs it. Only true misses on L2 hit the origin — and those are rare enough that the origin can handle them comfortably. The complexity trade-off: multi-level means you have three places to invalidate. This is manageable with an event-driven invalidation bus that fans out a delete to all levels simultaneously.

A cache that is not monitored is a silent source of production issues. Two metrics matter most. Hit ratio: what percentage of cache lookups result in a hit? Healthy caches run at 90%+ for hot workloads; a sudden drop signals either a keyspace change, a deploy that changed key names, or a memory constraint causing unexpected evictions. Eviction rate: how many keys per second is Redis discarding to make room? Zero evictions is normal. Sudden spikes in eviction indicate either a memory leak (keys accumulating without TTL) or a working-set size that exceeds your allocated memory. Alert at 5%+ drop in hit ratio, and alert at any non-zero eviction on production caches that should not be evicting. These two alerts catch 80% of cache production issues.

The most common cache technical debt: teams add caching as an optimization without designing an invalidation strategy, because "the TTL will handle it." This works until the data changes faster than the TTL — and then it does not. The right time to design invalidation is when you design the cache, not after you discover stale data in production. Ask: "What event causes this data to change? Who owns the write? How quickly does the read path need to see the update?" If the answer is "within the TTL window, staleness is acceptable," TTL-only is fine. If the answer is "immediately," plan explicit invalidation: the write path deletes the cache key, or publishes a "key invalidated" event. Do this upfront — retrofitting invalidation into a busy codebase is painful and error-prone.

The financial app incident in Section 21 is the canonical example of this rule. Some data — account balances, inventory counts, access control permissions, payment authorizations — must be accurate at the moment of use. Caching these values with any form of eventual consistency (TTL-only, lazy invalidation) introduces a window where the cached value is wrong and a business decision is made on wrong data. The rule: before caching any piece of data, ask "what is the worst-case cost of acting on a stale copy of this value?" If the answer involves money, security, or user safety, the cache must either be bypassed for authoritative reads or use a coherence mechanism (read-your-writes guarantees, cache invalidation on every write, or strong consistency via a distributed lock) that eliminates the staleness window.