Caching: cache hits, misses, write policies and invalidation

01Caching: definition, hits, misses and TTL

Caching stores a reusable copy of data or a computed result so later requests can avoid a slower or more expensive operation. A cache entry is addressed by a cache key, such as product:P7. The authoritative store holds the record the application treats as the source of truth, such as the product database. A cache holds a copy. Its freshness policy states how old that copy may be, and its access policy states who may read it.

A hit means the cache has an acceptable entry. A miss means the entry is absent or cannot be used. A time to live, or TTL, is how long an entry may remain usable under the cache policy. A TTL is not the same as a guarantee that the underlying value cannot change.

Suppose P7 costs $20 and thousands of people view it each minute. Reusing a small product record can reduce database work. During checkout, however, the service must check which price applies and whether stock is available under the agreed purchase rules. The displayed cached value is not automatically permission to charge an old price or sell unavailable stock.

02Cache placement: local, shared, distributed and CDN

Memory (RAM) is fast temporary working storage; a disk retains bytes with a different access cost. An application can cache in its own memory, avoiding a network call. It can also cache on local disk: slower than RAM, but useful for larger reusable files. With several application servers, these local caches are separate. If the next request reaches another server, that server may miss or hold a different version.

A shared cache gives applications a common network-accessible cache. A distributed cache spreads that cache's keys across several machines. “Shared” describes who can use it; “distributed” describes how its capacity is placed. Neither word specifies durability or the freshness protocol.

A browser cache stores a user's copy. A reverse-proxy cache serves requests in front of the application. A content delivery network, or CDN, keeps copies at edge locations nearer users. For a product image, the first edge request misses and fetches the object from the origin, the server or storage service that supplies the original content; later allowed requests reuse it.

A separate hostname for static content, such as static.shop.example, makes a later CDN migration easier. Initially it serves image files directly. Later that hostname can point through a CDN while the object paths remain stable. Cache keys, certificates, cache headers, and private-content policy still need configuration; changing DNS alone does not define correct caching.

03Worked example: cache-aside read of P7

Cache-aside means the application manages cache lookup and refill. Start with a database record {id:P7, price:20, version:8} and an empty cache.

1. Request R1 reads P7. The application checks key product:P7 and misses.
2. It reads version 8 from the database, stores a copy with a 30-second TTL, and returns $20.
3. Request R2 reads P7 one second later. The same cache key hits; no database read is needed for that product-page request.
4. When the TTL expires, the next request needs a refresh. The cached copy did not update itself when time passed.

04Cache read and write policies

Cache placement answers where a copy lives. A cache policy answers who loads a missing copy and how a write reaches durable storage and existing cached copies. These are separate decisions: invalidation marks or removes a cached value so later readers cannot keep using it as current. For P7, the policy must explain what happens both when a page read misses and when the seller changes the price.

Read-through describes loading reads; it is not itself a write policy. Write-through can make the write path more explicit, but two independent stores are not automatically one atomic transaction. If the database commits a $25 price while the cache update fails, readers need invalidation, version checks, or a declared staleness limit.

For the public product page, use cache-aside: set a TTL and invalidate the cached price after a database update commits. Accept brief display delays. Checkout must check current price and stock in the purchase transaction. Write-back may suit disposable counters, but important orders need a way to survive cache failure before the service reports success.

05Cache invalidation and the stale-refill race

The seller changes P7 from $20 to $25. Simply deleting the cache after the database write can still race with an earlier reader:

If the business accepts up to a stated stale-display interval, a TTL may be sufficient under a specified refresh policy. If deletion or permission revocation must be immediate, verify current authorization rather than treating a stale cached record as truth. The interview answer should first state how stale a read may be and how quickly a permission change must take effect, then choose a mechanism that meets those requirements.

Suppose the database confirms version 8 at 10:00:00 with permission to reuse it until 10:00:30. A reader receiving it at 10:00:20 has only ten seconds left. Starting a fresh 30-second timer would incorrectly extend use to 10:00:50. Allow for clock differences. After expiry, obtain a newly validated value or return an error if validation fails. This limits age under the stated clock and database assumptions; it still allows stale reads before expiry and does not revoke access immediately.

Recovery also needs a way to distinguish fills started before a cache restart from fills started afterward. A cache generation is an identifier for one such cache lifetime. A refill carries the generation it started in; after recovery selects a new generation, the cache rejects results from the old one even if their requests finally resume.

06Eviction policies: FIFO, LRU, LFU and alternatives

Invalidation removes data because it is no longer acceptable. Eviction removes data because the cache needs space. A perfectly fresh entry can be evicted.

Compare the policies on one trace

Take a two-entry cache: insert A, insert B, read A, then insert C. Before C arrives, insertion order is A then B; access recency is B then A.

State the insertion convention when illustrating LIFO/MRU.

Real implementations may approximate these policies to save CPU and memory. An LRU cache can perform poorly during a large one-time scan because scan entries evict frequently reused data.

Implementation choice and data that must not be evicted

For a disposable shared product cache, one practical option is Redis with an explicit maxmemory limit and a measured choice between allkeys-lru and allkeys-lfu. Redis approximates these policies; LFU also decays old popularity. A volatile-only policy considers only expiring keys, so it is a different capacity policy. Keep durable business records and correctness metadata out of an indiscriminately evictable cache. See the Redis eviction reference; the application still owns freshness and origin-overload protection.

07Cache stampedes, negative caching and outages

When a popular key expires, 10,000 simultaneous readers may all miss and query the database. This is a stampede.

Negative caching stores a short-lived “not found” result to reduce repeated lookups of missing keys. It needs a short enough lifetime to let newly created records become visible, and must not reveal to an unauthorized user whether a private record exists.

08HTTP cache control and conditional revalidation

HTTP caches use response directives and validators to make reuse decisions. These are distinct from an application cache’s own TTL configuration.

A directive is an instruction carried in response headers, usually Cache-Control, about whether and how caches may reuse the response. A validator, such as an ETag, identifies a representation version. Revalidation asks the origin whether that cached version is still usable, which can avoid sending the full body again.

For example, an origin returns ETag: "v8". On revalidation the cache sends If-None-Match: "v8"; a 304 response confirms that its selected representation can be reused without resending the body. Vary identifies request headers that select different representations, such as language. It does not perform authorization. Cache directives do not recall bytes already downloaded or replace access checks. See RFC 9111.

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.