Cache Hierarchies and Observability
Observe each cache layer with its own request population, key, freshness rule and failure policy. A high hit rate is useful only when the returned response is appropriate and timely. Browser hits reduce the requests an edge cache sees, and edge hits reduce those reaching a service cache, so their percentages cannot be added as if they share a denominator.
Start with a request path and an explicit correctness rule. Identify which responses may be reused, for whom, and for how long. Then measure load reduction, latency, freshness and failures together. A cache that returns the wrong user's response very quickly has not achieved a performance success.
TL;DR
- Observe each cache layer with its own request population, key, freshness rule, and failure policy.
- A high hit rate matters only if the response returned is correct and fresh enough.
- Layer hit rates have different denominators — browser hits shrink what the edge sees — so never add them.
- Measure load reduction, latency, freshness, and failures together along a real request path.
no-cachemeans revalidate before reuse;no-storemeans don't store at all.
Quick Example
This Python example accounts for a fictional thousand-request population. It performs no HTTP requests and implements no cache. The numbers explain nested denominators only.
The edge hit rate is 300 out of 400 requests reaching the edge, or 75 percent. The origin receives 50 out of the original thousand, or five percent. Browser, edge and service hit counts plus origin requests sum to the original population under the toy assumptions. Real retries, revalidation, background refreshes and request coalescing require additional categories.
Core Concepts
A cache key defines the reuse boundary
For HTTP caching, RFC 9111 defines storage and reuse behavior and describes matching nominated request fields through Vary. The representation a cache can safely reuse depends on more than a convenient URL string. Review headers, identity context and the actual cache implementation before sharing responses. HTTP caching standard
Application caches need an equally explicit key contract. Include the dimensions that change the result, such as tenant, permission-sensitive context, version or relevant parameters. Do not put private identifiers into unrestricted metric labels simply to debug a key problem. Use an approved observability representation with controlled access.
Freshness and validation are different from storage
The RFC's unqualified no-cache response directive requires successful validation before reuse; it does not simply mean that storage is forbidden. The unqualified private directive restricts shared-cache storage, while still allowing appropriate private-cache storage under other conditions. It is a storage control, not a complete privacy guarantee.
Those distinctions matter when interpreting a request trace. A cache may store a representation yet contact the origin to validate it. A report that classifies every origin contact as a complete miss can conceal useful reuse and its actual cost. Define categories such as fresh reuse, validation, full fetch and error fallback according to the observed system.
This editorial observation map is a starting point for instrumenting a real request path. The arithmetic example measures none of these runtime behaviors.
Eviction and expiry answer different questions
Expiry concerns when a stored response no longer meets a reuse rule. Eviction concerns removing entries, often under capacity or policy pressure. A layer can evict a fresh entry or retain an expired one for later validation or another documented purpose. Record what the implementation actually does rather than using both words for every disappearance.
When hit rate falls, investigate key cardinality, workload shifts, capacity pressure, expiry policy and fill failures. A lower hit rate can be the correct result of refusing unsafe reuse. Raising retention blindly may improve one metric while returning older or incorrectly shared data.
Best Practices
Draw one request path through the relevant caches and origin. Define each layer's requests, hits, validations, fetches and errors. Record whether background work is included and how retries are counted. Keep ratios beside their numerator, denominator, window and sampling coverage so later comparisons remain interpretable.
Use traces or controlled experiments to connect layers without assuming every request is visible everywhere. Browser caches can satisfy work before a server sees it. Shared caches may combine concurrent fills. Correlate only the identifiers needed for diagnosis and follow the organization's privacy and retention requirements for telemetry.
Test invalidation and freshness with a known disposable representation. Change the source, observe the actual layers, and verify the permitted delay or validation behavior. Include a permission or tenant-boundary scenario when the real application shares infrastructure. Correct response selection matters at least as much as avoiding the origin.
Exercise failure policy deliberately in a representative test environment. If stale data is permitted, specify its maximum acceptable age and the kinds of responses for which fallback is allowed. If it is not permitted, verify that the system fails appropriately rather than serving an attractive but invalid cached response. No failure injection was performed for this article.
Reverse a caching recommendation when invalidation complexity or sensitive reuse outweighs the measured benefit. A cache may be unnecessary for a cheap, infrequent query, or inappropriate for a response whose validity cannot be bounded safely. Start with the bottleneck and correctness requirement rather than adding layers to improve a dashboard number.
Common Mistakes
Adding hit rates across layers
Reporting only origin latency
Treating no-cache as no-store
Optimizing hit rate before correctness
FAQ
Can we add the three hit rates?
Not when each layer sees a different population. Account for requests reaching each layer and relate origin work to the intended overall denominator. The toy example illustrates that distinction without simulating a network.
Does no-cache prohibit storage?
The unqualified HTTP response directive requires validation before reuse. Review the complete response directives and RFC behavior rather than interpreting the name as a universal storage prohibition.
What should be on the first cache dashboard?
Request populations, reuse/validation/fetch categories, latency, freshness, eviction and failures with clear definitions. Add application correctness checks and review telemetry privacy. A single hit-rate number cannot describe the hierarchy's health.