2.1.2 - Caching and reusable components

Store results under the inputs that determine them

Caching stores data or computed results so that a later request can reuse them instead of repeating the original work. A cache key identifies the request. A hit finds a usable stored result; a miss requires the original computation or retrieval.

Suppose generating a thumbnail from an image takes 40 milliseconds, while looking up a cached thumbnail takes 1 millisecond. For a repeated request, a hit avoids generating it again. A first request still pays the lookup and generation costs. Caching does not remove the work of the first computation.

A result must be keyed by everything that can change it. A thumbnail depends on the source image, output width and transformation settings. Using only image ID would wrongly treat 100-pixel and 300-pixel requests as identical. A version identifier can distinguish edited source images.

An original small routine computes a square and caches it by integer input. Assume an unlimited cache, initially empty:

function cachedSquare(n)
    if cache contains key n then
        return cache[n]
    endif
    result = n * n
    cache[n] = result
    return result
endfunction

Membership matters, not whether the stored value is non-zero. A cached result of 0 is still a hit. This static pseudocode illustrates the logic; the source-only pseudocode editor in Testing preserves an answer but does not execute it.

Balance reuse against stale data and storage cost

A cached result becomes stale when the data it represents have changed. An invalidation policy determines when it must be removed or replaced.

A catalogue might cache the price of item R as 12 units. If the authoritative price changes to 15, returning 12 is fast but wrong. The system could invalidate R when its price changes, use a data-version key, or expire cached prices after a stated interval. Expiry permits stale results during that interval, so it is suitable only when the requirement tolerates that delay. Payment calculation may need a fresh authoritative check even if browsing can use an approximate cached price.

Storage is limited. An eviction policy chooses which entries to remove when a bounded cache is full, such as the least recently used entry. Eviction of a valid result is different from invalidating a stale result: both remove entries, but for different reasons. Cached copies can use substantial memory, and managing keys, expiry and eviction adds complexity.

Under a simple timing model, each request costs 1 millisecond for lookup; a miss adds 40 milliseconds for generation. For five thumbnail requests with three hits and two misses, total time is $5\times1+2\times40=85$ milliseconds, compared with $5\times40=200$ without caching. If all five requests differ, caching takes 205 milliseconds and uses extra storage. These are explicit assumptions, not a universal speed guarantee.