TTL Strategies (Hard, Soft, Jittered)

Time-to-live (TTL) is the fundamental mechanism for controlling data freshness in caches. Every cached entry has a lifespan -- when that lifespan expires, the entry must be refreshed from the source of truth. The TTL value directly controls the trade-off between data freshness and cache efficiency: short TTLs ensure data is never far from current but reduce hit rates and increase database load; long TTLs maximize hit rates but allow stale data to persist. Choosing the right TTL and expiration strategy is one of the most impactful decisions in caching architecture.

Hard TTL is the simplest strategy: when an entry's TTL expires, it is immediately deleted (or marked for lazy deletion on the next access). The next request for that key results in a cache miss, triggering a fresh fetch from the database. Hard TTL is easy to implement and reason about, but it creates a predictable vulnerability: when a popular cache entry expires, all concurrent requests for that key simultaneously miss the cache and hit the database. This is the thundering herd problem, and it is particularly dangerous for entries that were set at the same time (e.g., during cache warming) because they all expire simultaneously.

Soft TTL, also known as stale-while-revalidate, addresses the thundering herd by separating the concepts of 'stale' and 'expired.' An entry has two timestamps: a soft TTL (when it becomes stale) and a hard TTL (when it is truly expired). When a request arrives for a stale-but-not-expired entry, the cache serves the stale data immediately (fast response) and triggers an asynchronous background refresh. The next request gets the refreshed data. HTTP cache-control headers support this natively via the stale-while-revalidate directive. This pattern eliminates the cache miss latency penalty for popular keys because the user never waits for a refresh -- they always get a response immediately.

Jittered TTL adds random variance to expiration times to prevent synchronized expiry. Instead of setting TTL=300s for all entries, you set TTL=300s + random(-30s, +30s). This spreads expirations over a 60-second window instead of having all entries expire at exactly the same moment. Jitter is critical when cache warming or bulk loading populates many entries simultaneously -- without jitter, all those entries expire at once, creating a stampede. CDNs like Cloudflare use tiered TTLs with jitter across their caching layers: edge caches use shorter TTLs (60s) with jitter, regional tiers use medium TTLs (300s), and origin shields use longer TTLs (3600s). Sliding TTL is another variant that resets the expiration timer on every access, keeping hot entries alive indefinitely while cold entries naturally expire.

Think of TTL like expiration dates on milk cartons. Hard TTL: the milk is thrown away exactly on the expiration date, even if it is still fine. If everyone in the office bought milk on the same day, all the milk expires at once, and everyone rushes to the store simultaneously (thundering herd). Soft TTL: the milk is labeled 'best by' (soft TTL) and 'discard by' (hard TTL). After the 'best by' date, you still drink the milk but ask someone to pick up a fresh carton (background refresh). Jittered TTL: instead of all cartons showing the same date, each one has a slightly different expiration, so the office never runs out of all milk at once.