Another common problem in caching is the "thundering herd". You have a bit of data that is expensive to compute, very frequently requested, and fine to be a little bit stale.
The naive approach is to follow the normal pattern: use the cached version of it's there, and otherwise compute and stick it in the cache with a TTL. The problem arises when you have a fleet of web nodes going at full bore, and the cache expires. If the data takes 10 seconds to compute, then for the whole next 10 seconds, every new request will find the data missing from the cache since it has expired and not yet been replaced, and so each of those requests will take on the expensive and redundant work of repopulating that data.
It can be a performance blip at best, or a total disaster at worst, causing cascading failures and downtime.
One relatively easy solution here is to just have a separate worker process that runs in an interval and preemptively freshens the data in the cache. In this case you can set with no TTL, and web front ends just use whatever they find. You also need to deal gracefully with the rare case where they may find nothing, like in the case of a cold cache boot.
The naive approach is to follow the normal pattern: use the cached version of it's there, and otherwise compute and stick it in the cache with a TTL. The problem arises when you have a fleet of web nodes going at full bore, and the cache expires. If the data takes 10 seconds to compute, then for the whole next 10 seconds, every new request will find the data missing from the cache since it has expired and not yet been replaced, and so each of those requests will take on the expensive and redundant work of repopulating that data.
It can be a performance blip at best, or a total disaster at worst, causing cascading failures and downtime.
One relatively easy solution here is to just have a separate worker process that runs in an interval and preemptively freshens the data in the cache. In this case you can set with no TTL, and web front ends just use whatever they find. You also need to deal gracefully with the rare case where they may find nothing, like in the case of a cold cache boot.