Add TTL for Cache::Ci::ProjectPipelineStatus Redis keys

As discussed in gitlab-com/gl-infra/production#15875 (comment 1441352754), Cache::Ci::ProjectPipelineStatus does not set a TTL.

As suggested by @reprazent:

This means that we could have very old keys living in Redis that we need to clean up. The approach for that would be

  • Adjust the code to set a TTL (perhaps on write and on access)
  • Add a background migration, to scan through the keys and add a TTL to them to make sure we clean up after ourselves.

Doing so will reduce memory pressure on redis-cache and the upcoming redis-cluster-cache.

This issue will be broken into 2 parts

  1. Introduce expiry and a script-based method to backfill -- !124439 (merged)
  2. Depending on (1), decide on how the backfill can be rolled out to SM users.
Edited by Sylvester Chin