Support Dual Write to the container registry redis clusters

Context

The GitLab Container Registry currently uses two separate Redis clusters:

• A legacy cluster used by the Redis cache
• A newer cluster used for database load balancing

Following the work in epic gitlab-com/gl-infra/data-access/durability#24 (closed), the registry will consolidate to a single dedicated Redis cluster (the newer one currently used for database load balancing). This migration requires implementing a dual-write approach to ensure data consistency and minimize risk during the transition.

Implementation Requirements

1. Interface Abstraction

   • Create a CacheInterface that abstracts current Cache operations

2. Dual-Write Capability

   • Implement DualWriteCache that satisfies CacheInterface
   • Support simultaneous writes to both Redis clusters
   • Handle secondary write failures gracefully (non-blocking - via warn logs)

3. Feature Flags

   • Read Control FF: Toggle between primary/secondary for reads
   • Dual-Write FF: Enable/disable dual-write functionality

4. Code Updates

   • Fix all code directly referencing concrete Cache type to now reference CacheInterface
   • Use the DualWriteCache implementation of the CacheInterface for the redis cache operations only
   • Update tests and code paths to work with interface

5. New Tests

   • Verify dual-write functionality
   • Test read switching behaviour
   • Validate error handling scenarios

With the implementation above we can enable a phased migration approach: first writing to both clusters, then switching reads to the new cluster, and finally disabling writes to the old cluster after validation.

After successfully migrating to the new Redis cluster, we'll need a follow-up issue to streamline our configuration and remove the dual write code. This should consolidate our Redis settings to reference a single Redis configuration rather than maintaining separate configurations for cache and database load balancing.