Draft: POC: Optimize destroy_old_pipelines query

What does this MR do and why?

• PipelineDeletionCutoffCache - this class is responsible for reading, writing, clearing, and invalidating (gets invalidated automatically when config changes) the cut_off timestamp for a project
• PipelineDeletionCutoffCacheQueue - This queue maintains the list of project_ids that have to be pushed to PipelineDeletionCutoffCacheWorker to calculate the cut-off cache (timestamp for a project to start in DestroyOldPipelinesWorker)
• PipelineDeletionCutoffCacheWorker - A LimitedCapacity::Worker which runs 10 jobs concurrently based on the queue size. This is triggered with the existing Ci::ScheduleOldPipelinesRemovalCronWorker

Flow:

graph TD
    A[Cron Job Runs] --> B[ScheduleOldPipelinesRemovalCronWorker]
    B --> C[Start Both Workers]
    
    C --> D[Ci::DestroyOldPipelinesWorker]
    C --> E[PipelineDeletionCutoffCacheWorker]
    
    D --> F{Has `last_processed_updated_at` Timestamp? `PipelineDeletionCutoffCache#read`}
    F -->|No| G[Push to `PipelineDeletionCutoffCacheQueue`]
    F -->|Yes| H[Delete Pipelines]
    
    E --> I{`PipelineDeletionCutoffCacheQueue` Empty?}
    I -->|No| J[Process from `PipelineDeletionCutoffCacheQueue`]
    I -->|Yes| K[Fill from `ProjectsCleanupQueue`]
    K --> J
    
    G --> L[Will be re-enqueued to `ProjectsCleanupQueue` in the next runs]
    J --> M[Calculate & Store Timestamp `PipelineDeletionCutoffCache#write`]
    
    style E fill:#90EE90
    style D fill:#87CEEB

References

#572278

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.