Virtual Registry: cache entry lifecycle last_download_at rule

🔥 Problem

Virtual registry cache entries are currently not handled automatically. In that sense, once created, a cache entry is not destroyed unless done so by a user. This can be problematic as cache records reference files on object storage and its usage is not free.

This issue is to implement a rule based on the downloaded_at field. This field is updated every time the associated file is accessed.

The goal is that users can set a rule of type "cache entries not accessed in the last X days can be removed".

🚒 Solution

• lifecycle settings linked to the top level group model. This is to store the lifecycle settings (for now, we only need keep_n_days_after_download), when the cleanup was executed and metrics of the last cleanup (entries deleted and storage saved), the cadence (every week, every day, every two weeks, every month, every three months). Also, we will need the next_run_at field and a status to know if the policy is being processed or not.
• background cron job that runs every hour or half hour to know if there is a policy to execute. If there is enqueue the cleanup background job
• cleanup background job (limited capacity) that takes the next available policy for execution and execute it
  • it would detect the cache entries that can be removed and should update their status to pending_destruction (batch update)
  • update the last execution metrics on the lifecycle settings and schedule the next run.
• Update the API so that the cleanup settings can be updated (default policy is a disabled one).
  • user permissions are: group_owner since the settings are located on the packages and settings page. See this comment.

🔮 Follow ups

• Cleanup policy for orphan upstreams.
• Cleanup preview (see the designs).
• Cleanup estimated size for the next run.

Design Requirements

UI/UX Design Needs

• Design a policy configuration interface that balances simplicity with flexibility
• Create visual representations of policy impact (e.g., storage savings projections)
• Design notification interfaces for policy enforcement events
• Develop audit log visualization for policy-driven actions

Mockups Needed

• Policy creation/configuration modal with time-based expiration controls
• Policy management dashboard showing active policies and their impact
• Notification design for policy enforcement events

User Experience Considerations

• Platform engineers should be able to understand policy impacts before implementation
• Policy configuration should be straightforward while allowing for customization
• Users should receive appropriate notifications about significant cleanup events
• Audit trail should be easily accessible for compliance and troubleshooting