Add feature to delete inactive projects

What does this MR do and why?

1. This MR currently targets !86211 (merged), once that MR is merged to master we can update the base branch to master for this MR as well. The dependent MR has been merged and this MR now targets master.
2. Add inactive projects deletion feature.
3. Add a cronjob that runs once daily, finds the inactive projects and perform the following action:
4. Send deletion warning email to the inactive project's maintainers and owners. Store the date when the email was sent in redis hash.
5. If we've already sent an email then it checks whether the project is still inactive for a duration greater than inactive_projects_delete_after_months defined in the application_settings table. In case the project is still inactive it permanently deletes the project.
6. When an activity is performed on an inactive project after the deletion warning email was sent, we remove that project's key from the redis hash.
7. The documentation is added in a separate MR !86907 (merged)

Screenshots or screen recordings

HTML version	Plain text

Instance audit events	Project audit events

How to set up and validate locally

1. Create a project under a group A. Let's call it as Project A.
2. Create another project in group B. Let's call it Project B.
3. Modify the last_activity_at of both the projects so that they are considered as inactive. For example:

project = Project.find(<id of Project A>)
project.last_activity_at = 3.years.ago
project.save!

4. Enable feature flag for group A only. Feature.enable(:inactive_projects_deletion, Namespace.without_project_namespaces.find_by_full_path('<name of Group A>'))
5. Enable inactive projects deletion settings using the application_settings API. Example : https://gdk.test:3000/api/v4/application/settings?delete_inactive_projects=true&inactive_projects_send_warning_email_after_months=6&inactive_projects_delete_after_months=10&inactive_projects_min_size_mb=0
6. Connect to your local redis using CLI - redis-cli -s <path-to-gitlab-development-kit>/redis/redis.socket. Verify that redis hash is empty by running hgetall inactive_projects_deletion_warning_email_notified inside the redis CLI.
7. Manually modify 1_settings.rb file and update the inactive_projects_deletion_cron_worker cron timing to run every minute. This will help us test the feature fast. Restart the background jobs using gdk restart rails-background-jobs
   • Settings.cron_jobs['inactive_projects_deletion_cron_worker']['cron'] ||= '0 1 * * *'
   • Settings.cron_jobs['inactive_projects_deletion_cron_worker']['cron'] ||= '* * * * *'
8. Wait for Projects::InactiveProjectsDeletionCronWorker to run. You can tail the logs gdk tail rails-background-jobs | grep -i inactive.
9. After the cron has executed once, visit https://gdk.test:3000/rails/letter_opener/ and verify that we've received the deletion warning email for Project A. Also verify that no email should have been received for Project B (we didn't enable the feature flag for this project).
10. Verify that the redis key has been updated. Run hgetall inactive_projects_deletion_warning_email_notified in redis-cli and verify that the key for project A exists. The value should be the current date which signifies the date when the deletion warning email was sent.
11. Manually update the redis value to simulate that the deletion warning email was sent more than (inactive_projects_delete_after_months - inactive_projects_send_warning_email_after_months) months ago (4 months as per our example). Run the following to update via redis-cli hset inactive_projects_deletion_warning_email_notified project:<project_if of Project A> 2021-04-25
12. Wait for Projects::InactiveProjectsDeletionCronWorker to run again. This time the project should get deleted immediately or should get marked for deletion if the project had enabled delayed project deletion.
13. Visit the homepage of Project A and verify that the project is either deleted or marked for deletion.
14. Visit the homepage of Project B and verify that the project is neither deleted nor marked for deletion.
15. Verify that the redis key has been deleted. hgetall inactive_projects_deletion_warning_email_notified
16. Visit the instance audit events dashboard and verify that audit events for deletion warning emails are captured successfully.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/357376