Add feature to delete inactive projects

What does this MR do and why?

1. This MR currently targets !86211 (merged), once that MR is merged to master we can update the base branch to master for this MR as well. The dependent MR has been merged and this MR now targets master.
2. Add inactive projects deletion feature.
3. Add a cronjob that runs once daily, finds the inactive projects and perform the following action:
4. Send deletion warning email to the inactive project's maintainers and owners. Store the date when the email was sent in redis hash.
5. If we've already sent an email then it checks whether the project is still inactive for a duration greater than inactive_projects_delete_after_months defined in the application_settings table. In case the project is still inactive it permanently deletes the project.
6. When an activity is performed on an inactive project after the deletion warning email was sent, we remove that project's key from the redis hash.
7. The documentation is added in a separate MR !86907 (merged)

Screenshots or screen recordings

HTML version	Plain text

Instance audit events	Project audit events

How to set up and validate locally

1. Create a project under a group A. Let's call it as Project A.
2. Create another project in group B. Let's call it Project B.
3. Modify the last_activity_at of both the projects so that they are considered as inactive. For example:

project = Project.find(<id of Project A>)
project.last_activity_at = 3.years.ago
project.save!

4. Enable feature flag for group A only. Feature.enable(:inactive_projects_deletion, Namespace.without_project_namespaces.find_by_full_path('<name of Group A>'))
5. Enable inactive projects deletion settings using the application_settings API. Example : https://gdk.test:3000/api/v4/application/settings?delete_inactive_projects=true&inactive_projects_send_warning_email_after_months=6&inactive_projects_delete_after_months=10&inactive_projects_min_size_mb=0
6. Connect to your local redis using CLI - redis-cli -s <path-to-gitlab-development-kit>/redis/redis.socket. Verify that redis hash is empty by running hgetall inactive_projects_deletion_warning_email_notified inside the redis CLI.
7. Manually modify 1_settings.rb file and update the inactive_projects_deletion_cron_worker cron timing to run every minute. This will help us test the feature fast. Restart the background jobs using gdk restart rails-background-jobs
   • Settings.cron_jobs['inactive_projects_deletion_cron_worker']['cron'] ||= '0 1 * * *'
   • Settings.cron_jobs['inactive_projects_deletion_cron_worker']['cron'] ||= '* * * * *'
8. Wait for Projects::InactiveProjectsDeletionCronWorker to run. You can tail the logs gdk tail rails-background-jobs | grep -i inactive.
9. After the cron has executed once, visit https://gdk.test:3000/rails/letter_opener/ and verify that we've received the deletion warning email for Project A. Also verify that no email should have been received for Project B (we didn't enable the feature flag for this project).
10. Verify that the redis key has been updated. Run hgetall inactive_projects_deletion_warning_email_notified in redis-cli and verify that the key for project A exists. The value should be the current date which signifies the date when the deletion warning email was sent.
11. Manually update the redis value to simulate that the deletion warning email was sent more than (inactive_projects_delete_after_months - inactive_projects_send_warning_email_after_months) months ago (4 months as per our example). Run the following to update via redis-cli hset inactive_projects_deletion_warning_email_notified project:<project_if of Project A> 2021-04-25
12. Wait for Projects::InactiveProjectsDeletionCronWorker to run again. This time the project should get deleted immediately or should get marked for deletion if the project had enabled delayed project deletion.
13. Visit the homepage of Project A and verify that the project is either deleted or marked for deletion.
14. Visit the homepage of Project B and verify that the project is neither deleted nor marked for deletion.
15. Verify that the redis key has been deleted. hgetall inactive_projects_deletion_warning_email_notified
16. Visit the instance audit events dashboard and verify that audit events for deletion warning emails are captured successfully.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/357376

changed milestone to %15.0

added backend devopsmanage featureaddition groupcompliance sectiondev typefeature labels

assigned to @huzaifaiftikhar1

Suggested Reviewers (beta)

This is an experimental ML-based code reviewer recommendation system created by ~"group::applied ml".

The individuals below may be good candidates to participate in the review based on various factors.

After you review all recommendations, please assign reviewers manually, as this is not done automatically.

You can use slash commands in comments to quickly assign /assign_reviewer @user1.

Reviewers
@rspeicher, @DouweM, @kushalpandya, @rymai, @smcgivern

If you do not believe these recommendations are useful or if you do not want to use any of the suggestions, please apply the label Bad Suggested Reviewer. You can also provide feedback for this feature on this issue: https://gitlab.com/gitlab-org/gitlab/-/issues/357923.

Automatically generated by Suggested Reviewers Bot

@mwoolf Can you please take an initial look into this MR? This is still in draft and I need to add RSpecs and documentation changes. Would you help in validating that there are no design flaws?

requested review from @mwoolf

changed the description

	5 Warnings
	This MR changes code in ee/, but its Changelog commit is missing the EE: true trailer. Consider adding it to your Changelog commits.
	This merge request is quite big (790 lines changed), please consider splitting it into multiple merge requests.
	c55d158c: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines.
	604d0faa: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines.
	featureaddition and featureenhancement merge requests normally have a documentation change. Consider adding a documentation update or confirming the documentation plan with the Technical Writer counterpart. For more information, see: The Handbook page on merge request types. The definition of done documentation.

If needed, you can retry the danger-review job that generated this comment.

Reviewer roulette

Changes that require review have been detected!

Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:

Category	Reviewer	Maintainer
backend	Tarun Vellishetty (@tvellishetty) (UTC+5.5, same timezone as @huzaifaiftikhar1)	Luke Duncalfe (@.luke) (UTC+12, 6.5 hours ahead of @huzaifaiftikhar1)

To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.

To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.

Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.

Sidekiq queue changes

This merge request contains changes to Sidekiq queues. Please follow the documentation on changing a queue's urgency.

These queues were added:

• cronjob:projects_inactive_projects_deletion_cron
• projects_inactive_projects_deletion_notification

Generated by Danger

Allure report

allure-report-publisher generated test report!

review-qa-blocking: test report

review-qa-blocking: test report for 17569f44

expand test summary

+-------------------------------------------------------------------+
|                          suites summary                           |
+----------------------+--------+--------+---------+-------+--------+
|                      | passed | failed | skipped | flaky | result |
+----------------------+--------+--------+---------+-------+--------+
| Create               | 24     | 0      | 2       | 22    | ❗     |
| Manage               | 31     | 0      | 2       | 32    | ❗     |
| Plan                 | 41     | 0      | 1       | 41    | ❗     |
| Verify               | 12     | 0      | 1       | 12    | ❗     |
| Configure            | 0      | 0      | 1       | 0     | ➖     |
| Protect              | 2      | 0      | 0       | 2     | ❗     |
| Version sanity check | 0      | 0      | 1       | 0     | ➖     |
| Package              | 0      | 0      | 1       | 0     | ➖     |
+----------------------+--------+--------+---------+-------+--------+
| Total                | 110    | 0      | 9       | 109   | ❗     |
+----------------------+--------+--------+---------+-------+--------+

package-and-qa-ff-enabled: test report for 17569f44

expand test summary

+-------------------------------------------------------------------+
|                          suites summary                           |
+----------------------+--------+--------+---------+-------+--------+
|                      | passed | failed | skipped | flaky | result |
+----------------------+--------+--------+---------+-------+--------+
| Manage               | 91     | 0      | 6       | 0     | ✅     |
| Verify               | 33     | 0      | 8       | 5     | ❗     |
| Create               | 143    | 6      | 6       | 11    | ❌     |
| Secure               | 13     | 0      | 6       | 7     | ❗     |
| Package              | 0      | 0      | 3       | 0     | ➖     |
| Plan                 | 52     | 1      | 0       | 2     | ❌     |
| Release              | 6      | 0      | 0       | 0     | ✅     |
| Protect              | 2      | 0      | 0       | 0     | ✅     |
| Fulfillment          | 2      | 0      | 10      | 0     | ✅     |
| Non-devops           | 2      | 0      | 0       | 0     | ✅     |
| Configure            | 0      | 0      | 3       | 0     | ➖     |
| Version sanity check | 0      | 0      | 1       | 0     | ➖     |
+----------------------+--------+--------+---------+-------+--------+
| Total                | 344    | 7      | 43      | 25    | ❌     |
+----------------------+--------+--------+---------+-------+--------+

package-and-qa-ff-disabled: test report for 17569f44

expand test summary

+-------------------------------------------------------------------+
|                          suites summary                           |
+----------------------+--------+--------+---------+-------+--------+
|                      | passed | failed | skipped | flaky | result |
+----------------------+--------+--------+---------+-------+--------+
| Create               | 144    | 5      | 6       | 11    | ❌     |
| Verify               | 33     | 0      | 8       | 5     | ❗     |
| Manage               | 91     | 0      | 6       | 0     | ✅     |
| Secure               | 13     | 0      | 6       | 7     | ❗     |
| Non-devops           | 2      | 0      | 0       | 0     | ✅     |
| Plan                 | 52     | 1      | 0       | 2     | ❌     |
| Release              | 6      | 0      | 0       | 0     | ✅     |
| Configure            | 0      | 0      | 3       | 0     | ➖     |
| Package              | 0      | 0      | 3       | 0     | ➖     |
| Fulfillment          | 2      | 0      | 10      | 0     | ✅     |
| Version sanity check | 0      | 0      | 1       | 0     | ➖     |
| Protect              | 2      | 0      | 0       | 0     | ✅     |
+----------------------+--------+--------+---------+-------+--------+
| Total                | 345    | 6      | 43      | 25    | ❌     |
+----------------------+--------+--------+---------+-------+--------+

removed review request for @mwoolf

added 533 commits

• 1ef46cb6...d9c59dea - 532 commits from branch master
• 07ddaef1 - Add inactive projects deletion feature

Compare with previous version

added 1 commit

• 5bf97665 - Add inactive projects deletion feature

Compare with previous version

changed the description

added 1 commit

• 8c228b31 - Add inactive projects deletion feature

Compare with previous version

added 1 commit

• bc84c9b1 - Add inactive projects deletion feature

Compare with previous version