Turn delayed project deletion into default behaviour and remove option to delete immediately

This is a follow-up to a recent incident captured here.

Problem

Premium users can make use of delayed project deletion. However, our support team still gets requests from users that accidentally deleted projects. This usually requires a time consuming recovery process, and leaves customers frustrated when they face this issue.

Current setting

Proposal

To prevent this from happening, we propose to make delayed project deletion the default workflow behaviour for all premium and ultimate customers. Currently this is a setting that needs to be enabled. However, it is safer for our users to have this available as the default behaviour without the option to turn it off.

• Update admin setting
  
• Remove deletion protection setting from group settings

Implementation plan

• Leave the setting in the UI, but remove the option None, delete immediately.
• Remove the dropdown option groups only and turn the remaining dropwdown selection groups and projects into text
• The option Keep deleted groups and projects for XX days appears without the radio button and just as plain text.
• The time frame can still be adjusted between 0 < XX < 90 days, as before.
• Remove the delayed_project_deletion and delayed_group_deletion parameters from the /application/settings API
• Update the documentation to state that this is now default workflow behaviour. If customers want to remove a project immediately, they can still do that by using delete now.
  • https://docs.gitlab.com/ee/api/settings.html#list-of-settings-that-can-be-accessed-via-api-calls
    • Also note that the section regarding delayed_project_deletion and delayed_group_deletion for the deletion_adjourned_period parameter should also be removed
  • https://docs.gitlab.com/ee/user/gitlab_com/index.html#delayed-project-deletion (Manoj: Not changing this part of the documentation setting because this part of the docs talks about gitlab.com settings, and this is not enabled on gitlab.com yet. We should merge the change for this doc only in 16.0)
  • https://docs.gitlab.com/ee/user/group/manage.html#enable-delayed-project-deletion
• Ensure that the DELETE /groups/:id endpoint with the params permanently_remove=true and full_path still works as expected, which is used by E2E tests to clean up resources. See Availability and Testing section below.
• Announce to support team when the change goes into production

How to approach it technically?

• Delayed project deletion exists on instance and group level
• Create a feature flag
• Behind the feature flag, we should:
  • Stop using this setting in the code
  • Hide UI setting. - Added frontend implementation plan in #389557 (closed)
• When it works on SaaS, we should default enable feature flag before release.
• In the next milestone we can start column removal process according to the database docs.

Availability and Testing

• Please ensure that the API endpoint DELETE /groups/:id endpoint with the params permanently_remove=true and full_path continues to work as expected, as this is used by our test data deletion scripts for our E2E tests
  • The qa/qa/tools/delete_subgroups.rb rake task can be used to test this endpoint. You can create a top level group and several subgroups on your local GDK that you want to delete. Then, the task can be run using GITLAB_QA_ACCESS_TOKEN=<access token for group owner or admin> GITLAB_ADDRESS=<gdk address> TOP_LEVEL_GROUP_NAME=<name of group with subgroups to delete> PERMANENTLY_DELETE=true bundle exec rake delete_subgroups
• When the feature flag is enabled on staging, a full E2E test run should be kicked off (the results can then be found in the #qa-staging Slack channel) Please observe the results of the delete-test-resources job in the post-test stage from that pipeline to ensure no issues occur when the script permanently deletes subgroups.
• Feel free to reach out to @vburton regarding questions or assistance on the above