Change Oauth Access and Grant ArchiveWorkers to hard delete records
What does this MR do and why?
Context
- In hhttps://gitlab.com/gitlab-org/gitlab/-/merge_requests/207837 and !202767 (merged) we introduced a worker that enforces a data retention policy on
oauth_access_tokensandoauth_access_grants. - The workers runs once a day
- The workers stores old rows in an archive table (
oauth_access_token_archived_recordsandoauth_access_grant_archived_recordsrespectively) and then deletes the old rows from the related table - The archival was temporary to enable soft deleting records
- We've verified no negative impact from deleting
oauth_access_tokensandoauth_access_grantsand no longer need the archive table
This MR
Change Oauth Access and Grant ArchiveWorkers to hard delete records
Change OauthAccessTokenArchiveWorker and OauthAccessGrantArchiveWorker to hard delete records.
Previously, the OauthAccessTokenArchiveWorker archived old revoked OAuth access tokens to oauth_access_token_archived_records before deleting them.
The OauthAccessGrantArchiveWorker archived to oauth_access_grant_archived_records.
This was a temporary soft deletion strategy.
This change removes the archival step and directly deletes old revoked tokens and grants, bringing us closer to dropping the archive table.
Relates to #562373
Relates to #562376
Additional notes
- The worker gets enqueued in all environments but the logic is gated behind a feature flag and application setting checks (https://gitlab.com/gitlab-org/gitlab/-/blob/bf10545b4086aad456e62a8a819a7a6816a59030/app/workers/authn/data_retention/oauth_access_grant_archive_worker.rb#L24-L25), so in practice the job only deletes records in the GitLab.com environment
- Therefore this change is only expected to impact GitLab.com and not other environments.
References
OAuth Access Tokens:
- Issue: #562373
- Extracted from: Change OauthAccessTokenArchiveWorker deletion strategy from soft deletion to hard deletion
- Worker: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/562376
- Related cleanup: !215367 (merged)
- Query plan: !215726 (comment 2942717164)
- Previous query plan: !202767 (comment 2774688387)
OAuth Access Grants
- Issue: #207837 (closed)
- Extracted from: Change OauthAccessGrantArchiveWorker deletion strategy from soft deletion to hard deletion
- Worker: !202767 (merged)
- Related cleanup: !213902 (merged)
How to set up and validate locally
- Seed the local environment with the script bundle exec rake 'db:seed:oauth_tokens[1]' and bundle exec rake 'db:seed:oauth_grants[1]'
- Run the worker via Rails console:
Authn::DataRetention::OauthAccessTokenArchiveWorker.new.perform
Authn::DataRetention::OauthAccessGrantArchiveWorker.new.perform
- Observe the worker didn't delete anything, because the feature flag and application setting is disabled.
- Enable FF and ApplicationSetting
Feature.enable(:archive_revoked_access_tokens)
Feature.enable(:archive_revoked_access_grants)
# With a valid access token from http://gdk.test:3000/-/user_settings/personal_access_tokens with `api` scope:
curl -X PUT "http://gdk.test:3000/api/v4/application/settings" \
-H "Authorization: Bearer <YOUR_TOKEN>" \
-H "Content-Type: application/json" \
-d '{"authn_data_retention_cleanup_enabled": true}'
- Re-enqueue the job via Rails console:
Authn::DataRetention::OauthAccessTokenArchiveWorker.new.perform
Authn::DataRetention::OauthAccessGrantArchiveWorker.new.perform
- The worker will delete the records from the related table without archiving into the archival table
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.