Add worker to delete expired and revoked OAuth Access Tokens

What does this MR do and why?

Adds a CronJob worker to delete expired OAuth Access Tokens.

The oauth_access_tokens table size has grown substantially and does not have a proper cleanup mechanism. There are over a 100 million expired and revoked tokens each. This MR adds a worker to delete the existing invalid tokens. It doesn't explicitly delete revoked tokens, for simplicity. These tokens will also eventually be covered by the expiry logic. The cron schedule is set to once a month for now since the table size is too large and it can take a while to clean up the existing stale tokens. Once the table size is under control, we need a subsequent change to increase the frequency of the cron job to atleast once a day, if not more.

https://gitlab.com/gitlab-org/gitlab/-/issues/521855

Changelog: changed

References

Example queries fired for each run:

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/40557/commands/124790

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/40557/commands/124791

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/40557/commands/124792

Screenshots or screen recordings

NA

How to set up and validate locally

• Create some expired and live OauthAccessTokens using rails console

def create_oauth_tokens
  batch_size = 10_000
  expired_count = 20000
  application = Doorkeeper::Application.first
  user = User.first
  organization_id = user.organizations.first.id
    
  # Create expired tokens
  (0...expired_count).each_slice(batch_size) do |batch|
    tokens = batch.map do |i|
      {
        application_id: application.id,
        expires_in: 2.hours,
        resource_owner_id: user.id,
        token: Doorkeeper::OAuth::Helpers::UniqueToken.generate,
        scopes: application.scopes.to_s,
        organization_id: organization_id,
        created_at: DateTime.yesterday
      }
    end
    
    OauthAccessToken.insert_all(tokens)
    puts "Created #{batch.last + 1 - expired_count} expired tokens..." if (batch.last + 1) % 100_000 == 0
  end

  # Create live tokens
  (1...1000).each_slice(1000) do |batch|
    tokens = batch.map do |i|
      {
        application_id: application.id,
        resource_owner_id: user.id,
        token: Doorkeeper::OAuth::Helpers::UniqueToken.generate,
        scopes: application.scopes.to_s,
        organization_id: organization_id
      }
    end
    
    OauthAccessToken.insert_all(tokens)
    puts "Created #{batch.last + 1} live tokens..." if (batch.last + 1) % 1000 == 0
  end  
end

# Run the creation
create_oauth_tokens

• Trigger the worker from rails console - Authn::OauthAccessTokenCleanupWorker.perform_in(1.seconds)
• Verify that only the expired tokens are deleted

select count(id) from oauth_access_tokens where created_at + (expires_in * INTERVAL '1 SECOND') < NOW(); ---> should be 0
select count(id) from oauth_access_tokens where created_at + (expires_in * INTERVAL '1 SECOND') > NOW(); ---> should be 999

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #521855