Add background migration to expiry all OAuth tokens
What does this MR do and why?
In !86362 (merged), we are removing the backward compatibility for non-expiring OAuth tokens introduced. This MR makes sure that all existing tokens have an expiry, otherwise they would never expire.
Related to #340848 (closed)
Screenshots or screen recordings
These are strongly recommended to assist reviewers and reduce the time to merge your change.
Migration output
up
$ rake db:migrate:up:main VERSION=20220428133724
== 20220428133724 ScheduleExpireOAuthTokens: migrating ========================
== 20220428133724 ScheduleExpireOAuthTokens: migrated (0.0431s) ===============
down
$ rake db:migrate:up:main VERSION=20220428133724
== 20220428133724 ScheduleExpireOAuthTokens: migrating ========================
== 20220428133724 ScheduleExpireOAuthTokens: migrated (0.0431s) ===============
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Merge request reports
Activity
changed milestone to %15.0
added groupauthentication and authorization [DEPRECATED] typefeature + 1 deleted label
assigned to @ifarkas
Suggested Reviewers (beta)
This is an experimental ML-based code reviewer recommendation system created by ~"group::applied ml".
The individuals below may be good candidates to participate in the review based on various factors.
After you review all recommendations, please assign reviewers manually, as this is not done automatically.
You can use slash commands in comments to quickly assign
/assign_reviewer @user1
.Reviewers @stomlinson
,@a_akgun
,@abrandl
,@alinamihaila
,@dbalexandre
If you do not believe these recommendations are useful or if you do not want to use any of the suggestions, please apply the label Bad Suggested Reviewer. You can also provide feedback for this feature on this issue:
https://gitlab.com/gitlab-org/gitlab/-/issues/357923
.Automatically generated by Suggested Reviewers Bot
Edited by GitLab Reviewer-Recommender Bot1 Warning New migrations added but db/structure.sql wasn't updated Usually, when adding new migrations, db/structure.sql should be
updated too (unless the migration isn't changing the DB schema
and isn't the most recent one).Reviewer roulette
Changes that require review have been detected!
Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:
Category Reviewer Maintainer backend Josianne Hyson ( @jhyson
) (UTC+12, 10 hours ahead of@ifarkas
)Marc Shaw ( @marc_shaw
) (UTC+12, 10 hours ahead of@ifarkas
)database Pavel Shutsin ( @pshutsin
) (UTC+2, same timezone as@ifarkas
)Simon Tomlinson ( @stomlinson
) (UTC-5, 7 hours behind@ifarkas
)~migration No reviewer available No maintainer available To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.
Generated by
DangerAllure report
allure-report-publisher
generated test report!review-qa-blocking:
test report for 3b3bde42expand test summary
+-------------------------------------------------------------------+ | suites summary | +----------------------+--------+--------+---------+-------+--------+ | | passed | failed | skipped | flaky | result | +----------------------+--------+--------+---------+-------+--------+ | Create | 23 | 0 | 2 | 18 | ❗ | | Plan | 36 | 0 | 1 | 29 | ❗ | | Protect | 2 | 0 | 0 | 2 | ❗ | | Manage | 26 | 0 | 2 | 21 | ❗ | | Verify | 10 | 0 | 1 | 10 | ❗ | | Configure | 0 | 0 | 1 | 0 | ➖ | | Package | 0 | 0 | 1 | 0 | ➖ | | Version sanity check | 0 | 0 | 1 | 0 | ➖ | +----------------------+--------+--------+---------+-------+--------+ | Total | 97 | 0 | 9 | 80 | ❗ | +----------------------+--------+--------+---------+-------+--------+
Setting label(s) devopsmanage sectiondev based on ~"group::authentication and authorization".
added devopsmanage sectiondev labels
- Resolved by Imre Farkas
@ifarkas, please can you answer the question: Should this have a feature flag? to help with code review for the Authentication and Authorization group.This nudge was added by this triage-ops policy.
added 1 commit
- e2a524e9 - Add background migration to expiry all OAuth tokens
added 1 commit
- 628d6a88 - Add background migration to expiry all OAuth tokens
- Resolved by Imre Farkas
- Resolved by Imre Farkas
- Resolved by Imre Farkas
@alexbuijs, could you please review for backend?
requested review from @alexbuijs
requested review from @jdrpereira
added databasereviewed label
@jdrpereira
, thanks for approving this merge request.This is the first time the merge request is approved. To ensure full test coverage, a new pipeline has been started.
For more info, please refer to the following links:
- Resolved by Imre Farkas
requested review from @krasio
- Resolved by Imre Farkas
requested review from @.luke and removed review request for @alexbuijs
- Resolved by Imre Farkas
- Resolved by Imre Farkas
- Resolved by Imre Farkas
Database migrations
Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).
Migration Type Total runtime Result DB size change 20220428133724 - ScheduleExpireOAuthTokens Post deploy 1.5 s +0.00 B Runtime Histogram for all migrations
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 4 0.1 seconds - 1 second 0 1 second - 5 minutes 0 5 minutes + 0 Migration: 20220428133724 - ScheduleExpireOAuthTokens
- Type: Post deploy
- Duration: 1.5 s
- Database size change: +0.00 B
Query Calls Total Time Max Time Mean Time Rows SELECT MAX("id")
FROM "oauth_access_tokens"
/*application:test,db_config_name:main*/1 5.8 ms 5.8 ms 5.8 ms 1 INSERT INTO "batched_background_migrations" ("created_at", "updated_at", "max_value", "batch_size", "sub_batch_size", "interval", "status", "job_class_name", "table_name", "column_name", "total_tuple_count", "started_at") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12) RETURNING "id" /*application:test,db_config_name:main*/
1 0.1 ms 0.1 ms 0.1 ms 1 SELECT $1 AS one
FROM "batched_background_migrations"
WHERE "batched_background_migrations"."job_class_name" = $2 AND "batched_background_migrations"."table_name" = $3 AND "batched_background_migrations"."column_name" = $4 AND (job_arguments = $5)
LIMIT $6 /*application:test,db_config_name:main*/1 0.0 ms 0.0 ms 0.0 ms 0 SELECT $1 AS one
FROM "batched_background_migrations"
WHERE "batched_background_migrations"."job_arguments" = $2 AND "batched_background_migrations"."job_class_name" = $3 AND "batched_background_migrations"."table_name" = $4 AND "batched_background_migrations"."column_name" = $5
LIMIT $6 /*application:test,db_config_name:main*/1 0.0 ms 0.0 ms 0.0 ms 0 Histogram for ScheduleExpireOAuthTokens
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 4 0.1 seconds - 1 second 0 1 second - 5 minutes 0 5 minutes + 0
Background migrations
Other migrations pending on GitLab.com
Migration Type Total runtime Result DB size change 20220413235818 - AddMaxSeatsUsedChangedAtToGitlabSubscriptions Regular 1.2 s +0.00 B 20220414203622 - AddIndexForColumnsUserCustomAttribute Regular 17.3 s +188.74 MiB 20220419223906 - AddArkoseNamespaceToApplicationSettings Regular 2.2 s +0.00 B 20220420173247 - AddGroupInheritanceTypeToPeAuthorizable Regular 1.3 s +0.00 B 20220422220507 - RemoveTmpIndexSupportingLeakyRegexCleanup Regular 2.8 s -88.00 KiB 20220324165436 - ScheduleBackfillProjectSettings Post deploy 1.8 s +0.00 B Clone Details
Clone ID Clone Created At Clone Data Timestamp Expected Removal Time database-testing-1185746
2022-05-05T02:19:42Z 2022-04-26T01:21:49Z 2022-05-05 14:23:20 +0000 Artifacts
Brought to you by gitlab-org/database-team/gitlab-com-database-testing. Epic
- A deleted user
added database-testing-automation label
- Resolved by Imre Farkas
- Resolved by Luke Duncalfe
removed review request for @.luke
added breaking change label
@ifarkas thanks for adding the breaking change label!
This merge request introduces breaking changes. Learn more about breaking changes.
It's important to identify how the breaking change was introduced. To estimate the impact, try to assess the following:
- Are there existing users depending on this feature?
- Are self-managed customers affected?
- To verify and quantify usage, use Grafana or Kibana.
- If you're not sure about how to query the data, contact the infrastructure team on their Slack channel, #infrastructure-lounge
- Was sufficient time given to communicate the change?
- Changes in the permissions, the API schema, and the API response might affect existing 3rd party integrations.
- Reach out to the Support team or Technical Account Managers and ask about the possible impact of this change.
- Are there existing users depending on this feature?
added 1014 commits
-
628d6a88...b6a449ec - 1012 commits from branch
master
- e7b092bc - Add background migration to expiry all OAuth tokens
- 18a2b7e8 - Address reviewer feedback
-
628d6a88...b6a449ec - 1012 commits from branch
requested review from @.luke
removed review request for @.luke
- Resolved by Imre Farkas
added 225 commits
-
18a2b7e8...12c7d7cf - 223 commits from branch
master
- 34f40fa8 - Add background migration to expiry all OAuth tokens
- 561cb0c2 - Address reviewer feedback
-
18a2b7e8...12c7d7cf - 223 commits from branch
added 57 commits
-
561cb0c2...c2336c2c - 55 commits from branch
master
- c862c9f3 - Add background migration to expiry all OAuth tokens
- 3b3bde42 - Address reviewer feedback
-
561cb0c2...c2336c2c - 55 commits from branch
added databaseapproved label and removed databasereviewed label
enabled an automatic merge when the pipeline for 8a420731 succeeds
mentioned in commit 832d65d8
added workflowstaging-canary label