Add cursor batched background migration testing

What does this MR do and why?

Adds preliminary support for cursor-based background migrations to the batched background migration testing system

References

Please include cross links to any resources that are relevant to this MR This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.

• !171337 (closed) - initial (closed) draft showing this working against merge_request_diff_commits
• !168897 - merge request diff commits backfill MR.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

added groupdatabase maintenancescalability typemaintenance labels

assigned to @stomlinson

added pipelinetier-1 label

added Category:Database devopsdata stores sectioncore platform labels

added 1 commit

• 784212e5 - Add cursor batched background migration testing

Compare with previous version

changed the description

removed Category:Database devopsdata stores pipelinetier-1 sectioncore platform labels

added Category:Database devopsdata stores sectioncore platform labels

@krasio @mattkasa I've split the base cursor bbm testing work out from Draft: Add cursor batched background migration ... (!171337 - closed), now that cursor-based batched background migrations are merged this should be good to go. Can you give it a review?

requested review from @krasio and @mattkasa

added backend database databasereview pending labels

	3 Warnings
	This merge request is quite big (628 lines changed), please consider splitting it into multiple merge requests.
	c21895ec: The commit subject must contain at least 3 words. For more information, take a look at our Commit message guidelines.
	This merge request does not refer to an existing milestone.

	1 Message
	CHANGELOG missing: If this merge request needs a changelog entry, add the Changelog trailer to the commit message you want to add to the changelog. If this merge request doesn't need a CHANGELOG entry, feel free to ignore this message.

Reviewer roulette

Category	Reviewer	Maintainer
backend	@bucoleary (UTC-5, 1 hour ahead of author)	@rymai (UTC+1, 7 hours ahead of author)
database	@irina.bronipolsky (UTC+0, 6 hours ahead of author)	@ahegyi (UTC+1, 7 hours ahead of author)

Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

If needed, you can retry the danger-review job that generated this comment.

Generated by Danger

mentioned in merge request !171337 (closed)

added 1 commit

• 07ff9c10 - Add cursor batched background migration testing

Compare with previous version

approved this merge request

added databaseapproved label and removed databasereview pending label

added pipeline:mr-approved label

added pipelinetier-3 pipeline:run-e2e-omnibus-once labels

added 2 commits

• e3a10228 - Add cursor batched background migration testing
• 34462169 - Fix traditional bbm testing specs

Compare with previous version

reset approvals from @krasio by pushing to the branch

E2E Test Result Summary

allure-report-publisher generated test report!

e2e-test-on-gdk: test report for c21895ec

expand test summary

+------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Release     | 5      | 0      | 0       | 0     | 5     | ✅     |
| Plan        | 76     | 0      | 0       | 0     | 76    | ✅     |
| Data Stores | 33     | 0      | 1       | 0     | 34    | ✅     |
| Create      | 129    | 0      | 22      | 0     | 151   | ✅     |
| Verify      | 43     | 0      | 2       | 0     | 45    | ✅     |
| Package     | 24     | 0      | 11      | 0     | 35    | ✅     |
| Analytics   | 2      | 0      | 0       | 0     | 2     | ✅     |
| Govern      | 75     | 0      | 3       | 0     | 78    | ✅     |
| Monitor     | 8      | 0      | 0       | 0     | 8     | ✅     |
| Fulfillment | 2      | 0      | 0       | 0     | 2     | ✅     |
| Secure      | 4      | 0      | 0       | 0     | 4     | ✅     |
| Manage      | 1      | 0      | 1       | 0     | 2     | ✅     |
| Ai-powered  | 0      | 0      | 1       | 0     | 1     | ➖     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 402    | 0      | 41      | 0     | 443   | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+

e2e-test-on-cng: test report for c21895ec

expand test summary

+------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Create      | 140    | 0      | 20      | 0     | 160   | ✅     |
| Verify      | 49     | 0      | 16      | 0     | 65    | ✅     |
| Data Stores | 33     | 0      | 10      | 0     | 43    | ✅     |
| Monitor     | 8      | 0      | 12      | 0     | 20    | ✅     |
| Plan        | 86     | 0      | 8       | 0     | 94    | ✅     |
| Package     | 24     | 0      | 14      | 0     | 38    | ✅     |
| Manage      | 1      | 0      | 9       | 0     | 10    | ✅     |
| Analytics   | 2      | 0      | 0       | 1     | 2     | ✅     |
| Ai-powered  | 0      | 0      | 2       | 0     | 2     | ➖     |
| Govern      | 84     | 0      | 9       | 1     | 93    | ✅     |
| Fulfillment | 2      | 0      | 7       | 1     | 9     | ✅     |
| Secure      | 2      | 0      | 5       | 0     | 7     | ✅     |
| Release     | 5      | 0      | 1       | 0     | 6     | ✅     |
| Growth      | 0      | 0      | 2       | 0     | 2     | ➖     |
| Configure   | 0      | 0      | 3       | 0     | 3     | ➖     |
| ModelOps    | 0      | 0      | 1       | 0     | 1     | ➖     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 436    | 0      | 119     | 3     | 555   | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+

approved this merge request

added 1 commit

• c21895ec - Fix rubocop

Compare with previous version

reset approvals from @mattkasa by pushing to the branch

approved this merge request

added groupglobal search label and removed groupdatabase label

added devopsfoundations label and removed devopsdata stores label

resolved all threads

enabled automatic add to merge train when checks pass

started a merge train

merged

mentioned in commit 4df8cc87

Hello @stomlinson

The database team is looking for ways to improve the database review process and we would love your help!

If you'd be open to someone on the database team reaching out to you for a chat, or if you'd like to leave some feedback asynchronously, just post a reply to this comment mentioning:

@gitlab-org/database-team

And someone will be by shortly!

Thanks for your help!

This message was generated automatically. Improve it or delete it.

added workflowstaging-canary label

added workflowcanary label and removed workflowstaging-canary label

added workflowstaging label and removed workflowcanary label

added workflowproduction label and removed workflowstaging label

changed milestone to %17.7

added workflowpost-deploy-db-staging label and removed workflowproduction label

added workflowpost-deploy-db-production label and removed workflowpost-deploy-db-staging label

added devopsdata_access groupdatabase sectioninfrastructure platforms labels and removed devopsfoundations groupglobal search sectioncore platform labels

added releasedcandidate label

added releasedpublished label and removed releasedcandidate label

added groupdatabase frameworks label and removed groupdatabase [DEPRECATED] label