Project repository replication via SSF
What does this MR do and why?
This MR migrates Project repositories to SSF Geo framework.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Merge request reports
Activity
assigned to @vsizov
- Resolved by Aakriti Gupta
Proper labels assigned to this merge request. Please ignore me.
@vsizov - please see the following guidance and update this merge request.1 Error Please add typebug typefeature, or typemaintenance label to this merge request. Edited by 🤖 GitLab Bot 🤖
- A deleted user
added backend feature flag groupgeo labels
2 Warnings This merge request is quite big (1123 lines changed), please consider splitting it into multiple merge requests. 2f2906e0: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines. 3 Messages CHANGELOG missing: If you want to create a changelog entry for GitLab FOSS, add the
Changelog
trailer to the commit message you want to add to the changelog.If you want to create a changelog entry for GitLab EE, also add the
EE: true
trailer to your commit message.If this merge request doesn't need a CHANGELOG entry, feel free to ignore this message.
This merge request adds or changes files that require a review from the Database team. This merge request adds or changes documentation files. A review from the Technical Writing team before you merge is recommended. Reviews can happen after you merge. This merge request requires a database review. To make sure these changes are reviewed, take the following steps:
- Ensure the merge request has database and databasereview pending labels. If the merge request modifies database files, Danger will do this for you.
- Prepare your MR for database review according to the docs.
- Assign and mention the database reviewer suggested by Reviewer Roulette.
The following files require a review from the Database team:
ee/app/finders/geo/project_repository_registry_finder.rb
Documentation review
The following files require a review from a technical writer:
-
doc/administration/monitoring/prometheus/gitlab_metrics.md
(Link to current live version) -
doc/api/graphql/reference/index.md
(Link to current live version) -
doc/api/geo_nodes.md
(Link to current live version)
The review does not need to block merging this merge request. See the:
-
Metadata for the
*.md
files that you've changed. The first few lines of each*.md
file identify the stage and group most closely associated with your docs change. - The Technical Writer assigned for that stage and group.
- Documentation workflows for information on when to assign a merge request for review.
Reviewer roulette
Changes that require review have been detected!
Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:
Category Reviewer Maintainer backend Minahil Nichols (
@minahilnichols
) (UTC-4, 5 hours behind@vsizov
)Jessie Young (
@jessieay
) (UTC+8, 7 hours ahead of@vsizov
)database Dmytro Biryukov (
@dbiryukov
) (UTC+2, 1 hour ahead of@vsizov
)Dylan Griffith (
@DylanGriffith
) (UTC+10, 9 hours ahead of@vsizov
)frontend Alex Pennells (
@apennells
) (UTC-4, 5 hours behind@vsizov
)Tristan Read (
@tristan.read
) (UTC+12, 11 hours ahead of@vsizov
)test for spec/features/*
John McDonnell (
@john.mcdonnell
) (UTC+1, same timezone as@vsizov
)Maintainer review is optional for test for spec/features/*
To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
Dangeradded devopssystems sectioncore platform labels
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@7d98ca82
- Resolved by Michael Kozono
Allure report
allure-report-publisher
generated test report!e2e-review-qa:
test report for 29a8c2f8expand test summary
+-----------------------------------------------------------------------+ | suites summary | +------------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +------------------+--------+--------+---------+-------+-------+--------+ | Plan | 8 | 42 | 1 | 0 | 51 | ❌ | | Govern | 7 | 17 | 0 | 0 | 24 | ❌ | | Create | 8 | 19 | 1 | 0 | 28 | ❌ | | Verify | 0 | 10 | 0 | 0 | 10 | ❌ | | Manage | 3 | 7 | 1 | 2 | 11 | ❌ | | Data Stores | 5 | 17 | 0 | 0 | 22 | ❌ | | Framework sanity | 9 | 0 | 1 | 0 | 10 | ✅ | | Package | 0 | 0 | 1 | 0 | 1 | ➖ | | Monitor | 0 | 4 | 0 | 0 | 4 | ❌ | +------------------+--------+--------+---------+-------+-------+--------+ | Total | 40 | 116 | 5 | 2 | 161 | ❌ | +------------------+--------+--------+---------+-------+-------+--------+
e2e-package-and-test:
test report for 28757958expand test summary
+------------------------------------------------------------+ | suites summary | +-------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +-------+--------+--------+---------+-------+-------+--------+ +-------+--------+--------+---------+-------+-------+--------+ | Total | 0 | 0 | 0 | 0 | 0 | ➖ | +-------+--------+--------+---------+-------+-------+--------+
e2e-test-on-gdk:
test report for 28757958expand test summary
+------------------------------------------------------------+ | suites summary | +-------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +-------+--------+--------+---------+-------+-------+--------+ +-------+--------+--------+---------+-------+-------+--------+ | Total | 0 | 0 | 0 | 0 | 0 | ➖ | +-------+--------+--------+---------+-------+-------+--------+
- A deleted user
added documentation label
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@a0dfaf57
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@b0c781b1
added 1 commit
- c7257e2e - Implement the GraphQL API for Project Repository
- A deleted user
added database databasereview pending labels
added 1 commit
- 126eae15 - Implement the GraphQL API for Project Repository
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@59ef3adc
added 1581 commits
-
126eae15...c59a6417 - 1579 commits from branch
master
- 9f5f460d - Project repository replication via SSF
- 09548f03 - Implement the GraphQL API for Project Repository
-
126eae15...c59a6417 - 1579 commits from branch
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@c0e281b9
added 803 commits
-
09548f03...205250e4 - 800 commits from branch
master
- 16080334 - Project repository replication via SSF
- 6638633d - Implement the GraphQL API for Project Repository
- 6ef9d66b - Fix specs
Toggle commit list-
09548f03...205250e4 - 800 commits from branch
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@49b2cd05
added 351 commits
-
6ef9d66b...c0b15c92 - 348 commits from branch
master
- fe4ab8f7 - Project repository replication via SSF
- ccb3f1f6 - Implement the GraphQL API for Project Repository
- a19912ab - Fix specs
Toggle commit list-
6ef9d66b...c0b15c92 - 348 commits from branch
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@01377a6f
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@24a49fb7
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@5d4415ce
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@5b355e46
added 2071 commits
Toggle commit list- A deleted user
added frontend label
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@3a3a364e
mentioned in issue #413603 (closed)
- Resolved by Aakriti Gupta
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@3b049d6c
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@1f81a4a6
added 877 commits
-
ab758839...44918cf8 - 875 commits from branch
master
- 8a15965b - Project repository replication via SSF
- 941c7608 - Fix specs
-
ab758839...44918cf8 - 875 commits from branch
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@dbffdcc8
added 1 commit
- 5afa6c90 - Add conditional Project category in admin area
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@16d944e4
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@99dbf5c1
- Resolved by Aakriti Gupta
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@62f914da
- Resolved by Valery Sizov
- Resolved by Aakriti Gupta
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@05ea542f
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@e102a1ab
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@1f6dfdf5
requested review from @aakriti.gupta
@aakriti.gupta It’s ready for review, I still didn't finish my manual testing though.
added featureaddition typefeature labels
- Resolved by Valery Sizov
- Resolved by Valery Sizov
- Resolved by Valery Sizov
- Resolved by Valery Sizov
- Resolved by Aakriti Gupta
- Resolved by Valery Sizov
- Resolved by Aakriti Gupta
- Resolved by Aakriti Gupta
- Resolved by Valery Sizov
Hey @vsizov I have reviewed about the half the MR. I still need to review the tests, but wanted to get this out to you sooner, because my day is split and it might take me sometime today, to finish the review (most likely by my EoD)
mentioned in issue #415488 (closed)
@vsizov Some end-to-end (E2E) tests have been selected based on the stage label on this MR. If not run already, please run thee2e:package-and-test-ee
job in theqa
stage and review the results before merging this MR. (E2E tests are not run automatically on some MRs due to runner resource constraints.)If you would like to run all e2e tests, please apply the pipeline:run-all-e2e label and restart the pipeline.
Once done, please apply the
emoji on this comment.For any questions or help in reviewing the E2E test results, please reach out on the internal #quality Slack channel.
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@61f59589
added 556 commits
-
26c034d4...2e6bc4bb - 554 commits from branch
master
- 7be6a7f4 - Project repository replication via SSF
- 12e9ec60 - Apply review suggestions
-
26c034d4...2e6bc4bb - 554 commits from branch
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@7c03b6b1
- Resolved by Valery Sizov
- Resolved by Valery Sizov
- Resolved by Michael Kozono
@aakriti.gupta
, thanks for approving this merge request.This is the first time the merge request is approved. To ensure full test coverage, a new pipeline will be started shortly.
For more info, please refer to the following links:
added pipeline:mr-approved label
removed review request for @aakriti.gupta
added 281 commits
-
12e9ec60...aad16f9d - 279 commits from branch
master
- b05a101a - Project repository replication via SSF
- 408caae3 - Apply review suggestions
-
12e9ec60...aad16f9d - 279 commits from branch
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@42df3720
mentioned in issue #415614 (closed)
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@4ff02692
requested review from @dbalexandre
- Resolved by Michael Kozono
@dbalexandre Can you review it please? I still need to test it manually. Just need to fix my GDK.
@vsizov Could you please test if project repo verification passes on primary when the git repo is missing?
I am adding in an instruction for it here !124174 (merged) (assigned to you for review).
Please see this bug report of an error that occurs for design repos, but misght be happening for project repos as well: #385977 (closed)
changed milestone to %16.2
- Resolved by Valery Sizov
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Valery Sizov
removed review request for @dbalexandre
added 380 commits
-
50ed5dae...0d430d24 - 378 commits from branch
master
- 01a359d0 - Project repository replication via SSF
- 509b097d - PorojectState: use default primary key
-
50ed5dae...0d430d24 - 378 commits from branch
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@feaa672b
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@8dfa85b2
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@76a00787
added 471 commits
-
e77331bd...2b163ecf - 467 commits from branch
master
- 4227a895 - Project repository replication via SSF
- 1e30d371 - PorojectState: use default primary key
- b9b96dda - Code format
- 5f50e0df - Fix graphQL query in admin area
Toggle commit list-
e77331bd...2b163ecf - 467 commits from branch
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@4c32227e
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@f05b65ee
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@cb4f04a4
added 246 commits
-
02da025c...4599ee34 - 242 commits from branch
master
- fe8c790e - Project repository replication via SSF
- edd4caf6 - PorojectState: use default primary key
- efa9ab46 - Code format
- 7cedd00f - Fix graphQL query in admin area
Toggle commit list-
02da025c...4599ee34 - 242 commits from branch
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@d47006c8
requested review from @dbalexandre
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@d61fb84d
mentioned in issue gitlab-com/www-gitlab-com#8670 (closed)
added 159 commits
-
ed55c1c6...e87025c1 - 158 commits from branch
master
- 3d8092c4 - Project repository replication via SSF
-
ed55c1c6...e87025c1 - 158 commits from branch
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@eccf5e13
requested review from @mkozono and removed review request for @dbalexandre
- Resolved by Michael Kozono
@mkozono Assigning you as a maintainer reviewer because @dbalexandre is OOO.
mentioned in merge request !125044 (merged)
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Valery Sizov
- Resolved by Michael Kozono
removed review request for @mkozono
mentioned in issue #367926 (closed)
added 1383 commits
-
b532ea4f...e6eac0ca - 1382 commits from branch
master
- cfecf86e - Project repository replication via SSF
-
b532ea4f...e6eac0ca - 1382 commits from branch
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@58273e77
requested review from @mkozono
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@efd189be
- Resolved by Valery Sizov
removed review request for @mkozono
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@e6fae114
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@a4bbd292
requested review from @mkozono
marked the checklist item I have evaluated the MR acceptance checklist for this MR. as completed
Note: Danger requests frontend, database, and test review. This is the final MR migrating datatypes to the new code, so most of these changes have been reviewed many times. To state it out loud, I don't think blocking on those reviews is worth the person-hours it would cost.
- Database review is triggered by our adding a Finder class which only
include
s a single module. - I think test review is triggered by adding a feature flag stub to a feature spec.
- Technical writing review is suggested, but the changes are all boilerplate, some of which are autogenerated, so I won't request that review.
- frontend changes are triggered by adding a feature flag to code which is only relevant when the feature flag is off.
Edited by Michael Kozono- Database review is triggered by our adding a Finder class which only
@vsizov Thanks for getting this over the line!
Setting MWPSenabled an automatic merge when the pipeline for 26a79fa6 succeeds
Failure looks unrelated. Retrying the job.
1st Try error in ./spec/features/issues/service_desk_spec.rb:203: expected to find text "by GitLab Support Bot" in "Skip to content GitLab / Help P Project-1 Name Project information Issues 2 List Boards Service Desk Milestones CI/CD Security and Compliance Deployments Packages and registries Infrastructure Monitor Analytics Wiki Snippets Settings Collapse sidebar Sidney Jones2 Project-1 Name Service Desk Open 1 Closed 0 All 1 Toggle history Please select at least one filter to see results" Timeout (30s) reached while running a waiting Capybara finder. Consider using a non-waiting finder. See https://www.cloudbees.com/blog/faster-rails-tests Unexpected browser console output: http://127.0.0.1:39655/assets/webpack/dll.ce7eabf38a7a/vendor.dll.bundle.js 285138:26 "Cache data may be lost when replacing the issues field of a Project object.\n\nTo address this problem (which is not a bug in Apollo Client), either ensure all objects of type IssueConnection have an ID or a custom merge function, or define a custom merge function for the Project.issues field, so InMemoryCache can safely merge these objects:\n\n existing: {\"__persist\":true,\"pageInfo\":{\"__typename\":\"PageInfo\",\"hasNextPage\":false,\"hasPreviousPage\":false,\"startCursor\":\"eyJjcmVhdGVkX2F0IjoiMjAyMy0wNy0wNiAxOTowODoyMC44NzMzNzIwMDAgKzAwMDAiLCJpZCI6IjIyIn0\",\"endCursor\":\"eyJjcmVhdGVkX2F0IjoiMjAyMy0wNy0wNiAxOTowODoyMC44NzMzNzIwMDAgKzAwMDAiLCJpZCI6IjIyIn0\"},\"nodes\":[{\"__ref\":\"Issue:gid://gitlab/Issue/22\"}]}\n incoming: {\"__typename\":\"IssueConnection\",\"count\":1}\n\nFor more information about these options, please refer to the documentation:\n\n * Ensuring entity objects have IDs: https://go.apollo.dev/c/generating-unique-identifiers\n * Defining custom merge functions: https://go.apollo.dev/c/merging-non-normalized-objects\n" RSpec::Retry: 2nd try ./spec/features/issues/service_desk_spec.rb:203 HTML screenshot: /builds/gitlab-org/gitlab/tmp/capybara/service_desk_issue_tracker_issues_list_when_service_desk_vue_list_feature_flag_is_enabled_when_there_are_issues_issues_list_shows_service_desk_reply_to.html Image screenshot: /tmp/capybara/service_desk_issue_tracker_issues_list_when_service_desk_vue_list_feature_flag_is_enabled_when_there_are_issues_issues_list_shows_service_desk_reply_to.png Failures: 1) Service Desk Issue Tracker issues list when service_desk_vue_list feature flag is enabled when there are issues issues list shows service_desk_reply_to in issues list Got 1 failure and 1 other error: 1.1) Failure/Error: expect(page).to have_text('by GitLab Support Bot') expected to find text "by GitLab Support Bot" in "Skip to content GitLab / Help P Project-1 Name Project information Issues 2 List Boards Service Desk Milestones CI/CD Security and Compliance Deployments Packages and registries Infrastructure Monitor Analytics Wiki Snippets Settings Collapse sidebar Sidney Jones2 Project-1 Name Service Desk Open 1 Closed 0 All 1 Toggle history Please select at least one filter to see results" Timeout (30s) reached while running a waiting Capybara finder. Consider using a non-waiting finder. See https://www.cloudbees.com/blog/faster-rails-tests # ./spec/features/issues/service_desk_spec.rb:204:in `block (6 levels) in <main>' # ./spec/spec_helper.rb:444:in `block (3 levels) in <top (required)>' # ./spec/support/sidekiq_middleware.rb:18:in `with_sidekiq_server_middleware' # ./spec/spec_helper.rb:435:in `block (2 levels) in <top (required)>' # ./spec/spec_helper.rb:431:in `block (3 levels) in <top (required)>' # ./lib/gitlab/application_context.rb:66:in `with_raw_context' # ./spec/spec_helper.rb:431:in `block (2 levels) in <top (required)>' # ./spec/spec_helper.rb:268:in `block (2 levels) in <top (required)>' # ./spec/support/system_exit_detected.rb:7:in `block (2 levels) in <main>' # ./spec/support/fast_quarantine.rb:23:in `block (2 levels) in <main>' # ./spec/support/database/prevent_cross_joins.rb:106:in `block (3 levels) in <main>' # ./spec/support/database/prevent_cross_joins.rb:60:in `with_cross_joins_prevented' # ./spec/support/database/prevent_cross_joins.rb:106:in `block (2 levels) in <main>' 1.2) Failure/Error: raise JSConsoleError, message JSConsoleError: Unexpected browser console output: http://127.0.0.1:39655/assets/webpack/dll.ce7eabf38a7a/vendor.dll.bundle.js 285138:26 "Cache data may be lost when replacing the issues field of a Project object.\n\nTo address this problem (which is not a bug in Apollo Client), either ensure all objects of type IssueConnection have an ID or a custom merge function, or define a custom merge function for the Project.issues field, so InMemoryCache can safely merge these objects:\n\n existing: {\"__persist\":true,\"pageInfo\":{\"__typename\":\"PageInfo\",\"hasNextPage\":false,\"hasPreviousPage\":false,\"startCursor\":\"eyJjcmVhdGVkX2F0IjoiMjAyMy0wNy0wNiAxOTowODoyMC44NzMzNzIwMDAgKzAwMDAiLCJpZCI6IjIyIn0\",\"endCursor\":\"eyJjcmVhdGVkX2F0IjoiMjAyMy0wNy0wNiAxOTowODoyMC44NzMzNzIwMDAgKzAwMDAiLCJpZCI6IjIyIn0\"},\"nodes\":[{\"__ref\":\"Issue:gid://gitlab/Issue/22\"}]}\n incoming: {\"__typename\":\"IssueConnection\",\"count\":1}\n\nFor more information about these options, please refer to the documentation:\n\n * Ensuring entity objects have IDs: https://go.apollo.dev/c/generating-unique-identifiers\n * Defining custom merge functions: https://go.apollo.dev/c/merging-non-normalized-objects\n" # ./spec/support/capybara.rb:220:in `block (2 levels) in <main>' # ./spec/spec_helper.rb:444:in `block (3 levels) in <top (required)>' # ./spec/support/sidekiq_middleware.rb:18:in `with_sidekiq_server_middleware' # ./spec/spec_helper.rb:435:in `block (2 levels) in <top (required)>' # ./spec/spec_helper.rb:431:in `block (3 levels) in <top (required)>' # ./lib/gitlab/application_context.rb:66:in `with_raw_context' # ./spec/spec_helper.rb:431:in `block (2 levels) in <top (required)>' # ./spec/spec_helper.rb:268:in `block (2 levels) in <top (required)>' # ./spec/support/system_exit_detected.rb:7:in `block (2 levels) in <main>' # ./spec/support/fast_quarantine.rb:23:in `block (2 levels) in <main>' # ./spec/support/database/prevent_cross_joins.rb:106:in `block (3 levels) in <main>' # ./spec/support/database/prevent_cross_joins.rb:60:in `with_cross_joins_prevented' # ./spec/support/database/prevent_cross_joins.rb:106:in `block (2 levels) in <main>' Finished in 1 minute 14 seconds (files took 39.95 seconds to load) 1 example, 1 failure Failed examples: rspec ./spec/features/issues/service_desk_spec.rb:203 # Service Desk Issue Tracker issues list when service_desk_vue_list feature flag is enabled when there are issues issues list shows service_desk_reply_to in issues list
I also retried a
start-as-if-jh
job, which succeeded and allowed another job to run, and eventually the whole pipeline succeeded.Edited by Michael Kozonomentioned in commit gitlab-org-sandbox/gitlab-jh-validation@26a79fa6
mentioned in commit 9e988f5c
added workflowstaging-canary label
added workflowcanary label and removed workflowstaging-canary label
added workflowstaging label and removed workflowcanary label
added workflowproduction label and removed workflowstaging label
added releasedcandidate label
mentioned in issue #427493 (closed)
mentioned in issue #467255 (closed)
mentioned in issue gitlab-org/quality/triage-reports#20616 (closed)
mentioned in issue gitlab-org/quality/triage-reports#20981 (closed)
mentioned in issue gitlab-org/quality/triage-reports#21533 (closed)
mentioned in issue gitlab-org/quality/triage-reports#22038