Geo - Decouple verification from replication feature flags on primary
What does this MR do and why?
Adds a verification feature flag for each replicator class, decoupling the verification from replication feature flags. This allows the checksumming feature to be enabled on the Geo primary sites with replication disabled.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related issues
Related to #469426 (closed)
Merge request reports
Activity
changed milestone to %17.2
added backend data integrity devopssystems featureenhancement groupgeo missed:17.0 missed:17.1 sectioncore platform typefeature workflowin dev + 1 deleted label
assigned to @dbalexandre
added pipelinetier-1 label
- A deleted user
added feature flag label
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
6 Warnings This merge request is quite big (1018 lines changed), please consider splitting it into multiple merge requests. f864a4d7: The commit subject must contain at least 3 words. For more information, take a look at our Commit message guidelines. 4daff8e7: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines. e5dd7b78: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines. d6349669: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines. featureaddition and featureenhancement merge requests normally have a documentation change. Consider adding a documentation update or confirming the documentation plan with the Technical Writer counterpart.
For more information, see:
- The Handbook page on merge request types.
- The definition of done documentation.
17 Messages ee/config/feature_flags/ops/geo_ci_secure_file_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_container_repository_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_dependency_proxy_blob_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_dependency_proxy_manifest_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_design_management_repository_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_group_wiki_repository_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_job_artifact_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_lfs_object_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_merge_request_diff_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_package_file_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_pages_deployment_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_pipeline_artifact_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_project_repository_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_project_wiki_repository_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_snippet_repository_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_terraform_state_version_force_primary_checksumming.yml: Consider filling feature_issue_url:
ee/config/feature_flags/ops/geo_upload_force_primary_checksumming.yml: Consider filling feature_issue_url:
Reviewer roulette
Category Reviewer Maintainer backend @arpitgogia
(UTC+5.5, 5.5 hours ahead of author)
@bala.kumar
(UTC+5.5, 5.5 hours ahead of author)
~"Tooling" Reviewer review is optional for ~"Tooling" @jennli
(UTC-7, 7 hours behind author)
Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
Dangeradded 1 commit
- f3fc1aec - Add a verification feature flag for each replicator class
removed missed:17.0 label
removed missed:17.1 label
added 165 commits
-
d8275161...87eddb43 - 163 commits from branch
master
- 69d0da9f - Add a verification feature flag for each replicator class
- 049528bf - Verify replicable after create/update
-
d8275161...87eddb43 - 163 commits from branch
@mkozono These are the changes we talked about during our call yesterday. Do you mind taking a look?
requested review from @mkozono
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Douglas Barbosa Alexandre
- Resolved by Michael Kozono
- Resolved by Rémy Coutable
@dbalexandre Thank you! I only have a couple questions. Also, does it kind of work locally, or is there a lot of other stuff that breaks when "verification_enabled" returns true with only a primary node?
added 234 commits
-
20d32de3...5f635214 - 232 commits from branch
master
- 7cd3577f - Add a verification feature flag for each replicator class
- 850e7be7 - Verify replicable after create/update
-
20d32de3...5f635214 - 232 commits from branch
added 1 commit
- 6845730f - Remove Geo::BaseRegistry.replication_enabled? method
added 1 commit
- b591798a - Keep verification tied to replication FF on secondaries
added 1 commit
- eadc4ed5 - fixup! Rename enabled? to replication_enabled?
marked this merge request as draft from eadc4ed5
added 297 commits
-
eadc4ed5...7e58dd70 - 290 commits from branch
master
- e2e9ea14 - Add a verification feature flag for each replicator class
- 86824a69 - Verify replicable after create/update
- 60b48d96 - Rename enabled? to replication_enabled?
- 7b34f43a - Remove Geo::BaseRegistry.replication_enabled? method
- 2a09f84b - Keep verification tied to replication FF on secondaries
- 7d5cd621 - Remove unused method
- 64bd4d1c - fixup! Rename enabled? to replication_enabled?
Toggle commit list-
eadc4ed5...7e58dd70 - 290 commits from branch
marked this merge request as draft from 64bd4d1c
added 2 commits
added pipeline:run-all-e2e label
added pipeline:run-all-rspec label
E2E Test Result Summary
allure-report-publisher
generated test report!e2e-test-on-gdk:
test report for 115459d8expand test summary
+------------------------------------------------------------------+ | suites summary | +-------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +-------------+--------+--------+---------+-------+-------+--------+ | Create | 128 | 0 | 12 | 0 | 140 | ✅ | | Verify | 44 | 0 | 3 | 0 | 47 | ✅ | | Plan | 67 | 0 | 2 | 0 | 69 | ✅ | | Data Stores | 27 | 0 | 4 | 0 | 31 | ✅ | | Package | 19 | 0 | 12 | 0 | 31 | ✅ | | Govern | 65 | 0 | 0 | 0 | 65 | ✅ | | Release | 5 | 0 | 0 | 0 | 5 | ✅ | | Fulfillment | 1 | 0 | 0 | 0 | 1 | ✅ | | Monitor | 8 | 0 | 0 | 0 | 8 | ✅ | | Analytics | 2 | 0 | 0 | 0 | 2 | ✅ | | Manage | 0 | 0 | 1 | 0 | 1 | ➖ | +-------------+--------+--------+---------+-------+-------+--------+ | Total | 366 | 0 | 34 | 0 | 400 | ✅ | +-------------+--------+--------+---------+-------+-------+--------+
test report for 115459d8expand test summary
+-------------------------------------------------------------+ | suites summary | +--------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +--------+--------+--------+---------+-------+-------+--------+ | Plan | 8 | 0 | 0 | 0 | 8 | ✅ | | Create | 8 | 0 | 0 | 0 | 8 | ✅ | | Govern | 4 | 0 | 0 | 0 | 4 | ✅ | +--------+--------+--------+---------+-------+-------+--------+ | Total | 20 | 0 | 0 | 0 | 20 | ✅ | +--------+--------+--------+---------+-------+-------+--------+
test report for 9dcbe7b3expand test summary
+------------------------------------------------------------------+ | suites summary | +-------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +-------------+--------+--------+---------+-------+-------+--------+ | Govern | 78 | 1 | 9 | 1 | 88 | ❌ | | Plan | 83 | 0 | 11 | 0 | 94 | ✅ | | Create | 138 | 0 | 16 | 0 | 154 | ✅ | | Verify | 52 | 0 | 14 | 0 | 66 | ✅ | | Release | 5 | 0 | 1 | 0 | 6 | ✅ | | Package | 23 | 0 | 15 | 0 | 38 | ✅ | | Configure | 0 | 0 | 3 | 0 | 3 | ➖ | | Fulfillment | 2 | 0 | 8 | 0 | 10 | ✅ | | Data Stores | 30 | 0 | 13 | 0 | 43 | ✅ | | Manage | 2 | 0 | 8 | 0 | 10 | ✅ | | Monitor | 8 | 0 | 12 | 0 | 20 | ✅ | | Analytics | 2 | 0 | 0 | 0 | 2 | ✅ | | Secure | 3 | 0 | 2 | 0 | 5 | ✅ | | Ai-powered | 0 | 0 | 1 | 0 | 1 | ➖ | | ModelOps | 0 | 0 | 1 | 0 | 1 | ➖ | | Growth | 0 | 0 | 2 | 0 | 2 | ➖ | +-------------+--------+--------+---------+-------+-------+--------+ | Total | 426 | 1 | 116 | 1 | 543 | ❌ | +-------------+--------+--------+---------+-------+-------+--------+
added 232 commits
-
9ef4a0fe...ae01b921 - 226 commits from branch
master
- db7f61e8 - Add a verification feature flag for each replicator class
- 4ce574f7 - Verify replicable after create/update
- f51f204e - Rename enabled? to replication_enabled?
- 55905ad9 - Remove Geo::BaseRegistry.replication_enabled? method
- 2df93213 - Keep verification tied to replication FF on secondaries
- f8057e40 - Remove unused method
Toggle commit list-
9ef4a0fe...ae01b921 - 226 commits from branch
added 1 commit
- 1257e6a5 - fixup! Keep verification tied to replication FF on secondaries
marked this merge request as draft from 1257e6a5
Generated bygitlab_quality-test_tooling
.
Slow tests detected in this merge request. These slow tests might be related to this merge request's changes.Click to expand
Job File Name Duration Expected duration #7260610016 ee/spec/features/projects/pipelines/pipeline_spec.rb#L167
Pipeline GET /:project/-/pipelines/:id identity verification requirement when pipeline failed with user_not_verified status prompts the user to verify their account 50.76 s < 50.13 s #7268971332 ee/spec/features/projects/pipelines/pipeline_spec.rb#L167
Pipeline GET /:project/-/pipelines/:id identity verification requirement when pipeline failed with user_not_verified status prompts the user to verify their account 50.16 s < 50.13 s - A deleted user
added rspec:slow test detected label
added 1 commit
- 055c5543 - Fix Geo sites UI when replication feature flags are off
added 230 commits
-
055c5543...fd663769 - 223 commits from branch
master
- 9c6b9b40 - Add a verification feature flag for each replicator class
- 0a29e750 - Verify replicable after create/update
- 692270e9 - Rename enabled? to replication_enabled?
- 1c053c95 - Remove Geo::BaseRegistry.replication_enabled? method
- 7c634860 - Keep verification tied to replication FF on secondaries
- 02293f5e - Remove unused methods
- 94922612 - Fix Geo sites UI when replication feature flags are off
Toggle commit list-
055c5543...fd663769 - 223 commits from branch
added 171 commits
-
94922612...83b21eef - 164 commits from branch
master
- ab06f0f7 - Add a verification feature flag for each replicator class
- a5426ccd - Verify replicable after create/update
- bdaaf503 - Rename enabled? to replication_enabled?
- c56ebd0b - Remove Geo::BaseRegistry.replication_enabled? method
- 6b52dbe9 - Keep verification tied to replication FF on secondaries
- 45c85f52 - Remove unused methods
- 28f66451 - Fix Geo sites UI when replication feature flags are off
Toggle commit list-
94922612...83b21eef - 164 commits from branch
- Resolved by Michael Kozono
requested review from @mkozono
- Resolved by Douglas Barbosa Alexandre
- Resolved by Michael Kozono
- Resolved by Douglas Barbosa Alexandre
added pipeline:mr-approved label
added pipelinetier-2 label and removed pipelinetier-1 label
- Resolved by Michael Kozono
Before you set this MR to auto-merge
This merge request will progress on pipeline tiers until it reaches the last tier: pipelinetier-3. We will trigger a new pipeline for each transition to a higher tier.
Before you resolve this discussion, please check the following:
- You are the last maintainer of this merge request
- The latest pipeline for this merge request is pipelinetier-3 (You can find which tier it is in the pipeline name)
- This pipeline is recent enough (created in the last 8 hours)
If all the criteria above apply, please resolve this discussion and the set auto-merge for this merge request.
See pipeline tiers and merging a merge request for more details.
requested review from @mkozono and removed approval
added 456 commits
-
28f66451...154a6436 - 447 commits from branch
master
- 609c53c5 - Add a verification feature flag for each replicator class
- 7fa6cec5 - Verify replicable after create/update
- 070ec529 - Rename enabled? to replication_enabled?
- 07608ddb - Remove Geo::BaseRegistry.replication_enabled? method
- a440db10 - Keep verification tied to replication FF on secondaries
- 575adb2d - Remove unused methods
- cab97e0b - Fix Geo sites UI when replication feature flags are off
- 8316e79e - Change geo_handle_after_* guard clauses to return nil
- b76b4b3a - Rename primary checksumming feature flags
Toggle commit list-
28f66451...154a6436 - 447 commits from branch
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
added 1 commit
- 33d81d43 - Rename force_primary_checksumming_feature_key
mentioned in issue #471684 (closed)
mentioned in issue #471685 (closed)
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
- Resolved by Michael Kozono
requested review from @aakriti.gupta
@dbalexandre As far as testing locally, is this pretty much what we need to do, and what we are looking for?
-
Set up a new GDK (without Geo)
-
Add a Premium or higher license
-
Create a GeoNode manually:
GeoNode.create(name: 'first one', url: 'http://127.0.0.1:3000', primary: true)
-
Add this to
gdk.yml
and thengdk reconfigure && gdk restart
:geo: enabled: true secondary: false node_name: 'first one'
I see checksum progress bars
:Edited by Michael Kozono-
removed review request for @aakriti.gupta
removed workflowin dev label
reset approvals from @mkozono and @aakriti.gupta by pushing to the branch
requested review from @mkozono
requested review from @aakriti.gupta
added 1 commit
- 39b8de13 - fixup! Change geo_handle_after_* guard clauses to return nil
marked this merge request as draft from 39b8de13
added 433 commits
-
39b8de13...e73f1754 - 422 commits from branch
master
- d6349669 - 1 earlier commit
- 4fcddeb8 - Verify replicable after create/update
- 1d504760 - Rename enabled? to replication_enabled?
- 91495ab5 - Remove Geo::BaseRegistry.replication_enabled? method
- 33d89d12 - Keep verification tied to replication FF on secondaries
- 83295170 - Remove unused methods
- b7e63979 - Fix Geo sites UI when replication feature flags are off
- eb0e3be9 - Change geo_handle_after_* guard clauses to return nil
- f42d35c4 - Rename primary checksumming feature flags
- 32aff847 - Rename force_primary_checksumming_feature_key
- 929b4a17 - Fix some typos
Toggle commit list-
39b8de13...e73f1754 - 422 commits from branch
removed review request for @aakriti.gupta
added pipelinetier-3 label and removed pipelinetier-2 label
requested review from @rymai
- Resolved by Michael Kozono
@dbalexandre It looks like there are some publish-related spec failures.
removed pipeline:run-all-e2e label
removed pipeline:run-all-rspec label
reset approvals from @mkozono by pushing to the branch
reset approvals from @mkozono by pushing to the branch
added workflowstaging-canary label
added workflowcanary label and removed workflowstaging-canary label
added workflowstaging label and removed workflowcanary label
added workflowproduction label and removed workflowstaging label
added workflowpost-deploy-db-staging label and removed workflowproduction label
added releasedcandidate label
mentioned in issue #473540
mentioned in merge request kubitus-project/kubitus-installer!3178 (merged)
mentioned in issue gitlab-com/gl-infra/production#18319 (closed)
added releasedpublished label and removed releasedcandidate label
mentioned in merge request !172163 (closed)
mentioned in issue #504702 (closed)