2021-03-15 - Triage report for "group::geo"
Hi, @fzimmer @nhxnguyen
This is a group or stage level triage report that aims to summarize the feature proposals and bugs which have not been scheduled or triaged. For more information please refer to the handbook:
Scheduling the workload is a collaborative effort by the Product Managers and Engineering Managers for that group. Please work together to provide a best estimate on priority and milestone assignments. For each issue please:
- Determine if the issue should be closed if it is no longer relevant or a duplicate.
- If it is still relevant please assign either a best estimate versioned milestone, the %Backlog or the %Awaiting further demand milestone.
- Specifically for ~bug, if there is no priority or clarity on a versioned milestone, please add a Priority label. Priority labels have an estimate SLO attached to them and help team members and the wider community understand roughly when it will be considered to be scheduled.
- Once a milestone has been assigned please check off the box for that issue.
- Please work with your team to complete the list by the due date set.
Feature Proposal Section
For the following feature proposals. Please either close or assign either a versioned milestone, the %Backlog or the %Awaiting further demand milestone.
customer
Unscheduled ~feature with- gitlab-org/gitlab#324172 GEO: pausing and resuming replication using external databases Enterprise Edition, GitLab Premium, customer, ~"devops::enablement", ~"feature", groupgeo, ~"section::enablement"
- gitlab-org/gitlab#321606 Provide a backup upload-only rake task Category:Backup/Restore of GitLab instances, customer, ~"devops::enablement", ~"feature", groupgeo, ~"section::enablement"
- gitlab-org/gitlab#292227 Allow location aware URL for Registry with AWS Route 53 Enterprise Edition, GitLab Premium, customer, ~"devops::enablement", ~"feature", groupgeo, ~"section::enablement"
Unscheduled ~feature (non-customer)
- gitlab-org/gitlab#324417 (closed) Geo: Verify Snippets (feature parity with Package Files) Enterprise Edition, GitLab Premium, ~"devops::enablement", ~"feature", featureaddition, groupgeo, ~"section::enablement"
- gitlab-org/gitlab#324208 (closed) Geo: On high retry count, alternate between snapshot and Git fetch Enterprise Edition, GitLab Premium, ~"devops::enablement", ~"feature", ~"feature::maintenance", groupgeo, ~"section::enablement", workflowscheduling
- gitlab-org/gitlab#323286 (closed) Geo: Self Service Framework - Verify LFS Objects Enterprise Edition, GitLab Premium, ~"devops::enablement", ~"feature", groupgeo, ~"section::enablement"
- gitlab-org/gitlab#323285 (closed) Geo: Self Service Framework - Verify Merge request Diffs Enterprise Edition, GitLab Premium, ~"devops::enablement", ~"feature", geoactive, groupgeo, ~"section::enablement", workflowready for development
- gitlab-org/gitlab#323238 Geo: improve snapshot approach to preserve existing copy on secondary in case primary is corrupt Category:Disaster Recovery, Enterprise Edition, Geo Administration, GitLab Premium, data integrity, ~"devops::enablement", ~"feature", groupgeo, ~"section::enablement"
- gitlab-org/gitlab#322886 (closed) Geo: Self Service Framework - Verify terraform states Enterprise Edition, GitLab Premium, ~"devops::enablement", ~"feature", geoactive, groupgeo, ~"section::enablement", workflowready for development
- gitlab-org/gitlab#322691 (closed) Docs: Development guidelines for where to do permission checks Architecture decision, development guidelines, ~"devops::enablement", docs-only, documentation, ~"feature", geoactive, groupgeo, ~"performance", ~"section::enablement", security, workflowin review
- gitlab-org/gitlab#320940 Follow-up from "Add geo node status metrics to usage ping" Enterprise Edition, GitLab Premium, ~"devops::enablement", ~"feature", groupgeo, ~"section::enablement"
- gitlab-org/gitlab#301247 (closed) Geo: Successful sync should cause verification Enterprise Edition, GitLab Premium, ~"devops::enablement", ~"feature", featureaddition, groupgeo, ~"section::enablement"
- gitlab-org/gitlab#301244 (closed) Geo: Verification failure should cause resync, not retry of verification Enterprise Edition, GitLab Premium, ~"devops::enablement", ~"feature", featureaddition, groupgeo, ~"section::enablement"
- gitlab-org/gitlab#299824 (closed) Geo: Implement UI SSF verification details Enterprise Edition, GitLab Premium, backend, ~"devops::enablement", ~"feature", featureaddition, frontend, groupgeo, ~"section::enablement"
- gitlab-org/gitlab#296979 Add caution text for handling background jobs ~"Accepting merge requests", backend, ~"devops::enablement", ~"feature", groupgeo, missed:13.8, quad-planningcomplete-no-action, ~"section::enablement", workflowready for development
- gitlab-org/gitlab#296822 (closed) Geo: Update admin UI to align with terminology UX, ~"devops::enablement", ~"feature", ~"feature::maintenance", frontend, groupgeo, ~"section::enablement"
- gitlab-org/gitlab#293426 create gitlab-backup at two or more locations Category:Backup/Restore of GitLab instances, automation:ml, ~"devops::enablement", ~"feature", groupgeo, ~"section::enablement"
- gitlab-org/gitlab#292691 Geo: When a Snippet is not yet replicated, it should redirect to the primary Enterprise Edition, GitLab Premium, ~"devops::enablement", ~"feature", featureaddition, groupgeo, ~"section::enablement"
- gitlab-org/gitlab#23567 Consider hair_trigger gem as way to use/implement db specific triggers Category:Geo Replication, Enterprise Edition, backstage [DEPRECATED], database, ~"devops::enablement", ~"feature", geotriage, groupgeo, ~"section::enablement"
- gitlab-org/gitlab#22082 (closed) Always keep a copy of the currently-deployed version of a Pages site ~"Accepting merge requests", Category:Geo Replication, Category:Pages, Enterprise Edition, ~"devops::enablement", ~"feature", groupgeo, ~"section::enablement"
Bug Section
For the following bugs. Please either close or assign either a versioned milestone, the %Backlog or the %Awaiting further demand milestone and ensure that a priority label is set.
- Engineering Managers: Please add a severity label for those issues without one
- Product Designers: Please add a severity label to UX ~bug issues without one
Heatmap for all bugs
Bugs for their priority and severity label are counted here. Every bug should have severity and priority labels applied. Please take a look at the bugs which fall into the columns indicating that the priority or severity labels are currently missing.
severity1 | severity2 | severity3 | severity4 | No severity | |
---|---|---|---|---|---|
priority1 | 1 | 0 | 0 | 0 | 0 |
priority2 | 0 | 4 | 0 | 0 | 0 |
priority3 | 0 | 1 | 7 | 3 | 1 |
priority4 | 0 | 0 | 5 | 13 | 0 |
No priority | 0 | 0 | 11 | 10 | 6 |
customer bugs
Heatmap forseverity1 | severity2 | severity3 | severity4 | No severity | |
---|---|---|---|---|---|
priority1 | 0 | 0 | 0 | 0 | 0 |
priority2 | 0 | 1 | 0 | 0 | 0 |
priority3 | 0 | 1 | 3 | 0 | 1 |
priority4 | 0 | 0 | 4 | 2 | 0 |
No priority | 0 | 0 | 2 | 2 | 1 |
customer
Unscheduled ~bug with- gitlab-org/gitlab#323750 (closed) Geo project/wiki sync shows no errors when Gitaly is down ~"Accepting merge requests", Enterprise Edition, GitLab Premium, ~"bug", customer, ~"devops::enablement", groupgeo, ~"section::enablement", severity3
- gitlab-org/gitlab#323084 Zero-downtime updated for a Gitaly cluster on secondary Geo site cause read-only repositories Category:Geo Replication, Geo Administration, GitLab Ultimate, ~"bug", customer, ~"devops::enablement", groupgeo, needs investigation, priority3, ~"section::enablement", self-managed
- gitlab-org/gitlab#297472 (closed) Geo: Secondaries are orphaning artifact files ~"bug", customer, ~"devops::enablement", groupgeo, priority3, regression, regression:13.6, ~"section::enablement", severity3
- gitlab-org/gitlab#230881 Backups not automatically setting "skipped" for disabled registries Category:Backup/Restore of GitLab instances, ~"bug", customer, ~"devops::enablement", groupgeo, ~"section::enablement", severity3
- gitlab-org/gitlab#202693 (confidential) ~"(confidential)"
- gitlab-org/gitlab#37135 (closed) Sidekiq workers get automatically re-enabled when disabled from the UI ~"bug", customer, devopsmanage, groupgeo, priority4, severity3, vintage, workflowsolution validation
- gitlab-org/gitlab#9803 Geo: Handle moving project repository storage Category:Geo Replication, Enterprise Edition, ~"bug", customer, ~"devops::enablement", groupgeo, priority4, ~"section::enablement", severity4, workflowsolution validation
- gitlab-org/gitlab#22249 (closed) gitlab:backup:restore does not clean tmp directory in backups directory ~"Accepting merge requests", Category:Backup/Restore of GitLab instances, ~"bug", customer, ~"devops::enablement", groupgeo, ~"section::enablement", severity4, vintage
-
gitlab-org/gitlab#5267 Unclean kill of sidekiq meant the repository sync didn’t happen.
Failed to obtain exclusive lease
~"Accepting merge requests", Category:Geo Replication, Enterprise Edition, ~"bug", customer, ~"devops::enablement", groupgeo, priority4, ~"section::enablement", severity3, vintage - gitlab-org/gitlab#20018 (closed) Backup skips repositories where HEAD pointing to a non-existent master branch Category:Backup/Restore of GitLab instances, Create [DEPRECATED], Manage [DEPRECATED], awaiting feedback, backend, ~"bug", customer, default-priority, default-severity, ~"devops::enablement", groupgeo, priority3, rake tasks, ~"section::enablement", severity3, vintage
- gitlab-org/gitlab#19347 (closed) Backup restore SKIP option (omnibus) Category:Backup/Restore of GitLab instances, Create [DEPRECATED], Manage [DEPRECATED], backend, ~"bug", customer, default-priority, default-severity, ~"devops::enablement", groupgeo, priority3, ~"section::enablement", severity3, vintage
Unscheduled ~bug (non-customer)
- gitlab-org/gitlab#322707 (closed) Default branch setting lost during backup/restore process Category:Backup/Restore of GitLab instances, ~"bug", ~"devops::enablement", groupgeo, ~"section::enablement", severity3
- gitlab-org/gitlab#321779 (closed) backup_keep_time doesn't work with back_upload_connection ~"bug", ~"devops::enablement", groupgeo, ~"section::enablement", severity3
- gitlab-org/gitlab#320809 (closed) Geo: Preflight checks blocked by repository checked status ~"bug", ~"devops::enablement", geoactive, groupgeo, ~"section::enablement", severity3, workflowready for development
- gitlab-org/gitlab#301249 (closed) Geo: Fail syncs which exceed a timeout Enterprise Edition, GitLab Premium, ~"bug", ~"devops::enablement", geoactive, groupgeo, ~"section::enablement", severity3, workflowin dev
- gitlab-org/gitlab#300761 (closed) Output of gitlab-ctl promote-db is misleading when there is nothing to do ~"bug", ~"devops::enablement", groupgeo, ~"section::enablement", severity4, workflowscheduling
- gitlab-org/gitlab#294486 Geo: Sync requests are rate limited by the primary Enterprise Edition, GitLab Premium, ~"bug", ~"devops::enablement", groupgeo, priority4, ~"section::enablement", severity4
- gitlab-org/gitlab#294074 (closed) Geo: Snippet repository replication should be limited by "Repository synchronization concurrency limit" Enterprise Edition, GitLab Premium, ~"bug", ~"devops::enablement", groupgeo, ~"section::enablement", severity4
- gitlab-org/gitlab#293676 (closed) Gitlab restore fails without error message Category:Backup/Restore of GitLab instances, ~"approaching-SLO", automation:ml, awaiting feedback, ~"bug", ~"devops::enablement", geotriage, groupgeo, ~"missed-SLO", priority2, ~"section::enablement", severity2
- gitlab-org/gitlab#292690 Geo: When repo is not yet replicated, the Git fetch redirect URL should not contain the word "push" UX, ~"bug", ~"devops::enablement", groupgeo, priority4, ~"section::enablement", severity4
- gitlab-org/gitlab#292688 (closed) Geo: Fetching a personal snippet repo from a secondary always redirects the request to the primary ~"bug", ~"devops::enablement", groupgeo, priority4, ~"section::enablement", severity4
- gitlab-org/gitlab#288021 (closed) Gitlab::Git::CommandError: 2:NoMethodError: undefined method `relative_path’ for nil:NilClass. prevents project migration to Hashed Storage Category:Gitaly, ~"bug", devopscreate, groupgeo, needs investigation, repository, sectiondev
- gitlab-org/gitlab#276216 (closed) Geo - Enabling the Tracking Database requires two reconfigures ~"bug", ~"devops::enablement", groupgeo, ~"section::enablement", severity3
- gitlab-org/gitlab#260456 Geo: Feature flag changes do not propagate immediately to secondaries ~"bug", ~"devops::enablement", groupgeo, priority4, ~"section::enablement", severity4
- gitlab-org/gitlab#246567 (closed) Restore hashed repositories from backup Category:Backup/Restore of GitLab instances, ~"bug", ~"devops::enablement", groupgeo, ~"section::enablement", severity3
- gitlab-org/gitlab#241761 (closed) Geo: Repo update events are created after project import or mirror updates even if there is no repo ~"Accepting merge requests", Enterprise Edition, GitLab Premium, ~"bug", ~"devops::enablement", groupgeo, priority4, ~"section::enablement", severity4
- gitlab-org/gitlab#233635 (closed) The "repository_updated_max_id" metric doesn't make sense anymore ~"bug", deprecation, ~"devops::enablement", groupgeo, ~"section::enablement", severity4
- gitlab-org/gitlab#230958 (closed) Geo: Make selective sync exclude design thumbnails Enterprise Edition, For Scheduling, GitLab Premium, ~"bug", ~"devops::enablement", geoactive, groupgeo, priority2, ~"section::enablement", severity2, workflowready for development
- gitlab-org/gitlab#230338 (closed) A secondary node fails to identify itself with relative URLs awaiting feedback, ~"bug", ~"devops::enablement", groupgeo, ~"section::enablement", severity3
- gitlab-org/gitlab#222261 (closed) After restoring backup, Gitlab begins to have 500 internal server errors Category:Backup/Restore of GitLab instances, ~"bug", ~"devops::enablement", groupgeo, ~"section::enablement"
- gitlab-org/gitlab#220026 (closed) Projects 404 after a backup restore Category:Backup/Restore of GitLab instances, ~"bug", ~"devops::enablement", groupgeo, needs investigation, ~"section::enablement", severity3
- gitlab-org/gitlab#217980 (closed) Backup restore task does not create backup directory automatically Category:Backup/Restore of GitLab instances, ~"bug", ~"devops::enablement", groupgeo, priority4, ~"section::enablement", severity4
- gitlab-org/gitlab#217880 (closed) Geo: Endpoints that facilitate proxying Git SSH requests may need to check the Geo IP whitelist ~"bug", ~"devops::enablement", groupgeo, priority4, ~"section::enablement", security, severity4
-
gitlab-org/gitlab#217631 (closed) ActiveRecord is not responding
connected?
with true ~"bug", ~"devops::enablement", groupgeo, priority3, rails6, ~"section::enablement", severity3 - gitlab-org/gitlab#215085 Geo GraphQL API: querying geoNode by name not working ~"bug", ~"devops::enablement", groupgeo, ~"section::enablement", severity4
- gitlab-org/gitlab#212936 (closed) GitLab sometimes failed in processing the backups, particularly in the phase of uploading Category:Backup/Restore of GitLab instances, ~"bug", ~"devops::enablement", groupgeo, ~"section::enablement"
- gitlab-org/gitlab#207023 (closed) Backup performance Category:Backup/Restore of GitLab instances, ~"bug", ~"devops::enablement", groupgeo, ~"performance", ~"section::enablement", severity4, vintage
- gitlab-org/gitlab#201877 (closed) GitLab restore hangs / is stuck on epoll_wait Category:Backup/Restore of GitLab instances, backend, ~"bug", ~"devops::enablement", groupgeo, ~"section::enablement", stale, vintage
- gitlab-org/gitlab#11173 (closed) Git push SSH proxy may not handle relative_url_root ~"Accepting merge requests", Enterprise Edition, Geo Administration, ~"bug", ~"devops::enablement", groupgeo, priority3, ~"section::enablement", severity4, vintage
Heatmap for ~missed-SLO bugs
severity1 | severity2 | severity3 | severity4 | No severity | |
---|---|---|---|---|---|
priority1 | 0 | 0 | 0 | 0 | 0 |
priority2 | 0 | 1 | 0 | 0 | 0 |
priority3 | 0 | 1 | 0 | 0 | 0 |
priority4 | 0 | 0 | 0 | 0 | 0 |
No priority | 0 | 0 | 0 | 0 | 0 |
vintage bugs
Heatmap forseverity1 | severity2 | severity3 | severity4 | No severity | |
---|---|---|---|---|---|
priority1 | 0 | 0 | 0 | 0 | 0 |
priority2 | 0 | 0 | 0 | 0 | 0 |
priority3 | 0 | 0 | 3 | 2 | 0 |
priority4 | 0 | 0 | 2 | 3 | 0 |
No priority | 0 | 0 | 0 | 2 | 2 |
Open Quarantined Flaky Specs Issues
- gitlab-org/gitlab#241701 (closed) Backup::Repository spec looks stuck on CI Category:Backup/Restore of GitLab instances, ~"devops::enablement", failureflaky-test, groupgeo, missed:13.6, missed:13.7, ~"quarantined test", ~"section::enablement", test
This is a group level triage report that aims to collate the latest bug reports (for frontend and otherwise) and feature proposals. For more information please refer to the handbook:
If assignees or people mentioned in this individual triage report need to be amended, please edit group-definition.yml.