Pipeline Triage Report 2020-07-06 to 2020-07-10
DRI
EMEA | AMER | APAC | |
---|---|---|---|
Primary | @niskhakova | @sliaquat | |
Secondary | @tmslvnkc | @dchevalier2 |
Issues carried over from last week
-
all_monitor_features_spec / create_project_with_auto_devops_spec / all_monitor_core_features_spec
: gitlab-org/gitlab#217942 (closed) -
all_monitor_core_features_spec
: gitlab-org/gitlab#223031 (closed) - N+1 jobs: gitlab-org/gitlab#36452 (closed)
-
merge_merge_request_from_fork_spec
: gitlab-org/gitlab#224111 (closed) -
Failure in add_file_template_spec | name "gitlab-ci.yml" is already taken
: gitlab-org/gitlab#225638 (closed)
Summary
Highlights
- A few infra related issues that are now fixed:
- Unable to Merge MRs on staging:
Source branch does not exist. Please restore it or use a different source branch
- Multiple timeout failures on end to end tests
- Multiple e2e test failures when creating a MR via UI after push - "Create merge request" button missing on project show page.
- Determine failures in staging in relation to urgent-cpu-bound shard
- Unable to Merge MRs on staging:
(Also see Remaining Issues below)
✅ / ❌
Staging - Sanity run was green by end of Friday (2020-07-10).
- Full run only had gitlab-org/gitlab#227672 (closed) which is now fixed
✅ / ❌
Staging Orchestrated Mostly green through out the week.
✅
Pre-Production
✅
Production
❌
Nightly - N+1 jobs: gitlab-org/gitlab#36452 (closed)
❌
Master - Intermittent failure in
all_monitor_core_features_spec
: gitlab-org/gitlab#221283 (closed)
Remaining Issues
-
500 in canary and production on GET /api/v4/groups/:group_id API call | 100% CPU spike
: gitlab-org/gitlab#227374 (closed) -
N+1 jobs
are still failing on nightly. Per Stan's suggestion, I enabled instance image jobs in nightly withoutenforce_requests_limits
to compare results with thenplus1-instance-image
jobs -
all_monitor_core_features_spec.rb
: gitlab-org/gitlab#223031 (closed) - Multiple failures due to
registry.gitlab.com
incident gitlab-com/gl-infra/production#2381 (closed)
Edited by Sanad Liaquat