Failure in ee/api/14_model_ops/code_suggestions_spec.rb | ModelOps Code Suggestions on the GitLab API with PAT auth returns a suggestion
Test metadata (don't modify)
Field | Value |
---|---|
File | qa/qa/specs/features/ee/api/14_model_ops/code_suggestions_spec.rb#L40 |
Description | ModelOps Code Suggestions on the GitLab API with PAT auth returns a suggestion |
Test level | |
Hash | 6d6e335edb9f2d6e74d6cf84b9398a62e689f254867bf0ad6fc4cd1bc8cdcdf0 |
Duration | 1.76 seconds |
Expected duration | < 240 seconds |
Test case | https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases/420973 |
Stack trace
Failure/Error: expect(response.code).to be(200), "Request returned (#{response.code}): `#{response}`"
Request returned (401): `{"message":"401 Unauthorized"}`
Reports (11)
- 2023-11-09: https://ops.gitlab.net/gitlab-org/quality/staging/-/jobs/11835202 (https://ops.gitlab.net/gitlab-org/quality/staging/-/pipelines/2504511)
- 2023-11-09: https://ops.gitlab.net/gitlab-org/quality/canary/-/jobs/11835456 (https://ops.gitlab.net/gitlab-org/quality/canary/-/pipelines/2504569)
- 2023-11-09: https://ops.gitlab.net/gitlab-org/quality/canary/-/jobs/11835694 (https://ops.gitlab.net/gitlab-org/quality/canary/-/pipelines/2504569)
- 2023-11-09: https://ops.gitlab.net/gitlab-org/quality/canary/-/jobs/11835726 (https://ops.gitlab.net/gitlab-org/quality/canary/-/pipelines/2504569)
- 2023-11-09: https://ops.gitlab.net/gitlab-org/quality/canary/-/jobs/11836898 (https://ops.gitlab.net/gitlab-org/quality/canary/-/pipelines/2504569)
- 2023-11-09: https://ops.gitlab.net/gitlab-org/quality/staging/-/jobs/11838077 (https://ops.gitlab.net/gitlab-org/quality/staging/-/pipelines/2505198)
- 2023-11-09: https://ops.gitlab.net/gitlab-org/quality/canary/-/jobs/11838511 (https://ops.gitlab.net/gitlab-org/quality/canary/-/pipelines/2504569)
- 2023-11-09: https://ops.gitlab.net/gitlab-org/quality/staging-canary/-/jobs/11838519 (https://ops.gitlab.net/gitlab-org/quality/staging-canary/-/pipelines/2505197)
- 2023-11-09: https://ops.gitlab.net/gitlab-org/quality/staging/-/jobs/11838594 (https://ops.gitlab.net/gitlab-org/quality/staging/-/pipelines/2505350)
- 2023-11-09: https://ops.gitlab.net/gitlab-org/quality/staging/-/jobs/11839027 (https://ops.gitlab.net/gitlab-org/quality/staging/-/pipelines/2505510)
Timeline
-
2023-11-09 05:43:51 UTC
test passed on staging in full pipeline https://ops.gitlab.net/gitlab-org/quality/staging/-/jobs/11833840#L61 -
2023-11-09 07:41 UTC
gstg deploy started: https://ops.gitlab.net/gitlab-com/gl-infra/deployer/-/pipelines/2504421 - note that this is the deploy that includes the test as a smoke test, although it ran previously in the full suite. -
2023-11-09 08:09 UTC
deploy finished -
2023-11-09 08:16:52 UTC
first failure - The execution failure history issue shows that it passed at least a few times after that first failure.
-
2023-11-09 09:43:51 UTC
example pass: https://ops.gitlab.net/gitlab-org/quality/staging/-/jobs/11836474#L208 -
2023-11-09 11:52:00 UTC
Quarantine MR merged: !136438 (merged) -
2023-11-09 13:43:00 UTC
Nailia noted it passed when she ran it from her machine against Staging. -
2023-11-10 02:00 UTC
test passes on each environment in each pipeline: #431317 (comment 1642037321)
Data
- 2023-11-21: https://ops.gitlab.net/gitlab-org/quality/staging-canary/-/jobs/11982618 (https://ops.gitlab.net/gitlab-org/quality/staging-canary/-/pipelines/2551750)
Investigation summary
- Rails rejected authentication without sending the request to the AI-gateway.
- The API token has the
api
scope and hasn't changed. - No changes were made to the user profile or group settings.
- The test was running successfully until I added it to the smoke test suite. No change was made to the test itself, or the pipelines that run the test.
- There was a Ruby 3.1 test rollout around the time of the failures, but the tests passed many times in subsequent Ruby 3.1 test rollouts.
- The test passed several hundreds of times when run every 30 minutes the same CI conditions that failed before (in staging, staging-canary, canary, and production pipelines).
I think it's reasonable to add the tests back to the smoke suite, given the inconclusive investigation above and the many successful executions since the failures.
Plus, fast quarantine works in those pipelines now so we can more easily unblock them if needed.
Edited by Mark Lapierre