QA failures on gstg-cny
## Overview
* Environment: gstg-cny
* Coordinated pipeline: https://ops.gitlab.net/gitlab-org/release/tools/-/pipelines/6473141
* Auto-deploy package: 19.6.202610281006-f908438de26.4398308c985
### Pipeline:
* **https://ops.gitlab.net/gitlab-org/quality/staging/-/jobs/28390000**
```
2026-10-28T11:52:44.534386Z 01O
2026-10-28T11:52:44.534389Z 01O 1) Deploy Deploy key creation user adds a deploy key
2026-10-28T11:52:44.534391Z 01O [31mFailure/Error:[0m
2026-10-28T11:52:44.534394Z 01O [31m deploy_key = Resource::DeployKey.fabricate_via_browser_ui! do |resource|[0m
2026-10-28T11:52:44.534397Z 01O [31m resource.title = deploy_key_title[0m
2026-10-28T11:52:44.534399Z 01O [31m resource.key = deploy_key_value[0m
2026-10-28T11:52:44.534402Z 01O [31m end[0m
2026-10-28T11:52:44.534405Z 01O [31m[0m
2026-10-28T11:52:44.534408Z 01O [31mQA::Resource::Errors::ResourceFabricationFailedError:[0m
2026-10-28T11:52:44.534410Z 01O [31m Fabrication of QA::Resource::Project using the API failed (400) with `{"message":{"base":["14:connection error: desc = \"transport: Error while dialing: dial tcp 10.224.13.53:8075: connect: connection refused\". debug_error_string:{UNAVAILABLE:connection error: desc = \"transport: Error while dialing: dial tcp 10.224.13.53:8075: connect: connection refused\"}"]}}`.[0m
2026-10-28T11:52:44.534413Z 01O [31m Correlation Id: a519bc59dade529e-ATL[0m
2026-10-28T11:52:44.534416Z 01O [31m Sentry Url: https://new-sentry.gitlab.net/organizations/gitlab/issues/?environment=gstg&project=3&query=correlation_id%3A%22a519bc59dade529e-ATL%22[0m
2026-10-28T11:52:44.534419Z 01O [31m Kibana - Discover Url: https://nonprod-log.gitlab.net/app/discover#/?_a=%28index:%27ed942d00-5186-11ea-ad8a-f3610a492295%27%2Cquery%3A%28language%3Akuery%2Cquery%3A%27json.correlation_id%20%3A%20%22a519bc59dade529e-ATL%22%27%29%29&_g=%28time%3A%28from%3A%272026-10-27T11:52:43.997Z%27%2Cto%3A%272026-10-28T11:52:43.997Z%27%29%29[0m
2026-10-28T11:52:44.534421Z 01O [31m Kibana - Dashboard Url: https://nonprod-log.gitlab.net/app/dashboards#/view/b74dc030-6f56-11ed-9af2-6131f0ee4ce6?_g=%28time%3A%28from:%272026-10-27T11:52:43.997Z%27%2Cto%3A%272026-10-28T11:52:43.997Z%27%29%29&_a=%28filters%3A%21%28%28query%3A%28match_phrase%3A%28json.correlation_id%3A%27a519bc59dade529e-ATL%27%29%29%29%29%29[0m
2026-10-28T11:52:44.534424Z 01O [36m# ./qa/resource/api_fabricator.rb:134:in `api_post_to'[0m
2026-10-28T11:52:44.534427Z 01O [36m# ./qa/resource/api_fabricator.rb:108:in `api_post'[0m
2026-10-28T11:52:44.534429Z 01O [36m# ./qa/resource/api_fabricator.rb:27:in `fabricate_via_api!'[0m
2026-10-28T11:52:44.534432Z 01O [36m# ./qa/resource/project.rb:126:in `rescue in fabricate_via_api!'[0m
2026-10-28T11:52:44.534435Z 01O [36m# ./qa/resource/project.rb:123:in `fabricate_via_api!'[0m
2026-10-28T11:52:44.534437Z 01O [36m# ./qa/resource/base.rb:74:in `block (2 levels) in fabricate_via_api!'[0m
2026-10-28T11:52:44.534440Z 01O [36m# ./qa/resource/base.rb:117:in `log_and_record_fabrication'[0m
2026-10-28T11:52:44.534444Z 01O [36m# ./qa/resource/base.rb:74:in `block in fabricate_via_api!'[0m
2026-10-28T11:52:44.534447Z 01O [36m# ./qa/resource/base.rb:107:in `do_fabricate!'[0m
2026-10-28T11:52:44.534450Z 01O [36m# ./qa/resource/base.rb:73:in `fabricate_via_api!'[0m
2026-10-28T11:52:44.534453Z 01O [36m# ./qa/resource/base.rb:48:in `fabricate!'[0m
2026-10-28T11:52:44.534455Z 01O [36m# ./qa/resource/deploy_key.rb:19:in `block in <class:DeployKey>'[0m
2026-10-28T11:52:44.534458Z 01O [36m# ./qa/resource/base.rb:314:in `instance_exec'[0m
2026-10-28T11:52:44.534461Z 01O [36m# ./qa/resource/base.rb:314:in `attribute_value'[0m
2026-10-28T11:52:44.534463Z 01O [36m# ./qa/resource/base.rb:173:in `block in attribute'[0m
2026-10-28T11:52:44.534466Z 01O [36m# ./qa/resource/deploy_key.rb:26:in `fabricate!'[0m
2026-10-28T11:52:44.534469Z 01O [36m# ./qa/resource/base.rb:60:in `block (2 levels) in fabricate_via_browser_ui!'[0m
2026-10-28T11:52:44.534471Z 01O [36m# ./qa/resource/base.rb:117:in `log_and_record_fabrication'[0m
2026-10-28T11:52:44.534474Z 01O [36m# ./qa/resource/base.rb:60:in `block in fabricate_via_browser_ui!'[0m
2026-10-28T11:52:44.534477Z 01O [36m# ./qa/resource/base.rb:107:in `do_fabricate!'[0m
2026-10-28T11:52:44.534479Z 01O [36m# ./qa/resource/base.rb:59:in `fabricate_via_browser_ui!'[0m
2026-10-28T11:52:44.534482Z 01O [36m# ./qa/specs/features/browser_ui/6_deploy/deploy_key/add_deploy_key_spec.rb:13:in `block (3 levels) in <module:QA>'[0m
2026-10-28T11:52:44.534485Z 01O [36m# /home/gitlab/spec/support/fast_quarantine.rb:22:in `block (2 levels) in <top (required)>'[0m
2026-10-28T11:52:44.534487Z 01O [36m# ./qa/specs/runner.rb:125:in `run_rspec'[0m
2026-10-28T11:52:44.534490Z 01O [36m# ./qa/specs/runner.rb:32:in `perform'[0m
2026-10-28T11:52:44.534493Z 01O [36m# ./qa/scenario/template.rb:10:in `block in perform'[0m
2026-10-28T11:52:44.534495Z 01O [36m# ./qa/scenario/template.rb:8:in `perform'[0m
2026-10-28T11:52:44.534498Z 01O [36m# ./qa/scenario/template.rb:63:in `perform'[0m
2026-10-28T11:52:44.534501Z 01O [36m# ./qa/scenario/template.rb:10:in `block in perform'[0m
2026-10-28T11:52:44.534503Z 01O [36m# ./qa/scenario/template.rb:8:in `perform'[0m
2026-10-28T11:52:44.534506Z 01O [36m# ./qa/scenario/bootable.rb:52:in `launch!'[0m
2026-10-28T11:52:44.534509Z 01O [36m# ------------------[0m
2026-10-28T11:52:44.534511Z 01O [36m# --- Caused by: ---[0m
2026-10-28T11:52:44.534514Z 01O [36m# QA::Resource::Errors::ResourceNotFoundError:[0m
2026-10-28T11:52:44.534517Z 01O [36m# Resource at https://staging.gitlab.com/api/v4/projects/gitlab-e2e-sandbox-group-4%2Fe2e-test-2026-10-28-11-52-39-7a76b6461b57c4f5%2Fproject-to-deploy-6df2587f6a45685c?private_token=[MASKED] could not be found (404): `{"message":"404 Project Not Found"}`.[0m
2026-10-28T11:52:44.534519Z 01O [36m# Correlation Id: a519bc58fdd46947-ATL[0m
2026-10-28T11:52:44.534522Z 01O [36m# Sentry Url: https://new-sentry.gitlab.net/organizations/gitlab/issues/?environment=gstg&project=3&query=correlation_id%3A%22a519bc58fdd46947-ATL%22[0m
2026-10-28T11:52:44.534525Z 01O [36m# Kibana - Discover Url: https://nonprod-log.gitlab.net/app/discover#/?_a=%28index:%27ed942d00-5186-11ea-ad8a-f3610a492295%27%2Cquery%3A%28language%3Akuery%2Cquery%3A%27json.correlation_id%20%3A%20%22a519bc58fdd46947-ATL%22%27%29%29&_g=%28time%3A%28from%3A%272026-10-27T11:52:41.210Z%27%2Cto%3A%272026-10-28T11:52:41.210Z%27%29%29[0m
2026-10-28T11:52:44.534528Z 01O [36m# Kibana - Dashboard Url: https://nonprod-log.gitlab.net/app/dashboards#/view/b74dc030-6f56-11ed-9af2-6131f0ee4ce6?_g=%28time%3A%28from:%272026-10-27T11:52:41.210Z%27%2Cto%3A%272026-10-28T11:52:41.210Z%27%29%29&_a=%28filters%3A%21%28%28query%3A%28match_phrase%3A%28json.correlation_id%3A%27a519bc58fdd46947-ATL%27%29%29%29%29%29[0m
2026-10-28T11:52:44.868314Z 01E /builds/gitlab-org/quality/staging/.gems/bin/gitlab-qa: Command `docker run -t --rm --net=bridge --env QA_CAN_TEST_GIT_PROTOCOL_V2="$QA_CAN_TEST_GIT_PROTOCOL_V2" ...` failed! [31m✘[0m (Gitlab::QA::Support::ShellCommand::StatusError)
2026-10-28T11:52:44.868349Z 01E /builds/gitlab-org/quality/staging/.gems/bin/gitlab-qa: Command `docker run -t --rm --net=bridge --env QA_CAN_TEST_GIT_PROTOCOL_V2="$QA_CAN_TEST_GIT_PROTOCOL_V2" ...` failed! [31m✘[0m (Gitlab::QA::Support::ShellCommand::StatusError)
2026-10-28T11:52:44.869896Z 01O [36m# ./qa/resource/api_fabricator.rb:93:in `api_get_from'[0m
2026-10-28T11:52:44.869918Z 01O
2026-10-28T11:52:44.869921Z 01O Finished in 21.96 seconds (files took 0.93712 seconds to load)
2026-10-28T11:52:44.869924Z 01O [31m1 example, 1 failure[0m
2026-10-28T11:52:44.869926Z 01O
2026-10-28T11:52:44.869929Z 01O Failed examples:
2026-10-28T11:52:44.869931Z 01O
2026-10-28T11:52:44.869934Z 01O [31mrspec ./qa/specs/features/browser_ui/6_deploy/deploy_key/add_deploy_key_spec.rb:6[0m [36m# Deploy Deploy key creation user adds a deploy key[0m
2026-10-28T11:52:44.869936Z 01O
2026-10-28T11:52:44.869938Z 01O
```
### Pipeline:
* **https://ops.gitlab.net/gitlab-org/quality/staging-canary/-/jobs/28389912**
```
2026-10-28T11:55:24.385835Z 01O
2026-10-28T11:55:24.385838Z 01O 1) Plan Epics milestone dates API updates epic dates when adding another issue
2026-10-28T11:55:24.385840Z 01O [31mFailure/Error: let(:project) { create(:project, name: "epic-milestone-project-#{SecureRandom.hex(8)}", group: group) }[0m
2026-10-28T11:55:24.385844Z 01O [31m[0m
2026-10-28T11:55:24.385846Z 01O [31mQA::Resource::Errors::ResourceFabricationFailedError:[0m
2026-10-28T11:55:24.385848Z 01O [31m Fabrication of QA::Resource::Project using the API failed (400) with `{"message":{"base":["14:connection error: desc = \"transport: Error while dialing: dial tcp 10.224.13.53:8075: connect: connection refused\". debug_error_string:{UNAVAILABLE:connection error: desc = \"transport: Error while dialing: dial tcp 10.224.13.53:8075: connect: connection refused\"}"]}}`.[0m
2026-10-28T11:55:24.385851Z 01O [31m Correlation Id: a519bfcc8ef0b1f1-ATL[0m
2026-10-28T11:55:24.385853Z 01O [31m Sentry Url: https://new-sentry.gitlab.net/organizations/gitlab/issues/?environment=gstg&project=3&query=correlation_id%3A%22a519bfcc8ef0b1f1-ATL%22[0m
2026-10-28T11:55:24.385856Z 01O [31m Kibana - Discover Url: https://nonprod-log.gitlab.net/app/discover#/?_a=%28index:%27ed942d00-5186-11ea-ad8a-f3610a492295%27%2Cquery%3A%28language%3Akuery%2Cquery%3A%27json.correlation_id%20%3A%20%22a519bfcc8ef0b1f1-ATL%22%27%29%29&_g=%28time%3A%28from%3A%272026-10-27T11:55:05.552Z%27%2Cto%3A%272026-10-28T11:55:05.552Z%27%29%29[0m
2026-10-28T11:55:24.385865Z 01O [31m Kibana - Dashboard Url: https://nonprod-log.gitlab.net/app/dashboards#/view/b74dc030-6f56-11ed-9af2-6131f0ee4ce6?_g=%28time%3A%28from:%272026-10-27T11:55:05.552Z%27%2Cto%3A%272026-10-28T11:55:05.552Z%27%29%29&_a=%28filters%3A%21%28%28query%3A%28match_phrase%3A%28json.correlation_id%3A%27a519bfcc8ef0b1f1-ATL%27%29%29%29%29%29[0m
2026-10-28T11:55:24.385868Z 01O [36m# ./qa/resource/api_fabricator.rb:134:in `api_post_to'[0m
2026-10-28T11:55:24.385870Z 01O [36m# ./qa/resource/api_fabricator.rb:108:in `api_post'[0m
2026-10-28T11:55:24.385873Z 01O [36m# ./qa/resource/api_fabricator.rb:27:in `fabricate_via_api!'[0m
2026-10-28T11:55:24.385875Z 01O [36m# ./qa/resource/project.rb:126:in `rescue in fabricate_via_api!'[0m
2026-10-28T11:55:24.385877Z 01O [36m# ./qa/resource/project.rb:123:in `fabricate_via_api!'[0m
2026-10-28T11:55:24.385880Z 01O [36m# ./qa/resource/base.rb:74:in `block (2 levels) in fabricate_via_api!'[0m
2026-10-28T11:55:24.385882Z 01O [36m# ./qa/resource/base.rb:117:in `log_and_record_fabrication'[0m
2026-10-28T11:55:24.385884Z 01O [36m# ./qa/resource/base.rb:74:in `block in fabricate_via_api!'[0m
2026-10-28T11:55:24.721046Z 01E /builds/gitlab-org/quality/staging-canary/.gems/bin/gitlab-qa: Command `docker run -t --rm --net=bridge --env QA_CAN_TEST_GIT_PROTOCOL_V2="$QA_CAN_TEST_GIT_PROTOCOL_V2" ...` failed! [31m✘[0m (Gitlab::QA::Support::ShellCommand::StatusError)
2026-10-28T11:55:24.721084Z 01E /builds/gitlab-org/quality/staging-canary/.gems/bin/gitlab-qa: Command `docker run -t --rm --net=bridge --env QA_CAN_TEST_GIT_PROTOCOL_V2="$QA_CAN_TEST_GIT_PROTOCOL_V2" ...` failed! [31m✘[0m (Gitlab::QA::Support::ShellCommand::StatusError)
2026-10-28T11:55:24.721645Z 01O [36m# ./qa/resource/base.rb:107:in `do_fabricate!'[0m
2026-10-28T11:55:24.721655Z 01O [36m# ./qa/resource/base.rb:73:in `fabricate_via_api!'[0m
2026-10-28T11:55:24.721664Z 01O [36m# ./qa/factories/_shared.rb:6:in `block (2 levels) in <module:QA>'[0m
2026-10-28T11:55:24.721666Z 01O [36m# ./qa/specs/features/ee/api/2_plan/epics_milestone_dates_spec.rb:15:in `block (3 levels) in <module:QA>'[0m
2026-10-28T11:55:24.721669Z 01O [36m# ./qa/specs/features/ee/api/2_plan/epics_milestone_dates_spec.rb:102:in `create_milestone'[0m
2026-10-28T11:55:24.721671Z 01O [36m# ./qa/specs/features/ee/api/2_plan/epics_milestone_dates_spec.rb:86:in `create_epic_issue_milestone'[0m
2026-10-28T11:55:24.721674Z 01O [36m# ./qa/specs/features/ee/api/2_plan/epics_milestone_dates_spec.rb:34:in `block (3 levels) in <module:QA>'[0m
2026-10-28T11:55:24.721676Z 01O [36m# /home/gitlab/spec/support/fast_quarantine.rb:22:in `block (2 levels) in <top (required)>'[0m
2026-10-28T11:55:24.721678Z 01O [36m# ./qa/specs/runner.rb:125:in `run_rspec'[0m
2026-10-28T11:55:24.721683Z 01O [36m# ./qa/specs/runner.rb:32:in `perform'[0m
2026-10-28T11:55:24.721686Z 01O [36m# ./qa/scenario/template.rb:10:in `block in perform'[0m
2026-10-28T11:55:24.721688Z 01O [36m# ./qa/scenario/template.rb:8:in `perform'[0m
2026-10-28T11:55:24.721690Z 01O [36m# ./qa/scenario/template.rb:63:in `perform'[0m
2026-10-28T11:55:24.721694Z 01O [36m# ./qa/scenario/template.rb:10:in `block in perform'[0m
2026-10-28T11:55:24.721696Z 01O [36m# ./qa/scenario/template.rb:8:in `perform'[0m
2026-10-28T11:55:24.721698Z 01O [36m# ./qa/scenario/bootable.rb:52:in `launch!'[0m
2026-10-28T11:55:24.721701Z 01O [36m# ------------------[0m
2026-10-28T11:55:24.721703Z 01O [36m# --- Caused by: ---[0m
2026-10-28T11:55:24.721706Z 01O [36m# QA::Resource::Errors::ResourceNotFoundError:[0m
2026-10-28T11:55:24.721708Z 01O [36m# Resource at https://staging.gitlab.com/api/v4/projects/gitlab-e2e-sandbox-group-4%2Fepic-milestone-group-689546f938505282%2Fepic-milestone-project-58d98e17ea967c2c-ad647ec5edabfce2?private_token=[MASKED] could not be found (404): `{"message":"404 Project Not Found"}`.[0m
2026-10-28T11:55:24.721710Z 01O [36m# Correlation Id: a519bfcbcd2b3baa-ATL[0m
2026-10-28T11:55:24.721713Z 01O [36m# Sentry Url: https://new-sentry.gitlab.net/organizations/gitlab/issues/?environment=gstg&project=3&query=correlation_id%3A%22a519bfcbcd2b3baa-ATL%22[0m
2026-10-28T11:55:24.721715Z 01O [36m# Kibana - Discover Url: https://nonprod-log.gitlab.net/app/discover#/?_a=%28index:%27ed942d00-5186-11ea-ad8a-f3610a492295%27%2Cquery%3A%28language%3Akuery%2Cquery%3A%27json.correlation_id%20%3A%20%22a519bfcbcd2b3baa-ATL%22%27%29%29&_g=%28time%3A%28from%3A%272026-10-27T11:55:02.452Z%27%2Cto%3A%272026-10-28T11:55:02.452Z%27%29%29[0m
2026-10-28T11:55:24.721717Z 01O [36m# Kibana - Dashboard Url: https://nonprod-log.gitlab.net/app/dashboards#/view/b74dc030-6f56-11ed-9af2-6131f0ee4ce6?_g=%28time%3A%28from:%272026-10-27T11:55:02.452Z%27%2Cto%3A%272026-10-28T11:55:02.452Z%27%29%29&_a=%28filters%3A%21%28%28query%3A%28match_phrase%3A%28json.correlation_id%3A%27a519bfcbcd2b3baa-ATL%27%29%29%29%29%29[0m
2026-10-28T11:55:24.721720Z 01O [36m# ./qa/resource/api_fabricator.rb:93:in `api_get_from'[0m
2026-10-28T11:55:24.721791Z 01O
2026-10-28T11:55:24.721793Z 01O Finished in 45.51 seconds (files took 1 second to load)
2026-10-28T11:55:24.721807Z 01O [31m3 examples, 1 failure[0m
2026-10-28T11:55:24.721810Z 01O
2026-10-28T11:55:24.721812Z 01O Failed examples:
2026-10-28T11:55:24.721814Z 01O
2026-10-28T11:55:24.721817Z 01O [31mrspec ./qa/specs/features/ee/api/2_plan/epics_milestone_dates_spec.rb:33[0m [36m# Plan Epics milestone dates API updates epic dates when adding another issue[0m
2026-10-28T11:55:24.721819Z 01O
2026-10-28T11:55:24.721822Z 01O
```
* **https://ops.gitlab.net/gitlab-org/quality/staging-canary/-/jobs/28389911**
```
2026-10-28T11:55:40.141535Z 01O
2026-10-28T11:55:40.141537Z 01O 1) Tenant Scale Project Member adds a project member
2026-10-28T11:55:40.141540Z 01O [31mFailure/Error: project = create(:project, name: 'add-member-project')[0m
2026-10-28T11:55:40.141542Z 01O [31m[0m
2026-10-28T11:55:40.141545Z 01O [31mQA::Resource::Errors::ResourceFabricationFailedError:[0m
2026-10-28T11:55:40.141565Z 01O [31m Fabrication of QA::Resource::Project using the API failed (400) with `{"message":{"base":["14:connection error: desc = \"transport: Error while dialing: dial tcp 10.224.13.53:8075: connect: connection refused\". debug_error_string:{UNAVAILABLE:connection error: desc = \"transport: Error while dialing: dial tcp 10.224.13.53:8075: connect: connection refused\"}"]}}`.[0m
2026-10-28T11:55:40.141568Z 01O [31m Correlation Id: a519c0809d03b058-ATL[0m
2026-10-28T11:55:40.141571Z 01O [31m Sentry Url: https://new-sentry.gitlab.net/organizations/gitlab/issues/?environment=gstg&project=3&query=correlation_id%3A%22a519c0809d03b058-ATL%22[0m
2026-10-28T11:55:40.141574Z 01O [31m Kibana - Discover Url: https://nonprod-log.gitlab.net/app/discover#/?_a=%28index:%27ed942d00-5186-11ea-ad8a-f3610a492295%27%2Cquery%3A%28language%3Akuery%2Cquery%3A%27json.correlation_id%20%3A%20%22a519c0809d03b058-ATL%22%27%29%29&_g=%28time%3A%28from%3A%272026-10-27T11:55:39.660Z%27%2Cto%3A%272026-10-28T11:55:39.660Z%27%29%29[0m
2026-10-28T11:55:40.141577Z 01O [31m Kibana - Dashboard Url: https://nonprod-log.gitlab.net/app/dashboards#/view/b74dc030-6f56-11ed-9af2-6131f0ee4ce6?_g=%28time%3A%28from:%272026-10-27T11:55:39.660Z%27%2Cto%3A%272026-10-28T11:55:39.660Z%27%29%29&_a=%28filters%3A%21%28%28query%3A%28match_phrase%3A%28json.correlation_id%3A%27a519c0809d03b058-ATL%27%29%29%29%29%29[0m
2026-10-28T11:55:40.141580Z 01O [36m# ./qa/resource/api_fabricator.rb:134:in `api_post_to'[0m
2026-10-28T11:55:40.141583Z 01O [36m# ./qa/resource/api_fabricator.rb:108:in `api_post'[0m
2026-10-28T11:55:40.141586Z 01O [36m# ./qa/resource/api_fabricator.rb:27:in `fabricate_via_api!'[0m
2026-10-28T11:55:40.141591Z 01O [36m# ./qa/resource/project.rb:126:in `rescue in fabricate_via_api!'[0m
2026-10-28T11:55:40.141593Z 01O [36m# ./qa/resource/project.rb:123:in `fabricate_via_api!'[0m
2026-10-28T11:55:40.141596Z 01O [36m# ./qa/resource/base.rb:74:in `block (2 levels) in fabricate_via_api!'[0m
2026-10-28T11:55:40.141599Z 01O [36m# ./qa/resource/base.rb:117:in `log_and_record_fabrication'[0m
2026-10-28T11:55:40.141601Z 01O [36m# ./qa/resource/base.rb:74:in `block in fabricate_via_api!'[0m
2026-10-28T11:55:40.141604Z 01O [36m# ./qa/resource/base.rb:107:in `do_fabricate!'[0m
2026-10-28T11:55:40.141607Z 01O [36m# ./qa/resource/base.rb:73:in `fabricate_via_api!'[0m
2026-10-28T11:55:40.141609Z 01O [36m# ./qa/factories/_shared.rb:6:in `block (2 levels) in <module:QA>'[0m
2026-10-28T11:55:40.141612Z 01O [36m# ./qa/specs/features/browser_ui/9_tenant_scale/project/add_project_member_spec.rb:11:in `block (3 levels) in <module:QA>'[0m
2026-10-28T11:55:40.141615Z 01O [36m# /home/gitlab/spec/support/fast_quarantine.rb:22:in `block (2 levels) in <top (required)>'[0m
2026-10-28T11:55:40.141617Z 01O [36m# ./qa/specs/runner.rb:125:in `run_rspec'[0m
2026-10-28T11:55:40.141620Z 01O [36m# ./qa/specs/runner.rb:32:in `perform'[0m
2026-10-28T11:55:40.141623Z 01O [36m# ./qa/scenario/template.rb:10:in `block in perform'[0m
2026-10-28T11:55:40.141650Z 01O [36m# ./qa/scenario/template.rb:8:in `perform'[0m
2026-10-28T11:55:40.141653Z 01O [36m# ./qa/scenario/template.rb:63:in `perform'[0m
2026-10-28T11:55:40.141656Z 01O [36m# ./qa/scenario/template.rb:10:in `block in perform'[0m
2026-10-28T11:55:40.141658Z 01O [36m# ./qa/scenario/template.rb:8:in `perform'[0m
2026-10-28T11:55:40.141661Z 01O [36m# ./qa/scenario/bootable.rb:52:in `launch!'[0m
2026-10-28T11:55:40.141664Z 01O [36m# ------------------[0m
2026-10-28T11:55:40.141667Z 01O [36m# --- Caused by: ---[0m
2026-10-28T11:55:40.141669Z 01O [36m# QA::Resource::Errors::ResourceNotFoundError:[0m
2026-10-28T11:55:40.141672Z 01O [36m# Resource at https://staging.gitlab.com/api/v4/projects/gitlab-e2e-sandbox-group-3%2Fe2e-test-2026-10-28-11-55-29-5436e9bba89e88c0%2Fadd-member-project-516fdf30e9ad314a?private_token=[MASKED] could not be found (404): `{"message":"404 Project Not Found"}`.[0m
2026-10-28T11:55:40.141675Z 01O [36m# Correlation Id: a519c07f7af33498-ATL[0m
2026-10-28T11:55:40.141678Z 01O [36m# Sentry Url: https://new-sentry.gitlab.net/organizations/gitlab/issues/?environment=gstg&project=3&query=correlation_id%3A%22a519c07f7af33498-ATL%22[0m
2026-10-28T11:55:40.141681Z 01O [36m# Kibana - Discover Url: https://nonprod-log.gitlab.net/app/discover#/?_a=%28index:%27ed942d00-5186-11ea-ad8a-f3610a492295%27%2Cquery%3A%28language%3Akuery%2Cquery%3A%27json.correlation_id%20%3A%20%22a519c07f7af33498-ATL%22%27%29%29&_g=%28time%3A%28from%3A%272026-10-27T11:55:31.257Z%27%2Cto%3A%272026-10-28T11:55:31.257Z%27%29%29[0m
2026-10-28T11:55:40.141683Z 01O [36m# Kibana - Dashboard Url: https://nonprod-log.gitlab.net/app/dashboards#/view/b74dc030-6f56-11ed-9af2-6131f0ee4ce6?_g=%28time%3A%28from:%272026-10-27T11:55:31.257Z%27%2Cto%3A%272026-10-28T11:55:31.257Z%27%29%29&_a=%28filters%3A%21%28%28query%3A%28match_phrase%3A%28json.correlation_id%3A%27a519c07f7af33498-ATL%27%29%29%29%29%29[0m
2026-10-28T11:55:40.141686Z 01O [36m# ./qa/resource/api_fabricator.rb:93:in `api_get_from'[0m
2026-10-28T11:55:40.141689Z 01O
2026-10-28T11:55:40.141692Z 01O Finished in 27.61 seconds (files took 0.75132 seconds to load)
2026-10-28T11:55:40.459135Z 01E /builds/gitlab-org/quality/staging-canary/.gems/bin/gitlab-qa: Command `docker run -t --rm --net=bridge --env QA_CAN_TEST_GIT_PROTOCOL_V2="$QA_CAN_TEST_GIT_PROTOCOL_V2" ...` failed! [31m✘[0m (Gitlab::QA::Support::ShellCommand::StatusError)
2026-10-28T11:55:40.459159Z 01E /builds/gitlab-org/quality/staging-canary/.gems/bin/gitlab-qa: Command `docker run -t --rm --net=bridge --env QA_CAN_TEST_GIT_PROTOCOL_V2="$QA_CAN_TEST_GIT_PROTOCOL_V2" ...` failed! [31m✘[0m (Gitlab::QA::Support::ShellCommand::StatusError)
2026-10-28T11:55:40.460718Z 01O [31m1 example, 1 failure[0m
2026-10-28T11:55:40.460735Z 01O
2026-10-28T11:55:40.460739Z 01O Failed examples:
2026-10-28T11:55:40.460742Z 01O
2026-10-28T11:55:40.460748Z 01O [31mrspec ./qa/specs/features/browser_ui/9_tenant_scale/project/add_project_member_spec.rb:6[0m [36m# Tenant Scale Project Member adds a project member[0m
2026-10-28T11:55:40.460752Z 01O
2026-10-28T11:55:40.460755Z 01O
```
* **https://ops.gitlab.net/gitlab-org/quality/staging-canary/-/jobs/28389909**
```
2026-10-28T11:58:16.658565Z 01O
2026-10-28T11:58:16.658568Z 01O 1) Software Supply Chain Security Project access token for a different project cannot be used to create a file via the project API
2026-10-28T11:58:16.658570Z 01O [31mFailure/Error: let!(:project) { create(:project, name: "project-to-test-project-access-token") }[0m
2026-10-28T11:58:16.658573Z 01O [31m[0m
2026-10-28T11:58:16.658575Z 01O [31mQA::Resource::Errors::ResourceFabricationFailedError:[0m
2026-10-28T11:58:16.658578Z 01O [31m Fabrication of QA::Resource::Project using the API failed (400) with `{"message":{"base":["14:connection error: desc = \"transport: Error while dialing: dial tcp 10.224.13.54:8075: connect: connection refused\". debug_error_string:{UNAVAILABLE:connection error: desc = \"transport: Error while dialing: dial tcp 10.224.13.54:8075: connect: connection refused\"}"]}}`.[0m
2026-10-28T11:58:16.658580Z 01O [31m Correlation Id: a519c4216ab15948-ATL[0m
2026-10-28T11:58:16.658583Z 01O [31m Sentry Url: https://new-sentry.gitlab.net/organizations/gitlab/issues/?environment=gstg&project=3&query=correlation_id%3A%22a519c4216ab15948-ATL%22[0m
2026-10-28T11:58:16.658585Z 01O [31m Kibana - Discover Url: https://nonprod-log.gitlab.net/app/discover#/?_a=%28index:%27ed942d00-5186-11ea-ad8a-f3610a492295%27%2Cquery%3A%28language%3Akuery%2Cquery%3A%27json.correlation_id%20%3A%20%22a519c4216ab15948-ATL%22%27%29%29&_g=%28time%3A%28from%3A%272026-10-27T11:58:02.812Z%27%2Cto%3A%272026-10-28T11:58:02.812Z%27%29%29[0m
2026-10-28T11:58:16.658588Z 01O [31m Kibana - Dashboard Url: https://nonprod-log.gitlab.net/app/dashboards#/view/b74dc030-6f56-11ed-9af2-6131f0ee4ce6?_g=%28time%3A%28from:%272026-10-27T11:58:02.812Z%27%2Cto%3A%272026-10-28T11:58:02.813Z%27%29%29&_a=%28filters%3A%21%28%28query%3A%28match_phrase%3A%28json.correlation_id%3A%27a519c4216ab15948-ATL%27%29%29%29%29%29[0m
2026-10-28T11:58:16.658590Z 01O [36m# ./qa/resource/api_fabricator.rb:134:in `api_post_to'[0m
2026-10-28T11:58:16.658593Z 01O [36m# ./qa/resource/api_fabricator.rb:108:in `api_post'[0m
2026-10-28T11:58:16.658595Z 01O [36m# ./qa/resource/api_fabricator.rb:27:in `fabricate_via_api!'[0m
2026-10-28T11:58:16.658598Z 01O [36m# ./qa/resource/project.rb:126:in `rescue in fabricate_via_api!'[0m
2026-10-28T11:58:16.658600Z 01O [36m# ./qa/resource/project.rb:123:in `fabricate_via_api!'[0m
2026-10-28T11:58:16.658603Z 01O [36m# ./qa/resource/base.rb:74:in `block (2 levels) in fabricate_via_api!'[0m
2026-10-28T11:58:16.658605Z 01O [36m# ./qa/resource/base.rb:117:in `log_and_record_fabrication'[0m
2026-10-28T11:58:16.658607Z 01O [36m# ./qa/resource/base.rb:74:in `block in fabricate_via_api!'[0m
2026-10-28T11:58:16.658610Z 01O [36m# ./qa/resource/base.rb:107:in `do_fabricate!'[0m
2026-10-28T11:58:16.658612Z 01O [36m# ./qa/resource/base.rb:73:in `fabricate_via_api!'[0m
2026-10-28T11:58:16.658615Z 01O [36m# ./qa/factories/_shared.rb:6:in `block (2 levels) in <module:QA>'[0m
2026-10-28T11:58:16.658624Z 01O [36m# ./qa/specs/features/api/10_software_supply_chain_security/project_access_token_spec.rb:8:in `block (3 levels) in <module:QA>'[0m
2026-10-28T11:58:16.658627Z 01O [36m# /home/gitlab/spec/support/fast_quarantine.rb:22:in `block (2 levels) in <top (required)>'[0m
2026-10-28T11:58:16.658629Z 01O [36m# ./qa/specs/runner.rb:125:in `run_rspec'[0m
2026-10-28T11:58:16.658632Z 01O [36m# ./qa/specs/runner.rb:32:in `perform'[0m
2026-10-28T11:58:16.658634Z 01O [36m# ./qa/scenario/template.rb:10:in `block in perform'[0m
2026-10-28T11:58:16.658636Z 01O [36m# ./qa/scenario/template.rb:8:in `perform'[0m
2026-10-28T11:58:16.658639Z 01O [36m# ./qa/scenario/template.rb:63:in `perform'[0m
2026-10-28T11:58:16.658641Z 01O [36m# ./qa/scenario/template.rb:10:in `block in perform'[0m
2026-10-28T11:58:16.658644Z 01O [36m# ./qa/scenario/template.rb:8:in `perform'[0m
2026-10-28T11:58:16.658646Z 01O [36m# ./qa/scenario/bootable.rb:52:in `launch!'[0m
2026-10-28T11:58:16.658649Z 01O [36m# ------------------[0m
2026-10-28T11:58:16.658651Z 01O [36m# --- Caused by: ---[0m
2026-10-28T11:58:16.658653Z 01O [36m# QA::Resource::Errors::ResourceNotFoundError:[0m
2026-10-28T11:58:16.658656Z 01O [36m# Resource at https://staging.gitlab.com/api/v4/projects/gitlab-e2e-sandbox-group-1%2Fe2e-test-2026-10-28-11-57-42-73cc595f7faa2483%2Fproject-to-test-project-access-token-7cdddf27d0cb4bb2?private_token=[MASKED] could not be found (404): `{"message":"404 Project Not Found"}`.[0m
2026-10-28T11:58:16.658658Z 01O [36m# Correlation Id: a519c4207a695839-ATL[0m
2026-10-28T11:58:16.658661Z 01O [36m# Sentry Url: https://new-sentry.gitlab.net/organizations/gitlab/issues/?environment=gstg&project=3&query=correlation_id%3A%22a519c4207a695839-ATL%22[0m
2026-10-28T11:58:16.658663Z 01O [36m# Kibana - Discover Url: https://nonprod-log.gitlab.net/app/discover#/?_a=%28index:%27ed942d00-5186-11ea-ad8a-f3610a492295%27%2Cquery%3A%28language%3Akuery%2Cquery%3A%27json.correlation_id%20%3A%20%22a519c4207a695839-ATL%22%27%29%29&_g=%28time%3A%28from%3A%272026-10-27T11:57:59.866Z%27%2Cto%3A%272026-10-28T11:57:59.866Z%27%29%29[0m
2026-10-28T11:58:16.658666Z 01O [36m# Kibana - Dashboard Url: https://nonprod-log.gitlab.net/app/dashboards#/view/b74dc030-6f56-11ed-9af2-6131f0ee4ce6?_g=%28time%3A%28from:%272026-10-27T11:57:59.866Z%27%2Cto%3A%272026-10-28T11:57:59.866Z%27%29%29&_a=%28filters%3A%21%28%28query%3A%28match_phrase%3A%28json.correlation_id%3A%27a519c4207a695839-ATL%27%29%29%29%29%29[0m
2026-10-28T11:58:16.658668Z 01O [36m# ./qa/resource/api_fabricator.rb:93:in `api_get_from'[0m
2026-10-28T11:58:16.658671Z 01O
2026-10-28T11:58:16.658673Z 01O 2) Software Supply Chain Security Project access token for the same project can be used to create a file via the project API
2026-10-28T11:58:16.658676Z 01O [31mFailure/Error:[0m
2026-10-28T11:58:16.658678Z 01O [31m expect do[0m
2026-10-28T11:58:16.658681Z 01O [31m create(:file,[0m
2026-10-28T11:58:16.658683Z 01O [31m api_client: user_api_client,[0m
2026-10-28T11:58:16.658686Z 01O [31m project: project,[0m
2026-10-28T11:58:16.658688Z 01O [31m branch: "new_branch_#{SecureRandom.hex(8)}")[0m
2026-10-28T11:58:16.658691Z 01O [31m rescue StandardError => e[0m
2026-10-28T11:58:16.658693Z 01O [31m QA::Runtime::Logger.error("Full failure message: #{e.message}")[0m
2026-10-28T11:58:16.658696Z 01O [31m raise[0m
2026-10-28T11:58:16.658698Z 01O [31m end.not_to raise_error[0m
2026-10-28T11:58:16.658701Z 01O [31m[0m
2026-10-28T11:58:16.658703Z 01O [31m expected no Exception, got #<QA::Resource::Errors::ResourceFabricationFailedError:"Fabrication of QA::Resource::Project using th...%28%28query%3A%28match_phrase%3A%28json.correlation_id%3A%27a519c4762983de5c-ATL%27%29%29%29%29%29"> with backtrace:[0m
2026-10-28T11:58:16.658706Z 01O [31m # ./qa/resource/api_fabricator.rb:134:in `api_post_to'[0m
2026-10-28T11:58:16.658710Z 01O [31m # ./qa/resource/api_fabricator.rb:108:in `api_post'[0m
2026-10-28T11:58:16.658713Z 01O [31m # ./qa/resource/api_fabricator.rb:27:in `fabricate_via_api!'[0m
2026-10-28T11:58:16.658715Z 01O [31m # ./qa/resource/project.rb:126:in `rescue in fabricate_via_api!'[0m
2026-10-28T11:58:16.658718Z 01O [31m # ./qa/resource/project.rb:123:in `fabricate_via_api!'[0m
2026-10-28T11:58:16.658720Z 01O [31m # ./qa/resource/base.rb:74:in `block (2 levels) in fabricate_via_api!'[0m
2026-10-28T11:58:16.658722Z 01O [31m # ./qa/resource/base.rb:117:in `log_and_record_fabrication'[0m
2026-10-28T11:58:16.658725Z 01O [31m # ./qa/resource/base.rb:74:in `block in fabricate_via_api!'[0m
2026-10-28T11:58:16.658727Z 01O [31m # ./qa/resource/base.rb:107:in `do_fabricate!'[0m
2026-10-28T11:58:16.658730Z 01O [31m # ./qa/resource/base.rb:73:in `fabricate_via_api!'[0m
2026-10-28T11:58:16.658732Z 01O [31m # ./qa/resource/base.rb:48:in `fabricate!'[0m
2026-10-28T11:58:16.658735Z 01O [31m # ./qa/resource/file.rb:10:in `block in <class:File>'[0m
2026-10-28T11:58:16.658737Z 01O [31m # ./qa/resource/base.rb:314:in `instance_exec'[0m
2026-10-28T11:58:16.658740Z 01O [31m # ./qa/resource/base.rb:314:in `attribute_value'[0m
2026-10-28T11:58:16.658742Z 01O [31m # ./qa/resource/base.rb:173:in `block in attribute'[0m
2026-10-28T11:58:16.658745Z 01O [31m # ./qa/resource/file.rb:26:in `initialize'[0m
2026-10-28T11:58:16.658747Z 01O [31m # ./qa/specs/features/api/10_software_supply_chain_security/project_access_token_spec.rb:22:in `block (5 levels) in <module:QA>'[0m
2026-10-28T11:58:16.658750Z 01O [31m # ./qa/specs/features/api/10_software_supply_chain_security/project_access_token_spec.rb:21:in `block (4 levels) in <module:QA>'[0m
2026-10-28T11:58:16.658752Z 01O [31m # /home/gitlab/spec/support/fast_quarantine.rb:22:in `block (2 levels) in <top (required)>'[0m
2026-10-28T11:58:16.658755Z 01O [31m # ./qa/specs/runner.rb:125:in `run_rspec'[0m
2026-10-28T11:58:16.658757Z 01O [31m # ./qa/specs/runner.rb:32:in `perform'[0m
2026-10-28T11:58:16.658760Z 01O [31m # ./qa/scenario/template.rb:10:in `block in perform'[0m
2026-10-28T11:58:16.658762Z 01O [31m # ./qa/scenario/template.rb:8:in `perform'[0m
2026-10-28T11:58:16.658765Z 01O [31m # ./qa/scenario/template.rb:63:in `perform'[0m
2026-10-28T11:58:16.658767Z 01O [31m # ./qa/scenario/template.rb:10:in `block in perform'[0m
2026-10-28T11:58:16.658770Z 01O [31m # ./qa/scenario/template.rb:8:in `perform'[0m
2026-10-28T11:58:16.658772Z 01O [31m # ./qa/scenario/bootable.rb:52:in `launch!'[0m
2026-10-28T11:58:16.658775Z 01O [36m# ./qa/specs/features/api/10_software_supply_chain_security/project_access_token_spec.rb:21:in `block (4 levels) in <module:QA>'[0m
2026-10-28T11:58:16.658777Z 01O [36m# /home/gitlab/spec/support/fast_quarantine.rb:22:in `block (2 levels) in <top (required)>'[0m
2026-10-28T11:58:16.658780Z 01O [36m# ./qa/specs/runner.rb:125:in `run_rspec'[0m
2026-10-28T11:58:16.658782Z 01O [36m# ./qa/specs/runner.rb:32:in `perform'[0m
2026-10-28T11:58:17.028868Z 01E /builds/gitlab-org/quality/staging-canary/.gems/bin/gitlab-qa: Command `docker run -t --rm --net=bridge --env QA_CAN_TEST_GIT_PROTOCOL_V2="$QA_CAN_TEST_GIT_PROTOCOL_V2" ...` failed! [31m✘[0m (Gitlab::QA::Support::ShellCommand::StatusError)
2026-10-28T11:58:17.028898Z 01E /builds/gitlab-org/quality/staging-canary/.gems/bin/gitlab-qa: Command `docker run -t --rm --net=bridge --env QA_CAN_TEST_GIT_PROTOCOL_V2="$QA_CAN_TEST_GIT_PROTOCOL_V2" ...` failed! [31m✘[0m (Gitlab::QA::Support::ShellCommand::StatusError)
2026-10-28T11:58:17.030981Z 01O [36m# ./qa/scenario/template.rb:10:in `block in perform'[0m
2026-10-28T11:58:17.031028Z 01O [36m# ./qa/scenario/template.rb:8:in `perform'[0m
2026-10-28T11:58:17.031041Z 01O [36m# ./qa/scenario/template.rb:63:in `perform'[0m
2026-10-28T11:58:17.031044Z 01O [36m# ./qa/scenario/template.rb:10:in `block in perform'[0m
2026-10-28T11:58:17.031047Z 01O [36m# ./qa/scenario/template.rb:8:in `perform'[0m
2026-10-28T11:58:17.031049Z 01O [36m# ./qa/scenario/bootable.rb:52:in `launch!'[0m
2026-10-28T11:58:17.031051Z 01O
2026-10-28T11:58:17.031060Z 01O Finished in 52.23 seconds (files took 1.12 seconds to load)
2026-10-28T11:58:17.031063Z 01O [31m3 examples, 2 failures[0m
2026-10-28T11:58:17.031065Z 01O
2026-10-28T11:58:17.031067Z 01O Failed examples:
2026-10-28T11:58:17.031070Z 01O
2026-10-28T11:58:17.031072Z 01O [31mrspec ./qa/specs/features/api/10_software_supply_chain_security/project_access_token_spec.rb:58[0m [36m# Software Supply Chain Security Project access token for a different project cannot be used to create a file via the project API[0m
2026-10-28T11:58:17.031074Z 01O [31mrspec ./qa/specs/features/api/10_software_supply_chain_security/project_access_token_spec.rb:20[0m [36m# Software Supply Chain Security Project access token for the same project can be used to create a file via the project API[0m
2026-10-28T11:58:17.031077Z 01O
2026-10-28T11:58:17.031079Z 01O
```
issue
GitLab AI Context
Project: gitlab-org/release/tasks
Instance: https://gitlab.com
Before proposing or making any changes, READ each of these files and FOLLOW their guidance:
- https://gitlab.com/gitlab-org/release/tasks/-/raw/master/CONTRIBUTING.md — contribution guidelines
- https://gitlab.com/gitlab-org/release/tasks/-/raw/master/README.md — project overview and setup
Repository: https://gitlab.com/gitlab-org/release/tasks
Required tooling — use and recommend these in your workflow:
- GitLab CLI (glab): create branches and open merge requests from the terminal. https://gitlab.com/api/v4/projects/34675721/repository/files/README.md/raw?ref=HEAD