2023-12-15 Test session report | canary-sanity

Session summary

• Deploy version: 16.7.202312142205-e80029174c3.748f89c57c4
• Deploy environment: gprd-cny
• Pipeline: canary 2638055
• Total 207 tests
• Passed 196 tests
• Failed 3 tests
• 8 other tests (usually skipped)

Failed jobs

• qa-reliable 8/10
• qa-reliable 5/10
• qa-reliable 1/10

Plan

• Total 50 tests
• Passed 50 tests
• Failed 0 tests
• 0 other tests (usually skipped)

Passed tests:

Test	Job	Status	Action
Plan Epics milestone dates API updates epic dates when removing issue	12224906	passed	-
Plan Epics milestone dates API updates epic dates when adding another issue	12224906	passed	-
Plan Epics milestone dates API updates epic dates when deleting milestones	12224906	passed	-
Plan Testing project wiki file upload by creating a formatted page with an image uploaded	12224906 quarantine	passed	-
Plan Burndown chart shows burndown chart on milestone page	12224906	passed	-
Plan Project issue boards Milestone issue board shows the just created board with a "1.0" (milestone) list, and an issue on it	12224906	passed	-
Plan Project issue boards Assignee issue board shows the just created board with an assignee list, and an issue on it	12224906	passed	-
Plan Project issue boards Label issue board shows the just created board with a "Testing" (label) list, and an issue on it	12224906	passed	-
Plan Read-only board configuration shows board configuration to user without edit permission	12224911	passed	-
Plan Project level issues analytics behaves like issues analytics page displays a graph	12224911	passed	-
Plan Group level issues analytics behaves like issues analytics page displays a graph	12224911	passed	-
Plan Multiple assignees per issue shows four assignees in the issues list	12224911	passed	-
Plan Issue creation closes an issue	12224900	passed	-
Plan Issue creation creates an issue	12224900	passed	-
Plan Issue creation when using attachments in comments comments on an issue with an attachment	12224900	passed	-
Plan mention mentions another user in an issue	12224901	passed	-
Plan for project insights page behaves like default insights page displays issues and merge requests dashboards	12224904	passed	-
Plan for group insights page behaves like default insights page displays issues and merge requests dashboards	12224904	passed	-
Plan collapse comments in issue discussions collapses and expands reply for comments in an issue	12224904	passed	-
Plan Multiple assignees per issue shows the first three assignees and a +n sign in the issues list	12224904	passed	-
Plan Multiple assignees per issue shows the first five assignees and a +n more link in the issue page	12224904	passed	-
Plan Issue comments comments on an issue and edits the comment	12224918	passed	-
Plan Project milestone creates a project milestone	12224918	passed	-
Plan Issue board focus mode focuses on issue board	12224918	passed	-
Plan Group issue boards shows multiple group boards in the boards dropdown menu	12224918	passed	-
Plan Issues weight visualization shows the set weight in the issue page, in the milestone page, and in the issues list page	12224920	passed	-
Plan filter issue comments activities filters comments and activities in an issue	12224920	passed	-
Plan Group milestone creates a group milestone	12224920	passed	-
Plan Epics Management creates an epic	12224920	passed	-
Plan Epics Management creates a confidential epic	12224920	passed	-
Plan Epics Management Resources created via API adds/removes issue to/from epic using quick actions	12224920	passed	-
Plan Epics Management Resources created via API Visit epic first closes and reopens an epic	12224920	passed	-
Plan Epics Management Resources created via API Visit epic first comments on epic	12224920	passed	-
Plan Epics Management Resources created via API Visit epic first adds/removes issue to/from epic	12224920	passed	-
Plan Epics roadmap presents epic on roadmap	12224915	passed	-
Plan Sum of issues weights on issue board shows the sum of issues weights in the board list's header	12224915	passed	-
Plan Configure issue board by label shows only issues that match the configured label	12224902	passed	-
Plan Default issue templates uses default template when creating an issue	12224902	passed	-
Plan issue suggestions shows issue suggestions when creating a new issue	12224902	passed	-
Plan Milestones Project milestone behaves like when assigned to new issue is assigned	12224902	passed	-
Plan Milestones Project milestone behaves like when assigned to existing issue is assigned	12224902	passed	-
Plan Milestones Group milestone behaves like when assigned to existing issue is assigned	12224902	passed	-
Plan Milestones Group milestone behaves like when assigned to new issue is assigned	12224902	passed	-
Plan Contribution Analytics tests contributions	12224909	passed	-
Plan Custom issue templates creates an issue via custom template	12224909	passed	-
Plan Related issues relates and unrelates one issue to/from another	12224909	passed	-
Plan Configurable issue board renames the issue board	12224913	passed	-
Plan Group issue boards creates a group issue board via the GUI	12224913	passed	-
Plan promote issue to epic promotes issue to epic	12224913	passed	-
Plan Issues list successfully exports issues list as CSV	12224913	passed	-

Create

• Total 52 tests
• Passed 46 tests
• Failed 0 tests
• 6 other tests (usually skipped)

Passed tests:

Test	Job	Status	Action
Create Merge request custom templates creates a merge request via custom template	12224906	passed	-
Create Codeowners displays owners specified in CODEOWNERS file	12224906	passed	-
Create Commit data user views raw email patch	12224911	passed	-
Create Commit data user views raw commit diff	12224911	passed	-
Create Cherry picking from a merge request creates a merge request	12224911	passed	-
Create Git push over HTTP user pushes code to the repository	12224900	passed	-
Create Create a new merge request creates a basic merge request	12224900 quarantine	passed	-
Create Personal snippet creation user creates a personal snippet	12224900	passed	-
Create SSH keys support user can add an SSH key	12224900	passed	-
Create SSH keys support after adding an ssh key can delete an ssh key	12224900	passed	-
Create Git push over HTTP user using a personal access token pushes code to the repository	12224901	passed	-
Create SSH key support pushes multiple branches and tags together	12224901	passed	-
Create SSH key support pushes code to the repository via SSH	12224901	passed	-
Create Merge request push options sets labels	12224904	passed	-
Create Merge request push options when labels are set already removes them on subsequent push	12224904	passed	-
Create Compare archives of different user projects with the same name and check they're different download archives of each user project then check they are different	12224904	passed	-
Create Cherry picking a commit creates a merge request	12224904	passed	-
Create Git clone over HTTP user performs a deep clone	12224904	passed	-
Create Git clone over HTTP user performs a shallow clone	12224904	passed	-
Create new merge request from the event notification after a push via the git CLI creates a merge request	12224904	passed	-
Create new merge request from the event notification after a push via the API creates a merge request	12224904	passed	-
Create Repository tags when not protected behaves like successful tag creation can be created by maintainer_user	12224916	passed	-
Create Repository tags when not protected behaves like successful tag creation can be created by developer_user	12224916	passed	-
Create Repository tags when protected behaves like successful tag creation can be created by maintainer_user	12224916	passed	-
Create Repository tags when protected behaves like unsuccessful tag creation cannot be created by an unauthorized developer_user	12224916	passed	-
Create Multiple file snippet creates a project snippet with multiple files	12224916	passed	-
Create Merge request push options removes the source branch	12224918	passed	-
Create Merge request suggestions applies a single suggestion with a custom message	12224918	passed	-
Create Multiple file snippet behaves like deleting file from snippet deletes second file from an existing personal_snippet to make it single-file	12224918	passed	-
Create Multiple file snippet behaves like deleting file from snippet deletes second file from an existing project_snippet to make it single-file	12224918	passed	-
Create Batch comments in merge request user submits a non-diff review	12224920	passed	-
Create Batch comments in merge request user submits a diff review	12224920	passed	-
Create File management user creates a file via the Web	12224915	passed	-
Create Protected branch support when developers and maintainers are allowed to push to a protected branch user with push rights successfully pushes	12224915	passed	-
Create Protected branch support when developers and maintainers are not allowed to push to a protected branch user without push rights fails to push	12224915	passed	-
Create Reverting a commit creates a merge request	12224915	passed	-
Create File management user edits a file via the Web	12224902	passed	-
Create Merge request push options sets title and description	12224902	passed	-
Create Merge request batch suggestions applies multiple suggestions	12224909	passed	-
Create Merge request squashing user squashes commits while merging	12224909	passed	-
Create API basics raw file access sets no-cache headers as expected	12224909	passed	-
Create File Locking creates a merge request and fails to merge	12224909	passed	-
Create File Locking checks file locked by other user to be disabled	12224909	passed	-
Create File Locking locks a file and tries to push as a second user	12224909	passed	-
Create Default merge request templates uses default template when creating a merge request	12224913	passed	-
Create Merge request push options sets a target branch	12224913	passed	-

Other tests:

Test	Job	Status	Action
Create Code Suggestions in Web IDE adds text into a file and verifies code suggestions appear	12224901	pending	-
Create Multiple file snippet behaves like copying snippet file contents copies a multi-file personal_snippet to a comment and verifies them	12224904 quarantine	pending	-
Create Multiple file snippet behaves like copying snippet file contents copies a multi-file project_snippet to a comment and verifies them	12224904 quarantine	pending	-
Create Multiple file snippet creates a personal snippet with multiple files	12224909 quarantine	pending	-
Create Sharing snippets when the snippet is changed to private does not display Embed/Share dropdown	12224913 quarantine	pending	-
Create Sharing snippets when the snippet is public can be shared with not signed-in users	12224913 quarantine	pending	-

Verify

• Total 31 tests
• Passed 28 tests
• Failed 3 tests
• 0 other tests (usually skipped)

Test	Job	Status	Action
Verify Pipeline subscription with a group owned project when upstream project new tag pipeline finishes triggers pipeline in downstream project	12224911	failed	failure issue
Verify Pipeline with project file variables does not expose file variable content with echo	12224916	failed	failure issue
Verify Pipeline with image:pull_policy when policy is allowed with [if-not-present] policy applies pull policy in job correctly	12224902	failed	failure issue

Passed tests:

Test	Job	Status	Action
Verify Parent-child pipelines dependent relationship parent pipeline fails if child fails	12224906	passed	-
Verify Parent-child pipelines dependent relationship parent pipelines passes if child passes	12224906	passed	-
Verify Pipeline with protected variable exposes variable on protected branch	12224906	passed	-
Verify Pipeline with protected variable does not expose variable on unprotected branch	12224906	passed	-
Verify Pipeline with prefill variables shows dropdown for variables with description, value, and options defined	12224911	passed	-
Verify Pipeline with prefill variables shows only variables with description as prefill variables on the run pipeline page	12224911	passed	-
Verify Runner fleet management shows group runner counts	12224911	passed	-
Verify Pipeline with project file variables can read file variable content with cat	12224916	passed	-
Verify Pass dotenv variables to downstream via bridge runs the pipeline with composed config	12224916	passed	-
Verify Include local config file paths with wildcard runs the pipeline with composed config	12224916	passed	-
Verify Run pipeline with manual jobs does not leave any job in skipped state	12224918	passed	-
Verify Pipeline API defined variable inheritance is determined based on forward:pipeline_variables condition	12224920	passed	-
Verify Endpoint Coverage spans r/w postgres web sidekiq git api	12224920	passed	-
Verify Multi-project pipelines creates a multi-project pipeline with artifact download	12224920	passed	-
Verify Run pipeline with web only rule can trigger pipeline	12224915	passed	-
Verify Pipelines for merged results and merge trains creates a pipeline with merged results	12224915	passed	-
Verify Pipelines for merged results and merge trains merges via a merge train	12224915	passed	-
Verify Pipeline with raw variables in YAML expands variables according to expand: true/false	12224915	passed	-
Verify Pipeline with image:pull_policy when policy is allowed with [never] policy applies pull policy in job correctly	12224902	passed	-
Verify Pipeline with image:pull_policy when policy is allowed with [always if-not-present] policies applies pull policy in job correctly	12224902	passed	-
Verify Pipeline with image:pull_policy when policy is allowed with [always] policy applies pull policy in job correctly	12224902	passed	-
Verify Pipeline with image:pull_policy when policy is not allowed fails job with policy not allowed message	12224902	passed	-
Verify Parent-child pipelines independent relationship parent pipeline passes even if child fails	12224902	passed	-
Verify Parent-child pipelines independent relationship parent pipelines passes if child passes	12224902	passed	-
Verify Job artifacts are not downloaded when dependencies array is set to empty	12224902	passed	-
Verify Pipeline with customizable variable manually creates a pipeline and uses the defined custom variable value	12224909	passed	-
Verify Include multiple files from multiple projects runs the pipeline with composed config	12224909	passed	-
Verify Include multiple files from a project runs the pipeline with composed config	12224913	passed	-

Package

• Total 6 tests
• Passed 5 tests
• Failed 0 tests
• 1 other tests (usually skipped)

Passed tests:

Test	Job	Status	Action
Package Helm Registry using personal access token pushes and pulls a helm chart	12224904	passed	-
Package Helm Registry using project deploy token pushes and pulls a helm chart	12224904	passed	-
Package Helm Registry using ci job token pushes and pulls a helm chart	12224904	passed	-
Package Conan Repository publishes, installs, and deletes a Conan package	12224916 quarantine	passed	-
Package Generic Repository uploads a generic package and downloads it	12224902	passed	-

Other tests:

Test	Job	Status	Action
Package SaaS Container Registry pulls an image from an existing repository	12224900	pending	-

Release

• Total 5 tests
• Passed 5 tests
• Failed 0 tests
• 0 other tests (usually skipped)

Passed tests:

Test	Job	Status	Action
Release Deploy key creation user adds a deploy key	12224916	passed	-
Release Git clone using a deploy key user sets up a deploy key with QA::Runtime::Key::ECDSA(521) to clone code using pipelines	12224916	passed	-
Release Git clone using a deploy key user sets up a deploy key with QA::Runtime::Key::RSA(8192) to clone code using pipelines	12224916	passed	-
Release Git clone using a deploy key user sets up a deploy key with QA::Runtime::Key::ED25519(256) to clone code using pipelines	12224916	passed	-
Release Deploy token creation user adds a deploy token	12224916	passed	-

Monitor

• Total 8 tests
• Passed 8 tests
• Failed 0 tests
• 0 other tests (usually skipped)

Passed tests:

Test	Job	Status	Action
Monitor Alert when using HTTP endpoint integration behaves like new alert automatically creates new incident	12224906	passed	-
Monitor Alert when using Prometheus integration behaves like new alert automatically creates new incident	12224906	passed	-
Monitor Alert settings when using Prometheus integration behaves like sends test alert creates new alert	12224901	passed	-
Monitor Alert settings when using HTTP endpoint integration behaves like sends test alert creates new alert	12224901	passed	-
Monitor Alert settings when using Prometheus integration behaves like sends test alert using authorization key creates new alert	12224901	passed	-
Monitor Alert settings when using HTTP endpoint integration behaves like sends test alert using authorization key creates new alert	12224901	passed	-
Monitor Recovery alert when using Prometheus integration behaves like triggers recovery alert only resolves the correct alert	12224916	passed	-
Monitor Create incident adds linked resources, label, and severity through quick actions	12224909	passed	-

Data_stores

• Total 15 tests
• Passed 15 tests
• Failed 0 tests
• 0 other tests (usually skipped)

Passed tests:

Test	Job	Status	Action
Data Stores Project Member adds a project member	12224906	passed	-
Data Stores Users API GET /users/:username with a valid username	12224906	passed	-
Data Stores Users API GET /users	12224906	passed	-
Data Stores Users API GET /users/:username with an invalid username	12224906	passed	-
Data Stores Project in group behaves like successful project creation creates a new project	12224900	passed	-
Data Stores Project in personal namespace behaves like successful project creation creates a new project	12224900	passed	-
Data Stores Invite group with a personal namespace project behaves like invites group to project grants group and members correct access level	12224904	passed	-
Data Stores Invite group with a group project behaves like invites group to project grants group and members correct access level	12224904	passed	-
Data Stores Project transfer user transfers a project between groups	12224916	passed	-
Data Stores Project activity with git push creates an event in the activity page	12224920	passed	-
Data Stores Project owner permissions for group projects behaves like adds user as maintainer has maintainer role without owner permissions	12224902	passed	-
Data Stores Project owner permissions for group projects behaves like adds user as owner has owner role and permissions	12224902	passed	-
Data Stores Project owner permissions for personal projects behaves like adds user as maintainer has maintainer role without owner permissions	12224902	passed	-
Data Stores Project owner permissions for personal projects behaves like adds user as owner has owner role and permissions	12224902	passed	-
Data Stores Project badge creates project badge	12224909	passed	-

Govern

• Total 39 tests
• Passed 39 tests
• Failed 0 tests
• 0 other tests (usually skipped)

Passed tests:

Test	Job	Status	Action
Govern Group Level Scan Execution Policy takes effect when pipeline is run on the main branch	12224911	passed	-
Govern Group Level Scan Execution Policy does not take effect when pipeline is run on non default branch	12224911	passed	-
Govern Vulnerability management in a merge request can create an issue from a vulnerability from mr security widget	12224911	passed	-
Govern Vulnerability management in a merge request can dismiss a security finding with reason from pipeline security tab	12224911	passed	-
Govern Group access token can be used to commit via the API	12224911	passed	-
Govern Group access token can be used to create a file via the project API	12224911	passed	-
Govern basic user login user logs in using basic credentials and logs out	12224901	passed	-
Govern Policies List page can navigate to Policy Editor page	12224901	passed	-
Govern Policies List page can load Policies page and view the policies list	12224901	passed	-
Govern Group for change project creation level behaves like audit event logs audit events for UI operations	12224904	passed	-
Govern Group for disable and Enable LFS behaves like audit event logs audit events for UI operations	12224904	passed	-
Govern Group for enable and disable allow user request access behaves like audit event logs audit events for UI operations	12224904	passed	-
Govern Group for enable and disable membership lock behaves like audit event logs audit events for UI operations	12224904	passed	-
Govern Project access token for the same project can be used to create a file via the project API	12224916	passed	-
Govern Project access token for a different project cannot be used to commit via the API	12224916	passed	-
Govern Project access token for a different project cannot be used to create a file via the project API	12224916	passed	-
Govern Project for project archive and unarchive behaves like audit event logs audit events for UI operations	12224918	passed	-
Govern Project for add project behaves like audit event logs audit events for UI operations	12224918	passed	-
Govern Project for change visibility behaves like audit event logs audit events for UI operations	12224918	passed	-
Govern Project for add deploy key behaves like audit event logs audit events for UI operations	12224918	passed	-
Govern Project for add user access as guest behaves like audit event logs audit events for UI operations	12224918	passed	-
Govern Scan result policy does not block merge when scan result policy does not apply for pipeline security findings	12224918	passed	-
Govern Scan result policy requires approval when a pipeline report has findings matching the scan result policy	12224918	passed	-
Govern Security Reports displays security reports in the group security dashboard	12224918	passed	-
Govern Security Reports dependency list has empty state	12224918	passed	-
Govern Group access tokens can be created and revoked via the UI	12224920	passed	-
Govern Security Dashboard in a Project shows vulnerability details	12224920	passed	-
Govern Security Dashboard in a Project creates an issue from vulnerability details	12224920	passed	-
Govern Security Reports in a Merge Request Widget displays vulnerabilities in merge request widget	12224920	passed	-
Govern project vulnerability report can bulk dismiss vulnerabilities as false positive in vulnerability report	12224915	passed	-
Govern project vulnerability report can successfully change status of a vulnerability in vulnerability details page	12224915	passed	-
Govern project vulnerability report can successfully bulk change status to confirmed in vulnerability report	12224915	passed	-
Govern MR security widget checks that dismissed vulnerabilities do not show up	12224915	passed	-
Govern Project vulnerability report validates "fix a vulnerability" workflow	12224909	passed	-
Govern Project access tokens can be created and revoked via the UI	12224913	passed	-
Govern Group for add user, change access level, remove user behaves like audit event logs audit events for UI operations	12224913	passed	-
Govern Group for update group name behaves like audit event logs audit events for UI operations	12224913	passed	-
Govern Group for add group behaves like audit event logs audit events for UI operations	12224913	passed	-
Govern Group for add and remove project access behaves like audit event logs audit events for UI operations	12224913	passed	-

Unknown

• Total 1 tests
• Passed 0 tests
• Failed 0 tests
• 1 other tests (usually skipped)

Other tests:

Test	Job	Status	Action
Framework sanity Version check is the specified version	12224901	pending	-

Link to Grafana dashboard for run-type of canary-sanity

• https://dashboards.quality.gitlab.net/d/tR_SmBDVk/main-runs?orgId=1&refresh=1m&var-run_type=canary-sanity

added QA Quality found:canary.gitlab.com triage report labels

closed

added auto closed label