Update secure analyzer projects to use consistent QA expectation location
Problem to solve
Secure analyzers currently store expectation files in qa/expect/<project-name>-<branch-name>
, for example:
The above expectation is located in typescript-yarn-tsx
, where typescript-yarn
is the test project name and the branch is tsx-FREEZE
.
The purpose of this issue is to formally document a naming convention in the README.md
for the tests/common project, as well as update all existing dependency scanning and SAST analyzers to use this pattern.
Intended users
Proposal
Determine the best location pattern to use. Currently we're using qa/expect/<project-name>-<branch-name>
, however, this naming convention is ambiguous. For example, if we consider the path java-maven-cli-opts
, it's not possible to know if the project name is java-maven-cli
and the branch name is opts
or if the project is java-maven
and the branch is cli-opts
. For this reason, I would prefer to use the pattern <project-name>/<branch-name>
, which in this case would result in the unambiguous pattern java-maven/cli-opts
.
Implementation Plan
Use the following expectation location pattern:
-
qa/expect/<project-name>/gl-<report_type>-report.json
for the default branch and default test case -
qa/expect/<project-name>/<branch-name|test-case>/gl-<report_type>-report.json
for a specific branch and/or a specific test case
Ensure the QA tests of the following projects use the provided expectation location pattern:
-
SAST -
Dependency Scanning See this comment for a list of complete merge requests for moving the expectation files.
Further details
See discussion here
Documentation
Documentation will be added to tests/common/README.md
What does success look like, and how can we measure that?
Secure analyzer qa expectation location pattern is documented and consistently enforced throughout our analyzers.
What is the type of buyer?
Enterprise Edition GitLab Ultimate
Is this a cross-stage feature?
Yes, this affects ~"Category:Dependency Scanning" and Category:SAST