Host Debian, Alpine packages of Secure analyzers
Problem to solve
In the context of &3923 (closed), some SAST and Dependency Scanning analyzers need to be released as distribution packages. Package URLs must be predictable, based on the analyzer name, analyzer version, and package type (Debian or Alpine).
This is NOT a two-way door decision. Whatever the solution we choose, we'll have to maintain it for a long time, to ensure distro packages are served to older versions of GitLab.
Proposal
Use GitLab Generic Packages Repository (proposal B). See &4209 (closed) for development status.
GitLab API has two endpoints to publish and download generic packages:
PUT /projects/:id/packages/generic/:package_name/:package_version/:file_name?status=:status
GET /projects/:id/packages/generic/:package_name/:package_version/:file_name
package_name
could be the package type, so either debian
or alpine
.
package_version
is the analyzer version.
file_name
is a stable, predictable file name, like gemnasium.deb
. There's no need to repeat the package version in the file name.
BLOCKER: Right now only authenticated users can download generic packages. We need to provide public access, so that scanning jobs running on behalf of users can fetch and install these packages. This should be solved by !55978 (merged).
See other proposals:
Create a GitLab project to store the distro packages of Secure analyzers: https://gitlab.com/gitlab-org/security-products/packages In the analyzer pipeline, the release job pushes a new git commit to the TBD: Set a convention for the package path: This is a good enough solution but we proposal B seems better, and we'll have to maintain both solution if we eventually decide to go for B. Pros: Cons:Proposal A
packages
repo, to add a new package file. The release job uses the GitLab Bot token to connect to GitLab./analyzer_name/package_name.extension
packages
and fetch all packages from that fork.
Use raw package binary repository, a new GitLab feature described in #208712 (closed). In the analyzer pipeline, the release job pushes the package file the built-in raw package repo. The access token is either the token of the GitLab Bot or the This seems to be the ideal solution but there are unknowns. Pros: Cons:Proposal B
CI_JOB_TOKEN
- this is to be defined in #208712 (comment 392161680).
Use an external storage service, like Amazon S3. In the analyzer pipeline, the release job pushes the package file to S3. It uses an access token set at the project group level. Pros: Cons:Proposal C
Use release asset links. In the analyzer pipeline, the build job uploads the distro package as a CI job artifact, and the release job creates a project release with an asset link that points to this artifact. The release job uses the access token of the GitLab to access the GitLab API and create the asset links. Pros: Blocker:Proposal D
quoting comment: Pros: Cons:Proposal E
*-latest-*
instead of *-vX.Y.Z-*
latest
branch re-exposing the latest tagged build's artifacts
Further details
Permissions and Security
Proposal A: The GitLab bot can push commits to the packages
project, making possible to publish releases from the pipeline of the analyzer project (SAST or Dependency Scanning project).
Members of devopssecure are members of the packages
project, with developer
access.
Documentation
To be documented in analyzer projects using the packages
project and/or in the ci-templates which defines the shared CI configuration.
Availability & Testing
N/A
What is the type of buyer?
N/A
Is this a cross-stage feature?
No
Architectural Support
Scope checklist
-
Does not involve architectural decisions -
Is after-the-fact -
Is not already covered by architecture guidelines/handbook -
Has a broad impact within #secure -
Is a new unit of work -
Is strictly #secure -
Could not come to an agreement (escalation) -
Involves architectural decisions
== in-scope
Reviewed by
-
@cam_swords -
@d0c-s4vage -
@fcatteau(author) -
@markrian -
@theoretick