Resolve "Modify PublishProvenanceService so that it calls cosign, performing attestation." (!202816) · Merge requests · GitLab.org / GitLab

Background

What are "SLSA provenance statements"?

The grouppipeline security is working towards providing users with SLSA Level 3 Provenance Attestations. Quoting from the SLSA documentation, it states that attestations are:

It’s the verifiable information about software artifacts describing where, when, and how something was produced. For higher SLSA levels and more resilient integrity guarantees, provenance requirements are stricter and need a deeper, more technical understanding of the predicate. Describe how an artifact or set of artifacts was produced so that:

Consumers of the provenance can verify that the artifact was built according to expectations.

Others can rebuild the artifact, if desired.

As a simplified TL;DR, in the context of GitLab, a provenance statement is a JSON document that correlates the sha256 sum of an artifact with the build information. A worker then performs a digital signature, which is called a provenance attestation. This is a highly sought-after feature, particularly for our GitLab Ultimate customers.

References

Why is this change required

As described above, we need to perform a provenance attestation. All the necessary information for checking this provenance attestation will be stored within a Sigstore bundle.

In this merge request, I am implementing the necessary code to invoke the cosign command to generate the provenance attestation within PublishProvenanceService. This service will be invoked by PublishProvenanceWorker. This will only happen if [FF] slsa_provenance_statement is enabled, along with other preconditions such as a YAML variable being set, artifacts existing and the project the build belongs to being public.

This merge request also retrieves the bundle itself and logs it for manual verification in a production environment. In a follow-up ticket, this will be persisted through a file uploader. Metadata will be stored in the database.

Security considerations

Appsec review has been requested here, where I have documented the steps I've taken to ensure we don't introduce vulnerabilities such as remote code execution, server-side request forgery or path traversal.

✅ Review by appsec given in thread below.

Testing

Requirements:

To test this end-to-end, we need an OIDC provider that Fulcio can reach and is allowlisted. In production, this is simple because gitlab.com is an authorised OIDC provider. Locally, there are some prerequisites.

Install sigstore, as documented in Sigstore Local. This will install Fulcio, Rekor and TUF.
You need to configure GDK to have runners enabled as documented in doc/howto/runner.md · main · GitLab.org / GitLab Development Kit · GitLab
You need to configure your local cosign command to use the local TUF by running regenerating TUF, cd tuf/repository, executing python3 -m http.server, and then cosign initialize --mirror http://localhost:8000 --root ~/root.json
You need to configure a build that generates an artifact and uses our temporary environment variable GENERATE_PROVENANCE to trigger a build, and properly populates the SIGSTORE_ID_TOKEN variable. The project associated with the build must be public.
The build must have an age of less than 60 minutes. This is because SIGSTORE_ID_TOKEN expires after that time.

The following .gitlab.yml file complies with the requirements above.

build-job:
  stage: build
  variables:
    GENERATE_PROVENANCE: true
  id_tokens:
    SIGSTORE_ID_TOKEN:
      aud: sigstore
  script:
    - echo "Hello, $GITLAB_USER_LOGIN!"
    - echo "Hello, $GITLAB_USER_LOGIN!" > test.txt
  artifacts:
    paths:
      - test.txt

Testing:

After that, an end-to-end attestation bundle can be generated with the following commands:

% COSIGN_FULCIO_URL="http://sigstore.local:5555" COSIGN_REKOR_URL="http://sigstore.local:3090" bundle exec rails c
> build = Ci::Build.last
> pps = Ci::Slsa::PublishProvenanceService.new(build)
> pps.execute
=> #<ServiceResponse:0x000000032779a1b0 @http_status=:ok, @message="OK", @payload={}, @reason=nil, @status=:success>

The resultint attestation will be logged. In 18.5, we are looking to persist the attestation metadata to the database, and the attestation itself using a file uploader.

Example logs:

{"severity":"INFO","time":"2025-09-04T22:48:57.606Z","class":"Ci::Slsa::PublishProvenanceService","message":"Performing attestation for artifact","hash":"3c5bba498d6f7a2cb4c195cf0873c8b68c9407f04dfa9acaad7fe4875e5e93f1","path":"test.txt"}
{"severity":"INFO","time":"2025-09-04T22:48:58.015Z","class":"Ci::Slsa::PublishProvenanceService","message":"Attestation successful","hash":"3c5bba498d6f7a2cb4c195cf0873c8b68c9407f04dfa9acaad7fe4875e5e93f1","blob_name":"test.txt","attestation":"{\"mediaType\":\"application/vnd.dev.sigstore.bundle.v0.3+json\",\"verificationMaterial\":{\"certificate\":{\"rawBytes\":\"MIIFuTCCBT6gAwIBAgIUdKcB6jz6YorVc0Q6RWxLl2iAySowCgYIKoZIzj0EAwMwaTEMMAoGA1UEBhMDVVNBMREwDwYDVQQIEwhBbnlQbGFjZTEQMA4GA1UEBxMHQW55dG93bjEUMBIGA1UECRMLMTIzIE1haW4gU3QxDzANBgNV
[...] truncated

Manually verifying attestation

At this stage, the attestation is not persisted to the database, and needs to be retrieved from the logs. We will work on this with the 'Persist attestation to file and database' issue.

For now, you can then verify the attestation as follows.

# /tmp/test.txt is the file we attested, see .gitlab.yml
cat log/application_json.log | grep "Attestation successful" --color=never | tail -n 1 | jq -r '.attestation' > /tmp/attestation.bundle
cosign verify-blob-attestation --new-bundle-format --bundle /tmp/attestation.bundle --type slsaprovenance1 /tmp/test.txt --certificate-identity "https://gdk.test:3000/root/test_project//.gitlab-ci.yml@refs/heads/main" --certificate-oidc-issuer "http://gdk.test:3000"
setting TUF refresh period to 24h0m0s
Verified OK

Related to #559192 (closed)

Edited Sep 07, 2025 by Sam Roque-Worcel

Resolve "Modify PublishProvenanceService so that it calls cosign, performing attestation."