Skip to content

Keyless signing with sigstore not working in child pipelines

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Summary

Keyless signing using Sigstore fails in the context of child pipelines. cosign sign fails when it connects to Fulcio API.

In the context of child pipelines, ID tokens don't have the following claims:

  • ci_config_ref_uri
  • ci_config_sha

These claims are required by Fulcio. See #422146 (comment 2417136928)

Further details

This exact error is raised by the Fulcio server when the ID token is invalid. See https://github.com/sigstore/fulcio/blob/v1.6.6/pkg/server/grpc_server.go#L84-L88

	// Authenticate OIDC ID token by checking signature
	principal, err := g.IssuerPool.Authenticate(ctx, token)
	if err != nil {
		return nil, handleFulcioGRPCError(ctx, codes.InvalidArgument, err, invalidIdentityToken)
	}

The following line calls the API endpoint, and raises/wraps the error: https://github.com/sigstore/cosign/blob/v2.4.3/cmd/cosign/cli/sign/sign.go#L546

The API client is implemented in https://github.com/sigstore/cosign/blob/v2.4.3/cmd/cosign/cli/fulcio/fulcio.go.

Token verification occurs in https://github.com/sigstore/fulcio/blob/v1.6.6/pkg/identity/authorize.go#L28.

GitLab ID tokens are handled by https://github.com/sigstore/fulcio/blob/v1.6.6/pkg/identity/gitlabcom/issuer.go#L34.

See investigation in thread: #422146 (comment 2417136928)

Steps to reproduce

Example Project

Pipeline: https://gitlab.com/balasankarc/test-gitlab-id-tokens/-/pipelines/969074576

Failed job: https://gitlab.com/balasankarc/test-gitlab-id-tokens/-/jobs/4880949570#L77

CI/CD config 
stages:
  - build
  - verify

build_and_sign:
  stage: build
  image: docker:latest
  services:
    - docker:dind
  variables:
    COSIGN_YES: "true"
  id_tokens:
    SIGSTORE_ID_TOKEN:
      aud: sigstore
  before_script:
    - apk add --update cosign
    - docker login -u "gitlab-ci-token" -p "${CI_JOB_TOKEN}" ${CI_REGISTRY}
  script:
    - docker build --pull -t "${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHORT_SHA}" .
    - docker push "${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHORT_SHA}"
    - IMAGE_DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHORT_SHA})
    - cosign sign $IMAGE_DIGEST

trigger_child_pipeline:
  stage: build
  trigger:
    include: '.gitlab-ci.yml'
    strategy: depend
  rules:
    - if: '$CI_PIPELINE_SOURCE == "parent_pipeline"'
      when: never
    - when: always

What is the current bug behavior?

cosign sign fails to sign.

What is the expected correct behavior?

cosign sign signs using the ID token stored in SIGSTORE_ID_TOKEN (environment variable).

Relevant logs and/or screenshots

getting signer: getting key from Fulcio: retrieving cert: POST https://fulcio.sigstore.dev/api/v1/signingCert returned 400 Bad Request: "{\"code\":3, \"message\":\"There was an error processing the identity token\", \"details\":[]}"

API endpoint: https://fulcio.sigstore.dev/api/v1/signingCert

Error code: 400 Bad Request

Error response:

{"code":3, "message":"There was an error processing the identity token", "details":[]}

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Possible fixes

  • Change the backend so that ID tokens generated in child pipelines has ci_config_ref_uri and ci_config_sha in its claims.

Implementation plan

Edited by 🤖 GitLab Bot 🤖