Keyless signing with Keystore missing ci_config_ref_uri on specific configurations

Background

Sigstore mantainer has reached out with the following bug.

Thanks Billy for making the connection. Sam, nice to meet you! Currently, I see an error in our logs that ci_config_ref_uri is empty. It's not incredibly frequent, but I do see these errors throughout the day. For example, over the last day, we have 56 instances. I unfortunately don't have much information as we don't log the entire token, so I'm not able to provide which pipelines are causing this issue. Would that be helpful info? I can add that logging and get an update rolled out. [8:27 AM] For the most recent time I saw this error in the log:

> {
>   "insertId": "vjga2g9sa5nwj4jz",
>   "jsonPayload": {
>     "caller": "server/error.go:45",
>     "requestID": "SSLlfhJe",
>     "error": "template: :1:11: executing \"\" at <.ci_config_ref_uri>: map has no entry for key \"ci_config_ref_uri\"",
>     "stacktrace": "github.com/sigstore/fulcio/pkg/server.handleFulcioGRPCError\n\tgithub.com/sigstore/fulcio/pkg/server/error.go:45\ngithub.com/sigstore/fulcio/pkg/server.(*grpcaCAServer).CreateSigningCertificate\n\tgithub.com/sigstore/fulcio/pkg/server/grpc_server.go:231\ngithub.com/sigstore/fulcio/pkg/server.(*legacyGRPCCAServer).CreateSigningCertificate\n\tgithub.com/sigstore/fulcio/pkg/server/legacy_server.go:100\ngithub.com/sigstore/fulcio/pkg/generated/protobuf/legacy._CA_CreateSigningCertificate_Handler.func1\n\tgithub.com/sigstore/fulcio/pkg/generated/protobuf/legacy/fulcio_legacy_grpc.pb.go:155\ngithub.com/grpc-ecosystem/go-grpc-prometheus.init.(*ServerMetrics).UnaryServerInterceptor.func3\n\tgithub.com/grpc-ecosystem/go-grpc-prometheus@v1.2.1-0.20210315223345-82c243799c99/server_metrics.go:108\ngithub.com/sigstore/fulcio/cmd/app.createLegacyGRPCServer.ChainUnaryServer.func4.1\n\tgithub.com/grpc-ecosystem/go-grpc-middleware@v1.4.0/chain.go:48\ngithub.com/sigstore/fulcio/cmd/app.createLegacyGRPCServer.PassFulcioConfigThruContext.func2\n\tgithub.com/sigstore/fulcio/cmd/app/grpc.go:75\ngithub.com/sigstore/fulcio/cmd/app.createLegacyGRPCServer.ChainUnaryServer.func4.1\n\tgithub.com/grpc-ecosystem/go-grpc-middleware@v1.4.0/chain.go:48\ngithub.com/grpc-ecosystem/go-grpc-middleware/logging/zap.UnaryServerInterceptor.func1\n\tgithub.com/grpc-ecosystem/go-grpc-middleware@v1.4.0/logging/zap/server_interceptors.go:31\ngithub.com/sigstore/fulcio/cmd/app.createLegacyGRPCServer.ChainUnaryServer.func4.1\n\tgithub.com/grpc-ecosystem/go-grpc-middleware@v1.4.0/chain.go:48\ngoa.design/goa/v3/grpc/middleware.UnaryRequestID.func1\n\tgoa.design/goa/v3@v3.20.1/grpc/middleware/requestid.go:35\ngithub.com/sigstore/fulcio/cmd/app.createLegacyGRPCServer.ChainUnaryServer.func4.1\n\tgithub.com/grpc-ecosystem/go-grpc-middleware@v1.4.0/chain.go:48\ngithub.com/grpc-ecosystem/go-grpc-middleware/recovery.UnaryServerInterceptor.func1\n\tgithub.com/grpc-ecosystem/go-grpc-middleware@v1.4.0/recovery/interceptors.go:33\ngithub.com/sigstore/fulcio/cmd/app.createLegacyGRPCServer.ChainUnaryServer.func4\n\tgithub.com/grpc-ecosystem/go-grpc-middleware@v1.4.0/chain.go:53\ngithub.com/sigstore/fulcio/pkg/generated/protobuf/legacy._CA_CreateSigningCertificate_Handler\n\tgithub.com/sigstore/fulcio/pkg/generated/protobuf/legacy/fulcio_legacy_grpc.pb.go:157\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\tgoogle.golang.org/grpc@v1.72.0/server.go:1405\ngoogle.golang.org/grpc.(*Server).handleStream\n\tgoogle.golang.org/grpc@v1.72.0/server.go:1815\ngoogle.golang.org/grpc.(*Server).serveStreams.func2.1\n\tgoogle.golang.org/grpc@v1.72.0/server.go:1035",
>     "message": "template: :1:11: executing \"\" at <.ci_config_ref_uri>: map has no entry for key \"ci_config_ref_uri\"",
>     "clientMessage": "template: :1:11: executing \"\" at <.ci_config_ref_uri>: map has no entry for key \"ci_config_ref_uri\"",
>     "code": "InvalidArgument",
>     "ts": 1761923275.213548
>   },
>   "resource": {
>     "type": "k8s_container",
>     "labels": {
>       "cluster_name": "sigstore-prod",
>       "namespace_name": "fulcio-system",
>       "pod_name": "fulcio-server-65bdf4d645-jxk57",
>       "project_id": "project-rekor",
>       "container_name": "fulcio-server",
>       "location": "us-central1"
>     }
>   },
>   "timestamp": "2025-10-31T15:07:55.213768969Z",
>   "severity": "ERROR",
>   "logName": "projects/project-rekor/logs/stderr",
>   "receiveTimestamp": "2025-10-31T15:07:57.188634602Z"
> }

[8:28 AM] Out of this, the timestamp is probably the only useful value. [8:30 AM] From what you've said, it sounds like we shouldn't be seeing tokens with empty ci_config_ref_uri , correct? Could this be pipelines that haven't updated something? If this issue has been resolved on your side, it sounds like there's nothing we need to do and that this should self resolve. I am happy to add more logging though if you'd like the specific pipelines that are having issues.

This likely is a bug in our implementation. Internally, this information is mapped from the ProjectConfig based on a ClaimsMapper object. For example, for a Bridge project config (lib/gitlab/ci/project_config/bridge.rb) we use a (lib/gitlab/ci/jwt_v2/claim_mapper/bridge.rb). We currently support the repository (i.e. normal pipeline) and bridge (i.e. child pipeline). However there are several ProjectConfig types that are not supported. One of these may be the root cause of the issue.

Relevant links

Non-functional requirements

  • Documentation:
  • Feature flag:
  • Performance:
  • Testing:

Implementation plan

Verification steps

Edited by 🤖 GitLab Bot 🤖