Job using JWT/OIDC ran fine on trunk, fails on tag pipeline
Hey, I am a 101% puzzled by this behaviour. I don't know whether there's some kind of bug lurking there or if I'm just stupid. Here's what's happening:
We used to deploy trunk (master) straight to prod. The corresponding jobs authenticate against AWS via CI_JOB_JWT_V2 or the new GITLAB_OIDC_TOKEN. We now deploy to prod on tags, nothing else really changed, and it broke. I now see
An error occurred (AccessDenied) when calling the AssumeRoleWithWebIdentity operation: Not authorized to perform sts:AssumeRoleWithWebIdentity
and
Unable to locate credentials. You can configure credentials by running "aws configure".
and in later terraform steps
Error: No valid credential sources found
The exact same jobs run. It's the same environment. I checked the role, it's still the same. The script portion itself is just terraform init plan and apply. When trying GITLAB_OIDC_TOKEN I add
id_tokens:
GITLAB_OIDC_TOKEN:
aud: https://gitlab.com
to the job spec and the auth script can handle both env vars. This works just fine in other projects.
There's also two projects with the exact same behaviour. I even created a branch, dropped the rules from the deployment job and it did successfully auth agains AWS and deploy the stuff. Nothing else changed.
I don't see what should be different for a pipeline triggered by a tag than one for a push to any branch or MR.
Any ideas?