GitLab managed Terraform state file not deletable with $CI_JOB_TOKEN

What is this about?

We have a GitLab managed Terraform state in our environment.
We are also using Environments and Deployments, and every deployment has its own Terraform-State in GitLab. When we stop a deployment, we also want to delete the Terraform-State for this deployment with a CI-Job as described here: https://docs.gitlab.com/ee/user/infrastructure/iac/terraform_state.html#remove-a-state-file-by-using-the-api.
When we use the $CI_JOB_TOKEN as access token for the operation, we get an error {"message":"401 Unauthorized"}.
Is this intentional that the $CI_JOB_TOKEN cannot delete Terraform-State files? My expectation was that I can delete the Terraform-State with the $CI_JOB_TOKEN as access token in the command curl --header "Private-Token: <your_access_token>" --request DELETE "https://gitlab.example.com/api/v4/projects/<your_project_id>/terraform/state/<your_state_name>".

Thank you in advance for clarifying
Regards,
Manuel

Implementation Guide

  1. update the DELETE endpoint:

    - job_token_allowed: :basic_auth
+ job_token_allowed: true

  2. Update Terraform state docs: https://docs.gitlab.com/ee/user/infrastructure/iac/terraform_state.html#remove-a-state-file-by-using-the-api

    • Note that the header for CI_JOB_TOKEN is Job-Token instead of Private-Token.

  3. Update CI_JOB_TOKEN docs: https://docs.gitlab.com/ee/ci/jobs/ci_job_token.html

  4. Add specs to guarantee that only maintainers can delete states with the CI_JOB_TOKEN. This should be done on the spec/requests/api/terraform/state_spec.rb file (DELETE state section).

Useful references

We've implemented a very similar feature for packages: !91437 (merged)

Edited by João Alexandre Cunha