Unexpected cache suffix depending on branch protection

Summary

Our cache isn't shared anymore between our default branch (protected) and our feature branches (unprotected) because of a new dynamic suffix added after cache-<index> that depend on ref protection.

---

• <prefix>-<key>-<index>-protected
• <prefix>-<key>-<index>-non_protected

Steps to reproduce

• Use GitLab Caching
• Try to get cache of a protected branch on a non protected branch using FALLBACK_KEY or a lockfile as key ❌

.gitlab-ci.yml

rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
      variables:
        CACHE_FALLBACK_KEY: '$CI_JOB_NAME:$CI_MERGE_REQUEST_TARGET_BRANCH_NAME'

  cache:
    key:
      files:
        - yarn.lock
      prefix: project
    paths:
      - .yarn-cache/

  cache:
      key: '$CI_JOB_NAME:$CI_COMMIT_REF_NAME'
      paths:
        - .jestcache/

Actual behavior

A -non_protected suffix is added on the feature branch job:cache:key trying to get the default branch job:cache:key that was created with the -protected suffix. See logs

Expected behavior

A way to delete this suffix.

A new variable could fix it on FALLBACK_KEY but i can't see how to fix it using lockfile as key.

Note that the cache-<index> already causes problems on FALLBACK_KEY and ours depends on $CI_MERGE_REQUEST_TARGET_BRANCH_NAME

Relevant logs and/or screenshots

Default branch job log

Checking cache for jest:mobile:master-3-protected...

Checking cache for project-ea268a7e34ce64f1a07ba41343c9c84cd6a48bb0-3-protected...

Feature branch job log

Checking cache for jest:mobile:1703-link-faq-details-3-non_protected...
WARNING: file does not exist                       
Failed to extract cache
Checking cache for jest:mobile:master...

Checking cache for project-ea268a7e34ce64f1a07ba41343c9c84cd6a48bb0-3-non_protected...

Used GitLab Runner version

Running with gitlab-runner 14.10.0~beta.50.g1f2fe53e (1f2fe53e)
  on blue-4.shared.runners-manager.gitlab.com/default J2nyww-s

Workarounds

• disabling the splitting of the caches per project, if you're OK with the security implications: Add option to disable separated caches (gitlab!86876 - merged)
• Using lockfile key: generate cache on a non protected branch for each lockfile updates...
• Using fallback key: adding -protected or non_protected manually as cache-<index>...
• #29033 (comment 928173695)