Scheduled pipeline execution policy job fails to download artifacts from a previous job
Summary
Reported by @kumaresh.s in Slack (internal):
With scheduled pipeline execution policies, a dependency_scanning job in a test stage fails to start due to missing permissions to download artifacts of a previous job from .pipeline-policy-pre stage.
Steps to reproduce
Example Project
PEP YAML:
# This configuration generates Gradle lock files
# and runs dependency scanning
stages:
- .pipeline-policy-pre # Special stage that runs BEFORE project stages
- build
- test
# Job to generate Gradle lock files
gradle_lockfile_generation:
stage: .pipeline-policy-pre
image: gradle:9.1-jdk25
# Only run this job for projects with Gradle files
rules:
- if: '$CI_COMMIT_BRANCH'
exists:
- '**/build.gradle'
- '**/build.gradle.kts'
before_script:
# Create the Nebula plugin initialization script
- |
cat << 'EOF' > nebula.gradle
initscript {
repositories {
mavenCentral()
}
dependencies {
classpath 'com.netflix.nebula:gradle-dependency-lock-plugin:16.0.1'
}
}
allprojects {
apply plugin: nebula.plugin.dependencylock.DependencyLockPlugin
}
EOF
# Make gradlew executable if it exists
- chmod +x gradlew 2>/dev/null || echo "No gradlew found, will use system gradle"
script:
# Generate lock file with transitive dependencies
- |
if [ -f "./gradlew" ]; then
./gradlew --init-script nebula.gradle \
-PdependencyLock.includeTransitives=true \
-PdependencyLock.lockFile=dependencies.lock \
generateLock saveLock
else
gradle --init-script nebula.gradle \
-PdependencyLock.includeTransitives=true \
-PdependencyLock.lockFile=dependencies.lock \
generateLock saveLock
fi
# Generate lock file with direct dependencies only
- |
if [ -f "./gradlew" ]; then
./gradlew --init-script nebula.gradle \
-PdependencyLock.includeTransitives=false \
-PdependencyLock.lockFile=dependencies.direct.lock \
generateLock saveLock
else
gradle --init-script nebula.gradle \
-PdependencyLock.includeTransitives=false \
-PdependencyLock.lockFile=dependencies.direct.lock \
generateLock saveLock
fi
# Verify lock files were created
- find . -name "dependencies*.lock" -type f
after_script:
# Clean up temporary build artifacts to save space
- find . -path '*/build/dependencies*.lock' -delete
# Export lock files as artifacts for dependency scanning job
artifacts:
paths:
- '**/dependencies.lock'
- '**/dependencies.direct.lock'
expire_in: 1 day
allow_failure: false
# Include dependency scanning
include:
- component: $CI_SERVER_FQDN/components/dependency-scanning/main@0
# Enable the new SBOM-based analyzer
variables:
DS_ENFORCE_NEW_ANALYZER: "true"What is the current bug behavior?
The job fails to start due to 403 Forbidden - This action cannot be performed by internal users which happens when trying to download artifacts of a previous job gradle_lockfile_generation.
The job works fine when it's executed as a regular user.
What is the expected correct behavior?
The permissions should be granted for the policy bot user to download the artifacts.
Relevant logs and/or screenshots
https://gitlab.com/gl-demo-ultimate-kumaresh/demos/java/simple-java-gradle-app/-/jobs/11785690029
Job logs
Running with gitlab-runner 18.4.0~pre.195.g5de42c65 (5de42c65)
on blue-1.saas-linux-small.runners-manager.gitlab.com/default j1aLDqxSn, system ID: s_ccdc2f364be8
Resolving secrets
Preparing the "docker+machine" executor 00:09
Using Docker executor with image registry.gitlab.com/security-products/dependency-scanning:v0 ...
Using effective pull policy of [always] for container registry.gitlab.com/security-products/dependency-scanning:v0
Authenticating with credentials from job payload (GitLab Registry)
Pulling docker image registry.gitlab.com/security-products/dependency-scanning:v0 ...
Using docker image sha256:599c80b67eb7f0ae923949bcc838551455ff76c13112ed6e1c179705b813cf38 for registry.gitlab.com/security-products/dependency-scanning@sha256:0d4972b6dfbc76796adecdd1f29004ff67f51c07675a1c7ce53e2dc89b36bee3 ...
Preparing environment 00:04
Using effective pull policy of [always] for container sha256:1c7ef7eba67dc4ec18094f5f5c67fcdc7bb9c4a15e08ca51ca3b6177e2e43f0c
Running on runner-j1aldqxsn-project-75092203-concurrent-0 via runner-j1aldqxsn-s-l-s-amd64-1761033359-4e4a3a80...
Getting source from Git repository 00:01
Gitaly correlation ID: e00e64deb3324c56b0d37713b27e6b4d
Fetching changes with git depth set to 20...
Initialized empty Git repository in /builds/gl-demo-ultimate-kumaresh/demos/java/simple-java-gradle-app/.git/
Created fresh repository.
Checking out 49678023 as detached HEAD (ref is main)...
Skipping Git submodules setup
$ git remote set-url origin "${CI_REPOSITORY_URL}" || echo 'Not a git repository; skipping'
Downloading artifacts 00:01
Downloading artifacts for gradle_lockfile_generation (11785690009)...
ERROR: Downloading artifacts from coordinator... forbidden correlation_id=208189ca0d074597819c8e8a55f5055f host=us-east1-c.ci-gateway.int.gprd.gitlab.net:8989 id=11785690009 responseStatus=403 Forbidden status=GET https://us-east1-c.ci-gateway.int.gprd.gitlab.net:8989/api/v4/jobs/11785690009/artifacts: 403 Forbidden (403 Forbidden - This action cannot be performed by internal users) token=eyJraWQiO
FATAL: permission denied
Retrying in 5s
Uploading artifacts for failed job 00:00
Uploading artifacts...
WARNING: **/gl-sbom-*.cdx.json: no matching files. Ensure that the artifact path is relative to the working directory (/builds/gl-demo-ultimate-kumaresh/demos/java/simple-java-gradle-app)
ERROR: No files to upload
Uploading artifacts...
WARNING: gl-dependency-scanning-report.json: no matching files. Ensure that the artifact path is relative to the working directory (/builds/gl-demo-ultimate-kumaresh/demos/java/simple-java-gradle-app)
ERROR: No files to upload
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: exit code 1Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)(we will only investigate if the tests are passing)
Implementation Plan
To resolve this issue, the security policy bot permissions need to be refactored and enhanced through the following steps:
Phase 1: Refactor to YAML Format
Task: #594740 (closed) - Refactor security policy bot permissions to use YAML format
Migrate all security policy bot authorization policies from Ruby-based policy classes to the new YAML format. This establishes a foundation for managing bot permissions in a more maintainable and transparent way.
Deliverables:
- Create
config/authz/roles/security_policy_bot.ymlwith all current permissions - Migrate existing Ruby policy classes to YAML format
- Ensure backward compatibility during transition
Phase 2: Update to Minimal Access
Task: #594741 (closed) - Update security policy bot role to minimal access
Replace the guest role with a minimal access role that only includes permissions necessary for the bot to function. This will be implemented behind a feature flag to allow testing and validation.
Deliverables:
- Implement feature flag
security_policy_bot_minimal_access - Update bot creation logic to use minimal access when flag is enabled
- Identify and document required permissions through testing
- Maintain backward compatibility with guest role as fallback
Phase 3: Grant API Access
Task: #594742 - Grant API access to security policy bot role
Once the bot has been migrated to minimal access, grant the necessary API permissions to allow the bot to download artifacts and perform other required operations.
Deliverables:
- Add API access permissions to the
security_policy_botrole - Document which API endpoints the bot needs access to
- Test API access in the minimal access context
- Verify no additional permissions are needed
Possible fixes
Patch release information for backports
If the bug fix needs to be backported in a patch release to a version under the maintenance policy, please follow the steps on the patch release runbook for GitLab engineers.
Refer to the internal "Release Information" dashboard for information about the next patch release, including the targeted versions, expected release date, and current status.
High-severity bug remediation
To remediate high-severity issues requiring an internal release for single-tenant SaaS instances, refer to the internal release process for engineers.