secure-test-project-orchestrator's dynamic_job fails
Summary
The dynamic_job
of the secure-test-project-orchestrator fails when processing what the GitLab API returns. As a result, daily integration tests executed by this orchestrator are skipped.
Error is:
lib/dynamic_jobs/dynamic_jobs.rb:44:in `block in project_branch_details': no implicit conversion of String into Integer (TypeError)
This is because the code assumes that "https://gitlab.com/api/v4/projects/#{project_id}/repository/branches"
returns an array of branches, but it some cases it returns {"message"=>"404 Not Found"}
.
Currently this bug is triggered by tests/container-scanning (id 11432834) because this project no longer has a git repo.
Further details
dynamic_job
queries the API to list all the Secure test project, creates a dynamic CI config, and upload this CI config as a job artifact.
Steps to reproduce
Trigger a new pipeline for https://gitlab.com/gitlab-org/quality/ci/secure-test-project-orchestrator/
Or clone the git repo and run ruby lib/dynamic_jobs/main.rb
. Here's how to reproduce the error using Docker and the alpine:latest
image instance:
secure-test-project-orchestrator % docker run -ti -v $PWD:/app -w /app alpine:latest
/app # apk update && apk add ruby bundler ruby-bigdecimal ruby-json
/app # gem install httparty
/app # BRANCH_REGEXP='^(develop' ruby ./lib/dynamic_jobs/main.rb
What is the current bug behavior?
dynamic_job
fails, and there are no notifications on the #s_secure-alerts
Slack channel.
See https://gitlab.com/gitlab-org/quality/ci/secure-test-project-orchestrator/-/jobs/1337949747#L46
What is the expected correct behavior?
dynamic_job
succeeds, and notifications are pushed to Slack after running all the downstream pipelines (job integration tests).
Relevant logs and/or screenshots
See failing pipeline
See failing job
$ ruby ./lib/dynamic_jobs/main.rb
/builds/gitlab-org/quality/ci/secure-test-project-orchestrator/lib/dynamic_jobs/dynamic_jobs.rb:44:in `block in project_branch_details': no implicit conversion of String into Integer (TypeError)
from /builds/gitlab-org/quality/ci/secure-test-project-orchestrator/lib/dynamic_jobs/dynamic_jobs.rb:44:in `each'
from /builds/gitlab-org/quality/ci/secure-test-project-orchestrator/lib/dynamic_jobs/dynamic_jobs.rb:44:in `map'
from /builds/gitlab-org/quality/ci/secure-test-project-orchestrator/lib/dynamic_jobs/dynamic_jobs.rb:44:in `project_branch_details'
from /builds/gitlab-org/quality/ci/secure-test-project-orchestrator/lib/dynamic_jobs/dynamic_jobs.rb:35:in `block in api_all_branches'
from /builds/gitlab-org/quality/ci/secure-test-project-orchestrator/lib/dynamic_jobs/dynamic_jobs.rb:34:in `each'
from /builds/gitlab-org/quality/ci/secure-test-project-orchestrator/lib/dynamic_jobs/dynamic_jobs.rb:34:in `api_all_branches'
from /builds/gitlab-org/quality/ci/secure-test-project-orchestrator/lib/dynamic_jobs/dynamic_jobs.rb:10:in `generate_dynamic_ci'
Possible fixes
Change dynamic_jobs.rb to make it more robust. It shouldn't break when the API doesn't return a JSON array of branches.
/cc @gonzoyumo @twoodham