Check for pipeline refs during pipeline jobs
What does this MR do and why?
When a job is being run against a secondary via gitlab-runner, compare the existence of refs/pipelines refs on primary and secondary, and redirect to primary if any differences.
Related to #415179 (closed)
How to set up and validate locally
- Install the package from this merge request into a test Geo installation using
- Unified URL
- Runner in the same region as the secondary
- Create a small test project with a CI pipeline.
- Example:
dnstest: variables: GIT_TRACE: 1 GIT_CURL_VERBOSE: 1 stage: build image: registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3 script: - dig ${CI_SERVER_HOST} - With the instance healthy
- run the job
- The job should not error
- The job should call
GETon standard git paths (provided there are not significant delays between primary and secondary) - The job should query the
/geo/project-X/pipeline_refsAPI on the primary node
- Run
GIT_TRACE=1 GIT_CURL_VERBOSE=1 git pullfrom the command line where it will hit the secondary instance http(s) url (can be run from the secondary node, or against the secondary internal url).- The command should not error
- The command should call
GETagainst standard git paths - The secondary should not query the
/geo/project-X/pipeline_refsAPI on the primary node
- run the job
- Stop sidekiq on the secondary instance
- Run the job
- The job should not error
- The job should call
GETon thepush_from_secondarypath to proxy the request to the primary - The job should query the
/geo/project-X/pipeline_refsAPI on the primary node
- Run
GIT_TRACE=1 GIT_CURL_VERBOSE=1 git pullfrom the command line where it will hit the secondary instance http(s) url (can be run from the secondary node, or against the secondary internal url).- The command should not error
- The command should call
GETagainst standard git paths - The secondary should not query the
/geo/project-X/pipeline_refsAPI on the primary node
- Run the job
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Ian Baum