Skip to content

Check for pipeline refs during pipeline jobs

What does this MR do and why?

When a job is being run against a secondary via gitlab-runner, compare the existence of refs/pipelines refs on primary and secondary, and redirect to primary if any differences.

Related to #415179 (closed)

How to set up and validate locally

  1. Install the package from this merge request into a test Geo installation using
    1. Unified URL
    2. Runner in the same region as the secondary
  2. Create a small test project with a CI pipeline.
    1. Example:
    dnstest:
    variables:
      GIT_TRACE: 1
      GIT_CURL_VERBOSE: 1
    stage: build
    image: registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3
    script:
      - dig ${CI_SERVER_HOST}
  3. With the instance healthy
    1. run the job
      1. The job should not error
      2. The job should call GET on standard git paths (provided there are not significant delays between primary and secondary)
      3. The job should query the /geo/project-X/pipeline_refs API on the primary node
    2. Run GIT_TRACE=1 GIT_CURL_VERBOSE=1 git pull from the command line where it will hit the secondary instance http(s) url (can be run from the secondary node, or against the secondary internal url).
      1. The command should not error
      2. The command should call GET against standard git paths
      3. The secondary should not query the /geo/project-X/pipeline_refs API on the primary node
  4. Stop sidekiq on the secondary instance
    1. Run the job
      1. The job should not error
      2. The job should call GET on the push_from_secondary path to proxy the request to the primary
      3. The job should query the /geo/project-X/pipeline_refs API on the primary node
    2. Run GIT_TRACE=1 GIT_CURL_VERBOSE=1 git pull from the command line where it will hit the secondary instance http(s) url (can be run from the secondary node, or against the secondary internal url).
      1. The command should not error
      2. The command should call GET against standard git paths
      3. The secondary should not query the /geo/project-X/pipeline_refs API on the primary node

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Ian Baum

Merge request reports