Skip to content

option to redirect GitLab Runner job output (stdout/stderr & files/artefacts) to the local storage

Proposal

Have an option to redirect GitLab Runner job output (stdout/stderr & any produced files/artefacts) to the local storage.

This would be to ensure data residency within restricted network zones to facilitate GitLab Continuous Deployment.

Example Context:
Everyone has restricted zones. There are especially important in financial institutions, where data from one jurisdictional zone (eg. Swiss/HR/Chinese or other jurisdictional data) should never leave the zone.

Our Gitlab installation runs in an unrestricted (green) zone. Gitlab runners pick up jobs and return job output to Gitlab. We need a Gitlab runner that we can install in restricted zones where data is not meant to leave the restricted zone.

The immediate required capability is therefore for data that is held in such restricted zones not to leave these restricted zones. This implies that runners must be installed inside those restricted zones (or in a zone authorized to cover those restricted zones) and that data in restricted zones must not be fed back to the master process in the green zone. One way of preventing such data flow would be by redirecting any job output (stdout/stderr & any produced files/artefacts) to the local storage on the runner (running in the restricted zone).

Solution

Introduce a configuration switch in GitLab Running, similar to -disableloguploads , which is available in Azure DevOps Agent. Basically, if the switch is set:

  • redirects the stdout/stderr (that would otherwise be sent back) to a file in a local directory, named after the unique job id
  • redirects any files the job produces (that would otherwise be sent back to Gitlab) to the same directory, named after the unique job id

 The success criteria are, using a simple test pipeline you specifically direct to that runner:

  • No stdout/stderr from any of the jobs is returned to Gitlab and instead exclusively stored in a directory named after the unique job id on the machine hosting the runner
    It is acceptable for the generic output of the runner initialising some job to be returned. Such output is not controllable from the pipeline.
  • If the job was successful, a simple ‘PASS’ should be returned back to Gitlab instead.
    If the job was unsuccessful, it should be a ‘FAIL’. And exclusively that.
  • Having the test pipeline declare the production of some file artefact in some job (e.g. via “echo test > artefact.txt” and then the usual artefacts: … paths: artefact.txt), artefact.txt is not returned back to Gitlab and instead stored exclusively in a local directory on the machine hosting the runner.