Make the airgap check in the QA runner able to fail
What does this MR do and why?
The airgapped E2E job checks that the runner is isolated after it registers it. That check has never been able to fail: it passes whether or not the runner reaches the internet. This MR makes it fail the job when a probe connects, which is the follow-up promised in gitlab-qa!1477 (merged) for the problem described in gitlab-qa#741.
- Each probe was written as
probe && (exit 1) || (exit 0). When the probe connects,(exit 1)ends only its own subshell, so the||branch runs and the check exits 0. Each probe is now anif, and one that connects exits 1. - The wget probe passed GNU-only flags such as
--retry-connrefused. The defaultgitlab-runner:alpineimage has busybox wget, which rejects them withunrecognized option, so the probe never sent a packet. It now uses-q -T 10 -O /dev/null, which busybox and GNU wget both accept.
flowchart TD
A["Runner container is up"] --> B{"Does registry.gitlab.com resolve where the tests run?"}
B -- no --> C["Nothing is probed, job continues"]
B -- yes --> D{"Does netcat connect to port 80?"}
D -- yes --> E["Job fails"]
D -- no --> F{"Does wget fetch the registry page?"}
F -- yes --> E
F -- no --> G["Job continues"]Before this change both "yes" branches also ended in "Job continues".
Review notes
- Only a probe that connects fails the job. A timeout, a refused connection or a failed lookup passes, as it did before, so a flaky resolver cannot fail the job. @john.mcdonnell said leak checks that can actually fail would be a clear improvement in gitlab-qa#741 (comment 4019937419), and this keeps the job from failing for any other reason.
- The
airgappedscenario creates the runner network with--internal(https://gitlab.com/gitlab-org/gitlab-qa/-/blob/5048bbbca1b33055b4719e5cc3afe3d9dd6f0107/lib/gitlab/qa/component/base.rb#L99-109), where both probes fail withNetwork unreachableorbad address. - The wget probe now runs, so a check on an internal network takes about 5 seconds longer, spent on the name lookup. On a network that drops packets without answering, it takes 20 seconds instead of 10.
- The early return when
registry.gitlab.comdoes not resolve is unchanged. qa/Gemfile.lockis untouched. The gitlab-qa probes from the MR above will reach this project'sairgappedjob separately, after a gem release.
Screenshots or screen recordings
Output of the command prove_airgap builds, run with docker exec in registry.gitlab.com/gitlab-org/gitlab-runner:alpine on Docker 29.9.3:
| Runner network | Before | After |
|---|---|---|
--internal, as the scenario creates it |
passes | passes |
| Packets dropped without an answer | passes after 10 seconds | passes after 20 seconds |
| Internet access | passes, with Airgapped network faulty in the output |
fails with Airgapped network faulty. Connectivity netcat check failed. |
With netcat pointed at an unroutable address, so that only wget can see the leak, a network with internet access passes with the old wget flags and fails with the new ones.
How to set up and validate locally
- Run
cd qa && bundle exec rspec --options .rspec_internal spec/service/docker_run/gitlab_runner_spec.rb. The new#prove_airgapexamples run the generated command with stubncandwgetprograms. Thewgetstub rejects long options, as busybox does for--retry-connrefused. - Replace
qa/qa/service/docker_run/gitlab_runner.rbwith the version on master and run step 1 again. The examples for a connecting netcat and a fetching wget fail.
An AI coding assistant wrote this change, ran the network arms above on a test VM, and ran the specs and RuboCop.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.