Log the destinations the airgapped scenario blocks
What does this MR do and why?
The airgapped scenario logs how many packets the OUTPUT policy dropped, but not where they were headed. A destination that none of the four probes names can leave the container, and the job log never shows it. @john.mcdonnell said on #741 that the log-only count "gives visibility without making the job flaky". This MR gives the destinations the same treatment.
- Right after the allowlist rules,
airgap_gitlab_commandsappends two rules to theOUTPUTchain. They record the destination of every packet that reaches the policy in a kernelrecentlist, up to 256 packets per destination. - After the specs,
report_dropped_destinationslogs each destination with its packets, the most first. A count at the limit prints as256+. - Nothing here can fail the job. A rule that cannot be added prints a message in the setup output. A list that cannot be read logs a warning, and so does one whose lines do not parse, with the count and the first such line. The line "none recorded" is kept for an empty list.
flowchart TB
subgraph run["From setup through the specs, in the GitLab container"]
direction LR
PK["Outbound packet"] --> AC["ACCEPT rules"]
AC -->|"no match"| R2["New rule: record the destination"]
R2 --> PD["OUTPUT policy DROP"]
end
R2 -.->|"adds the address to"| LS[("airgap_dropped list<br/>/proc/net/xt_recent/airgap_dropped")]
subgraph after["After the specs, in the ensure block of perform"]
direction LR
RP["report_dropped_packets<br/>the policy total, as today"]
RD["New: report_dropped_destinations<br/>one line per destination"]
end
LS -.->|"read through docker exec"| RD
RP --> LOG["Job log lines only: the job result is unchanged"]
RD --> LOGReview notes
- The destination counts matched the policy total in the Docker host run below (869 of 869) and fell 10 short in the CI job (492 of 502). I did not trace the 10. The likely source is packets dropped while the commands between the policy and the tracking rule ran. A destination past the limit, or more than 100 destinations, would also open a gap: the list holds 100 by default and drops the address seen least recently when it is full.
- The list holds addresses, so it cannot split one destination by port or protocol.
- Addresses inside the test network show too, because the rules only allow the praefect node, and the existing total already counts that traffic.
- The first rule only sizes the list. A rule that sets entries cannot carry a hit count, and an unsized list keeps 32 packets per destination (
net/netfilter/xt_recent.cin Linux v6.12, lines 357 to 360 and 372 to 377). On the 5.15 kernel of the GitLab-hosted runners these jobs used, a hit count of 1024 was refused and 255 was accepted, so this MR sizes the list at 256. - 300 packets sent to one address through these two rules listed 256, and 40 sent to another listed 40.
- A
LOGrule would write to the host's kernel log, and only for containers once the host setsnet.netfilter.nf_log_all_netns, which a job cannot count on. The list is read from inside the container instead.
Screenshots or screen recordings
With the rules intact, the scenario ran against gitlab/gitlab-ee:19.4.2-ee.0 on a Docker host, and the 11 smoke examples from five spec files passed. The job output carried the existing lines and the new one:
WARN -- Airgapped finding: gitlab.com port 8080 is reachable
INFO -- Airgapped packets dropped by the OUTPUT policy: policy DROP 869 packets, 59248 bytes
INFO -- Airgapped destinations blocked by the OUTPUT policy, with packets counted (at most 256 per destination)
172.18.0.3: 168
172.18.0.4: 166
172.18.0.5: 166
52.38.134.47: 108
184.34.154.174: 102
184.34.28.19: 89
172.65.251.78: 21
104.18.32.27: 14
104.20.23.154: 14
172.64.155.229: 14
8.8.8.8: 7In Docker-in-Docker 29.9.1 jobs on GitLab-hosted runners the scenario passed the same way, with 502 packets dropped and eleven destinations that account for 492 of them. With tcp port 443 open to every destination the job failed on Airgapped network faulty. 172.65.251.78 port 443 is reachable. in all three attempts and printed no destination line, as before this change. Without the change, the same 11 examples passed and no destination line appeared.
How to set up and validate locally
-
Run
bundle exec rspec spec/gitlab/qa/scenario/test/instance/airgapped_spec.rb. -
From this branch, with
EE_LICENSEset (andGITLAB_LICENSE_MODE=testfor a test license), run the scenario on the image and the five spec files those runs used. Look for theAirgapped destinations blockedline after the dropped packet count.S=qa/specs/features bundle exec exe/gitlab-qa Test::Instance::Airgapped EE:19.4.2-ee.0 -- --tag smoke \ $S/api/9_tenant_scale/users_spec.rb \ $S/api/3_create/repository/add_list_delete_branches_spec.rb \ $S/api/3_create/merge_request/push_options_spec.rb \ $S/browser_ui/4_verify/ci_variable/raw_variables_defined_in_yaml_spec.rb \ $S/api/10_software_supply_chain_security/project_access_token_spec.rb -
To see that a leak still fails the job, add
iptables -I OUTPUT 1 -p tcp --dport 443 -j ACCEPTafter the praefect rule inairgap_gitlab_commandsand run step 2 again.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
- Commits include
Changelog:trailer - I have evaluated the MR acceptance checklist for this MR.
An AI coding assistant wrote this change. The assistant also checked each claim against the Linux v6.12 xt_recent source, airgapped.rb and the CI job logs, and ran RuboCop and the specs on a test VM. It also ran the scenario against GitLab EE 19.4.2 on a test VM and in CI jobs.