Settle the caproni external reachability gate on a live instance

What this does

caproni, the default base, has no external reachability gate on main: both wait_external_http call sites sit behind [[ "${base}" == "omnibus" ]]. So every caproni check runs inside the VM or over SSH, and nothing asserts what an operator actually sees. This extends the gate to caproni and settles it on a live instance.

caproni answers on a hostname through the k3s ingress, so the probe asks for the ingress host with --resolve <host>:80:<ip> rather than hitting the bare IP. A bare-IP request does reach nginx and comes back 404 from the default backend: reachable, and a false negative for an omnibus-shaped check.

Verified on a live caproni instance

  • Both call sites returned external HTTP 200 after 0s during a real up.
  • Negative control: a higher-priority GCE ingress deny closed the path, and the gate failed and terminated with HTTP 000 rather than hanging. Restoring the path made it pass again. Polarity demonstrated both ways.
  • The omnibus probe is byte-identical to main, same argv and same log line, diffed under a stubbed curl.

The negative control in the original commit does not work, and that is the finding

That commit proposed "removing http/https from firewalld makes the gate fail rather than hang". On caproni it does not. Removing both services left the gate returning 200.

Nothing owns :80 in the host netns on this base. The request is DNAT'd in PREROUTING by the k8s and CNI-HOSTPORT-DNAT chains and forwarded to a pod, so it never traverses the INPUT path firewalld's http/https services filter, and the k8s chains sit at INPUT positions 1 to 4 ahead of firewalld's regardless. The enforcement point for traffic arriving from outside is the GCE rule, and a deny there took 39 seconds to become effective, consistent with the propagation window recorded for the airgap seal.

Two consequences, both fixed here:

  • The gate was giving caproni operators a false lead. Its failure text named firewalld as the second suspect, which cannot be the cause on this base. It now says so, and notes that an empty ss -lntp | grep :80 is normal on caproni rather than a fault. gce.md's troubleshooting table gains the same distinction.
  • Elapsed time was half the truth. waited counts only the sleeps, so when every probe also burns its full -m 10 connect timeout the loop reported "after 120s" having spent 240s. All three messages now report real wall clock.

Testing

just ci green. vale checked locally against the cached GitLab styles at parity with origin/main, because the CI vale job cannot run: it fails on HTTP 429 fetching its own style rules and, sitting in .pre as a hard gate, it skips static behind it. That is a repo-wide problem worth fixing separately.

The VM was torn down.

Merge request reports

Loading
Loading