feat(df): add glab df go

Description

Adds glab dependency-firewall go. It runs go with every module zip download checked against the project's Dependency Firewall policy, through the loopback module proxy from !4168 (merged). The server accepts the golang ecosystem since !4167 (merged), and !4178 (merged) makes a stalled policy API fail closed after 60 seconds instead of hanging the gate.

How a run works

  1. The engine's optional setup step now gets what a sidecar server needs (the policy checker, project ID, resolved go binary, executor and IO), and returns env, a stop function and its verdicts. Run calls stop exactly once, after go exits and the MITM proxy stops but before verdicts are read, so a verdict recorded while draining still reaches the CI log. Yarn's plugin setup moves to the new shape unchanged.
  2. The Go manager reads the user's effective settings with go env -C <empty dir> -json under GOTOOLCHAIN=local, so the go env file is honoured and no toolchain is downloaded.
  3. It starts the module proxy and points go at it with GOPROXY=<loopback URL with per-run token>.

What the child go gets, and why

Setting Value Why
GOPROXY the server, or off No ,direct and no |, so nothing can fall through to another source.
GONOPROXY none An empty value falls back to the go env file, where a GOPRIVATE would send private modules straight to git.
GO111MODULE on GOPATH-mode go get clones over git.
GIT_HTTP_USER_AGENT empty go skips toolchain checksum checks when it mentions proxy.golang.org.
GONOSUMDB user's Private module paths stay off the public checksum database.
GOTOOLCHAIN user's A toolchain line works. Toolchain zips are exempt from policy and still checksum-verified.

The fetcher behind the gate gets the user's GONOPROXY, GONOSUMDB, GOPRIVATE, GOSUMDB and GOINSECURE, so private modules are checked, then fetched the way they would be without glab.

GOPROXY=off is kept and starts no server. Any other non-default GOPROXY is refused before go starts, without echoing the value (it can carry credentials), until the next MR makes the user's proxy the upstream.

A run that checked no module zips prints a notice on stderr, for example a warm module cache, vendored code, or GOPROXY=off.

The Go manager keeps the engine's MITM wiring with a matcher that matches nothing, because no module traffic reaches it. The newBackend field on the manager is the test seam for a fixture backend: it's a struct field, not a package-level variable, so composed tests run in parallel without the network.

Known limits, for the next MR

  • A GOSUMDB with an explicit URL (including the sum.golang.google.cn alias) makes go contact that URL directly, through the MITM proxy. On macOS with go older than 1.27 that lookup fails TLS. It fails closed.
  • Version-manager shims (asdf, mise) choose the Go version from the working directory, and the fetcher runs from a temp directory.

How has this been tested?

  • Unit and composed tests. The composed tests drive pm.Run with the real go against a fixture module proxy and the fake policy checker. They cover:
    • the child environment
    • GOPRIVATE set only in the go env file
    • inherited GOFLAGS and GONOPROXY
    • GOPROXY=off
    • a refused custom GOPROXY (credentials never printed)
    • allowed and blocked modules
    • zero requests to the MITM proxy
    • stop ordering
    • the nothing-checked notice
  • Real glab df go with the fake checker, on macOS (go 1.27.3) and in golang:1.27-bookworm on Linux arm64:
Case Result
go mod download rsc.io/quote@v1.5.2 downloaded
rsc.io/sampler@v1.3.0 blocked go fails with 403 … blocked by the GitLab Dependency Firewall and the reason; summary box; exit 1
GOPRIVATE=github.com/pkg/errors, allowed version fetched over git after the check
same, blocked version refused with the reason
GOPROXY=off module lookup disabled by GOPROXY=off, plus the nothing-checked notice
GOPROXY=https://user:secret@corp.example refused before go starts; neither the host nor the secret is printed
go env GOPROXY (no download) nothing-checked notice
Ctrl-C during k8s.io/kubernetes@v1.31.0 exit 130 about 1s after the signal; no temp trees or processes left
  • Real glab df go mod download against GDK (gitlab-org/df-test, real policy API), built from this branch:
Module Result
golang.org/x/net@v0.0.0-20181017193950-04a2e542c03f blocked, exit 1
github.com/pkg/errors@v0.9.1 downloaded, exit 0
github.com/Masterminds/goutils@v1.1.0 blocked, exit 1: go shows "violates 'DepFw - Block malicious packages' policy: vulnerability count threshold exceeded (critical: 2 of 0)"; ci-log.json records github.com/Masterminds/goutils blocked with status 403

Each zip made one evaluate call (HTTP 200) with the module path's case kept, and no glab-df-gomodproxy-* directory was left behind.

  • Top-10K module soak against GDK with this branch's build (d614e06a5, go 1.27.3): pass. Of 10,000 modules, 9,795 downloaded, 137 were blocked by policy, 56 don't exist upstream and 12 had no version to request. Nothing failed closed, nothing was missing and no result was a glab bug. Every per-batch invariant held, the canary was blocked in all 11 batches it ran in, and all 5 Ctrl-C runs passed. Batches took 4.6h in total, against 4.4h for the !3910 (closed) PoC.

  • Code review covered correctness, project standards, testing, maintainability, security, reliability, agent-native and adversarial lenses. It returned no actionable findings; the advisory items are recorded above.

Edited by Arpit Gogia

Merge request reports

Loading
Loading