fix(df): match Maven packages served under a repository path prefix

Description

Dependencies fetched through a mirror or proxy repository (Nexus, Artifactory, GitLab's virtual registry) pass the Dependency Firewall unchecked. The mirror's path segments get folded into the parsed package name, so a rule written for the real package never matches.

This MR also checks every reading of the path that drops leading segments, so the real package is checked behind any mirror prefix of up to 10 segments. Gradle shares this matcher and is fixed by the same change.

flowchart LR
    A["/repository/public/org/slf4j/slf4j-api/2.0.9/…jar"] --> B(["as parsed, then shorter readings, shortest first"])
    B --> C0["repository.public.org.slf4j:slf4j-api: allowed"]
    B --> C1["slf4j:slf4j-api: allowed"]
    B --> C2["org.slf4j:slf4j-api: BLOCKED"]
    C2 --> D["request blocked, verdict names org.slf4j:slf4j-api"]
    Z["/maven2/org/slf4j/slf4j-api/2.0.9/…jar"] --> Y["exact Central layout: one check, no readings"]
  • A block on any candidate wins and stops the loop. Otherwise the first warning wins, and the request is allowed only when every candidate is allowed.
  • Over-blocking behind a mirror is deliberate: a block on any reading decides the request, even when that reading isn't the real package. A rule on foo:bar also blocks /repository/x/com/foo/bar/…, and the verdict names foo:bar. Shaded copies (com.vendor.org.slf4j:slf4j-api under a rule on org.slf4j:slf4j-api) are one case of this. It's the cost of enumerating, and https://gitlab.com/gitlab-org/gitlab/-/work_items/638414+s narrows it for repositories we can recognise.
  • The exact Central layout (leading /maven2/) gets no readings, because there's no prefix to recover. Readings there would let a rule on a relocated group (dom4j:dom4j) block its successor (org.dom4j:dom4j), which main allows. This assumes no mirror serves a prefixed layout under a literal leading /maven2/.
  • A reading is kept if it has at most 10 segments or drops at most 10 leading segments (mavenMaxGroupDepth), so there are at most 20. Any group behind a prefix of up to 10 segments is covered, however long the group is. A path where both the prefix and the group exceed 10 segments is not.
  • An allowed artifact behind a mirror costs one policy call per segment of the parsed group (prefix plus group), up to 21, and Central stays at one. The calls run one after another under one shared policy.CheckTimeout (60s), so on the deepest paths a backend slower than about 3s a call runs out the budget and fails closed. Concurrent checks would lose the early return on a block, so this MR keeps them serial. CachingChecker dedupes within a run.
  • A failed policy call fails closed, and the verdict names the requested package instead of whichever reading was being checked (policy.Result.CheckFailed, set by CachingChecker).

Independent of !4289 (closed) in code, but this should merge first. !4289 (closed) makes a caller's mirror take effect, which is what exposes this bug: while the mirror is discarded, requests come from Central and parse correctly. !4310 (merged) combines both for testing.

Relates to gitlab-org/gitlab#637331 (repository path prefix). Follow-up: #8715 (closed).

How has this been tested?

End to end with a real mvn 3.9.12 and the fake policy checker (GLAB_DF_FAKE_BLOCK), fresh local repository each run:

Run main this MR
-Pproxied (/repository/central mirror), block org.apache.logging.log4j:log4j-core nothing blocked org.apache.logging.log4j:log4j-core 2.14.1 blocked
Central, block dom4j:dom4j@2.1.4, fetch org.dom4j:dom4j 2.1.4 not blocked not blocked (921de61e, before the Central change, blocked it)
New tests
  • TestMavenMirrorCandidates: candidate lists for no prefix, a Nexus-style prefix, the GitLab virtual registry, a doubled slash, the exact Central layout (no readings), a group longer than the cap (exactly 20 readings), and an 11-segment group behind a prefix (its real name is a candidate).
  • TestMavenMirrorPolicyMatching: the real package blocked behind a mirror; no match; a warning followed by a block (block wins); only readings warn (first wins); a check error, and a check failure behind CachingChecker (verdict names the requested package); a dom4j:dom4j rule not blocking org.dom4j:dom4j on Central; all candidates sharing one deadline.
  • TestMavenMirrorThroughTunnel: block, warning and check failure through the real proxy tunnel and CachingChecker, asserting the status code and the verdict's package name.
  • TestCacheCheckFailedOnlyMarksFailures: CheckFailed is set on a check error and never on a real block.

Commands: go test -race ./internal/dependencyfirewall/..., golangci-lint run ./internal/dependencyfirewall/..., go run ./scripts/comment-overlap, lefthook run pre-push.

Screenshots (if appropriate):

N/A

Edited by Hannah Baker

Merge request reports

Loading
Loading