build: pin the Duo CLI binary by sha256 and parse the 9.x JSON contract (!4) · Merge requests · GitLab.com / Public Sector / STIG Analyzer

Summary

Restores a working GitLab Duo CLI path on the absorbed base, in two halves that have to land together:

Supply chain. Pin the Duo CLI compiled binary directly (exact version, SHA256-verified, hard-fail on mismatch at build) instead of routing through glab duo cli run.
Output contract. Adopt duo run --output-format json as the parse contract at every call site, because 9.x changed the output shape the analyzer parsed, and extend the fail-loudly guard so a contract break turns the pipeline red instead of degrading to silent Not_Reviewed verdicts.

Target branch master. !6 (merged) (the 1.2.4 absorb) has merged, so this no longer stacks on anything.

Why: the absorbed mechanism does not run, and the pin alone would not fix it

The 1.2.4 absorb (!6 (merged)) pulled in a fork-parent commit (fix(duo): migrate to glab-managed Duo CLI binary, 2026-06-23) that switched the analyzer from the npm @gitlab/duo-cli package to a glab-managed compiled binary, reasoning that the npm CLI failed GitLab's instance-version handshake in CI.

Two things are true about that mechanism as absorbed:

glab 1.105.0 gates glab duo cli run on Duo CLI major 8. It resolves the package registry's latest release, major 9, and refuses to drive it. The image's pre-downloaded 8.107.0 binary is consequently never invoked.
The build's own probe for this (glab duo cli version || echo "note: ...") is ||-swallowed, so the mismatch never fails the build. It just prints a note that nothing reads.

A live check against gitlab.com on 2026-08-12 shows 9.10.0 authenticates without issue; the June diagnosis that this CLI cannot authenticate did not reproduce.

The part a version pin alone gets wrong. Between 8.107.0 and 9.10.0 the CLI changed its non-interactive output contract:

all logging moved to stderr;
the per-message [RunController] logging is gone from the binary, not merely relocated;
text mode now writes only the bare final response text to stdout.

parse_gitlab_duo_output() required a literal [RunController] marker and brace-counted from it, so against real 9.10.0 output it raises. That error does not fail the pipeline: it becomes a per-rule Not_Reviewed verdict under a green pipeline — the exact failure mode this project's guards exist to prevent, reintroduced by the fix meant to remove it. The existing tests passed only because their fixtures encoded the dead 8.x text contract, so they proved nothing about the pinned binary.

--output-format json first shipped in 8.110.0, after the old pin. 9.10.0 has it, and its help text states the contract outright: "json emits a single JSON document to stdout and routes logs to stderr."

Approach

Supply chain

Download the compiled binary directly, no glab coupling. duo is pulled from the same GitLab Package Registry endpoint the absorbed Dockerfile already used (gitlab.com/api/v4/projects/46519181/packages/generic/duo-cli/..., the gitlab-org/editor-extensions/gitlab-lsp project), pinned to 9.10.0, SHA256-verified, and installed straight to /usr/local/bin/duo. No glab-managed download, no npm, no Node involved in installing it. Uses the base duo-linux-x64, not -modern: the AVX2-optimized build can SIGILL on runners whose CPUs lack AVX2, and we don't control which runner executes.
Hard build-time check, not a swallowed probe. duo --version is the last statement in the RUN, with no || anywhere in the chain: a wrong, corrupted, or unrunnable binary fails the image build.
glab narrows to one job. It stays in the image, pinned, solely to back the token-liveness probe in verify_duo_creds.sh (glab api user). glab config set --global duo_cli_auto_run/auto_download is removed so glab can never manage or overwrite the pinned binary — PATH-shadowing eliminated, not merely discouraged.
Exact pin, deliberately. gitlab-lsp ships one semantic version across its whole monorepo per commit, and v8.84.0 shipped a documented BREAKING CHANGES entry (removal of --connection-type) while bumping only minor (changelog). The Dockerfile comment now says the pin is an output-contract pin as much as a supply-chain one, and names the three places a bump must be re-checked against.
Renovate actually tracks the pin. ARG DUO_CLI_VERSION carries # renovate: datasource=gitlab-packages depName=gitlab-org/editor-extensions/gitlab-lsp:duo-cli versioning=semver. The binary is a Package Registry generic package, so gitlab-packages (packageName in project/path:package-name form) is the correct datasource, not gitlab-releases/gitlab-tags, which track releases and tags rather than package-registry contents. An annotation alone is inert here — the shared preset (gitlab>gitlab-com/public-sector/pipeline:renovate-config) scopes its ARG *_VERSION manager to containers/*.Containerfile — so this MR carries the renovate.json customManagers in with it, absorbed from !4 (merged) (see Related):
- one manager for both ARG pins in stig_tools/Dockerfile, widening !4 (merged)'s regex to a (GKG|DUO_CLI)_VERSION alternation with an optional versioning= capture, which is what lets the Duo CLI pin declare semver over a datasource whose raw file names are not semver-ordered. ARG GKG_VERSION gains the gitlab-releases annotation it needs, carried from !4 (merged) unchanged;
- one for the npm install --global @gitlab/duo-cli@<version> line in the sample application's notify:mac-gun job (app/.gitlab-ci.yml), carried from !4 (merged) along with its pin — that line was floating unpinned on master;
- a packageRule holding both tools at automerge: false, so a bump is always a merge request a maintainer reads;
- a PR note on Duo CLI bumps specifically, because DUO_CLI_SHA256 has to be recomputed by hand (Renovate cannot digest a generic Package Registry file), and because the output contract must be re-checked against both parsers and the canary. A version-only bump fails the build at sha256sum -c by design.
All three regexes were run against the real files to confirm they match: DUO_CLI_VERSION=9.10.0 (with versioning=semver captured), GKG_VERSION=v0.25.0, and @gitlab/duo-cli@9.10.0.

Output contract

One contract, everywhere. Every Duo CLI invocation in the repo passes --output-format json and parses the CLI's result document: schemaVersion "1.0", sessionId, exitCode, response, elements, status, plus error when status is "error".
parse_gitlab_duo_output() rewritten to that contract. json.loads(stdout), take response. The existing markdown-code-fence stripping is kept and applied to that response text, factored out as strip_markdown_fence() so it is separately testable. No text-mode fallback at all — the shape a fallback would guess at no longer exists in the binary, and guessing is how a contract break becomes a green pipeline full of unreviewed rules.
Fail-loudly guard extended. Until now only a rejected token aborted the run. Non-JSON stdout, or a document without a string response, now raises DuoOutputContractError, is flagged parse_contract_failed through call_gitlab_duo() and analyze_rule_with_ai(), and aborts main() with a diagnostic pointing at DUO_CLI_VERSION. A status: "error" document is a failed run, not a broken contract, so it stays an ordinary per-rule error — as does an empty response.
All three remaining call sites converted, the ones the previous draft of this description wrongly dismissed as "comments":
- stig_tools/remediate.py run_duo() — reports a stdout that is not the expected document as a failure, which aborts that finding before any branch, commit or merge request is created. (The protection is that no MR is opened at all; build_mr_body() never renders the Duo summary it is handed.) Missing or non-string response both fail, matching the analyzer. A non-zero exit reports stderr first, labelled and separated from stdout, since that is where the diagnostics live now.
- test_gkg_duo_mcp (.gitlab-ci.yml) and stig::gkg-smoke-test (templates/stig-compliance-analysis.yml) — both keep the streams separate rather than merging with 2>&1: the JSON document to stdout, and the [MCP Manager] knowledge-graph: Tool executed: ... lines the assertions grep for to stderr, where logging moved. Merging would leave the document interleaved with logs and unparseable. Both jobs additionally assert the contract against the real pinned binary, and keep the stdout document as an artifact.
The canary repointed, not left asserting a fiction. !5 (merged) landed duo_cli_canary while this branch was in flight. It asserted the [RunController] text shape as "TODAY'S CONTRACT" and JSON mode as a "migration target ... not yet consumed by the analyzer". Both labels invert here, and the text assertion would now fail on master — the only ref carrying GITLAB_DUO_TOKEN, hence the only place its live assertions run. The text assertion is dropped; the JSON one is promoted to the live contract and strengthened from "stdout parses as JSON" to "stdout is one object carrying schemaVersion, status, and a string response", the fields the parsers actually read.

Verified

Schema derived from the artifact, not assumed. A live run was attempted first with the supplied project access token: it authenticated (/api/v4/personal_access_tokens/self → 200) but the run aborted at initialization because the token carries only the ai_features scope and the CLI's getUser GraphQL query needs more, so stdout was empty and no document was captured. The schema therefore comes from binary inspection: the embedded RunResultWriter and its zod schema in the downloaded duo-linux-x64, which builds {schemaVersion, sessionId, exitCode, response, elements} and adds status: "success" or status: "error" + error, then JSON.stringify(doc, null, 2) to process.stdout. response is the content of the last elements[] entry with type: "message", role: "assistant", isComplete: true, or "" when there is none. Element types come from the binary's own enum ({MESSAGE, TOOL, ERROR, INFO, SKILL_CONFLICTS}). That failed run did independently confirm the review's other finding: every log line went to stderr, stdout stayed empty.
URL and hash, independently. Downloaded https://gitlab.com/api/v4/projects/46519181/packages/generic/duo-cli/9.10.0/duo-linux-x64 (HTTP 200, 103290216 bytes) and computed its SHA256: 77d35130174a87126340acce38101bdf53213b741103d492971f23bd50558dc9, matching the pin. file reports a valid ELF 64-bit x86-64 executable.
--output-format json behaviour, from the CLI itself. duo run --help at the pin documents it as emitting "a single JSON document to stdout" and routing "logs to stderr". The committed canary snapshot (tests/fixtures/duo_run_help_9.10.0.txt, generated from the npm package) matches duo run --help from the downloaded binary byte for byte. That is evidence the two distributions agree on the CLI surface at this pin, not a guarantee that they always will, and not a check of the binary the image ships — that binary's integrity is proven separately by the SHA256 verification and the hard duo --version. The job comment now says exactly that, rather than overclaiming equivalence.
duo --version reports 9.10.0 from the pinned artifact (run on the darwin-arm64 build of the same version; the linux-x64 build cannot execute on this host). The CI pipeline on this MR is the first real-hardware run of the new RUN block.
Tests. Full suite pytest tests/ (equivalently python3 -m unittest discover -s tests -p "test_*.py"): 70 → 108 passing, 0 failures. test_ai_* (the pattern the test_duo_parser CI job discovers): 18 → 46. Every added test exercises the new contract, including the fail-loudly paths: non-JSON stdout (8.x-style log text specifically), empty and whitespace-only stdout, a JSON array, a missing response, a non-string response, and the escalation from parser → call_gitlab_duo() → analyze_rule_with_ai(). The test_duo_parser job now also discovers test_remediat* so the second call site's contract is checked in CI rather than only locally.
CI config linted server-side. glab ci lint on both modified CI files (.gitlab-ci.yml and app/.gitlab-ci.yml): valid. renovate.json parses. Both job scripts were extracted from the YAML and passed sh -n, and the embedded contract-assertion Python was executed against a good document, a malformed one, and one with a non-string response (exit 0 / 1 / 1 as intended).
Refusal detection, against the real traces. The entitlement classifier and the canary's shell alternation are both tested against the verbatim stderr lines above (AgenticChatForbiddenError ... and the upstream 404 Namespace Not Found), asserting they classify as ENTITLEMENT, that invalid_token still classifies distinctly and wins when both appear, that unrecognized output (e.g. ECONNRESET) is not treated as a refusal, and that the flag propagates through call_gitlab_duo() and analyze_rule_with_ai() to the abort guard. One test asserts the canary greps for every signature in duo_auth.py, so the shell mirror cannot drift.
No dead contract left in the tree. grep -rn RunController returns only prose explaining why the old shape is gone, plus the one negative test fixture that asserts it must now raise. No live code parses or asserts it.

Known state: pipeline red, and it is not this diff

The pipeline on this branch is red for exactly one reason: the CI bot token has no GitLab Duo Agent Platform entitlement. Admin action is pending. Nothing in this diff is implicated, and no re-run will change it.

The token authenticates cleanly and is then refused by the server. The refusal surfaces on stderr as:

[error]: Failed to verify Agentic Chat access: Failed to create workflow: HTTP 404. 404 Namespace Not Found
[error]: AgenticChatForbiddenError: GitLab Duo Agent Platform is not available for this namespace or project

The first run of this branch handled that badly, and it has been fixed. The fail-fast guard recognized only invalid_token / "Token is invalid or expired", so an entitlement wall fell through to generic per-rule errors and surfaced only at the end, via the all-Not_Reviewed guard — correct outcome, but late and labelled as if the token were dead. The canary was worse: it printed "exited 1 without a recognizable auth error... This may be transient — re-run once" over a permanent refusal.

Both now name it. stig_tools/duo_auth.py holds the signatures (AgenticChatForbiddenError, GitLab Duo Agent Platform is not available, Failed to verify Agentic Chat access, Namespace Not Found, insufficient_scope), used by both Python call sites and mirrored in the canary's shell greps. They take the same fail-fast path as a bad token but report ENTITLEMENT rather than INVALID TOKEN, and say outright that refreshing the token will not help — the fix is a Duo seat for the token's owner, Duo enabled for the namespace/project, and the ai_features scope.

So the failure now reports itself as precisely what it is. What this pipeline still cannot prove is the happy path: no live Duo call has succeeded on this branch, so the JSON result document has been verified against the artifact and in unit tests, but not yet end to end in CI. That waits on the entitlement.

Scope decisions

Base image: left as node:22-slim, out of scope here. The image has had no functional dependency on Node/npm since before this MR: no node/npm/npx invocation anywhere in it, GKG is a Rust binary, and the stig_tools/*.sh/*.py scripts use sh/python3 only. node:22-slim traces back to the project's original design (RUN npm install --global @anthropic-ai/claude-code @gitlab/duo-cli, commit e6f8f0d) and stayed pinned as the base through every later migration. Switching it (to something like python:3.12-slim, matching app/Dockerfile and aws/Dockerfile) touches every apt install in the file and needs its own verification that GKG install, pip installs, and glibc compatibility with the Bun-compiled duo binary hold on the new base. Named as a follow-up. Note the duo_cli_canary job still uses node:22-slim deliberately — it installs the npm package and genuinely needs a Node runtime.
app/'s own Duo usage untouched. app/.gitlab-ci.yml and app/README.md reference duo run / glab duo ask; app/ is the sample application under scan, not this tool's invocation of the CLI.
Token is passed by environment, never on argv. The pre-redraft draft of this branch added --gitlab-auth-token <token> at four call sites. That was new exposure measured against master, whose call sites were env-only: a token in argv is readable from /proc/<pid>/cmdline by anything else in the job container. Removed everywhere; GITLAB_TOKEN is the sole channel. Confirmed from the pinned binary that the two are interchangeable — the option definition is {flags: "--gitlab-auth-token <token>", env: "GITLAB_TOKEN", default: process.env.GITLAB_OAUTH_TOKEN} and the credential provider labels the source "env var / --gitlab-auth-token" — and confirmed by running the artifact with the env var and no flag: the CLI logged [CredentialProvider] Using static token from env var / CLI flag and reached the same authenticated GraphQL call, where with no token at all it falls back to from config file. Tests assert the token reaches the subprocess environment and never argv.

!6 (merged) (!6 (merged)), "chore: absorb upstream 1.2.4 (fork parent)" — merged; this MR targets master directly.
!5 (merged) (!5 (merged)), "ci: add Duo CLI output-contract canary" — merged while this branch was in flight. Reconciled here rather than left conflicting: its "today's contract" and "migration target" labels are inverted by this change, and its text-mode assertion asserts a shape 9.10.0 does not emit.
!3 (merged) (!3 (merged)), "feat: extract CCI identifiers from XCCDF parsing" — merged while this branch was in flight. It added TestExtractRuleCcis to tests/test_ai_parsers.py, the same file this MR rewrites; the rebase keeps both (its CCI class and this MR's JSON-contract classes).
!4 (merged) (!4 (merged)), "build: pin @gitlab/duo-cli to an exact version" — absorbed into this branch, which reworks it in place. It independently reached the same version conclusion (9.10.0, same v8.84.0 breaking-change-in-minor evidence) but through the pre-absorb npm mechanism, which no longer exists in stig_tools/Dockerfile. Its two durable pieces are carried across rather than discarded: the renovate.json customManagers (its ARG GKG_VERSION manager widened to cover DUO_CLI_VERSION too, its npm manager kept for app/) and its pin of the floating npm install --global @gitlab/duo-cli in app/.gitlab-ci.yml. Its stig_tools/Dockerfile npm-install pin is superseded by the direct binary download.

Checklist

Edited Aug 18, 2026 by Andrew Dunn

build: pin the Duo CLI binary by sha256 and parse the 9.x JSON contract