CI: build-package-and-qa triggers omnibus master for stable-branch MRs, breaking FIPS builds

Summary

Gitaly's build-package-and-qa CI job triggers the omnibus-gitlab-mirror pipeline without specifying a branch, so it always runs against omnibus's default branch (master). For a merge request targeting a stable branch (e.g. 19-3-stable), this packages the stable Gitaly on omnibus master's builder image instead of the matching stable builder. The two can use different Go toolchains, which breaks the FIPS build.

Where

.gitlab-ci.yml, build-package-and-qa:

build-package-and-qa:
  trigger:
    project: gitlab-org/build/omnibus-gitlab-mirror
    strategy: depend          # no `branch:` -> omnibus default branch (master)
  variables:
    ALTERNATIVE_SOURCES: "true"
    GITALY_SERVER_VERSION: $CI_COMMIT_SHA
    GITALY_SERVER_ALTERNATIVE_REPO: $CI_PROJECT_URL
  rules:
    - if: $CI_MERGE_REQUEST_IID
      changes:
        - go.sum
      allow_failure: false

The job fires on go.sum changes and is blocking (allow_failure: false).

How it surfaced

A backport that bumps a dependency (so go.sum changes) on 19-3-stable triggered this job. It ran the omnibus master pipeline, which uses the builder image registry.gitlab.com/gitlab-org/gitlab-omnibus-builder/ubuntu_22.04_fips:5.73.0 (Go 1.27.3). The stable Gitaly's FIPS build expects the Go 1.26 toolchain (GOFIPS140=off, boringcrypto). Building it on the 1.27.3 (native-FIPS) builder fails:

package gitlab.com/gitlab-org/gitaly/v18/cmd/gitaly
    imports gitlab.com/gitlab-org/gitaly/v18/internal/cli/gitaly
    imports gitlab.com/gitlab-org/labkit/fips
    imports crypto/boring: build constraints exclude all Go files in /usr/local/go/src/crypto/boring

Gitaly's Makefile sets GOEXPERIMENT=boringcrypto when go env GOFIPS140 is off and the probe GOEXPERIMENT=boringcrypto go version succeeds. On Go 1.27.3 that probe still succeeds, so boringcrypto is forced, but the toolchain no longer ships a buildable crypto/boring. The other Go components (gitlab-shell, gitlab-pages, gitlab-kas, workhorse) build FIPS fine on the same image because they do not force the experiment.

Example failing jobs (omnibus ref: master, triggered with GITALY_SERVER_VERSION set to a 19-3-stable merge SHA):

Root cause

The trigger has no branch:, so stable-branch Gitaly MRs are packaged on the omnibus master builder. This is latent: it only bites once the master builder's Go toolchain diverges from what the stable Gitaly expects, which happened when the FIPS builder moved to Go 1.27.3 (native FIPS). Stable release pipelines are unaffected because they build on the matching omnibus stable branch.

Proposed fix

Pin the downstream branch to the MR's target branch so stable MRs package on the matching omnibus stable branch:

  trigger:
    project: gitlab-org/build/omnibus-gitlab-mirror
    branch: $OMNIBUS_BRANCH
    strategy: depend
  variables:
    OMNIBUS_BRANCH: "master"
  rules:
    - if: $CI_MERGE_REQUEST_IID
      changes:
        - go.sum
      variables:
        OMNIBUS_BRANCH: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
      allow_failure: false

19-3-stable -> omnibus 19-3-stable (Go 1.26 builder); master -> omnibus master. All of 19-3-stable, 19-4-stable, 19-5-stable, and master exist in omnibus-gitlab-mirror.

Fix on master, then backport to the active stable branches.

Note on native FIPS

This is not a Gitaly FIPS defect. Native Go FIPS 140-3 (GOFIPS140) toolchains are already supported and verified: #7305 (closed) landed (Makefile guards boringcrypto behind GOFIPS140 being empty/off, labkit bumped to a native-capable release) and the native -tags fips build of gitaly/praefect was confirmed green on 2026-08-20 (CNG FIPS pipeline, gitaly master + labkit v1.64.11).

So on a native Go 1.27 builder with GOFIPS140 set to the native module, the FIPS binaries build via that module; on a Go 1.26 builder with GOFIPS140=off they build via boringcrypto. The failure here is only that a stable-branch MR was packaged on the omnibus master builder, whose toolchain no longer matches what the stable Gitaly is built for (Go 1.26 / boringcrypto). Routing to the matching omnibus stable branch resolves it. If omnibus's own Go 1.27 FIPS builder does not set GOFIPS140, the boringcrypto fallback will fail there too, but that is an omnibus build-env matter, separate from this trigger fix.

Edited by Stan Hu