## Overview Implement the real build pipeline in bench, replacing the current stub build functions. For M1 this is the **per-language chassis build**: `bench build` produces a deployable OCI image for a single-language service (Go or Rust), wiring in the chassis and native LabKit. That image is bench's opaque-container contract with the rest of Theseus: Fairway schedules and observes it but does not model what runs inside (see the [Theseus Platform Vision](https://gitlab.com/gitlab-com/content-sites/handbook/-/merge_requests/19702), Lab Bench section, and ADR 006). Polyglot single-binary assembly is **deferred** (see the collapsed decision record below). This milestone also lands the scaffolding half of the Theseus CLI → Bench merge: a new `bench scaffold` command (Track A). Scaffolding is the entry to the developer journey and is independent of the build pipeline. ## Must cover - What `bench build` produces: given an assembly's staged services (all one language, per `Assembly.runtime`), generate the assembly entrypoint that composes those services onto the shared language-matched chassis and native LabKit (no FFI), running as one process behind the assembly's inbound listeners, then build it with the native toolchain (`go build` / `cargo build`) into a single binary. bench generates the wiring and drives the toolchain; it does not reimplement the compiler. Composing one-or-more same-language services into one process is the distinguishing work: a single-service assembly is close to an ordinary native build, and the multi-service case is what bench adds. - `bench serve` / `bench assembly serve`: running a built service locally - OCI image structure and tagging convention - The boundary with the paved-path template: `bench scaffold` wraps `common-template-copier`, which generates the Fairway manifest, CI config and CODEOWNERS. bench drives copier; it does not reimplement scaffolding or manifest generation (see Track A and the DM-1 "Note on Fairway vs Runway"). - SLSA L3 attestation plan: - Provenance generation for every OCI image produced by `bench build` - Hermetic builds: no network access during compilation - Build environment defined and pinned (toolchain versions, base images) - Signing and verification strategy for produced artifacts - Integration point with GitLab's hardened builds pipeline - Timeline derived from the program-level SLSA L3 target in the [LabBench SOA doc](https://docs.google.com/document/d/11Zj918LuZeY3fPcU50ZPhzJtcqzvyXaO0SDamW7cDc8) <details> <summary><b>Deferred: polyglot assembly</b> (decision record — LB-D5)</summary> The original design compiled Go / Ruby / Python / Rust into a single binary via C FFI, Magnus, and PyO3 (the Panoptic approach). The Modular GitLab Execution Weekly (2026-06-03) decided LabBench is **language-matched**: a Go chassis for Go services, a Rust chassis for Rust services. Library capabilities come from native LabKit, not FFI (an FFI metric increment benchmarked ~8x slower than native — see [Theseus MR !19702 note](https://gitlab.com/gitlab-com/content-sites/handbook/-/merge_requests/19702#note_3394124361)). Polyglot single-binary assembly (LB-D5) is therefore deferred to post-M1. When revisited, Panoptic's FFI machinery is the starting point and these PoC limitations must be addressed: handler-panic recovery, Go FFI OOB safety, Ruby VM init failure handling, Go signal-handler reset, Ruby worker-pool throughput, Go module isolation. Captured here so the analysis is not lost. Panoptic is a proof-of-concept, not a production blueprint: the transferable idea is build-time service discovery and route wiring; the FFI fusion itself is this deferred polyglot model, not what M1 builds. </details> ## Current state - bench CLI has stub build functions that log and return success - The build pipeline already stages services correctly (local copy or git-clone via `GitRef`); the entrypoint generation and per-language compile steps are the missing pieces ## Definition of done The work splits into four tracks, which together land both halves of the Theseus CLI → Bench merge: scaffolding (Track A) and the build/runtime half (Tracks B and C); the schema/config half is in DM-1. Note the deploy artifact (the Fairway manifest) is produced by the copier template in Track A, not by `bench build`. Scaffolding (Track A) is the entry to the developer journey, is independent of the build, and can start immediately. The two compile backends (Track B) are independent of each other, so staff DM-2c and DM-2d in parallel. Packaging (Track C) needs only one backend to land. Go remains the reference since it's the IAM-adopter language and the DM-3 reference. The SLSA plan (Track D) is a writing task with no code dependency and can start alongside everything else. ### Track A: scaffolding — `bench scaffold` wraps common-template-copier `bench init` only writes a `bench.yaml`. The Theseus CLI → Bench merge brings the scaffolding half of `theseus new` into bench as a separate command: `bench scaffold` bootstraps projects from [`common-template-copier`](https://gitlab.com/gitlab-com/gl-infra/common-template-copier), the paved-path template that already generates the Fairway manifest, CI config, and CODEOWNERS. It runs after `bench check`, reads each assembly's runtime from the validated `bench.yaml`, and scaffolds one project per assembly. Keeping it separate from `bench init` matters because `bench.yaml` is a specification built up after init — at init time the spec is empty, so scaffolding can't read it (see [MR !17](https://gitlab.com/gitlab-org/lab-bench/bench/-/merge_requests/17), the SOA doc §3.4, and LB-D7: copier and the bench CLI are "complementary, not alternatives"). bench wraps copier; it does not reimplement scaffolding or manifest generation (see the DM-1 "Note on Fairway vs Runway"). - [x] DM-2a (MR !17, in review): Add a `bench scaffold --group <gitlab-org/...> [assembly]` command that runs after `bench check` and shells out to `copier copy` against common-template-copier, once per assembly, pre-answering the template inputs (`project_name`, `gitlab_namespace` from `--group`, `golang` from each assembly's `Assembly.runtime`, the standard toggles). `bench init` stays config-only; `--group` is a scaffold-command argument, not a schema field (ownership is scaffold-time metadata in `.copier-answers.yml`). Committed scaffold for now; ephemeral/framework-side invocation (per the [May 28 weekly](https://docs.google.com/document/d/1jtLUUlyVsihK0IlITL29Oj-r8Cwo-gBDZEGT5PS-LH4/edit?tab=t.0#heading=h.ccibu7h7s837)) is a tracked follow-up, not this slice. - [x] https://gitlab.com/gitlab-org/lab-bench/bench/-/work_items/35+ ### Track B: compile backends — generate the assembly entrypoint, build with the native toolchain Per the build contract above, for each backend generate the assembly entrypoint (route/handler wiring) and shell out to the native toolchain to compile it. The backends share only the dispatch contract, so they are independent. **Reference implementation (Go).** `gitlab-org/auth/iam` is a hand-written example of the Go composition DM-2c generates: a registry of service constructors, a `ServiceManager` that runs the selected services in one process, and a thin `cmd/<name>/main.go` per build target. The `data-access` composite (Lookup + Update on a shared gRPC port) is the canonical multi-service case to validate DM-2c against. IAM is a reference to study, not a build target. Full walkthrough and the one contract difference to carry into the generator are in the collapsed note below. <details> <summary>IAM reference walkthrough (DM-2c implementation guidance)</summary> `gitlab-org/auth/iam` composes the same service packages (`auth`, `dataaccess/lookup`, `dataaccess/update`, `sts`) into different binaries through: - a registry of constructors: `service.Create(ctx, name)` in `service/service_factory.go` - a `ServiceManager` that runs the selected services in one process: `service/service_manager.go` - a thin `cmd/<name>/main.go` per build target (each ~15 lines: pick a name, then `Create` + `NewServiceManager` + `RunAll`) The canonical multi-service case to validate DM-2c against is the `data-access` composite: Lookup + Update behind one API on a shared gRPC port, built as `make iam-data-access`, deployed as the `iam-data-access-grpc` Runway target. It's the smallest real one, and a production deploy target rather than a toy. IAM is a reference to study, not a build target. It predates bench and is Copier-scaffolded, so DM-2c validates by building a bench assembly that composes equivalent Go services, not by driving IAM's `Makefile`. **One contract difference to carry into the generator.** IAM composes at the service-owns-its-listener granularity: each `Service` (`Name/Address/Serve/Shutdown/Logger`, see `service/serve.go`) serves its own socket, and `ServiceManager.RunAll` runs one goroutine per service. bench instead mounts handlers onto a shared chassis behind the assembly's declared listeners. Treat IAM's registry-plus-manager shape as the structural template, not a 1:1 lift. </details> - [x] DM-2c: (https://gitlab.com/gitlab-org/lab-bench/bench/-/merge_requests/41) Make `build_service_go` real. Generate a Go assembly entrypoint that mounts the staged Go services' handlers onto the chassis, then `go build` it into one binary, replacing the current print stub. - [x] DM-2d: (https://gitlab.com/gitlab-org/lab-bench/bench/-/merge_requests/30) Make `build_service_rust` real against the same contract: generate the Rust entrypoint composing the staged service crates onto the chassis, then `cargo build`, replacing its stub. ### Track C: packaging and run (starts once the first backend lands) - [x] DM-2e: (https://gitlab.com/gitlab-org/lab-bench/bench/-/merge_requests/73 and https://gitlab.com/gitlab-org/lab-bench/bench/-/merge_requests/85) Package a built artifact into a deployable OCI image in `build_assembly`. Document the image structure and tag convention, and push to `default_registry`. Needs either 2c or 2d since packaging consumes a compiled binary - **Packages one image per assembly** (the assembly's single entrypoint binary). This diverges from IAM's one-image-per-binary split (`iam-auth`, `iam-lookup`, …) because bench's shared-chassis model composes all of an assembly's services into one process; IAM predates bench and composes at the service-owns-its-listener granularity. Revisit if multi-binary assemblies land - **Locate the binary via a backend-returned path contract, not a fixed directory.** The Go backend's leading design builds into the scaffolded repo (`go build ./...` there), while the Rust backend builds into `target/benches/<bench>/<assembly>/`. Packaging must be backend-agnostic: each compile backend returns its built-binary path(s), and `build_assembly` packages whatever it is handed. - **Reconcile the pushed image reference with `.fairway/manifest.yaml` `spec.image`** — the manifest the copier template emits points at the canonical image; the tag/registry this step pushes to must line up. - Reference the RP-onboarded shape from [IAM !226](https://gitlab.com/gitlab-org/auth/iam/-/merge_requests/226): minimal base image, `COPY` the pre-built binary onto it (no in-image compile stage), non-root user, `ca-certificates`. ### Track D: SLSA L3 attestation plan (a written plan, no code, runs alongside the rest) - [ ] DM-2g: Write the SLSA L3 plan: provenance generation, hermetic builds, pinned toolchain and base images, a signing and verification strategy, and the hardened-builds integration point. Addresses the SLSA L3 portion of LB-D18 in the [epic Decision Index](https://gitlab.com/groups/gitlab-org/lab-bench/-/epics/1). - Include the hermeticity rule: no `latest` anywhere in the build graph. Every input is version-pinned and checksum-verified, with no live resolution during the build. Pin and let Renovate bump. See example `cargo-binstall` thread in [!7](https://gitlab.com/gitlab-org/lab-bench/bench/-/merge_requests/7#note_3443772190). - Pin the base image **by digest, as a single swappable reference**. This is the hardened-builds/[TUBE](https://gitlab.com/gitlab-com/content-sites/handbook/-/merge_requests/11660) integration point: a TUBE build stream is a curated, promotable set of the same pins this rule already mandates, so adopting TUBE's certified `CURRENT` stream later becomes a one-line digest swap rather than a build-definition rewrite. Leave room for an opt-in stream-validation mode (extra allowed-to-fail image builds against alternate pinned stream digests), which stays hermetic because each stream is itself fully pinned. Roll our own hermetic build now; TUBE is a future migration, not a dependency. - Real-world reference - [IAM !226](https://gitlab.com/gitlab-org/auth/iam/-/merge_requests/226) onboarded IAM to the Release Platform CI component (the TUBE + Fairway consumption side), decoupling binary production from image packaging. `bench` should copy IAM's exact structure, but adds the one discipline IAM skipped by pinning the base image by its content hash so the build is truly reproducible. ## References - [Panoptic](https://gitlab.com/gitlab-org/rust/experiments/panoptic): deferred polyglot PoC (see the collapsed decision record) - [bench cli/src/main.rs](https://gitlab.com/gitlab-org/lab-bench/bench/-/blob/main/cli/src/main.rs): the stub build functions this milestone makes real - [Theseus CLI](https://gitlab.com/gitlab-org/theseus): port source for Track A scaffolding - [Hardened builds](https://gitlab.com/gitlab-org/software-supply-chain-security/hardened-builds) - [IAM (`gitlab-org/auth/iam`)](https://gitlab.com/gitlab-org/auth/iam): Go reference implementation for Track B (see the walkthrough above) - [Phase 1 epic](https://gitlab.com/groups/gitlab-org/lab-bench/-/work_items/1)