Push deb packages to Pulp with one repository version per repo

🤖 beep boop — clanker comment.

What does this MR do?

Changes the deb push to Pulp so each repository gets one new version and one publish per job, instead of one per package per release.

Before

For every package and every release, one pulp deb content upload --repository ... --distribution ... call. Pulp creates the package plus the units that place it in the release, opens a new repository version, and because the repos have autopublish on, republishes the whole repo. Pulp serialises version changes to a repo, so the calls queue behind each other.

flowchart LR
    subgraph job["unstable pulp: [ubuntu]"]
        loop["for each of 6 releases<br/>for each of 9 packages"]
    end
    loop -->|"54 × upload with --repository"| pulp
    subgraph pulp["Pulp, one task per upload, serialised per repo"]
        direction TB
        a["store package"] --> b["create release_component,<br/>release_architecture, link"] --> c["new repository version"] --> d["autopublish whole repo<br/>~14k links, 3.5 to 5 min"]
    end

54 publishes × 3.5 to 5 minutes is the 2 to 4 hours. Debian is 45 publishes. The rpm flavors finish in 10 to 18 minutes because the rpm pusher already uploads once and links.

After

Each file is uploaded once, without a repository. The units that place a package in a release are created through the content API, which does not create a repository version. Everything is then added with one pulp deb repository content modify per repo.

flowchart LR
    subgraph job["unstable pulp: [ubuntu]"]
        direction TB
        u["9 × upload, no repository<br/>(parallel)"] --> l["per release: get-or-create<br/>release_component, release_architecture,<br/>one link per package<br/>(no repository version)"] --> m["1 × repository content modify<br/>with all hrefs"]
    end
    m --> pulp
    subgraph pulp["Pulp"]
        direction TB
        c["new repository version"] --> d["autopublish whole repo, once"]
    end

pulp-cli-deb 0.5.0 cannot create release_architecture or package_release_component units, so those two endpoints are called directly, using the base_url, username and password from the config that pulp:createConfig writes. Links are looked up before they are created, so a retried job adds nothing and creates no new version.

RUNNER_MAGE_CONCURRENCY was unset, so the existing worker pool ran one upload at a time. .release_pulp now sets it to 4, which applies to both the deb and the rpm push in those jobs. For rpm this means up to 4 packages upload in parallel, each to its own per-arch repository, followed by the existing link step; the retry on the artifact-checksum race is unchanged.

Why was this MR needed?

The ubuntu and debian pulp jobs are the last thing to finish in every main pipeline, and they get slower as the unstable repos grow.

What's the best way to test this MR?

I ran this against a sandbox Pulp (pulpcore 3.116.1, pulp_deb 3.10.0, the same plugin version as production) with the real deb artifacts from pipeline 2815418752. With the current code, pushing 9 packages to one release produced 9 repository versions and 9 publications. With this MR, pushing 9 packages to all 6 ubuntu releases produced one version and one publication, and the Packages and Release files were correct for every release and arch. Pushing the same packages again created no new version.

The next main pipeline is the real check: the ubuntu and debian pulp jobs should finish in minutes, and https://pulp.gitlab.com/runner/unstable/ubuntu/dists/<release>/main/binary-<arch>/Packages should list the new version for every release and arch.

What are the relevant issue numbers?

gitlab-com/gl-infra/production-engineering#29705