Stop uploading Omnibus packages to PackageCloud and split deploy into a separate job
What does this MR do?
Re-applies !9213 (merged) (reverted in !9233 (merged) due to an auto-deploy incident) with additional fixes for the root cause of that incident.
Re-apply: Stop uploading Omnibus packages to PackageCloud (!9213 (merged))
As we are decommissioning PackageCloud (PC), make the PC jobs optional so we can test and rollback if needed with only an env var change.
- Add
.skip_unless_packagecloud_enabledrule to skip jobs whenPACKAGECLOUD_ENABLEDis nottrue - Apply the rule to all jobs uploading to PackageCloud
- Refactor staging upload templates to separate packagecloud and pulp concerns:
-
.staging_upload_templateis now a shared base (common settings: stage, image, script, cache, retry, tags) -
.packagecloud_staging_upload_templateextends the base, withINCLUDE_PACKAGECLOUD: "true"and the packagecloud-specific rules -
.pulp_staging_upload_templateextends the base with pulp-specific settings - Derived packagecloud templates carry the
packagecloud_prefix:.packagecloud_fips_staging_upload_template, etc.
-
- Refactor production release templates with the same pattern:
-
.production_release_templateis now a shared base -
.packagecloud_production_release_templateextends the base withINCLUDE_PACKAGECLOUD: "true"and packagecloud-specific rules -
.pulp_production_release_templateextends the base with pulp-specific settings -
.fips_release_templaterenamed to.packagecloud_fips_release_template
-
Fix: Split deploy and upload into separate jobs (!9225 (merged))
The root cause of the auto-deploy incident (!9233 (merged)) was that .packagecloud_staging_upload_with_rc_deployment_template was hiding both upload and deploy logic inside a single job. When PackageCloud was disabled, the deploy step was silently skipped along with it.
- Remove
.packagecloud_staging_upload_with_rc_deployment_template -
Ubuntu-20.04-stagingnow extends.packagecloud_staging_upload_template(upload only) - Add
deployer-triggerjob in a newdeploystage, running afterUbuntu-20.04-staging-pulp, making the deployment step explicit and independent of PackageCloud -
deployer-triggeronly runs on RC pipelines whenDEPLOYER_TRIGGER_TOKENis set, expressed via CI rules rather than a bash condition
Related issues
Ref:
- https://gitlab.com/gitlab-org/omnibus-gitlab/-/work_items/9552+
- https://gitlab.com/gitlab-org/build/team-tasks/-/work_items/177+
Checklist
See Definition of done.
For anything in this list which will not be completed, please provide a reason in the MR discussion.
Required
- MR title and description are up to date, accurate, and descriptive.
- MR targeting the appropriate branch.
- Latest Merge Result pipeline is green.
- When ready for review, MR is labeled workflowready for review per the Distribution MR workflow.
-
The UBT version and corresponding checksum hash have been updated and referenced in the merge request if applicable.
-
UBT EE pipeline (
Trigger:ee-package-ubt) is green
-
UBT EE pipeline (
For GitLab team members
If you don't have access to this, the reviewer should trigger these jobs for you during the review process.
-
The manual
Trigger:ee-packagejobs have a green pipeline running against latest commit.- To debug QA failures, refer to the Investigate QA failures section.
-
If
config/softwareorconfig/patchesdirectories are changed, make sure thebuild-package-on-all-osjob within theTrigger:ee-packagedownstream pipeline succeeded. -
If you are changing anything SSL related, then the
Trigger:package:fipsmanual job within theTrigger:ee-packagedownstream pipeline must succeed. -
If CI configuration is changed, the branch must be pushed to
dev.gitlab.orgto confirm regular branch builds aren't broken.
Expected (please provide an explanation if not completing)
- Test plan indicating conditions for success has been posted and passes.
- Documentation created/updated.
- Tests added.
- Integration tests added to GitLab QA.
- Equivalent MR/issue for the GitLab Chart opened.
-
Validate potential values for new configuration settings. Formats such as integer
10, duration10s, URIscheme://user:passwd@host:portmay require quotation or other special handling when rendered in a template and written to a configuration file.