Guide developers on the usage of the `package-and-test` pipeline

Delivery is working to extend the GitLab maintenance policy to support bug fixes to the two previous monthly releases in addition to the current stable release. Part of the work to extend the maintenance policy will open up the stable branches to allow developers to merge bug fixes directly into the stable branch.

The execution of the package-and-test test suite will be the Quality indicator to guarantee stable branches are Quality-compliant and ready to be patched at any point.

Proposal: Guide developers on the usage of the package-and-test pipeline.

The package-and-test (formerly known as package-and-qa) pipeline is a manual downstream pipeline that is allowed to fail (there is some flakiness on these jobs that prevent them from acting as blocking).

Screenshot_2023-01-31_at_11.15.08

Merge requests targeting stable branches should execute these specs before merging, and if they fail, engineers should ask their SET counterparts for assistance. To accomplish this the Danger bot rules introduced on #2691 (closed) can be expanded to:

  • State the package-and-test should be executed before merging the merge request, and if there are failures the SET counterpart should be pinged to confirm they're not related to the merge request. This message will ping the author to ensure they are aware of this guideline.
  • Danger should fail if:
    • package-and-test job is not present, is in the manual or canceled state.
    • The pipeline:expedite label is used - It's not allowed on merge requests targeting stable branches because it causes the package-and-test job to be skipped.
  • On documentation merge requests (with documentation label) and merge requests fixing flaky failures (labeled with ~failure::flaky-testp) it is okay for the package-and-test to be skipped.
  • When the package-and-test pipeline is not manual or canceled, but also not success, a warning will be added to the top list of Danger warnings that says "The e2e:package-and-test job needs to succeed or have approval from a Software Engineer in Test. See the section below for more details."
Edited by Steve Abrams