Improve Staging Ref deployment stability

Occasionally Staging Ref deployment fails with transient issues:

  • Post-configure transient API requests failures:
    • 403 Forbidden account blocked - https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit-configs/staging-ref/-/issues/4
    • 502 error on Get Environment Settings API request. - https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit-configs/staging-ref/-/issues/13
  • Unattended Upgrades failed on PGBouncer - https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit-configs/staging-ref/-/issues/7
  • Dogfood GET docker image to increase stability and decrease runtime - https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit-configs/staging-ref/-/issues/8
  • Speed up Staging Ref with Geo deployments - https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit-configs/staging-ref/-/issues/23

The issue is to investigate these transient issues. Additionally we can explore adding a deployment troubleshooting docs to Staging Ref and set up a process for Delivery engineers to create issues if deployment fails.

Edited Apr 29, 2022 by Nailia Iskhakova
Assignee Loading
Time tracking Loading