Skip to content

Automatically clean up stale deployments prior to rollback checks

Robert Speicher requested to merge rs-stale-deploy-cleanup into master

What does this MR do?

Stale deployments (those marked as "running" for too long) can lead to invalid rollback checks.

It seems logical to clean up those stale deployments before we perform a rollback check so that we're working with valid state.

Related: gitlab-com/gl-infra/delivery#2249 (closed)

Local test run

% TEST=true ROLLBACK_CURRENT=gstg ROLLBACK_TARGET=gstg be rake auto_deploy:rollback_check
2022-02-23 11:37:36.077609 D [dry-run] ReleaseTools::GitlabClient -- [HTTParty] [2022-02-23 11:37:36 -0600] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/deployments" -
2022-02-23 11:37:36.078546 D [dry-run] ReleaseTools::Rollback::UpcomingDeployments -- Marking stale deployment as failed -- {:deployment=>7085, :updated_at=>"2022-02-21T18:40:10.237Z", :sha=>"a5835ae4cc98a6ff9e063ebc67e5fdc6c8940fa7"}
2022-02-23 11:37:36.078607 D [dry-run] ReleaseTools::Rollback::UpcomingDeployments -- Marking stale deployment as failed -- {:deployment=>7078, :updated_at=>"2022-02-21T16:15:24.524Z", :sha=>"898e870a0ebfb5720065402d84e861f48bd03ed6"}
2022-02-23 11:37:36.078664 D [dry-run] ReleaseTools::Rollback::UpcomingDeployments -- Marking stale deployment as failed -- {:deployment=>6994, :updated_at=>"2022-02-17T14:21:14.932Z", :sha=>"5f03ba0e4c7d3a350fef6c6a93bb8c43ff185c03"}

Author Check-list

Edited by Robert Speicher

Merge request reports