Fix review apps cleanups
What does this MR do and why?
Closes #359100 (closed) Closes https://gitlab.com/gitlab-org/quality/engineering-productivity-infrastructure/-/issues/18
There will be a follow-up issue (https://gitlab.com/gitlab-org/quality/engineering-productivity-infrastructure/-/issues/36) to clean up the orphaned resources in the cluster.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Merge request reports
Activity
changed milestone to %15.6
assigned to @ddieulivol
added typemaintenance label and removed typebug label
- Resolved by Rémy Coutable
- A deleted user
added backend label
1 Message CHANGELOG missing: If you want to create a changelog entry for GitLab FOSS, add the
Changelog
trailer to the commit message you want to add to the changelog.If you want to create a changelog entry for GitLab EE, also add the
EE: true
trailer to your commit message.If this merge request doesn't need a CHANGELOG entry, feel free to ignore this message.
Pipeline Changes
This merge request contains changes to the pipeline configuration for the GitLab project.
Please consider the effect of the changes in this merge request on the following:
- Effects on different pipeline types
- Effects on non-canonical projects:
gitlab-foss
security
dev
- personal forks
- Effects on pipeline performance
Please consider communicating these changes to the broader team following the communication guideline for pipeline changes
Reviewer roulette
Changes that require review have been detected!
Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:
Category Reviewer Maintainer backend Jon Jenkins (
@jon_jenkins
) (UTC-6, 7 hours behind@ddieulivol
)Igor Drozdov (
@igor.drozdov
) (UTC+1, same timezone as@ddieulivol
)maintenanceworkflow / maintenancepipelines for CI, Danger Maxime Orefice (
@morefice
) (UTC+1, same timezone as@ddieulivol
)Igor Drozdov (
@igor.drozdov
) (UTC+1, same timezone as@ddieulivol
)To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
Danger- Resolved by Rémy Coutable
- Resolved by Rémy Coutable
- Resolved by Rémy Coutable
- Resolved by Rémy Coutable
- Resolved by Rémy Coutable
So I ran
review-cleanup
but I think this error prevents the process from finishing and getting to the Helm cleanup:/usr/local/lib/ruby/gems/3.0.0/gems/gitlab-4.19.0/lib/gitlab/request.rb:71:in `validate': Server responded with code 500, message: 500 Internal Server Error. Request URI: https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/environments/12187117 (Gitlab::Error::InternalServerError) from /usr/local/lib/ruby/gems/3.0.0/gems/gitlab-4.19.0/lib/gitlab/request.rb:55:in `block (2 levels) in <class:Request>' from /usr/local/lib/ruby/gems/3.0.0/gems/gitlab-4.19.0/lib/gitlab/client/environments.rb:74:in `delete_environment' from /usr/local/lib/ruby/gems/3.0.0/gems/gitlab-4.19.0/lib/gitlab.rb:28:in `method_missing' from scripts/review_apps/automated_cleanup.rb:189:in `delete_environment' from scripts/review_apps/automated_cleanup.rb:89:in `block in perform_gitlab_environment_cleanup!' from /usr/local/lib/ruby/gems/3.0.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:41:in `<<' from /usr/local/lib/ruby/gems/3.0.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:41:in `each_page' from /usr/local/lib/ruby/gems/3.0.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:52:in `each' from /usr/local/lib/ruby/gems/3.0.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:52:in `each' from /usr/local/lib/ruby/gems/3.0.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:52:in `each' from /usr/local/lib/ruby/gems/3.0.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:52:in `each' from /usr/local/lib/ruby/gems/3.0.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:52:in `auto_paginate' from scripts/review_apps/automated_cleanup.rb:76:in `perform_gitlab_environment_cleanup!' from scripts/review_apps/automated_cleanup.rb:289:in `block in <main>' from scripts/review_apps/automated_cleanup.rb:266:in `timed' from scripts/review_apps/automated_cleanup.rb:288:in `<main>'
It seems the environment doesn't exist: https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/environments/12187117
- Resolved by David Dieulivol
I think we'll also need to fix
Checking for Helm releases that are failed or not updated in the last 7 days... Running command: `helm list --namespace "review-apps" --max 256 --offset 0 --output json --all --date` Running command: `helm list --namespace "review-apps" --max 256 --offset 256 --output json --all --date`
Our
Helm3Client
is only looking at thereview-apps
namespace:def helm @helm ||= Tooling::Helm3Client.new(namespace: review_apps_namespace) end
- Resolved by Rémy Coutable
added 1 commit
- 040c5d50 - Ignore 500 errors when deleting environments
- Resolved by Rémy Coutable
Hey @rymai, Could you please review this MR? Thanks in advance
requested review from @rymai
- Resolved by David Dieulivol
It looks like we have specs for the helm3 client
https://gitlab.com/gitlab-org/gitlab/-/jobs/3331487772@rymai
, thanks for approving this merge request.This is the first time the merge request is approved. To ensure full test coverage, a new pipeline will be started shortly.
For more info, please refer to the following links:
marked the checklist item I have evaluated the MR acceptance checklist for this MR. as completed
removed [deprecated] Accepting merge requests label
removed missed:15.3 label
removed missed:15.2 label
removed missed:15.4 label
removed missed:15.5 label
enabled an automatic merge when the pipeline for ad782a0e succeeds
mentioned in commit e29c8637
mentioned in issue #359100 (closed)
added workflowstaging-canary label
added workflowcanary label and removed workflowstaging-canary label
added workflowstaging label and removed workflowcanary label
added workflowproduction label and removed workflowstaging label
added workflowpost-deploy-db-staging label and removed workflowproduction label
added workflowpost-deploy-db-production label and removed workflowpost-deploy-db-staging label
added releasedcandidate label
mentioned in issue gitlab-org/quality/engineering-productivity/team#116 (closed)
Great job @ddieulivol!
added releasedpublished label and removed releasedcandidate label