An error occurred while fetching the assigned milestone of the selected merge_request.
This is an initial implementation of the problem and solutions described in this blueprint !134487 (merged) and it was extracted from a POC implementation in !134485 (closed) which contained other work we don't want to include in the first iteration.
This is a gem which can be run locally or in CI to do static and dynamic analysis of the codebase and using a list of predefined "keeps" it will automatically create merge requests for things that developers would have otherwise needed to remember to do themselves.
It is analogous to a mix of rubocop -a
and GitLab Dependency Bot.
The word "keep" is used to describe a specific rule to apply to the code to match a required change and actually edit the code. The word "keep" was chosen as it sounds like "cop" and is very similar to implementing a rubocop rule as well as code to autocorrect the rule.
Today we have a complete implementation of the OverdueFinalizeBackgroundMigration
keep which finds all batched background migrations added before the last required stop upgrade point and then checks in Postgres.ai to confirm that this migration is complete on GitLab.com and then generates a migration to finalize the batched background migration. It has generated many migration MRs already.
The code is organized in a very similar way to RuboCop in that we have a overall gem called gitlab-housekeeper
that contains the generic logic of looping over all keeps
(analogous to Cops) which are rules for how to detect changes that can be made to the code and then actually how to correct them.
This MR includes the following high level pieces:
gems/gitlab-housekeeper
which is the generic gem. It should be responsible for running keeps
which will return a Change
object. The housekeeper then uses this change object to create a git branch, push the branch, create a merge request
gitlab-housekeeper
command which just calls the Runner
Runner
which contains the high level operation of the gitlab-housekeeper
gem. It looks at the options provided and runs the correct keeps. It then takes the Change
results from the keeps and creates merge requestsGit
client is just a wrapper around shelling out to gitGitlabClient
is just a thin wrapper around HTTParty
to make HTTP requests to create merge requestsShell
class is just a wrapper around shelling out. Mainly this is useful for stubbing in specs for all the other code.keeps/
directory which is analogous to our rubocop
directory. It contains implementations of specific keeps and also a helpers
directory that we expect to be used in multiple keeps
keeps/helpers/postgres_ai.rb
is a wrapper around the PG
client that we can use to encapsulate code that looks up things from the current state of our production database. For now it is used to check if a batched background migration is finished in productionkeeps/overdue_finalize_background_migration.rb
: This is our first keep we've implemented. It has been used successfully to generate at least 15 merged migrations already. This is a pretty complicated piece of code but at a high level it is looking for db/docs/batched_background_migrations
files and trying to find ones that were added before the last required stop and have already finished on GitLab.com . This means it's safe to finalize them. It then generates a migration with the change necessary to finalize the BBM.Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
Numbered steps to set up and validate the change are strongly suggested.
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.