Add supporting classes for ClickHouse migrations
What does this MR do and why?
Currently, the ClickHouse tables are setup by ClickHouseTestRunner#ensure_schema, by running db/click_house/main/*.sql files against the database.
This MR adds basic support for ClickHouse migrations:
-
first commit: support classes inspired by the Active Record equivalent (https://github.com/rails/rails/blob/main/activerecord/lib/active_record/migration.rb and the clickhouse-activerecord
gem). I've chosen to not rely on the equivalent ActiveRecord classes since:
- the amount of code saved is not significant.
- we'd need to catch any ActiveRecord exceptions and raise equivalent
ClickHouseexceptions, otherwise the exception class might mislead users. - we would be exposed to potential bugs due to subtle changes in the base ActiveRecord classes.
-
second commit: Rake targets (
rake gitlab:clickhouse:migrateandrake gitlab:clickhouse:rollback, including support forVERSION,VERBOSE, andSCOPEenvironment variables). -
third commit: moves the existing
db/click_house/main/*.sqlfiles todb/click_house/migrate/*.rbfiles.
Closes #428124 (closed)
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
| Command | Before | After |
|---|---|---|
rake gitlab:clickhouse:migrate |
![]() |
![]() |
rake gitlab:clickhouse:rollback VERSION=20230808140217 |
![]() |
![]() |
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
-
Setup a development ClickHouse server, see https://docs.gitlab.com/ee/development/database/clickhouse/clickhouse_within_gitlab.html
-
Test several migration commands, such as:
rake gitlab:clickhouse:migraterake gitlab:clickhouse:rollback
You can also pass specific
VERSIONvalues.At any point, you can run
SELECT * FROM schema_migrations FINALon the ClickHouse client (clickhouse client -d gitlab_clickhouse_development) to see the internal state of the migrations.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.



