Development change: Database schema version handling outside of structure.sql
With the change made in #212425 (closed), we are changing the way we track the migration versions that are present in structure.sql
.
Going forward, the applied versions will be tracked outside of db/structure.sql
in the directory db/schema_migrations
, which should eliminate the frequent conflicts when multiple developers add new migrations on separate feature branches. This issue serves as the communication point to track any issues or questions we are running into once this change has been merged.
How does this impact me?
If you're an engineer and your merge requests contain database schema changes, ensure you get acquainted with the content of this announcement.
What does this mean for development?
Tracking of migration versions
- Old: Migration version information to be applied in a reload of the database was tracked in the
db/structure.sql
file - New: Migration version information to be applied in a reload of the database is tracked in a series of files under
db/schema_migrations
What does this mean in more detailed terms? First, standard rake tasks like db:structure:dump
and db:structure:load
should continue to function as they always have. The database schema (tables, indexes, constraints, etc.) will continue to be tracked in structure.sql
. There are two main scenarios.
up
migration
Running an In the old method, when a new migration was run, the structure.sql
file was updated with the new database structure, and the new migration version was added to the COPY
command that appears at the bottom of the file. The COPY
command would re-populate the schema_migrations
table with the correct versions when the structure file was loaded.
In the new method, when a new migration is run, the structure.sql
file will still be updated with the new physical structure, but the migration version will not be added to the file. Instead, a new blank file, with filename matching the version number, will be created under the db/schema_migrations
directory. The new file must be committed alongside the migration, so standard rake tasks will continue to work properly.
down
migration
Running a In the old method, when a migration was rolled back, the structure.sql
file was updated with the new database structure, and the version of the migration was removed from the large COPY
command at the bottom of the file.
In the new method, the structure.sql
file will still be updated with the new physical structure, but the migration version will not be tracked in the file. Instead, the existing file matching that version under db/schema_migrations
will be deleted. That deletion must also be committed, so standard rake tasks will work properly.
What do I have to do now?
- Please pull the latest changes on master, or alternatively, you can update your gdk to bring your development environment up-to-date
- Pull in changes or rebase your feature branch onto master (ignore conflicts in
db/structure.sql
versioning information at the bottom of the file and discard changes) - If you have new migrations in your branch, reflect their changes in
db/structure.sql
and thedb/schema_migrations
directory
For (3), the cleanest way of doing this is to reset the local database schema (rake db:reset
- also drops local database data) and re-run the migration on top of it. This results in relevant changes to the appropriate files, which should be committed.
What if I need to backport a change?
If you need to backport a fix to a stable branch that exists prior to the merge of this change, you should follow the standard process we have been following since the switch to using structure.sql
. Ensure the correct migration versions changes related to your fix are tracked properly in the structure.sql
file, and do not include any changes to version files created as part of the new process.
Why are we doing this?
The structure.sql
file is touched by every developer adding a migration. As a result, it's common to have conflicts on the version information tracked in the file, often resulting in merge conflicts that must be resolved (sometimes repeatedly before a change can be successfully merged). By moving that information into individual files, multiple migrations added in separate branches should seamlessly merge.
Q&A
Please leave any questions in this issue.
Artifacts
- Original issue proposing this: #212425 (closed)
- Change for GitLab: !30109 (merged)