Commit b7d0cc5c authored by Robert Speicher's avatar Robert Speicher 🌴

Merge branch 'background-migrations' into 'master'

Support for post deployment migrations

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/22133

See merge request !6572
parents f73f09b1 83c82411
Pipeline #4831973 failed with stages
in 61 minutes and 20 seconds
......@@ -72,6 +72,51 @@ Please view this file on the master branch, on stable branches it's out of date.
- Fix and improve `Sortable.highest_label_priority`. !7165
- Fixed sticky merge request tabs when sidebar is pinned. !7167
- Only remove right connector of first build of last stage. !7179
- Backups do not fail anymore when using tar on annex and custom_hooks only. !5814
- Adds user project membership expired event to clarify why user was removed (Callum Dryden)
- Trim leading and trailing whitespace on project_path (Linus Thiel)
- Prevent award emoji via notes for issues/MRs authored by user (barthc)
- Adds an optional path parameter to the Commits API to filter commits by path (Luis HGO)
- Fix extra space on Build sidebar on Firefox !7060
- Fix mobile layout issues in admin user overview page !7087
- Fix HipChat notifications rendering (airatshigapov, eisnerd)
- Refactor Jira service to use jira-ruby gem
- Add hover to trash icon in notes !7008 (blackst0ne)
- Only show one error message for an invalid email !5905 (lycoperdon)
- Fix sidekiq stats in admin area (blackst0ne)
- Created cycle analytics bundle JavaScript file
- API: Fix booleans not recognized as such when using the `to_boolean` helper
- Removed delete branch tooltip !6954
- Stop unauthorized users dragging on milestone page (blackst0ne)
- Restore issue boards welcome message when a project is created !6899
- Escape ref and path for relative links !6050 (winniehell)
- Fixed link typo on /help/ui to Alerts section. !6915 (Sam Rose)
- Fix filtering of milestones with quotes in title (airatshigapov)
- Refactor less readable existance checking code from CoffeeScript !6289 (jlogandavison)
- Update mail_room and enable sentinel support to Reply By Email (!7101)
- Add task completion status in Issues and Merge Requests tabs: "X of Y tasks completed" (!6527, @gmesalazar)
- Added support for executing post deployment migrations
- Simpler arguments passed to named_route on toggle_award_url helper method
- Fix typo in framework css class. !7086 (Daniel Voogsgerd)
- New issue board list dropdown stays open after adding a new list
- Fix: Backup restore doesn't clear cache
- API: Fix project deploy keys 400 and 500 errors when adding an existing key. !6784 (Joshua Welsh)
- Replace jquery.cookie plugin with js.cookie !7085
- Use MergeRequestsClosingIssues cache data on Issue#closed_by_merge_requests method
- Fix Sign in page 'Forgot your password?' link overlaps on medium-large screens
- Show full status link on MR & commit pipelines
- Fix documents and comments on Build API `scope`
- Refactor email, use setter method instead AR callbacks for email attribute (Semyon Pupkov)
- Shortened merge request modal to let clipboard button not overlap
- In all filterable drop downs, put input field in focus only after load is complete (Ido @leibo)
## 8.13.2
- Fix builds dropdown overlapping bug !7124
- Fix applying labels for GitHub-imported MRs !7139
- Fix importing MR comments from GitHub !7139
- Modify GitHub importer to be retryable !7003
- Fix and improve `Sortable.highest_label_priority`
- Fixed sticky merge request tabs when sidebar is pinned
## 8.13.1 (2016-10-25)
......
# Post deployment migrations are included by default. This file must be loaded
# before other initializers as Rails may otherwise memoize a list of migrations
# excluding the post deployment migrations.
unless ENV['SKIP_POST_DEPLOYMENT_MIGRATIONS']
path = Rails.root.join('db', 'post_migrate').to_s
Rails.application.config.paths['db/migrate'] << path
# Rails memoizes migrations at certain points where it won't read the above
# path just yet. As such we must also update the following list of paths.
ActiveRecord::Migrator.migrations_paths << path
end
......@@ -41,6 +41,7 @@
- [What requires downtime?](what_requires_downtime.md)
- [Adding database indexes](adding_database_indexes.md)
- [Post Deployment Migrations](post_deployment_migrations.md)
## Compliance
......
# Post Deployment Migrations
Post deployment migrations are regular Rails migrations that can optionally be
executed after a deployment. By default these migrations are executed alongside
the other migrations. To skip these migrations you will have to set the
environment variable `SKIP_POST_DEPLOYMENT_MIGRATIONS` to a non-empty value
when running `rake db:migrate`.
For example, this would run all migrations including any post deployment
migrations:
```bash
bundle exec rake db:migrate
```
This however will skip post deployment migrations:
```bash
SKIP_POST_DEPLOYMENT_MIGRATIONS=true bundle exec rake db:migrate
```
## Deployment Integration
Say you're using Chef for deploying new versions of GitLab and you'd like to run
post deployment migrations after deploying a new version. Let's assume you
normally use the command `chef-client` to do so. To make use of this feature
you'd have to run this command as follows:
```bash
SKIP_POST_DEPLOYMENT_MIGRATIONS=true sudo chef-client
```
Once all servers have been updated you can run `chef-client` again on a single
server _without_ the environment variable.
The process is similar for other deployment techniques: first you would deploy
with the environment variable set, then you'll essentially re-deploy a single
server but with the variable _unset_.
## Creating Migrations
To create a post deployment migration you can use the following Rails generator:
```bash
bundle exec rails g post_deployment_migration migration_name_here
```
This will generate the migration file in `db/post_migrate`. These migrations
behave exactly like regular Rails migrations.
## Use Cases
Post deployment migrations can be used to perform migrations that mutate state
that an existing version of GitLab depends on. For example, say you want to
remove a column from a table. This requires downtime as a GitLab instance
depends on this column being present while it's running. Normally you'd follow
these steps in such a case:
1. Stop the GitLab instance
2. Run the migration removing the column
3. Start the GitLab instance again
Using post deployment migrations we can instead follow these steps:
1. Deploy a new version of GitLab while ignoring post deployment migrations
2. Re-run `rake db:migrate` but without the environment variable set
Here we don't need any downtime as the migration takes place _after_ a new
version (which doesn't depend on the column anymore) has been deployed.
Some other examples where these migrations are useful:
* Cleaning up data generated due to a bug in GitLab
* Removing tables
* Migrating jobs from one Sidekiq queue to another
# See http://doc.gitlab.com/ce/development/migration_style_guide.html
# for more information on how to write migrations for GitLab.
class <%= migration_class_name %> < ActiveRecord::Migration
include Gitlab::Database::MigrationHelpers
DOWNTIME = false
# When using the methods "add_concurrent_index" or "add_column_with_default"
# you must disable the use of transactions as these methods can not run in an
# existing transaction. When using "add_concurrent_index" make sure that this
# method is the _only_ method called in the migration, any other changes
# should go in a separate migration. This ensures that upon failure _only_ the
# index creation fails and can be retried or reverted easily.
#
# To disable transactions uncomment the following line and remove these
# comments:
# disable_ddl_transaction!
def change
end
end
module Rails
class PostDeploymentMigrationGenerator < Rails::Generators::NamedBase
def create_migration_file
timestamp = Time.now.strftime('%Y%m%d%H%I%S')
template "migration.rb", "db/post_migrate/#{timestamp}_#{file_name}.rb"
end
def migration_class_name
file_name.camelize
end
end
end
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment