Deploy of 8.6.0-rc2
The deploy had the following issues:
- The installation and reconfigure run worked without a problem until db:migrate
- During db:migrate, the migration stalled during trigram migrations
- We couldn't see why the migration stalled but at some point, postgres was not doing anything
- We started checking if the migration is still running
- The database tables were partially migrated, while the migration was basically not done yet
- We confirmed that the database is not being written to
- We saw that the slave was up to date
- We then killed the migration run, confirmed that migration partially ran with db:migrate:status
- Yorick then went into the database to manually execute the migrations that were pending
- After we checked that the migrations that are reported as not up were in fact not applied, we ran db:migrate manually
- We checked if the migration was done properly
- We decided to continue with the deploy by running reconfigure once again on the blessed worker
- We confirmed that everything was done without an error
- Continued with deploy on the rest of the workers.