Skip to content

Canary deployment fails for rails project with DB_INITIALIZE

I followed the documentation https://docs.gitlab.com/ee/topics/autodevops/#postgresql-database-support and set both DB_INITIALIZE and DB_MIGRATE. However deployment of canary failed for me. It did not work until I removed DB_INITIALIZE variable and re-deployed canary. Not sure how this is related. Maybe rails db:setup fails.

Failed canary job => https://gitlab.com/dz-test-group/rails-sample-shop/-/jobs/257371991

Successful canary job => https://gitlab.com/dz-test-group/rails-sample-shop/-/jobs/260001988

To fix canary I did next:

  1. Removed DB_INITIALIZE env var.
  2. Removed deployment via Cloud shell:
export TILLER_NAMESPACE=rails-sample-shop-13323112

curl -sS "https://kubernetes-helm.storage.googleapis.com/helm-v2.13.1-linux-amd64.tar.gz" | tar zx
export HELM_HOST="localhost:44134"
linux-amd64/tiller -listen ${HELM_HOST} -alsologtostderr > /dev/null 2>&1 &

linux-amd64/helm delete --purge production-canary

Possible fixes

  1. Don't run DB_INITIALIZE for canary job (though this means first ever canary deployment will fail)
  2. Run DB_INITIALIZE before canary/incremental/production jobs

See https://gitlab.com/gitlab-org/gitlab-ce/issues/65171#note_196861965 for detailed diagnostic

Related improvements

  1. Reproduce the issue and remove our DB_INITIALIZE suggestion for Rails if it causes issues.
  2. Find a way to better report errors in deployment job. Current log is not helpful
  3. Set DB_MIGRATE env variable automatically for user on Rails project?
  4. Create a set of proven to work recipes for DB_INITIALIZE and DB_MIGRATE values per framework
Edited by 🤖 GitLab Bot 🤖