Environment variables not being picked when deploying using tags(solved)
Summary
tl;dr - Environment variables are not being set for AWS when using Tags.
Explanation - So after successfully deploying to the staging environment using git push, I am deploying my final code to production by creating version tags. Environment variables are working when I deploy for staging, but fails when the deploy is performed using version tags
Steps to reproduce
Steps
- After my staging enviroment is deployed...I go to Repository > Tags.
- I create a tag...say v1.2.3.
- The job 'deploy_production' is triggered after that
- It fails as environment key are not passed. More details in the log.
Here is my .gitlab-ci.yml file
stages:
- test
- deploy
variables:
AWS_DEFAULT_REGION: $AWS_DEFAULT_LOCATION
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
S3_BUCKET: $S3_BUCKET
S3_KEY: $S3_KEY
test:
image: "ruby:2.4.3-alpine3.7"
stage: test
cache:
untracked: false
variables:
BUILD_PACKAGES: "tzdata ruby-nokogiri build-base postgresql-client bash"
DEV_PACKAGES: "zlib-dev libxml2-dev libxslt-dev ruby-dev postgresql-dev"
RUBY_PACKAGES: "ruby-json nodejs-npm"
RACK_ENV: test
RAILS_ENV: test
POSTGRES_DB: xxx_test
DATABASE_URL: "postgresql://postgres:postgres@postgres:5432/$POSTGRES_DB"
services:
- postgres:9.6.6-alpine
before_script:
- apk upgrade && apk add --update --no-cache $BUILD_PACKAGES $DEV_PACKAGES $RUBY_PACKAGES && rm -rf /var/cache/apk/*
- echo -e 'http://dl-cdn.alpinelinux.org/alpine/edge/main\nhttp://dl-cdn.alpinelinux.org/alpine/edge/community\nhttp://dl-cdn.alpinelinux.org/alpine/edge/testing' > /etc/apk/repositories
- apk add --no-cache yarn
script:
- RAILS_ENV=test bundle install --jobs 20
- rake db:schema:load
- rspec spec
deploy_staging:
image: alpine:3.7
stage: deploy
environment: Staging
variables:
EB_APP_ENV: $STAGING
only:
- master
before_script:
- apk update && apk upgrade
- apk -v --update add python py-pip groff less zip bash
- pip install --upgrade awscli
script:
- chmod a+x bin/deploy.sh
- ./bin/deploy.sh
deploy_production:
image: alpine:3.7
stage: deploy
environment: Production
variables:
EB_APP_ENV: $PRODUCTION
only:
- tags
before_script:
- apk update && apk upgrade
- apk -v --update add python py-pip groff less zip bash
- pip install --upgrade awscli
script:
- chmod a+x bin/deploy.sh
- ./bin/deploy.sh
And this is my deploy.sh file
#!/bin/bash
ts=`date +%s`
fn="$EB_APP_NAME-$ts.zip"
zip -r $fn * -x node_modules/\* tmp/\* vendor/\* log/\* storage/\*
S3_KEY="$S3_KEY/$fn"
# Copy the files to S3
aws s3 cp $fn "s3://$S3_BUCKET/$S3_KEY"
# Create a new version in Elastic Beanstalk
echo "Creating ElasticBeanstalk Application Version ..."
aws elasticbeanstalk create-application-version \
--application-name $EB_APP_NAME \
--version-label "$EB_APP_NAME-$ts" \
--description "$EB_APP_NAME-$ts" \
--source-bundle S3Bucket="$S3_BUCKET",S3Key="$S3_KEY" --auto-create-application
# Update to that version
echo "Updating ElasticBeanstalk Application Version ..."
aws elasticbeanstalk update-environment \
--application-name $EB_APP_NAME \
--environment-name $EB_APP_ENV \
--version-label "$EB_APP_NAME-$ts"
Actual behavior
The environment variables are not transferred when I am deploying the app in the deploy_production job. It deploys perfectly in the deploy_staging job. The problem is shown in the logs....eg. $S3_BUCKET is not being set.
Expected behavior
Environment variables should be passed properly.
Relevant logs and/or screenshots
upload failed: ./my-app-name-1517346247.zip to s3://$S3_BUCKET/$S3_KEY/my-app-name-1517346247.zip Parameter validation failed:
Invalid bucket name "$S3_BUCKET": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$"
Creating ElasticBeanstalk Application Version ...
An error occurred (InvalidClientTokenId) when calling the CreateApplicationVersion operation: The security token included in the request is invalid.
Updating ElasticBeanstalk Application Version ...
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument --environment-name: expected one argument
ERROR: Job failed: exit code 2
Environment description
Using shared Runners on GitLab.com
Used GitLab Runner version
Running with gitlab-runner 10.4.0 (857480b6) on docker-auto-scale (4e4528ca) Using Docker executor with image alpine:3.7
Edited by Abhinav Mathur