Skip to content

Environment variables not being picked when deploying using tags(solved)

Summary

tl;dr - Environment variables are not being set for AWS when using Tags.

Explanation - So after successfully deploying to the staging environment using git push, I am deploying my final code to production by creating version tags. Environment variables are working when I deploy for staging, but fails when the deploy is performed using version tags

Steps to reproduce

Steps

  1. After my staging enviroment is deployed...I go to Repository > Tags.
  1. I create a tag...say v1.2.3.
  1. The job 'deploy_production' is triggered after that
  1. It fails as environment key are not passed. More details in the log.

Here is my .gitlab-ci.yml file

stages:
  - test
  - deploy

variables:
  AWS_DEFAULT_REGION: $AWS_DEFAULT_LOCATION
  AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
  AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
  S3_BUCKET: $S3_BUCKET
  S3_KEY: $S3_KEY

test:
  image: "ruby:2.4.3-alpine3.7"
  stage: test
  cache:
    untracked: false
  variables:
    BUILD_PACKAGES: "tzdata ruby-nokogiri build-base postgresql-client bash"
    DEV_PACKAGES: "zlib-dev libxml2-dev libxslt-dev ruby-dev postgresql-dev"
    RUBY_PACKAGES: "ruby-json nodejs-npm"
    RACK_ENV: test
    RAILS_ENV: test
    POSTGRES_DB: xxx_test
    DATABASE_URL: "postgresql://postgres:postgres@postgres:5432/$POSTGRES_DB"
  services:
    - postgres:9.6.6-alpine
  before_script:
    - apk upgrade && apk add --update --no-cache $BUILD_PACKAGES $DEV_PACKAGES $RUBY_PACKAGES && rm -rf /var/cache/apk/*
    - echo -e 'http://dl-cdn.alpinelinux.org/alpine/edge/main\nhttp://dl-cdn.alpinelinux.org/alpine/edge/community\nhttp://dl-cdn.alpinelinux.org/alpine/edge/testing' > /etc/apk/repositories
    - apk add --no-cache yarn

  script:
    - RAILS_ENV=test bundle install --jobs 20
    - rake db:schema:load
    - rspec spec

deploy_staging:
  image: alpine:3.7
  stage: deploy
  environment: Staging
  variables:
    EB_APP_ENV: $STAGING
  only:
    - master
  before_script:
    - apk update && apk upgrade
    - apk -v --update add python py-pip groff less zip bash
    - pip install --upgrade awscli
  script:
    - chmod a+x bin/deploy.sh
    - ./bin/deploy.sh

deploy_production:
  image: alpine:3.7
  stage: deploy
  environment: Production
  variables:
    EB_APP_ENV: $PRODUCTION
  only:
    - tags
  before_script:
    - apk update && apk upgrade
    - apk -v --update add python py-pip groff less zip bash
    - pip install --upgrade awscli
  script:
    - chmod a+x bin/deploy.sh
    - ./bin/deploy.sh

And this is my deploy.sh file

#!/bin/bash

ts=`date +%s`
fn="$EB_APP_NAME-$ts.zip"

zip -r $fn * -x node_modules/\* tmp/\* vendor/\* log/\* storage/\*
S3_KEY="$S3_KEY/$fn"
# Copy the files to S3
aws s3 cp $fn "s3://$S3_BUCKET/$S3_KEY"

# Create a new version in Elastic Beanstalk
echo "Creating ElasticBeanstalk Application Version ..."
aws elasticbeanstalk create-application-version \
  --application-name $EB_APP_NAME \
  --version-label "$EB_APP_NAME-$ts" \
  --description "$EB_APP_NAME-$ts" \
  --source-bundle S3Bucket="$S3_BUCKET",S3Key="$S3_KEY" --auto-create-application

# Update to that version
echo "Updating ElasticBeanstalk Application Version ..."
aws elasticbeanstalk update-environment \
  --application-name $EB_APP_NAME \
  --environment-name $EB_APP_ENV \
  --version-label "$EB_APP_NAME-$ts"

Actual behavior

The environment variables are not transferred when I am deploying the app in the deploy_production job. It deploys perfectly in the deploy_staging job. The problem is shown in the logs....eg. $S3_BUCKET is not being set.

Expected behavior

Environment variables should be passed properly.

Relevant logs and/or screenshots

upload failed: ./my-app-name-1517346247.zip to s3://$S3_BUCKET/$S3_KEY/my-app-name-1517346247.zip Parameter validation failed:
Invalid bucket name "$S3_BUCKET": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$"
Creating ElasticBeanstalk Application Version ...

An error occurred (InvalidClientTokenId) when calling the CreateApplicationVersion operation: The security token included in the request is invalid.
Updating ElasticBeanstalk Application Version ...
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --environment-name: expected one argument
ERROR: Job failed: exit code 2

Environment description

Using shared Runners on GitLab.com

Used GitLab Runner version

Running with gitlab-runner 10.4.0 (857480b6)
on docker-auto-scale (4e4528ca)
Using Docker executor with image alpine:3.7
Edited by Abhinav Mathur