## Problem to solve Our current Review apps implementation, requires a static URL that is used as the CI/CD variable `CI_ENVIRONMENT_SLUG`. Many use cases the environment variable is not static but dynamic, for example when using AWS , a user will probably want to use the environment name based on the stage `stage: qa-lambda-47378475` and it can be replaced. In order to run Review apps in such an environment , we would need to support dynamic URLs. ## Proposal We introduce a new report type of artifact - [`dotenv`](https://github.com/motdotla/dotenv). ```yml review: script: - DYNAMIC_ENVIRONMENT_URL=$(deploy-script) # In script, you get an environment URL. - echo "DYNAMIC_ENVIRONMENT_URL=$DYNAMIC_ENVIRONMENT_URL" >> deploy.env # Add the value to a dotenv file. artifacts: reports: dotenv: deploy.env # Report back dotenv file to rails. environment: name: review/$CI_COMMIT_REF_SLUG url: $DYNAMIC_ENVIRONMENT_URL # and set the variable produced in script to `environment:url` on_stop: stop_review stop_review: script: - ./teardown-environment when: manual environment: name: review/$CI_COMMIT_REF_SLUG # NOTE: Don't set `environment:url` as `dotenv` file is not generated in this job. action: stop ``` This opens an awesome possibility for later passing variables between different builds, by using `dotenv` syntax. The `.env` is a simple `KEY=VALUE` text file with env per line. ### Use cases AWS Lambda and the Serverless framework which give a dynamic URL based on the stage: ``` Serverless: Stack update finished... Service Information service: lambda-api stage: qa-lambda-47378475 region: us-east-1 stack: lambda-api-qa-lambda-47378475 api keys: None endpoints: POST - https://sg01n2q8ph.execute-api.us-east-1.amazonaws.com/qa-lambda-47378475/gql functions: graphqlServer: lambda-api-qa-lambda-47378475-graphqlServer layers: None ``` We have the same requirement to set `CI_ENVIRONMENT_URL` dynamically (based on data available in the job), but want to allow it to be overridden from `.gitlab-ci.yml` when it is manually set there. ## Technical Proposal - Runner returns `artifacts:report:dotenv: #{dot_env_file_path}` as a raw artifact. - In `Ci::BuildFinalizeWorker/Service`, Rails parses `dotenv` file and persists it into `ci_job_variables` table. - In `ci_builds.status == finalizing`, we processes the build. ## TODO - [ ] Persist DotEnv Variables From an artifact => https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26247 - [ ] Publish Feature (documentation and feature flag removal) ## Old Proposal for safe keeping: <details> <summary>Old Proposal for safe keeping:</summary> If you did support a script-created variable for URLs, I could do something like this :heart::  **Current Implementation** ``` create_env: stage: review when: manual script: echo "Create platform environment and create a dynamic variable called FANCY_SLUG" environment: name: env/$CI_BUILD_REF_SLUG url: https://$CI_BUILD_REF_SLUG-$FANCY_SLUG-$APPS_DOMAIN on_stop: delete_env ``` * `$APPS_DOMAIN` is a YAML-defined variable (not job-level) * `$FANCY_SLUG` is a variable that, ideally, could be set during the `script:` portion of the job **Limitation** * Environment urls in `.gitlab-ci.yml` definitions only expand CI-* variables **Example**  Using a popular hosting platform (https://platform.sh/) I can create environments pretty easily by installing their CLI and using authentication tokens. But the entire integration requires that I have some dynamic variable ability in setting the environment URL. Their URL has a token that is unique per branch, something that requires the script step of the review stage. ### Functional Work Around Let me upfront tell you that I'm probably the worst dev ops to tell you how or what you should do. The below works for me and I see it as a stop-gap measure and provide no warranty whatsoever. ``` <job>: stage: review when: manual script: - export ENVURL='http://replacethiswithanAPIgeneratedURL.com/' - export PRIVATE_TOKEN='I recommend you set a project variable called PRIVATE_TOKEN and drop your private token in there.' - export GITLAB_URL='https://gitlab.websites-r-us.com' # Using the API to bypass the limitation # @see https://docs.gitlab.com/ce/api/enviroments.html#environments # I clearly don't know how to script in this environment # See this issue for something better hopefully: - export ENVIRONMENT_REQ_HEADER="PRIVATE-TOKEN:"" "$PRIVATE_TOKEN - export ENV_ID_REQ_URL=$GITLAB_URL"/api/v3/projects/$CI_PROJECT_ID/environments" - export JSON=$(curl --header "$ENVIRONMENT_REQ_HEADER" "$ENV_ID_REQ_URL") - export PARSE_JSON='$r = json_decode(fgets(STDIN)); foreach ($r as $e) if ($e->name == "'$CI_ENVIRONMENT_NAME'") { echo $e->id; break;}' - export CI_ENVIRONMENT_ID=$(echo $JSON | php -r "$PARSE_JSON") - export ENV_SET_URL=$GITLAB_URL"/api/v3/projects/$CI_PROJECT_ID/environments/$CI_ENVIRONMENT_ID" - curl --request PUT --data "external_url=$ENVURL" --header "$ENVIRONMENT_REQ_HEADER" "$ENV_SET_URL" environment: name: $CI_BUILD_REF_SLUG on_stop: delete_env only: - branches except: - master ``` * Caveat: This will not work if your CI_ENVIRONMENT_NAME is not unique. Though, I can't imagine how that would happen * OMG! You used `something` that is stupid. ... I would really like the above snippet to either get better or for us to not have to revert to using the API to make a URL dynamic. * I've not tested on a new environment yet, but I expect it to not work. </details>