Single export file

Each exec step is given the opportunity to mutate the global environment by writing to an environment file. Within a step definition the file is referenced with the syntax ${{ export_file }}. The export and output file are both created together and are unique for each exec call.

However a step invocation like this doesn't work:

- name: add_go_to_path
  script: echo PATH=/usr/local/go/bin:$PATH | tee ${{ export_file }}
- name: check_go
  script: go version

This is because inputs are expanded before invoking the step: https://gitlab.com/gitlab-org/step-runner/-/blob/fb8e782787609d85ea439e24e8c5491350ae6a99/pkg/runner/runner.go#L322. So ${{ export_file }} expands to "" (empty string) since there is no export file for composite steps.

Is the problem that ${{ export_file }} should be evaluated after invocation? No. This behavior is expected as it's normal behavior for most modern programming languages to evaluate argument before function invocation. And expressions referencing outputs of previous steps must use the same StepsContext (same level) because steps are not allowed to reference the context of their caller (again normal behavior).

Should we create an export file for composite steps? No. We could but it would need to be read between each sub-step to make this ☝ example work. But the exec step already does exports so it's strange to do it again on the caller side.

We can sidestep this problem by using a single export file for the whole global context. Then whenever ${{ export_file }} is evaluated, exports will happen after each exec as expected. Each step has an opportunity to append to or replace the export file. Each export operation reads the file anew and updates the global environment. But it doesn't remove values. Steps cannot "unset" global environment variables.

Note: we don't need to worry about concurrent access to the file because parallelism will be done by multiple run requests, each of which will create a new and unique export file. See Parallel Execution and Other Strategies (#29) • Unassigned