Convert the output file format to JSON
# Problem Currently a step provides outputs by writing to an output file in the form `KEY=VALUE` where `VALUE` is either a raw string or a JSON value, depending on the declared output type. Having raw string as an output type creates a few problems, one of which is that it's confusing to have both `raw_string` and `string` output types. Callers don't really care about how the step returns a string. To address `raw_string` we are removing it: https://gitlab.com/gitlab-org/step-runner/-/merge_requests/118#note_2153234086. However that means that users always have to provide a JSON value when returning values. Even when those outputs are just a simple string. E.g. `KEY="VALUE"`. There has been some discussion about whether to "auto-detect" raw strings. But that leaves some confusing escaping situations, so we've decided to just do always JSON. This also brought up the point that it's weird to have `KEY=` JSON value instead of just a JSON value. E.g. `{"name":"KEY","value":"VALUE"}`. Historically outputs were only simple strings, so the dotenv format of `KEY=VALUE` fit the problem. However when adding complex types values because JSON strings. And there may be additional changes in the output format. For example, we might want to add metadata such as `sensitive=true`. This would require additional parsing, escaping or extension of the dotenv-like format. # Proposal In order to make the output format less surprising. And to enable future extension, we propose to make all outputs a single line of JSON. Specifically that line will be a JSON object with keys `name` and `value` which specify the outputs name and value (`KEY` and `VALUE`) above. This means the output file will conform to the well-known [JSON Lines format (JSONL)](https://jsonlines.org/). # Examples - before Currently, the output format is `KEY=[JSON value]`. ```yaml spec: outputs: foo: type: string --- # Implementation that outputs "bar". ``` Output file (`${{ output_file }}`): ```json foo="bar" ``` # Examples - after The following is how the output file would look after this issue is complete. ### Single string output ```yaml spec: outputs: foo: type: string --- # Implementation that outputs "bar". ``` Output file (`${{ output_file }}`): ```json {"name":"foo","value":"bar"} ``` ### Multiple complex outputs ```yaml spec: outputs: foo: type: string baz: type: struct duh: type: array --- # Implementation that outputs "bar", {"a":"b:} and [1,2,3] respectively. ``` Output file: ```json {"name":"foo","value":"bar"} {"name":"baz","value":{"a":"b"}} {"name":"duh","value":[1,2,3]} ``` ### Outputting a raw string with newlines ```yaml spec: outputs: ifconfig: type: string --- script: jq -c -n --arg value "$(ifconfig | jq -Rs .)" '{name:"foo",value:$value}' ``` Output file: ```json {"name":"foo","value":"(JSON escaped output of `ifconfig`)"} ```
issue