Fix YAML encoding of multi-line strings in step requests
What does this MR do?
The runner hands step-runner its steps as YAML: the run: keyword and the job script with FF_SCRIPT_TO_STEP_MIGRATION on. A multi-line string that starts with a blank line or a tab, or whose first line is indented more than the rest, fails to decode. A string indented on every line decodes without its indentation. This MR writes every string value in the request as a double-quoted scalar, so it decodes to the text it was built from.
flowchart LR
A["Step request:<br/>run keyword or<br/>script migration"] --> B["steps.NewRequest<br/>addStepsPreamble"]
B -->|before| C["yaml.Marshal writes a literal<br/>block scalar for a multi-line string"]
C --> D["step-runner decode fails,<br/>or indentation is lost"]
B -->|after| G["JSON round trip to plain maps<br/>and lists, numbers stay numbers"]
G --> E["Every string value written<br/>as a double-quoted scalar"]
E --> F["step-runner decodes the same strings"]addStepsPreambleconverts the step tree to plain maps and lists, then marshals it with every string value as a double-quoted scalar. Map keys stay plain.- Numbers, booleans and null encode as before. The conversion decodes JSON without
UseNumber, so numbers arefloat64, the typeRun.UnmarshalJSONalready delivers. - Quoting and sorted map keys change the expected strings in
Test_addStepsPreamble. Test_NewRequest_ScriptLinesSurviveStepRunnerDecodingbuilds a request throughNewRequest, decodes it with step-runner's decoder settings and compares each script line and an env value.Test_NewRequest_InputTypesSurviveStepRunnerDecodingandTest_NewRequest_RunKeywordStepsSurviveStepRunnerDecodingdo the same for numeric, boolean and null inputs and for arun:job payload, and check the decoded types.
A companion step-runner MR covers the other half. Step-runner validation re-marshals the decoded step with the same library, so a string that starts with a blank line or a tab, or whose first line is indented more than the rest, still fails there with this MR alone. A step-runner version bump in go.mod completes the fix once that change is released.
Why was this MR needed?
go.yaml.in/yaml/v3 v3.0.5 writes a multi-line string as a literal block scalar whose indentation does not match where the block sits inside a sequence. What the job sees depends on the script:
- First line blank, or indented more than the rest:
did not find expected '-' indicator. - First line starts with a tab (multi-line only):
found a tab character where an indentation space is expected. - Every line indented: the script runs with its indentation removed, so its output differs from the legacy script path.
The cause is go-yaml issue 76, fixed on go-yaml's main branch (the v4 line) by go-yaml PR 414. No release fixes this yet, so the runner quotes its strings.
- No v4 tag carries PR 414 yet, and no v3 release does.
- Step-runner still decodes with v3, so the runner's output has to suit the v3 decoder.
- Setting the encoder indent to 2 is the smaller change, and it still fails the leading-tab case.
- Quoting works with any encoder. It can go once go-yaml tags a v4 with the fix and both projects move to it.
The tests mirror step-runner's decoder settings (KnownFields(true), spec then step) instead of calling schema.ReadSteps. ReadSteps also validates, and validation still fails for a leading blank line or tab until the companion MR lands. The tests switch to schema.ReadSteps in the step-runner bump.
What's the best way to test this MR?
go test ./steps/... -run 'Test_NewRequest_|Test_addStepsPreamble'On main, Test_NewRequest_ScriptLinesSurviveStepRunnerDecoding fails for the blank-first-line, tab and indentation cases. The input-type tests pass on main and fail if a number reaches step-runner as a string.
End to end, each script ran as a job with FF_SCRIPT_TO_STEP_MIGRATION on, on a self-managed GitLab 19.5.0 instance, and its output was compared with the same job with the flag off:
| Job script | Unmodified 19.5.0 | This MR with the companion step-runner MR | This MR with unmodified step-runner |
|---|---|---|---|
| Ordinary, or a tab inside a line | runs | output equals legacy | output equals legacy |
| Leading blank line | decode error | output equals legacy | fails in step-runner validation |
| First line indented more than the rest | decode error | output equals legacy | fails in step-runner validation |
| Blank lines, then unindented text | decode error | output equals legacy | fails in step-runner validation |
| Leading tab | decode error | output equals legacy | fails in step-runner validation |
| Every line indented | runs, output differs from legacy | output equals legacy | output equals legacy |
What are the relevant issue numbers?
None. The upstream cause is tracked in the go-yaml issue linked above.
An AI coding assistant wrote this change and this description. The assistant also ran each script case in the table on a test VM running a self-managed GitLab 19.5.0 instance, and ran the project's tests and golangci-lint on the changed packages. Only the unit test covers numeric inputs.