Fix YAML encoding of multi-line strings in step requests

What does this MR do?

The runner hands step-runner its steps as YAML: the run: keyword and the job script with FF_SCRIPT_TO_STEP_MIGRATION on. A multi-line string that starts with a blank line or a tab, or whose first line is indented more than the rest, fails to decode. A string indented on every line decodes without its indentation. This MR writes every string value in the request as a double-quoted scalar, so it decodes to the text it was built from.

flowchart LR
    A["Step request:<br/>run keyword or<br/>script migration"] --> B["steps.NewRequest<br/>addStepsPreamble"]
    B -->|before| C["yaml.Marshal writes a literal<br/>block scalar for a multi-line string"]
    C --> D["step-runner decode fails,<br/>or indentation is lost"]
    B -->|after| G["JSON round trip to plain maps<br/>and lists, numbers stay numbers"]
    G --> E["Every string value written<br/>as a double-quoted scalar"]
    E --> F["step-runner decodes the same strings"]
  • addStepsPreamble converts the step tree to plain maps and lists, then marshals it with every string value as a double-quoted scalar. Map keys stay plain.
  • Numbers, booleans and null encode as before. The conversion decodes JSON without UseNumber, so numbers are float64, the type Run.UnmarshalJSON already delivers.
  • Quoting and sorted map keys change the expected strings in Test_addStepsPreamble.
  • Test_NewRequest_ScriptLinesSurviveStepRunnerDecoding builds a request through NewRequest, decodes it with step-runner's decoder settings and compares each script line and an env value.
  • Test_NewRequest_InputTypesSurviveStepRunnerDecoding and Test_NewRequest_RunKeywordStepsSurviveStepRunnerDecoding do the same for numeric, boolean and null inputs and for a run: job payload, and check the decoded types.

A companion step-runner MR covers the other half. Step-runner validation re-marshals the decoded step with the same library, so a string that starts with a blank line or a tab, or whose first line is indented more than the rest, still fails there with this MR alone. A step-runner version bump in go.mod completes the fix once that change is released.

Why was this MR needed?

go.yaml.in/yaml/v3 v3.0.5 writes a multi-line string as a literal block scalar whose indentation does not match where the block sits inside a sequence. What the job sees depends on the script:

  • First line blank, or indented more than the rest: did not find expected '-' indicator.
  • First line starts with a tab (multi-line only): found a tab character where an indentation space is expected.
  • Every line indented: the script runs with its indentation removed, so its output differs from the legacy script path.

The cause is go-yaml issue 76, fixed on go-yaml's main branch (the v4 line) by go-yaml PR 414. No release fixes this yet, so the runner quotes its strings.

  • No v4 tag carries PR 414 yet, and no v3 release does.
  • Step-runner still decodes with v3, so the runner's output has to suit the v3 decoder.
  • Setting the encoder indent to 2 is the smaller change, and it still fails the leading-tab case.
  • Quoting works with any encoder. It can go once go-yaml tags a v4 with the fix and both projects move to it.

The tests mirror step-runner's decoder settings (KnownFields(true), spec then step) instead of calling schema.ReadSteps. ReadSteps also validates, and validation still fails for a leading blank line or tab until the companion MR lands. The tests switch to schema.ReadSteps in the step-runner bump.

What's the best way to test this MR?

go test ./steps/... -run 'Test_NewRequest_|Test_addStepsPreamble'

On main, Test_NewRequest_ScriptLinesSurviveStepRunnerDecoding fails for the blank-first-line, tab and indentation cases. The input-type tests pass on main and fail if a number reaches step-runner as a string.

End to end, each script ran as a job with FF_SCRIPT_TO_STEP_MIGRATION on, on a self-managed GitLab 19.5.0 instance, and its output was compared with the same job with the flag off:

Job script Unmodified 19.5.0 This MR with the companion step-runner MR This MR with unmodified step-runner
Ordinary, or a tab inside a line runs output equals legacy output equals legacy
Leading blank line decode error output equals legacy fails in step-runner validation
First line indented more than the rest decode error output equals legacy fails in step-runner validation
Blank lines, then unindented text decode error output equals legacy fails in step-runner validation
Leading tab decode error output equals legacy fails in step-runner validation
Every line indented runs, output differs from legacy output equals legacy output equals legacy

What are the relevant issue numbers?

None. The upstream cause is tracked in the go-yaml issue linked above.

An AI coding assistant wrote this change and this description. The assistant also ran each script case in the table on a test VM running a self-managed GitLab 19.5.0 instance, and ran the project's tests and golangci-lint on the changed packages. Only the unit test covers numeric inputs.

Merge request reports

Loading
Loading