Properly handle really long lines from subprocess

Axel von Bertoldirequested to merge
avonbertoldi/443/buffer-overflow into main

Prevent Scanner buffer overruns by using a bufio.Reader instead of the original bufio.Scanner. bufio.Reader.ReadBytes() does the necessary buffering internally so we can't run into the original issue.

Note that we made a minor change in the read loop. Scanner.Bytes strips the trailing newline, which we UNCONDITIONALLY re-added by explicitly writing a newline character to the writer. This means that the file streamer was inserting a trailing newline where there might have not been one!!!

Now, instead, we keep the trailing newline returned by the reader, and no longer explicitly write the newline to the writer. This IS a change in behaviour, but I think it's a correction and the original behaviour was wrong. This also changes the possible error cases slightly since previously it was possible for the first write to fail, and the data written would not include the trailing newline. There are a few tests that either assume this or explicitly test for it, and we have to update those.

There's also a unit tests to reproduce error, which failed before the fix.

Fixes step-runner: Handle oversized log lines gracefully (#443 - closed)

Edited by Axel von Bertoldi

Merge request reports