Skip to content

Fix trace short writes when large masks are configured

Arran Walker requested to merge ajwalker/fix-trace-safe-token into main

What does this MR do?

Fixes an issue where trace's buffer.Write() can return a transform: short source error when exceeding an internal buffer.

If a large mask has been defined, each log line written is buffered until equal or more in length. This is to detect whether a secret lies within that buffer. However, we use text/transform to perform this, which has an internal buffer size of 4094 bytes. If a secret exceeds this limit, a short source error occurs as the transform asks for more data to be populated, but no more fits.

This MR fixes the short source error from bubbling up to the writer, but in certain situations, will have to reveal the tail of a secret (when the secret is over 4096 bytes). It turns out this is not a breaking change as such, as the older masking implementation had a similar edgecase, but would leak the entire secret, not just the tail. This needs to be documented as a technical limitation of masking.

Why was this MR needed?

Trace logs were being truncated and causing jobs to fail.

What's the best way to test this MR?

Manual QA

Have a job that will print a secret variable many, many times:

mask-job:
  script:
    - echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME
    - echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME
    - echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME
    - echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME
    - echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME; echo $MASK_ME
  • To test masking up to 4096 bytes, create a project variable MASK_ME with 4096 characters of content..
  • To test masking beyond 4096 bytes, where the tail of the secret might be revealed, set MASK_ME to a much larger secret.

Fuzzing

!2993 (merged) introduced a fuzz test that should already fail on this type of bug.

docker run -v $(pwd):/code --rm -it golang:1.13.8
go get github.com/dvyukov/go-fuzz/go-fuzz && go get github.com/dvyukov/go-fuzz/go-fuzz-build
cd /code/helpers/trace
mkdir corpus
cp testdata/corpus/* corpus/
go-fuzz-build
go-fuzz

What are the relevant issue numbers?

Closes #27964 (closed)

Edited by Arran Walker

Merge request reports