Improve trace masking performance (!2863) · Merge requests · GitLab.org / gitlab-runner

What does this MR do?

Reintroduces the trace improvements that were merged in !2677 (merged) but subsequently reverted due to problems with the masking setup.

I'm initially pushing this MR without the fix that required the revert. We've since added multiple unit and integration tests that will fail the pipeline.
- https://gitlab.com/gitlab-org/gitlab-runner/-/pipelines/287142595 failed well.
The second commit to this MR will fix these issues and the pipeline should then succeed.
- https://gitlab.com/gitlab-org/gitlab-runner/-/pipelines/287251091 succeeded.
- Worth noting that the fix doesn't require additional tests as they were already added to specifically catch this as part of our corrective actions to the regression.
For a manual QA:
- Add a secret with any key, but with the value SECRET
- Configure a .gitlab-ci.yml with:
```
job:
  script: |
     for ((i = 0 ; i <= 3000 ; i++)); do
       num=$((1+$RANDOM % 5000))
       text=$(printf "%${num}s\n" | tr " " "=")
       p=$(($RANDOM % $num))
       echo ${i}. ${text:0:p}__SECRET__${num}${text:p}
     done
```
  This script will print 1000 lines of text with the text _SECRET_ inside at random locations on random length lines.
  
  It will also likely print far too much data and the data will cut off at a certain limit, also testing that that works as expected.
- Open complete raw job log after the job finishes, and search for _SECRET_ it should return zero results.

Edited Apr 19, 2021 by Arran Walker