Log masking crashes on long data

Summary

We notarize DMG installer during CI build. Since 13.12 gitlab runner crashes on Apple's response. The response is written to STDOUT, it is a JSON with a ~5k chars long base64-encoded value.

Steps to reproduce

(not tested)

  • Create job to output long line (>4k chars) with content "value={base64 encoded data}" (cat or echo should be sufficient)
  • Perhaps any long line is sufficient?

Actual behavior

CI job fails, logs truncated

Expected behavior

CI job succeeds, logs intact

Relevant logs

log from 13.12.0 (7a6612da) 
Running with gitlab-runner 13.12.0 (7a6612da)
  on telco-builder-osx fbKwDrTe
Preparing the "shell" executor 00:00
Using Shell executor...
...
<<<lots of lines snipped off>>>
...
[2021/06/04 10:21:55] Size of data is 6293
[2021/06/04 10:21:55] JSON Response is: {
[2021/06/04 10:21:55]     records =     (
[2021/06/04 10:21:55]                 {
[2021/06/04 10:21:55]             created =             {
[2021/06/04 10:21:55]                 deviceID = 2;
[2021/06/04 10:21:55]                 timestamp = 1622794833871;
[2021/06/04 10:21:55]                 userRecordName = "...";
[2021/06/04 10:21:55]             };
[2021/06/04 10:21:55]             deleted = 0;
[2021/06/04 10:21:55]             fields =             {
[2021/06/04 10:21:55]                 signedTicket =                 {
[2021/06/04 10:21:55]                     type = BYTES;
Uploading artifacts for failed job 00:01
Uploading artifacts...
Runtime platform                                    arch=amd64 os=darwin pid=69095 revision=7a6612da version=13.12.0
build/*/test-results/*/*.xml: found 142 matching files and directories 
Uploading artifacts as "junit" to coordinator... ok  id=37107 responseStatus=201 Created token=-Sfb9Dy2
Cleaning up file based variables 00:00
ERROR: Job failed (system failure): transform: short internal buffer
log from 14.0.0~beta.133.g6b64d576 (6b64d576) 
Running with gitlab-runner 14.0.0~beta.133.g6b64d576 (6b64d576)
  on telco-builder-osx fbKwDrTe
Preparing the "shell" executor 00:00
Using Shell executor...
...
<<<lots of lines snipped off>>>
...
[2021/06/04 11:25:26] Size of data is 6293
[2021/06/04 11:25:26] JSON Response is: {
[2021/06/04 11:25:26]     records =     (
[2021/06/04 11:25:26]                 {
[2021/06/04 11:25:26]             created =             {
[2021/06/04 11:25:26]                 deviceID = 2;
[2021/06/04 11:25:26]                 timestamp = 1622798692294;
[2021/06/04 11:25:26]                 userRecordName = "...";
[2021/06/04 11:25:26]             };
[2021/06/04 11:25:26]             deleted = 0;
[2021/06/04 11:25:26]             fields =             {
[2021/06/04 11:25:26]                 signedTicket =                 {
[2021/06/04 11:25:26]                     type = BYTES;
[2021/06/04 11:25:26]                     value =
"czhj...+DNkwI/C

the line contains only the first 4096 chars.

log from 13.11.0 (7f7a4bb0) 
Running with gitlab-runner 13.11.0 (7f7a4bb0)
  on telco-builder-osx fbKwDrTe
Preparing the "shell" executor 00:00
Using Shell executor...
...
<<<lots of lines snipped off>>>
...
[2021/06/04 11:50:33] Size of data is 6293
[2021/06/04 11:50:33] JSON Response is: {
[2021/06/04 11:50:33]     records =     (
[2021/06/04 11:50:33]                 {
[2021/06/04 11:50:33]             created =             {
[2021/06/04 11:50:33]                 deviceID = 2;
[2021/06/04 11:50:33]                 timestamp = 1622800170772;
[2021/06/04 11:50:33]                 userRecordName = "_d28c74d190a3782e89496b0a13437fef";
[2021/06/04 11:50:33]             };
[2021/06/04 11:50:33]             deleted = 0;
[2021/06/04 11:50:33]             fields =             {
[2021/06/04 11:50:33]                 signedTicket =                 {
[2021/06/04 11:50:33]                     type = BYTES;
[2021/06/04 11:50:33]                     value = "czhjaAEAAADxBQAAWQoAADCCBe0wggL/MIICpKADAgECAgh70HPd0oNpUTAKBggqhkjO...
<<<few lines snipped off>>>
...
xEwsb+7OOI9X1OeXe22JQ3h8ggKxZ+BWnBrJwIhAIhpSaSgeHIVJv5VJMcrwmraiwLzJdSBlRGC6OqH9gVnAA==";
[2021/06/04 11:50:33]                 };
...
<<<lots of lines snipped off>>>
...
Job succeeded

Environment description

  • On premises gitlab installation
  • Shell executor on mac os 11.4.

Used GitLab Runner version

See the logs. Platform is darwin-amd64.

Tested with:

  • 14.0.0~beta.133.g6b64d576 (6b64d576) (broken; only shows first 4096 chars of the log line)
  • 13.12.0 (7a6612da) (broken; does not shows the long line at all)
  • 13.11.0 (7f7a4bb0) (good)

Possible fixes

The error message suggests that it is related to https://github.com/golang/text/blob/master/transform/transform.go#L37

Known workarounds

At the moment if you have a large variable that is 4096 or above we suggest you either remove the masking feature for that variable or move it to a file-based variable.

/cc @ajwalker

Edited by Katrin Leinweber