Log masking crashes on long data
Summary
We notarize DMG installer during CI build. Since 13.12 gitlab runner crashes on Apple's response. The response is written to STDOUT, it is a JSON with a ~5k chars long base64-encoded value.
Steps to reproduce
(not tested)
- Create job to output long line (>4k chars) with content "value={base64 encoded data}" (cat or echo should be sufficient)
- Perhaps any long line is sufficient?
Actual behavior
CI job fails, logs truncated
Expected behavior
CI job succeeds, logs intact
Relevant logs
log from 13.12.0 (7a6612da)
Running with gitlab-runner 13.12.0 (7a6612da)
on telco-builder-osx fbKwDrTe
Preparing the "shell" executor 00:00
Using Shell executor...
...
<<<lots of lines snipped off>>>
...
[2021/06/04 10:21:55] Size of data is 6293
[2021/06/04 10:21:55] JSON Response is: {
[2021/06/04 10:21:55] records = (
[2021/06/04 10:21:55] {
[2021/06/04 10:21:55] created = {
[2021/06/04 10:21:55] deviceID = 2;
[2021/06/04 10:21:55] timestamp = 1622794833871;
[2021/06/04 10:21:55] userRecordName = "...";
[2021/06/04 10:21:55] };
[2021/06/04 10:21:55] deleted = 0;
[2021/06/04 10:21:55] fields = {
[2021/06/04 10:21:55] signedTicket = {
[2021/06/04 10:21:55] type = BYTES;
Uploading artifacts for failed job 00:01
Uploading artifacts...
Runtime platform arch=amd64 os=darwin pid=69095 revision=7a6612da version=13.12.0
build/*/test-results/*/*.xml: found 142 matching files and directories
Uploading artifacts as "junit" to coordinator... ok id=37107 responseStatus=201 Created token=-Sfb9Dy2
Cleaning up file based variables 00:00
ERROR: Job failed (system failure): transform: short internal buffer
log from 14.0.0~beta.133.g6b64d576 (6b64d576)
Running with gitlab-runner 14.0.0~beta.133.g6b64d576 (6b64d576)
on telco-builder-osx fbKwDrTe
Preparing the "shell" executor 00:00
Using Shell executor...
...
<<<lots of lines snipped off>>>
...
[2021/06/04 11:25:26] Size of data is 6293
[2021/06/04 11:25:26] JSON Response is: {
[2021/06/04 11:25:26] records = (
[2021/06/04 11:25:26] {
[2021/06/04 11:25:26] created = {
[2021/06/04 11:25:26] deviceID = 2;
[2021/06/04 11:25:26] timestamp = 1622798692294;
[2021/06/04 11:25:26] userRecordName = "...";
[2021/06/04 11:25:26] };
[2021/06/04 11:25:26] deleted = 0;
[2021/06/04 11:25:26] fields = {
[2021/06/04 11:25:26] signedTicket = {
[2021/06/04 11:25:26] type = BYTES;
[2021/06/04 11:25:26] value =
"czhj...+DNkwI/C
the line contains only the first 4096 chars.
log from 13.11.0 (7f7a4bb0)
Running with gitlab-runner 13.11.0 (7f7a4bb0)
on telco-builder-osx fbKwDrTe
Preparing the "shell" executor 00:00
Using Shell executor...
...
<<<lots of lines snipped off>>>
...
[2021/06/04 11:50:33] Size of data is 6293
[2021/06/04 11:50:33] JSON Response is: {
[2021/06/04 11:50:33] records = (
[2021/06/04 11:50:33] {
[2021/06/04 11:50:33] created = {
[2021/06/04 11:50:33] deviceID = 2;
[2021/06/04 11:50:33] timestamp = 1622800170772;
[2021/06/04 11:50:33] userRecordName = "_d28c74d190a3782e89496b0a13437fef";
[2021/06/04 11:50:33] };
[2021/06/04 11:50:33] deleted = 0;
[2021/06/04 11:50:33] fields = {
[2021/06/04 11:50:33] signedTicket = {
[2021/06/04 11:50:33] type = BYTES;
[2021/06/04 11:50:33] value = "czhjaAEAAADxBQAAWQoAADCCBe0wggL/MIICpKADAgECAgh70HPd0oNpUTAKBggqhkjO...
<<<few lines snipped off>>>
...
xEwsb+7OOI9X1OeXe22JQ3h8ggKxZ+BWnBrJwIhAIhpSaSgeHIVJv5VJMcrwmraiwLzJdSBlRGC6OqH9gVnAA==";
[2021/06/04 11:50:33] };
...
<<<lots of lines snipped off>>>
...
Job succeeded
Environment description
- On premises gitlab installation
- Shell executor on mac os 11.4.
Used GitLab Runner version
See the logs. Platform is darwin-amd64.
Tested with:
- 14.0.0~beta.133.g6b64d576 (6b64d576) (broken; only shows first 4096 chars of the log line)
- 13.12.0 (7a6612da) (broken; does not shows the long line at all)
- 13.11.0 (7f7a4bb0) (good)
Possible fixes
The error message suggests that it is related to https://github.com/golang/text/blob/master/transform/transform.go#L37
Known workarounds
At the moment if you have a large variable that is 4096 or above we suggest you either remove the masking
feature for that variable or move it to a file-based variable.
/cc @ajwalker
Edited by Katrin Leinweber