Filter kubernetes trace to remove newline added for long logs in attach mode

What does this MR do?

This MR improves the processing of the logs when in attach mode. Before the change, when the buffer used by k8s is full, GitLab Runner Helper introduces a newline splitting in two a single line. With this MR, we make sure than the added newline is ignored before forwarding the log for processing.

Why was this MR needed?

The newline introduced in attach mode breaks the masking algorithm used within Runner and can create partial or complete reveal of masked variables or prefixed tokens.

What's the best way to test this MR?

Here is a link to the gitlab-ci.yml (as it is a huge file)

In this test we echo 10 blocks with a 3 seconds sleep between each echo.

• 1st line: < 16kb line
• 2nd line: < 16kb line with a newline within
• 3rd line: > 16kb line with newline at the beginning and at the end
• 4th line: > 39kb line without newline within
• 5th line: > 16kb line without newline within
• 6th line: > 18kb line without newline within
• 7th line: > 70kb line with newline within (splitting in two parts > 16kb)
• 8th line: > 34kb line without newline within
• 9th line: > 70kb line without newline within
• 10th line: > 70kb line without newline within. This line is a glpat- token

Test with docker executor

config.toml

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "Local GitLab Runner for tests and debugging"
  url = "https://gitlab.com"
  token = "__TOKEN__"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    image = "alpine:3.11"
    helper_image = "aaro301/gitlab-runner-helper:x86_64-aca69c61"

1. Log succeeds
2. No newline inserted if not needed
3. 10th line is masked

Test with k8s executor - exec mode

config.toml

concurrent = 1
check_interval = 1
log_level = "error"

[session_server]
  session_timeout = 1800

[[runners]]
  url = "https://gitlab.com/"
  token = "__TOKEN__"
  executor = "kubernetes"
  environment=["FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY=true"]
  [runners.kubernetes]
    image = "alpine:latest"
    helper_image = "aaro301/gitlab-runner-helper:x86_64-aca69c61"
    terminationGracePeriodSeconds = 0
    [runners.kubernetes.affinity]
    [runners.kubernetes.dns_config]
    [runners.kubernetes.container_lifecycle]

1. Log succeeds
2. No newline inserted if not needed
3. 10th line is masked

Test with k8s executor - attach mode

config.toml

concurrent = 1
check_interval = 1
log_level = "error"

[session_server]
  session_timeout = 1800

[[runners]]
  url = "https://gitlab.com/"
  token = "__TOKEN__"
  executor = "kubernetes"
  [runners.kubernetes]
    image = "alpine:latest"
    helper_image = "aaro301/gitlab-runner-helper:x86_64-aca69c61"
    terminationGracePeriodSeconds = 0
    [runners.kubernetes.affinity]
    [runners.kubernetes.dns_config]
    [runners.kubernetes.container_lifecycle]

1. Log succeeds
2. No newline inserted in not needed
3. 10th line is masked

What are the relevant issue numbers?

close #29320 (closed)