Secret masking interrupted by newline injected by container logs at 16K offsets
@ratchade recently detected that secret masking can be interrupted by a newline appearing from Docker and Kubernetes at 16K offsets from the previous newline.
This is because the container runtime's logging API has a line-length limit, and if 16K bytes is exceeded, a newline is automatically inserted. As a result, any secret that crosses this boundary is unable to be detected and will not be masked.
The larger the secret, the more chance it has of crossing this 16K boundary.
Due to the fixed offset, any public jobs relying on masking are potentially at risk, as MRs may be able to subtly inject a large amount of data to appear in a log, pushing a masked variable towards the boundary.
Proposal
The 16K limit is hard-coded and happens with both Docker and Kubernetes (although I'm not sure if the limit is in Kubernetes or containerd).
For these executors, we could have an input filter for masking that drops the newline character at the 16K boundary.