Limit has_timestamps? to reading the header prefix

What does this MR do and why?

Gitlab::Ci::Trace::Stream#has_timestamps? used to detect job log timestamps by seeking to offset 0, calling stream.readline, and matching the whole first line against TIMESTAMP_HEADER_REGEX. It now reads a fixed 32-byte prefix (TIMESTAMP_HEADER_LENGTH) with stream.read and matches that prefix instead.

has_timestamps? runs inside Stream#extract_coverage, which Ci::Build#update_coverage calls for every finished build that has a coverage regex. Both ChunkedIO#readline and Gitlab::HttpIO#readline loop chunk by chunk until they find a newline. If the first line of the trace has no newline, readline read the entire trace forward, from object storage or trace chunks, before the backward coverage scan even started. The runner's timestamp header is a fixed 32 bytes at the start of a line (a 27-byte ISO 8601 datetime, a space, a two-hex-digit stream id, E or O, and + or space), so reading those 32 bytes is enough to decide, in one chunk fetch instead of a full forward scan.

Behavior change: TIMESTAMP_HEADER_REGEX is unanchored, so previously a header appearing anywhere in the first line counted as timestamped. Now only a header at the very start of the trace does. GitLab Runner only ever writes the header at the start of a line, so this only affects traces not produced by GitLab Runner.

References

Screenshots or screen recordings

Not applicable, backend-only change.

How to set up and validate locally

  1. In a Rails console, pick a finished GDK build whose log you can overwrite, and give it a trace whose first line spans four chunks:

    build = Ci::Build.finished.last
    build.trace.erase!
    build.trace.set(('x' * 400_000) + "\nTOTAL   85%\n")
    build.update!(coverage_regex: '^TOTAL\s+(\d+)%$')
  2. Record which chunks are read during extraction:

    reads = Hash.new(0)
    Ci::BuildTraceChunk.prepend(Module.new do
      define_method(:data) { reads[chunk_index] += 1; super() }
    end)
    
    build.trace.extract_coverage(build.coverage_regex) # => "85"
    reads.keys.sort
  3. Before this change the result is [0, 1, 2, 3]: every chunk is read forward looking for the first newline. After it, the result is [0, 3]: the header prefix and the tail chunk holding the coverage line.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Hordur Freyr Yngvason

Merge request reports

Loading
Loading