Skip timestamps in log if present when generating HTML
What does this MR do and why?
NOTE: This MR is best reviewed commit-by-commit.
This MR prepares the Ansi2html
class to skip timestamps emitted by the runner, when the FF_TIMESTAMPS
feature flag is enabled (either on the runner config itself, or as an environment variable for the job).
Logs are processed by the Gitlab::Ci::Ansi2html
and Gitlab::Ci::Ansi2json
classes, to produce HTML and JSON variants. Recently, the runner has introduced a FF_TIMESTAMPS
feature flag, which will cause it to emit lines with a slightly modified logic:
E.g. 2024-05-14T11:19:20.000000Z 00O+Hey there!
. The first 32 characters are a timestamp prefix, and must be treated as optional. If present, Ansi2html
must not include it in its output.
Format:
<timestamp> <stream number><stream type><full line type>
-
timestamp
: UTC RFC3339 timestamp -
stream number
: 1 byte (2 hex chars) stream number -
stream type
:E
/O
(Err or Out) -
full line type
:+
if line is continuation of previous line,
Part of #455581 (closed)
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Performance benchmarks
runs = 100; branch_name = `git branch --show-current`.strip; Benchmark.bm do |x|
x.report("Large log with timestamps (#{branch_name})") do
runs.times do
json = File.open("~/Downloads/raw-large.txt", "r") { |stream| converter = Gitlab::Ci::Ansi2html::Converter.new.convert(stream, nil) }
end
end
x.report("Large log without timestamps (#{branch_name})") do
runs.times do
json = File.open("~/Downloads/raw-no-ts-large.txt", "r") { |stream| converter = Gitlab::Ci::Ansi2html::Converter.new.convert(stream, nil) }
end
end
end
Scenario | user | system | total | real |
---|---|---|---|---|
Large log with timestamps (master ) |
155.212844 | 1.004314 | 156.217158 | 159.158386 |
Large log without timestamps (master ) |
82.597942 | 0.550397 | 83.148339 | 84.355574 |
Large log with timestamps (this branch) | 82.770476 | 0.563733 | 83.334209 | 84.738522 |
Large log without timestamps (this branch) | 84.139308 | 0.487001 | 84.626309 | 86.217939 |
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
The tests should cover all the required scenarios.