Add Ansi2json V2 converter behind feature flag
What does this MR do and why?
In !145499 (merged), @mrincon added a JS renderer for full log rendering based on a PoC I'd written.
The hope was that perhaps eventually, we could move all log rendering over to JS. But that's a big task.
The best step for now is probably rewriting the Ruby-renderer.
This adds a V2 of the CI job log ANSI-to-JSON parser at
lib/gitlab/ci/ansi2json/v2/, gated by the new ci_ansi2json_v2
feature flag (default off) at the call site in Ci::BuildTrace.
V2 is a Ruby-idiomatic reimplementation structurally inspired by the JS
scanner at app/assets/javascripts/ci/job_log_viewer/lib/scanner.js.
Output is byte-identical to V1: the V1 spec body is wrapped in a shared
example group and runs against both implementations.
Internals
- A single stateful
AnsiEvaluator(case/when dispatch) replaces the per-commandParser.new+ Hash allocation in V1'sParser/Style. - SGR sequences are parsed with a byte cursor, eliminating the
.*?lazy-regex backtracking and capture allocation in V1's tokenizer. V2::Linebuilds segment hashes directly, dropping V1's intermediateSegmentclass and a layer of#to_hindirection.- A reused
StringScannerand a pooledLinecut aLine, segments array, andStringScannerper input line. V2::StatesubclassesV1::Stateto swap inV2::Line. HMAC encoding, signature handling, section tracking, and style restoration are inherited unchanged, so the state blob is interchangeable across versions (a request that started parsing under V1 can resume under V2 and vice-versa).
Performance
| Fixture | V1 | V2 | Speedup | Allocations cut |
|---|---|---|---|---|
| 192 KB / ~3.5k lines (existing fixture) | 33 ms | 13 ms | 2.5× | 63% |
| 88 KB / 1k lines (synthetic) | 19 ms | 8 ms | 2.4× | 66% |
1.04 MB / 6.8k lines (real gitlab-runner trace) |
132 ms | 54 ms | 2.4× | 59% |
| 2.56 MB / 30k lines (synthetic) | 635 ms | 275 ms | 2.3× | 66% |
References
- JS scanner used as architectural inspiration:
scanner.jsandansi_evaluator.jsunderapp/assets/javascripts/ci/job_log_viewer/lib/. - Feature flag rollout issue: #599447
How to set up and validate locally
- Enable the feature flag in a Rails console:
Feature.enable(:ci_ansi2json_v2) # or for canary: Feature.enable(:ci_ansi2json_v2, Project.find(<id>)) - Visit any job's log page, or hit the JSON endpoint directly:
GET /<namespace>/<project>/-/jobs/<id>/trace.json - The rendered log should be visually identical to V1.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist.