Add Ansi2json V2 converter behind feature flag

What does this MR do and why?

In !145499 (merged), @mrincon added a JS renderer for full log rendering based on a PoC I'd written.

The hope was that perhaps eventually, we could move all log rendering over to JS. But that's a big task.

The best step for now is probably rewriting the Ruby-renderer.

This adds a V2 of the CI job log ANSI-to-JSON parser at lib/gitlab/ci/ansi2json/v2/, gated by the new ci_ansi2json_v2 feature flag (default off) at the call site in Ci::BuildTrace.

V2 is a Ruby-idiomatic reimplementation structurally inspired by the JS scanner at app/assets/javascripts/ci/job_log_viewer/lib/scanner.js. Output is byte-identical to V1: the V1 spec body is wrapped in a shared example group and runs against both implementations.

Internals

  • A single stateful AnsiEvaluator (case/when dispatch) replaces the per-command Parser.new + Hash allocation in V1's Parser/Style.
  • SGR sequences are parsed with a byte cursor, eliminating the .*? lazy-regex backtracking and capture allocation in V1's tokenizer.
  • V2::Line builds segment hashes directly, dropping V1's intermediate Segment class and a layer of #to_h indirection.
  • A reused StringScanner and a pooled Line cut a Line, segments array, and StringScanner per input line.
  • V2::State subclasses V1::State to swap in V2::Line. HMAC encoding, signature handling, section tracking, and style restoration are inherited unchanged, so the state blob is interchangeable across versions (a request that started parsing under V1 can resume under V2 and vice-versa).

Performance

Fixture V1 V2 Speedup Allocations cut
192 KB / ~3.5k lines (existing fixture) 33 ms 13 ms 2.5× 63%
88 KB / 1k lines (synthetic) 19 ms 8 ms 2.4× 66%
1.04 MB / 6.8k lines (real gitlab-runner trace) 132 ms 54 ms 2.4× 59%
2.56 MB / 30k lines (synthetic) 635 ms 275 ms 2.3× 66%

References

  • JS scanner used as architectural inspiration: scanner.js and ansi_evaluator.js under app/assets/javascripts/ci/job_log_viewer/lib/.
  • Feature flag rollout issue: #599447

How to set up and validate locally

  1. Enable the feature flag in a Rails console:
    Feature.enable(:ci_ansi2json_v2)
    # or for canary:
    Feature.enable(:ci_ansi2json_v2, Project.find(<id>))
  2. Visit any job's log page, or hit the JSON endpoint directly: GET /<namespace>/<project>/-/jobs/<id>/trace.json
  3. The rendered log should be visually identical to V1.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

Edited by Arran Walker

Merge request reports

Loading