Skip to content

Improve job log rendering performance

Lukas 'ai-pi' Eipert requested to merge leipert-improve-ansi2html into master

What does this MR do?

While looking into: https://gitlab.com/gitlab-org/gitlab-ce/issues/65289, I have found three performance problems with our current ansi2html implementation since we have implemented: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/28642

  1. We send down span tags with an empty class, e.g. <span class="">. This adds unnecessary bytes to what could simply be <span>

  2. In certain edge cases we actually send down structures like this:

    Line 1
    Line 2
    Line 3
    Line 4

    turns to:

    <span>Line 1 <br/><span>Line 2 <br/><span>Line 3 <br/><span>Line 4 <br/></span></span></span></span>

    If we have several hundred lines of traces, this ends up in several hundred layers deep of elements. This is causing #65289 (closed), as Safari allows only around 500 levels of nesting elements. And it blows up our HTML because, we could send down:

    <span>Line 1 <br/>Line 2 <br/>Line 3 <br/>Line 4 <br/></span>
  3. Our collapsible job sections are prepending a span to each line, simply in order to add some indentation. This can be merged into the spans of the actual lines and safe a lot of bandwidth.

Take this GitLab CI YAML for example:

image: node:latest

build-1000:
  stage: build
  script:
    - node -e 'process.stdout.write([...Array(500)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(500)].map((x,i) => i).join("\r\n"))'

build-75000:
  stage: build
  script:
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'
    - node -e 'process.stdout.write([...Array(5000)].map((x,i) => i).join("\r\n"))'

We have two jobs, let's look at the size improvements we gain!

Commit 1000 lines size 75000 lines size Improvements
004b72fe 164695 bytes (2042 span elements) 11918137 Bytes (150064 span elements) -
90b68e06 164651 Bytes 11918093 Bytes -
141b26f6 164516 Bytes 11917958 Bytes -
90a32f9d 23332 Bytes (21 span elements) 1418142 Bytes (60 span elements) 7x - 8x times smaller, 97x to 2500x less span elements

Does this MR meet the acceptance criteria?

Conformity

Performance and testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Kamil Trzciński | At GitLab Summit

Merge request reports