Skip to content

Speed up path generation with build artifacts

Stan Hu requested to merge sh-speed-up-build-artifact-entry into master

Previously it would take seconds to render the pipeline index JSON (/gitlab-org/gitlab/pipelines.json) in the GitLab project. Apparently, we render over 5000 paths for BuildArtifactEntity, and generation of these paths dominated the request time.

The Rails path helpers are slow because they check the namespace and project parameters against large regular expressions, which include TOP_LEVEL_ROUTES and PROJECT_WILDCARD_ROUTES, respectively.

We can speed this up by generating the paths directly with strings. While this isn't ideal, this results in a 10x improvement in speed:

Calculating -------------------------------------
                orig    86.000  i/100ms
                 new     1.272k i/100ms
-------------------------------------------------
                orig      1.343k (±13.8%) i/s -      6.536k
                 new     13.756k (±15.3%) i/s -     67.416k

Comparison:
                 new:    13756.3 i/s
                orig:     1343.3 i/s - 10.24x slower

To prevent these helper methods from breaking if the routes ever change, we add tests to ensure that the results are the same.

Closes #121929 (closed)

Edited by Stan Hu

Merge request reports