Rails link generation is slow
Summary
Rails link generation is slow when parameter constraints are complex regular expressions. Affected parameters:
-
project_id
-Gitlab::PathRegex.project_route_regex
-
namespace_id
-Gitlab::PathRegex.full_namespace_route_regex
Performance test: Generate 10K links
project = Project.last
# preload route components
project.route
project.namespace.route
pipeline = Ci::Pipeline.last
N = 10000
# warmup
Gitlab::Routing.url_helpers.project_pipeline_url(project, pipeline)
Gitlab::Routing.url_helpers.new_user_session_url
Benchmark.bm(50) do |x|
x.report('project_pipeline_url(project, pipeline)') do
N.times do
Gitlab::Routing.url_helpers.project_pipeline_url(project, pipeline)
end
end
x.report('new_user_session') do
N.times do
Gitlab::Routing.url_helpers.new_user_session_url
end
end
end
Result:
project_pipeline_url(project, pipeline) 1.717725 0.007199 1.724924 ( 1.724931)
new_user_session 0.205695 0.001109 0.206804 ( 0.206806)
We need about 0.17ms to generate a URL using project_pipeline_url
helper. Generating one URL is quite expensive if we compare it to an average Redis cache call (fetching an integer value) which takes around 0.25ms.
Culprit: ActionDispatch::Jorney::Formatter#missing_keys
Important part from missing_keys
:
unless /\A#{tests[key]}\Z/ === parts[key]
missing_keys ||= []
missing_keys << key
end
-
missing_keys
method checks if all the required parameters are provided and valid. -
tests[key]
is the regex forproject_id
ornamespace_id
. - The regex is first casted to a string, then a new regex is created.
- The regex needs to be compiled (I think the most time is spent here).
- This process is executed for each link we generate.
Improvements
To reduce the computation time for the new regex, we should cache the newly created regex. This is tricky since the code is part of ActionDispatch
. Performance after overriding (caching) the missing_keys
method:
project_pipeline_url(project, pipeline) 0.396215 0.001382 0.397597 ( 0.397596)
new_user_session 0.214758 0.006118 0.220876 ( 0.220878)
Benefits
I'd expect a slight response time improvement (few ms tops), especially on large API calls (per_page=100
) that contains generated URLS.
Risks
Overriding framework code is risky. We should make sure that we have test cases for detecting changes in the framework code.
Involved components
URL generation, ActionDispatch