Backend: Optimize build.execute_hooks when creating pipeline
Summary
Using the steps defined in the main Epic, I downloaded the stackprof profile report and used it in https://www.speedscope.app/ to see the slowest part of the pipeline creation.
This looked interesting;
We are spending ~500ms seconds to run execute_hooks. And half of the time is spent on Gitlab::DataBuilder::Build.build(self) -> build.retries_count. We may get rid of retries_count if not used.
Proposal
See this thread; #452059 (comment 2245002810)
This is our current method;
def execute_hooks
return unless project
return if user&.blocked?
ActiveRecord::Associations::Preloader.new(records: [self], associations: { runner: :tags }).call
project.execute_hooks(build_data.dup, :job_hooks) if project.has_active_hooks?(:job_hooks)
project.execute_integrations(build_data.dup, :job_hooks) if project.has_active_integrations?(:job_hooks)
end
How about we could do this instead?
def execute_hooks
return unless project
return if user&.blocked?
ActiveRecord::Associations::Preloader.new(records: [self], associations: { runner: :tags }).call
Ci::ExecuteBuildHooksWorker.perform_async(project.id, build_data)
end
Building the data is still an expensive operation but at least we'd pass this execution to async. I think we are okay passing this big information to a worker.
Edited by Furkan Ayhan
