Backend: Optimize build.execute_hooks when creating pipeline

Summary

Using the steps defined in the main Epic, I downloaded the stackprof profile report and used it in https://www.speedscope.app/ to see the slowest part of the pipeline creation.

This looked interesting;

Screenshot_2024-10-15_at_16.18.26

We are spending ~500ms seconds to run execute_hooks. And half of the time is spent on Gitlab::DataBuilder::Build.build(self) -> build.retries_count. We may get rid of retries_count if not used.

Proposal

See this thread; #452059 (comment 2245002810)

This is our current method;

    def execute_hooks
      return unless project
      return if user&.blocked?

      ActiveRecord::Associations::Preloader.new(records: [self], associations: { runner: :tags }).call

      project.execute_hooks(build_data.dup, :job_hooks) if project.has_active_hooks?(:job_hooks)
      project.execute_integrations(build_data.dup, :job_hooks) if project.has_active_integrations?(:job_hooks)
    end

How about we could do this instead?


    def execute_hooks
      return unless project
      return if user&.blocked?

      ActiveRecord::Associations::Preloader.new(records: [self], associations: { runner: :tags }).call

      Ci::ExecuteBuildHooksWorker.perform_async(project.id, build_data)
    end

Building the data is still an expensive operation but at least we'd pass this execution to async. I think we are okay passing this big information to a worker.

Edited by Furkan Ayhan