Skip to content

Defer costly parameter assignments in Seed::Build

Fabio Pitino requested to merge ci-seed-defer-params-assignment into master

What does this MR do and why?

Related to #385850 (closed) (main issue)

Related to #386018 (closed) (ff rollout issue)

In Skip assign tag_list and build_needs in Seed::B... (#385850 - closed), while profiling the pipeline creation and looking at Chain::Seed step, we realized that:

  • Assigning tag_list to Ci::Build takes 5.4% (almost 1s) based on the profile data below.
  • Assigning needs_attributes to Ci::Build takes 4.9%.
  • In total we spend 2s in assigning attributes that then we persist in Bulk

image

In this MR we move the assignment of tag_list and needs_attributes from initial_attributes to deferred_attributes, after evaluating job rules. For example: let's supposed we have a .gitlab-ci.yml that defines 1000 jobs in total. After evaluating job rules, the pipeline contains 200 jobs.

  • initial_attributes are assigned to all 1000 jobs
  • deferred_attributes are assigned to the 200 jobs

There should be a performance gain by doing these heavy assignments only to jobs that are needed, given also the fact that tag_list and needs_attributes are not used for evaluating job rules.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Fabio Pitino

Merge request reports