Slow EnsureEnvironments on pipeline creation

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Summary

Ref: &14102 (comment 2785438645)

We must investigate and solve why EnsureEnvironments is slow.

We need to fix this holistically, but we have a few issues:

• We iterates over ALL jobs in the pipeline, but the service only does work for jobs with environments. Fix here: !210671 (diffs)
• possible N+1s:
  • safe_find_or_create_by for the environment
  • cluster agent
  • ci_access_authorizations_for_project query per job
• job.simple_variables.sort_and_expand_all ????
• batching could probably happen although this might be an enormous refactor idk