Parallelize fixture seeding

Problem

Seeding fixtures is currently run sequentially and takes some time. Fixtures have to be created every time GDK is setup or the database reset.

Proposed solution

While fixtures have to be run in alphabetical order (fixture files are prefixed with <number>_) each fixture file could benefit from inserting the data in parallel (via parallel gem).

For example, https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/fixtures/development/13_comments.rb. The notes could be inserted per user in parallel making the seed faster.

Implementation guide

• Measure how long it takes to seed development/production fixtures sequentially
• Create a fixture helper parallel_each and parallel_map to be used in fixtures
• Create a cop rule to enforce the use of this helper and create TODOs (.rubocop_todo/) to fix them
• Resolve all offenses
• ...
• Profit 🎉

Caveats

It's likely that some fixtures cannot be paralleized to the cop rule must be disabled inline.

Slowest development fixtures

Update 2025-10-02

$ curl -Lsq https://gitlab.com/gitlab-org/gitlab-development-kit/-/jobs/11572077365/raw | rg -o "==.*\d+\.\d+ seconds" | sort -k 4 -nr | uniq | head -15
== Seeding took 344.25 seconds
== ee/db/fixtures/development/20_burndown.rb took 61.39 seconds
== ee/db/fixtures/development/20_vulnerabilities.rb took 50.25 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/09_issues.rb took 48.84 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/03_project.rb took 29.54 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/14_pipelines.rb took 21.00 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/13_comments.rb took 19.38 seconds
== ee/db/fixtures/development/32_compliance_report_violations.rb took 16.69 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/24_forks.rb took 15.44 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/18_abuse_reports.rb took 14.03 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/15_award_emoji.rb took 11.98 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/12_snippets.rb took 9.39 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/06_teams.rb took 6.73 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/34_uploads.rb took 6.72 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/01_admin.rb took 6.56 seconds

Update 2025-09-18

$ curl -Lsq https://gitlab.com/gitlab-org/gitlab-development-kit/-/jobs/11385814434/raw | rg -o "==.*\d+\.\d+ seconds" | sort -k 4 -nr | uniq | head -15
== Seeding took 730.62 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/13_comments.rb took 193.22 seconds
== ee/db/fixtures/development/20_burndown.rb took 138.70 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/09_issues.rb took 84.33 seconds
== ee/db/fixtures/development/20_vulnerabilities.rb took 70.42 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/03_project.rb took 34.19 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/15_award_emoji.rb took 33.86 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/14_pipelines.rb took 29.84 seconds
== ee/db/fixtures/development/32_compliance_report_violations.rb took 20.76 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/24_forks.rb took 20.53 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/10_merge_requests.rb took 19.76 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/18_abuse_reports.rb took 15.35 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/02_users.rb took 13.22 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/34_uploads.rb took 11.55 seconds
== /home/gdk/gdk/gitlab/db/fixtures/development/12_snippets.rb took 9.87 seconds

Status