Skip to content

Speed up specs for fast spec serializer

Peter Leitzen requested to merge pl-spec-import-export-fast-hash-serializer into master

What does this MR do?

This MR improves the performance of fast hash serializer specs by using let_it_be and before_all where applicable.

Related to &3752.

Contributes to https://gitlab.com/gitlab-org/plan/-/issues/145.

Here's the output from test-prof when run with FPROF=1. Note:

  • Total amount of factories created went down from 1928 to 48 🚀
  • Total events went down from 31570 to 5333
  • Queries saved: 26237

Before

[TEST PROF INFO] EventProf results for sql.active_record

Total time: 00:42.808 of 01:23.704 (51.14%)
Total events: 31570

Top 5 slowest suites (by time):

Gitlab::Impor...HashSerializer (./spec/lib/gitlab/import_export/fast_hash_serializer_spec.rb:5) – 00:42.808 (31570 / 41) of 01:23.704 (51.14%)


Finished in 1 minute 8.6 seconds (files took 2.62 seconds to load)
41 examples, 0 failures

Randomized with seed 5252

[TEST PROF INFO] Factories usage

 Total: 1928
 Total top-level: 1436
 Total time: 68.7781s
 Total uniq factories: 28

   total   top-level     total time      time per call      top-level time               name

     246         205       10.9929s            0.0447s             3.2720s        ci_pipeline
     164          41       31.5298s            0.1923s             8.6126s            project
     164           0        8.0964s            0.0494s             0.0000s          namespace
     164          41        7.2285s            0.0441s             1.4242s               user
      82          82        0.6463s            0.0079s             0.6463s         label_link
      82          82        1.7930s            0.0219s             1.7930s          milestone
      82          82        1.9009s            0.0232s             1.9009s               note
      82          82        4.2879s            0.0523s             4.2879s resource_label_event
      82          82        0.4855s            0.0059s             0.4855s project_custom_attribute
      82          82        1.1775s            0.0144s             1.1775s      project_badge
      41          41       10.6696s            0.2602s            10.6696s            release
      41           0        1.5599s            0.0380s             0.0000s             author
      41          41        0.8963s            0.0219s             0.8963s              group
      41          41        2.0384s            0.0497s             2.0384s              issue
      41          41        1.1924s            0.0291s             1.1924s    project_snippet
      41          41        0.4040s            0.0099s             0.4040s              label
      41          41        0.3053s            0.0074s             0.3053s        group_label
      41          41        8.0079s            0.1953s             8.0079s     label_priority
      41          41        6.7024s            0.1635s             6.7024s      merge_request
      41          41        9.4616s            0.2308s             9.4616s           ci_build
      41          41        0.6713s            0.0164s             0.6713s      commit_status
      41          41        0.9423s            0.0230s             0.9423s    discussion_note
      41          41        1.7457s            0.0426s             1.7457s     note_on_commit
      41          41        0.4349s            0.0106s             0.4349s              event
      41          41        0.5829s            0.0142s             0.5829s            service
      41          41        0.7251s            0.0177s             0.7251s              board
      41          41        0.3902s            0.0095s             0.3902s               list
       1           1        0.0079s            0.0079s             0.0079s            license

After

[TEST PROF INFO] EventProf results for sql.active_record

Total time: 00:07.259 of 00:15.383 (47.19%)
Total events: 5333

Top 5 slowest suites (by time):

Gitlab::Impor...HashSerializer (./spec/lib/gitlab/import_export/fast_hash_serializer_spec.rb:5) – 00:07.259 (5333 / 41) of 00:15.383 (47.19%)



Finished in 21.24 seconds (files took 2.27 seconds to load)
41 examples, 0 failures

[TEST PROF INFO] Factories usage

 Total: 48
 Total top-level: 36
 Total time: 3.2380s
 Total uniq factories: 28

   total   top-level     total time      time per call      top-level time               name

       6           5        0.2706s            0.0451s             0.0849s        ci_pipeline
       4           1        0.2345s            0.0586s             0.1312s               user
       4           1        1.4406s            0.3602s             0.2139s            project
       4           0        0.1880s            0.0470s             0.0000s          namespace
       2           2        0.0147s            0.0074s             0.0147s         label_link
       2           2        0.1659s            0.0829s             0.1659s          milestone
       2           2        0.0602s            0.0301s             0.0602s               note
       2           2        0.0958s            0.0479s             0.0958s resource_label_event
       2           2        0.0244s            0.0122s             0.0244s project_custom_attribute
       2           2        0.1518s            0.0759s             0.1518s      project_badge
       1           1        0.0140s            0.0140s             0.0140s            license
       1           1        0.9167s            0.9167s             0.9167s            release
       1           0        0.0550s            0.0550s             0.0000s             author
       1           1        0.0298s            0.0298s             0.0298s              group
       1           1        0.1308s            0.1308s             0.1308s              issue
       1           1        0.0488s            0.0488s             0.0488s    project_snippet
       1           1        0.0281s            0.0281s             0.0281s              label
       1           1        0.0153s            0.0153s             0.0153s        group_label
       1           1        0.2836s            0.2836s             0.2836s     label_priority
       1           1        0.2438s            0.2438s             0.2438s      merge_request
       1           1        0.3477s            0.3477s             0.3477s           ci_build
       1           1        0.0102s            0.0102s             0.0102s      commit_status
       1           1        0.0424s            0.0424s             0.0424s    discussion_note
       1           1        0.0462s            0.0462s             0.0462s     note_on_commit
       1           1        0.0280s            0.0280s             0.0280s              event
       1           1        0.0560s            0.0560s             0.0560s            service
       1           1        0.0441s            0.0441s             0.0441s              board
       1           1        0.0097s            0.0097s             0.0097s               list

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Peter Leitzen

Merge request reports