Speed up project model specs by using factory default for namespace
What does this MR do?
This MR improves the performance of project model specs by reusing namespace via FactoryDefault (see https://docs.gitlab.com/ee/development/testing_guide/best_practices.html#optimize-factory-usage).
Related to &3752.
Contributes to https://gitlab.com/gitlab-org/plan/-/issues/145.
Here's the output from test-prof when run with FPROF=1. Note:
-
Totalamount of factories created went down from 2308 to 1870🚀 -
Total eventswent down from 61375 to 54576 - Queries saved: 6799
Before
[TEST PROF INFO] EventProf results for sql.active_record
Total time: 01:42.638 of 03:35.363 (47.66%)
Total events: 61375
Top 5 slowest suites (by time):
Project (./spec/models/project_spec.rb:5) – 01:42.638 (61375 / 1026) of 03:35.363 (47.66%)
Finished in 3 minutes 41.3 seconds (files took 4.71 seconds to load)
1028 examples, 0 failures
Failed examples:
rspec ./spec/models/project_spec.rb:5185 # Project#badges with nested_groups returns the project and the project nested groups badges
rspec ./spec/models/project_spec.rb:6498 # Project#enabled_group_deploy_keys when a project has a parent group and this group has a group deploy key enabled and this group has parent group which also has a group deploy key enabled returns both group deploy keys
[TEST PROF INFO] Factories usage
Total: 2308
Total top-level: 1359
Total time: 167.6076s
Total uniq factories: 75
total top-level total time time per call top-level time name
657 616 127.0594s 0.1934s 119.2710s project
456 17 24.4259s 0.0536s 1.1479s namespace
445 236 21.9498s 0.0493s 11.9747s user
122 118 5.4117s 0.0444s 5.2734s group
122 0 0.7705s 0.0063s 0.0000s namespace_settings
63 3 1.6181s 0.0257s 0.7314s ci_job_artifact
49 45 1.8672s 0.0381s 1.2004s ci_pipeline
34 30 3.0175s 0.0888s 2.2782s ci_build
27 27 0.3204s 0.0119s 0.3204s ci_variable
27 27 4.2109s 0.1560s 4.2109s merge_request
16 16 1.0465s 0.0654s 1.0465s note_on_commit
15 15 3.9543s 0.2636s 3.9543s project_member
14 10 1.3896s 0.0993s 1.1184s cluster
13 13 0.4225s 0.0325s 0.4225s ci_runner
12 12 0.7168s 0.0597s 0.7168s issue
11 0 0.1833s 0.0167s 0.0000s package_file
11 10 0.7616s 0.0692s 0.5944s environment
10 0 0.1851s 0.0185s 0.0000s conan_package_file
10 0 0.0669s 0.0067s 0.0000s conan_file_metadatum
9 9 0.2343s 0.0260s 0.2343s lfs_objects_project
9 9 0.1253s 0.0139s 0.1253s project_auto_devops
8 5 0.1993s 0.0249s 0.0493s lfs_object
8 8 0.0431s 0.0054s 0.0431s upload
8 8 0.8772s 0.1097s 0.8772s project_export_job
7 7 0.2035s 0.0291s 0.2035s deploy_token
6 6 0.4782s 0.0797s 0.4782s service
6 6 1.2773s 0.2129s 1.2773s jira_import_state
6 0 1.2238s 0.2040s 0.0000s label
6 6 0.1454s 0.0242s 0.1454s protected_branch
6 6 0.1436s 0.0239s 0.1436s prometheus_service
5 5 0.2582s 0.0516s 0.2582s import_state
5 5 0.0843s 0.0169s 0.0843s group_badge
5 5 0.3721s 0.0744s 0.3721s chat_name
5 5 0.4271s 0.0854s 0.4271s group_deploy_key
4 0 0.3254s 0.0814s 0.0000s issue_tracker_data
4 0 0.4357s 0.1089s 0.0000s project_feature_usage
4 4 0.0441s 0.0110s 0.0441s project_repository
4 4 0.0172s 0.0043s 0.0172s shard
4 4 0.3587s 0.0897s 0.3587s cluster_platform_kubernetes
4 4 0.2607s 0.0652s 0.2607s ci_instance_variable
4 4 0.0346s 0.0087s 0.0346s deploy_keys_project
4 4 0.6560s 0.1640s 0.6560s project_hook
4 4 0.0704s 0.0176s 0.0704s project_badge
3 3 2.2592s 0.7531s 2.2592s redmine_project
3 0 1.1085s 0.3695s 0.0000s redmine_service
3 3 0.0253s 0.0084s 0.0253s service_desk_setting
3 3 0.3425s 0.1142s 0.3425s deploy_key
3 3 0.1888s 0.0629s 0.1888s npm_package
2 2 0.6190s 0.3095s 0.6190s maven_package
2 0 0.4730s 0.2365s 0.0000s maven_metadatum
2 0 0.4456s 0.2228s 0.0000s package
2 2 0.0503s 0.0251s 0.0503s npm_package
2 2 0.5868s 0.2934s 0.5868s conan_package
2 0 0.3804s 0.1902s 0.0000s conan_metadatum
2 0 0.3614s 0.1807s 0.0000s conan_package
2 2 0.0894s 0.0447s 0.0894s event
2 2 0.4636s 0.2318s 0.4636s prometheus_project
2 2 0.0798s 0.0399s 0.0798s ci_bridge
2 2 0.0251s 0.0126s 0.0251s ci_sources_pipeline
2 2 0.0376s 0.0188s 0.0376s protected_tag
2 2 0.3547s 0.1773s 0.3547s project_empty_repo
2 2 0.0448s 0.0224s 0.0448s system_hook
2 2 0.4385s 0.2193s 0.4385s slack_service
2 2 0.4021s 0.2011s 0.4021s pool_repository
2 2 0.0760s 0.0380s 0.0760s alerts_service
1 1 0.0223s 0.0223s 0.0223s license
1 1 0.1128s 0.1128s 0.1128s container_expiration_policy
1 1 0.3752s 0.3752s 0.3752s jira_service
1 0 0.0779s 0.0779s 0.0000s jira_tracker_data
1 1 0.1470s 0.1470s 0.1470s container_repository
1 1 0.0510s 0.0510s 0.0510s lfs_file_lock
1 1 0.0844s 0.0844s 0.0844s custom_issue_tracker_service
1 1 0.2570s 0.2570s 0.2570s deployment
1 1 0.0520s 0.0520s 0.0520s jira_connect_subscription
1 0 0.0295s 0.0295s 0.0000s jira_connect_installationAfter
[TEST PROF INFO] EventProf results for sql.active_record
Total time: 01:17.632 of 02:48.726 (46.01%)
Total events: 54576
Top 5 slowest suites (by time):
Project (./spec/models/project_spec.rb:5) – 01:17.632 (54576 / 1026) of 02:48.726 (46.01%)
Finished in 2 minutes 54.4 seconds (files took 4.58 seconds to load)
1028 examples, 0 failures
Failed examples:
rspec ./spec/models/project_spec.rb:5187 # Project#badges with nested_groups returns the project and the project nested groups badges
rspec ./spec/models/project_spec.rb:6500 # Project#enabled_group_deploy_keys when a project has a parent group and this group has a group deploy key enabled and this group has parent group which also has a group deploy key enabled returns both group deploy keys
[TEST PROF INFO] Factories usage
Total: 1870
Total top-level: 1360
Total time: 126.8303s
Total uniq factories: 75
total top-level total time time per call top-level time name
657 616 91.1615s 0.1388s 85.7337s project
445 236 18.3535s 0.0412s 9.8218s user
122 118 4.8062s 0.0394s 4.6791s group
122 0 0.6691s 0.0055s 0.0000s namespace_settings
63 3 1.3141s 0.0209s 0.4726s ci_job_artifact
49 45 1.4773s 0.0301s 1.0644s ci_pipeline
34 30 2.7931s 0.0821s 2.3115s ci_build
27 27 0.2196s 0.0081s 0.2196s ci_variable
27 27 4.2999s 0.1593s 4.2999s merge_request
18 18 1.7683s 0.0982s 1.7683s namespace
16 16 0.6081s 0.0380s 0.6081s note_on_commit
15 15 2.9180s 0.1945s 2.9180s project_member
14 10 0.9792s 0.0699s 0.7063s cluster
13 13 0.5000s 0.0385s 0.5000s ci_runner
12 12 0.7008s 0.0584s 0.7008s issue
11 0 0.2788s 0.0253s 0.0000s package_file
11 10 0.4323s 0.0393s 0.3087s environment
10 0 0.2934s 0.0293s 0.0000s conan_package_file
10 0 0.1091s 0.0109s 0.0000s conan_file_metadatum
9 9 0.0973s 0.0108s 0.0973s lfs_objects_project
9 9 0.1118s 0.0124s 0.1118s project_auto_devops
8 5 0.0821s 0.0103s 0.0578s lfs_object
8 8 0.0360s 0.0045s 0.0360s upload
8 8 0.5759s 0.0720s 0.5759s project_export_job
7 7 0.1708s 0.0244s 0.1708s deploy_token
6 6 0.2239s 0.0373s 0.2239s service
6 6 0.9493s 0.1582s 0.9493s jira_import_state
6 0 0.9017s 0.1503s 0.0000s label
6 6 0.1414s 0.0236s 0.1414s protected_branch
6 6 0.1444s 0.0241s 0.1444s prometheus_service
5 5 0.0331s 0.0066s 0.0331s import_state
5 5 0.0795s 0.0159s 0.0795s group_badge
5 5 0.2226s 0.0445s 0.2226s chat_name
5 5 0.5115s 0.1023s 0.5115s group_deploy_key
4 0 0.4924s 0.1231s 0.0000s issue_tracker_data
4 0 0.2808s 0.0702s 0.0000s project_feature_usage
4 4 0.0353s 0.0088s 0.0353s project_repository
4 4 0.0128s 0.0032s 0.0128s shard
4 4 0.3411s 0.0853s 0.3411s cluster_platform_kubernetes
4 4 0.1108s 0.0277s 0.1108s ci_instance_variable
4 4 0.1402s 0.0350s 0.1402s deploy_keys_project
4 4 0.4741s 0.1185s 0.4741s project_hook
4 4 0.2041s 0.0510s 0.2041s project_badge
3 3 1.6482s 0.5494s 1.6482s redmine_project
3 0 1.2428s 0.4143s 0.0000s redmine_service
3 3 0.0282s 0.0094s 0.0282s service_desk_setting
3 3 0.2539s 0.0846s 0.2539s deploy_key
3 3 0.0800s 0.0267s 0.0800s npm_package
2 2 0.4890s 0.2445s 0.4890s maven_package
2 0 0.2842s 0.1421s 0.0000s maven_metadatum
2 0 0.2590s 0.1295s 0.0000s package
2 2 0.1062s 0.0531s 0.1062s npm_package
2 2 0.8836s 0.4418s 0.8836s conan_package
2 0 0.5636s 0.2818s 0.0000s conan_metadatum
2 0 0.5248s 0.2624s 0.0000s conan_package
2 2 0.0982s 0.0491s 0.0982s event
2 2 0.3475s 0.1738s 0.3475s prometheus_project
2 2 0.1041s 0.0520s 0.1041s ci_bridge
2 2 0.0315s 0.0158s 0.0315s ci_sources_pipeline
2 2 0.0343s 0.0172s 0.0343s protected_tag
2 2 0.2874s 0.1437s 0.2874s project_empty_repo
2 2 0.0434s 0.0217s 0.0434s system_hook
2 2 0.2839s 0.1420s 0.2839s slack_service
2 2 0.3595s 0.1798s 0.3595s pool_repository
2 2 0.0656s 0.0328s 0.0656s alerts_service
1 1 0.0152s 0.0152s 0.0152s license
1 1 0.0759s 0.0759s 0.0759s container_expiration_policy
1 1 0.3428s 0.3428s 0.3428s jira_service
1 0 0.1492s 0.1492s 0.0000s jira_tracker_data
1 1 0.0987s 0.0987s 0.0987s container_repository
1 1 0.0449s 0.0449s 0.0449s lfs_file_lock
1 1 0.0743s 0.0743s 0.0743s custom_issue_tracker_service
1 1 0.2095s 0.2095s 0.2095s deployment
1 1 0.0379s 0.0379s 0.0379s jira_connect_subscription
1 0 0.0224s 0.0224s 0.0000s jira_connect_installationDoes this MR meet the acceptance criteria?
Conformity
- [-] Changelog entry
- [-] Documentation (if required)
-
Code review guidelines -
Merge request performance guidelines -
Style guides - [-] Database guides
- [-] Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. - [-] Tested in all supported browsers
- [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
- [-] Label as security and @ mention
@gitlab-com/gl-security/appsec - [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
- [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Peter Leitzen