Optimize merge request refresh by using the database to check commit SHAs
Previously for a given merge request, we would:
- Create the array of commit SHAs involved in the push (A)
- Request all merge request commits and map the SHA (B)
- Reload the diff if there were any common commits between A and B
We can avoid additional database querying and overhead by checking with the database whether the merge request contains any of the commit SHAs.
Relates to https://gitlab.com/gitlab-org/gitlab-ce/issues/53213
In https://gitlab.com/gitlab-org/gitlab-ce/issues/49703#note_113602024, we see that a push to the staging www-gitlab-com
resulted in 204 queries for SELECT * FROM merge_request_diff_commits
.
Merge request reports
Activity
added Create [DEPRECATED] merge requests labels
mentioned in merge request !22729 (closed)
1 Error dbc03ce3: The commit subject may not be longer than 72 characters Commit message standards
One or more commit messages do not meet our Git commit message standards. For more information on how to write a good commit message, take a look at How to Write a Git Commit Message.
Here is an example of a good commit message:
Reject ruby interpolation in externalized strings When using ruby interpolation in externalized strings, they can't be detected. Which means they will never be presented to be translated. To mix variables into translations we need to use `sprintf` instead. Instead of: _("Hello #{subject}") Use: _("Hello %{subject}") % { subject: 'world' }
This is an example of a bad commit message:
updated README.md
This commit message is bad because although it tells us that README.md is updated, it doesn't tell us why or how it was updated.
Generated by
DangerEdited by 🤖 GitLab Bot 🤖added 142 commits
-
f77d6c2f...bb6b5653 - 141 commits from branch
master
- b32d1c3b - Optimize merge request refresh by using the database to check commit SHAs
-
f77d6c2f...bb6b5653 - 141 commits from branch
added 1 commit
- d5288618 - Optimize merge request refresh by using the database to check commit SHAs
- Resolved by Stan Hu
added 1 commit
- 296087e3 - Optimize merge request refresh by using the database to check commit SHAs
added 1 commit
- dbc03ce3 - Optimize merge request refresh by using the database to check commit SHAs
assigned to @DouweM
mentioned in merge request !22725 (merged)
Thanks @stanhu!
mentioned in commit 4d3ff28a
With a simulated push on the staging
www-gitlab-com
repository with https://gitlab.com/gitlab-org/gitlab-ce/issues/49703#note_113329681:- Memory usage went down from 150 MB to 120 MB
- Full ActiveRecord model loads dropped from 1,190 SQL queries to 969, not counting SELECT 1 and SELECT COUNT(*) etc.:
Total allocated: 119999038 bytes (1609014 objects) Total retained: 3257369 bytes (14142 objects) allocated memory by gem ----------------------------------- 67553020 activerecord-4.2.10 13792697 activesupport-4.2.10 9769264 activemodel-4.2.10 6840365 psych 3634513 arel-6.0.4 3615746 grpc-1.15.0-x86_64-linux 2835456 state_machines-0.5.0 2715759 gitlab-rails/lib 1888623 other 1162802 gitlab-rails/app
SQL queries:
$ cat profile_sql11.txt | sed -e "s/.*DEBUG -- ://g" | grep Load | sed -e "s/Load.*//g" | sort | uniq -c 339 MergeRequestDiff 235 Project 175 User 168 MergeRequest::Metrics 32 ForkNetwork 7 MergeRequest 7 MergeRequestDiffCommit 1 GitlabIssueTrackerService 1 License 1 Namespace 1 ProjectFeature 1 ProjectRepositoryState 1 Route
Edited by Stan Humentioned in issue gitlab-org/release/tasks#522 (closed)
added devopscreate label