GitLab failing to load the list of the merge requests if an MR contains a large amount of commit.
Summary
We had a customer where they encountered a problem accessing the projects merge requests page.
The production_json.log
shows this error:
"exception.class":"ActionView::Template::Error",
"exception.message":"8:grpc: received message larger than max (5691745 vs. 4194304).",
Click to expand full stack trace
"exception.class":"ActionView::Template::Error",
"exception.message":"8:grpc: received message larger than max (5691745 vs. 4194304).",
"exception.backtrace":[
"lib/gitlab/git/wraps_gitaly_errors.rb:15:in `rescue in wrapped_gitaly_errors'",
"lib/gitlab/git/wraps_gitaly_errors.rb:6:in `wrapped_gitaly_errors'",
"lib/gitlab/git/commit.rb:170:in `batch_by_oid'",
"lib/gitlab/git/rugged_impl/commit.rb:50:in `batch_by_oid'",
"app/models/repository.rb:130:in `commits_by'",
"app/models/commit.rb:142:in `block (2 levels) in lazy'",
"app/models/commit.rb:139:in `each'",
"app/models/commit.rb:139:in `block in lazy'",
"app/models/commit_collection.rb:102:in `block in enrich!'",
"app/models/commit_collection.rb:94:in `map'",
"app/models/commit_collection.rb:94:in `enrich!'",
"app/models/commit_collection.rb:36:in `block in without_merge_commits'",
"lib/gitlab/utils/strong_memoize.rb:30:in `strong_memoize'",
"app/models/commit_collection.rb:33:in `without_merge_commits'",
"app/models/commit_collection.rb:27:in `committers'",
"app/models/merge_request.rb:585:in `committers'",
"ee/app/models/approval_state.rb:40:in `filter_committers'",
"ee/app/models/approval_wrapped_rule.rb:118:in `filter_approvers'",
"ee/app/models/approval_wrapped_any_approver_rule.rb:10:in `approved_approvers'",
"ee/app/models/approval_wrapped_rule.rb:102:in `block in approvals_left'",
"lib/gitlab/utils/strong_memoize.rb:30:in `strong_memoize'",
"ee/app/models/approval_wrapped_rule.rb:101:in `approvals_left'",
"ee/app/models/approval_wrapped_any_approver_rule.rb:15:in `block in approved?'",
"lib/gitlab/utils/strong_memoize.rb:30:in `strong_memoize'",
"ee/app/models/approval_wrapped_any_approver_rule.rb:14:in `approved?'",
"ee/app/models/approval_state.rb:66:in `all?'",
"ee/app/models/approval_state.rb:66:in `block in approved?'",
"lib/gitlab/utils/strong_memoize.rb:30:in `strong_memoize'",
"ee/app/models/approval_state.rb:65:in `approved?'",
"ee/app/models/concerns/approvable.rb:23:in `approved?'",
"ee/app/views/projects/merge_requests/_approvals_count.html.haml:3",
"app/views/projects/merge_requests/_merge_request.html.haml:63",
"app/views/projects/merge_requests/_merge_requests.html.haml:3",
"app/views/projects/merge_requests/index.html.haml:26",
"app/controllers/application_controller.rb:136:in `render'",
The Gitaly log shows this error:
"error": "rpc error: code = ResourceExhausted desc = grpc: received message larger than max (5691745 vs. 4194304)",
After troubleshooting, we figure out that it was caused by an MR which contains ~100k commits.
Steps to reproduce
- Create an MR with a large amount of commits (e.g. ~100k commits).
- Access the project's merge requests page.
Example Project
What is the current bug behavior?
GitLab shows an error 500.
What is the expected correct behavior?
GitLab loads the list of merge requests.
Relevant logs and/or screenshots
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)
Possible fixes
@hchouraria recommends that the OIDs be sent in batches.
https://gitlab.com/gitlab-org/gitlab/blob/master/app/models/commit.rb#L123-128