Merge requests API is slow

https://gitlab.com/api/v4/projects/13083/merge_requests?private_token=[REDACTED]&per_page=100 took 20.56s

We need to figure out how to improve the performance so that it takes around ~300ms, similar to https://gitlab.com/gitlab-org/gitlab-ce/issues/33748