Skip to content

Reduce amount of API calls when retrieving merge requests

Peter Leitzen requested to merge pl-changeset-less-api-calls into master

What does this MR do and why?

When a security merge request is merged the commit messages of the following commits contain a reference to the same merge request.

This commit ensures that the merge request is retrieved (via API) only once.

This reduces the amount of API calls by half.

Re-record spec fixtures to contain duplicate MRs IIDs from version v16.2.6-ee to v16.2.7-ee.

Example and Screenshot

Diff v16.4.0 .. v16.4.1
Screenshot_from_2023-10-20_15-28-51

Both commit message contain a reference to https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/3583 but the information is now fetched only once!

How to verify locally

export RELEASE_BOT_PRODUCTION_TOKEN=$GITLAB_API_PRIVATE_TOKEN
irb -Ilib -rrelease_tools

and then 👇

Log this branch (18 API calls) 🚀

Each merge request is retrieve only once!

>> x = ReleaseTools::Qa::ProjectChangeset.new(project: ReleaseTools::Project::GitlabEe, from: "v16.4.0-ee", to: "v16.4.1-ee").merge_requests.uniq(&:id).size
2023-10-20 15:26:56.294028 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:56 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/repository/commits/v16.4.0-ee" -
2023-10-20 15:26:56.614757 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:56 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/repository/commits/v16.4.1-ee" -
2023-10-20 15:26:57.446479 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:57 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/repository/compare" -
2023-10-20 15:26:58.133481 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:58 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3583" -
2023-10-20 15:26:58.720759 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:58 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3588" -
2023-10-20 15:26:59.406361 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:59 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3593" -
2023-10-20 15:26:59.951032 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:59 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3553" -
2023-10-20 15:27:01.136226 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:27:01 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3586" -
2023-10-20 15:27:01.792360 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:27:01 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3594" -
2023-10-20 15:27:02.478253 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:27:02 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3591" -
2023-10-20 15:27:03.398693 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:27:03 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3605" -
2023-10-20 15:27:04.523375 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:27:04 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3587" -
2023-10-20 15:27:05.155664 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:27:05 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3607" -
2023-10-20 15:27:06.000295 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:27:06 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3611" -
2023-10-20 15:27:06.612170 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:27:06 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3604" -
2023-10-20 15:27:07.263789 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:27:07 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3603" -
2023-10-20 15:27:07.995233 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:27:07 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3602" -
=> 15
2023-10-20 15:27:08.623662 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:27:08 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3597" -

Log this branch (33 API calls) 🐢

Notice the redundant API calls to retrieve merge requests.

>> ReleaseTools::Qa::ProjectChangeset.new(project: ReleaseTools::Project::GitlabEe, from: "v16.4.0-ee", to: "v16.4.1-ee").merge_requests.uniq(&:id).size
2023-10-20 15:26:16.753954 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:16 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/repository/commits/v16.4.0-ee" -
2023-10-20 15:26:17.127406 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:17 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/repository/commits/v16.4.1-ee" -
2023-10-20 15:26:17.492394 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:17 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/repository/compare" -
2023-10-20 15:26:18.421783 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:18 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3583" -
2023-10-20 15:26:19.104141 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:19 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3583" -
2023-10-20 15:26:19.730490 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:19 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3588" -
2023-10-20 15:26:20.353946 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:20 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3588" -
2023-10-20 15:26:20.978725 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:20 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3593" -
2023-10-20 15:26:21.525161 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:21 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3593" -
2023-10-20 15:26:22.481696 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:22 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3553" -
2023-10-20 15:26:23.133016 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:23 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3553" -
2023-10-20 15:26:24.027368 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:24 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3586" -
2023-10-20 15:26:24.613191 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:24 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3586" -
2023-10-20 15:26:25.236339 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:25 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3594" -
2023-10-20 15:26:25.834949 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:25 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3594" -
2023-10-20 15:26:26.445897 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:26 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3591" -
2023-10-20 15:26:27.094706 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:27 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3591" -
2023-10-20 15:26:27.742678 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:27 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3605" -
2023-10-20 15:26:28.801320 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:28 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3605" -
2023-10-20 15:26:29.953791 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:29 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3587" -
2023-10-20 15:26:30.503522 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:30 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3587" -
2023-10-20 15:26:31.247904 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:31 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3607" -
2023-10-20 15:26:34.715148 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:34 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3607" -
2023-10-20 15:26:35.297140 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:35 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3611" -
2023-10-20 15:26:36.026618 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:36 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3611" -
2023-10-20 15:26:36.728818 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:36 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3604" -
2023-10-20 15:26:37.332349 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:37 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3604" -
2023-10-20 15:26:37.996396 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:37 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3603" -
2023-10-20 15:26:38.696212 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:38 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3603" -
2023-10-20 15:26:39.493626 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:39 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3602" -
2023-10-20 15:26:41.804720 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:41 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3602" -
2023-10-20 15:26:42.431022 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:42 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3597" -
=> 15
>> 2023-10-20 15:26:43.122037 D ReleaseTools::GitlabClient -- [HTTParty] [2023-10-20 15:26:43 +0200] 200 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fsecurity%2Fgitlab/merge_requests/3597" -

Author Check-list

  • [-] Has documentation been updated?
Edited by Peter Leitzen

Merge request reports