Memoize force_push for a push (!53536) · Merge requests · GitLab.org / GitLab

What does this MR do?

This merge request contains one improvement:

Here we memoize whether the push or forced or not, since this makes a gitaly call for every open merge request where the branch is either the source or target.

Using very basic merge requests, set up via:

for i in {1..500}
do
git checkout -b "$i";
echo "$i" >> $i.txt;
git add .; git commit -am 'test';
git push origin "$i";
done

for i in {1..500}
do
curl -X POST --header "PRIVATE-TOKEN: INSERT_TOKEN_HERE" --data "source_branch=$i&target_branch=master&title=test" http://127.0.0.1:3000/api/v4/projects/6/merge_requests
done

Baseline:

"2021-02-10_02":"34":52.64132[
   "34mrails-background-jobs":[
      0m{
         "severity":"INFO",
         "time":"2021-02-10T02:34:52.640Z",
         "class":"UpdateMergeRequestsWorker",
         "args":[
            "701",
            "1",
            "17a28bb7181854683abab97fa44ca1d3c488d3fb",
            "1db4f3fe00dd4305eb35a99afd9ee9148ca5ff17",
            "refs/heads/master"
         ],
         "retry":3,
         "queue":"update_merge_requests",
         "backtrace":true,
         "version":0,
         "jid":"2bb3a7265b747f34bff27548",
         "created_at":"2021-02-10T02:33:59.542Z",
         "correlation_id":"e924b208414a1ae799fbac81dd727e99",
         "enqueued_at":"2021-02-10T02:33:59.559Z",
         "pid":41351,
         "message":"UpdateMergeRequestsWorker JID-2bb3a7265b747f34bff27548: done: 53.079206 sec",
         "job_status":"done",
         "scheduling_latency_s":0.002266,
         "job_size_bytes":377,
         "cpu_s":29.159938,
         "gitaly_calls":1194,
         "gitaly_duration_s":14.589041,
         "rugged_calls":27,
         "rugged_duration_s":0.030981,
         "db_count":9591,
         "db_write_count":9,
         "db_cached_count":2428,
         "duration_s":53.079206,
         "completed_at":"2021-02-10T02:34:52.640Z",
         "db_duration_s":8.240358
      }

Memoizing Force Push

"118 2021-02-10_03":"04":12.21957 ^[
   [
      "34mrails-background-jobs":"^"[
         [
            0m{
               "severity":"INFO",
               "time":"2021-02-10T03:04:12.219Z",
               "class":"UpdateMergeRequestsWorker",
               "args":[
                  "701",
                  "1",
                  "17a28bb7181854683abab97fa44ca1d3c488d3fb",
                  "1db4f3fe00dd4305eb35a9    9afd9ee9148ca5ff17",
                  "refs/heads/master"
               ],
               "retry":3,
               "queue":"update_merge_requests",
               "backtrace":true,
               "version":0,
               "jid":"6d0f3bdcbcfd52bd70c99fec",
               "created_at":"2021-02-10T03:03:36.616Z",
               "correlation_id":"3763cb91565a4278dc6fc1978d3b42d    6",
               "enqueued_at":"2021-02-10T03:03:36.632Z",
               "pid":54377,
               "message":"UpdateMergeRequestsWorker JID-6d0f3bdcbcfd52bd70c99fec: done: 35.580233 sec",
               "job_status":"done",
               "scheduling_latency_s":0.006031,
               "job_size_bytes":378,
               "cpu_s":27.45684,
               "gitaly_calls":2,
               "gitaly_duration_s":0.06227,
               "rugged_calls":27,
               "rugged_duration_s":0.007904,
               "db_count":9591,
               "db_write_count":9,
               "db_cached_count":2428,
               "duration_s":35.580233,
               "completed_at":"2021-02-10T03:04:12.219Z",
               "db_duration_s":5.9    60271
            }"¬"

Screenshots (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
Tested in all supported browsers
Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

Label as security and @ mention @gitlab-com/gl-security/appsec
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
Security reports checked/validated by a reviewer from the AppSec team

Memoize force_push for a push