Resolve "Redis N+1 in /api/v4/groups/:id/projects - forks count key"

What does this MR do?

In #214510 (closed) was described how /groups/:id/projects api endpoint is making multiple calls to redis in order to gather all the relevant data. In order to mitigate that, I'm proposing:

1. use mget to get all available cached values of forks count
2. refresh cache for projects that are not cached yet
3. pass retrieved values to serializers, so there is no need to call redis again.

This solution is based on discussion with @ifarkas here

Screenshots

I was afraid if this will actually help - as we need to gather data and then put them into the serializers. But we can avoid one Redis call.

I was testing it locally using apache bench, on endpoint with 112 projects. Here are results for an old way and the new way, one in case when all cache is flushed before testing, and one when it was not flushed. Apache Bench command: ab -n 100 -c 2 -H 'private-token: _u33SKKCgmbVsPUyE36B' 'http://127.0.0.1:3000/api/v4/groups/57/projects' which means that there were 100 requests made, 2 at the time (as -c means "Number of multiple requests to perform at a time. Default is one request at a time.").

New way: with flushed cache:

Concurrency Level:      2
Time taken for tests:   65.971 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      5928792 bytes
HTML transferred:       5786100 bytes
Requests per second:    1.52 [#/sec] (mean)
Time per request:       1319.415 [ms] (mean)
Time per request:       659.708 [ms] (mean, across all concurrent requests)
Transfer rate:          87.76 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:  1106 1303 134.2   1255    1834
Waiting:     1106 1302 134.1   1254    1833
Total:       1106 1303 134.2   1255    1834

Percentage of the requests served within a certain time (ms)
  50%   1255
  66%   1274
  75%   1405
  80%   1417
  90%   1507
  95%   1550
  98%   1648
  99%   1834
 100%   1834 (longest request)

with cache:

Concurrency Level:      2
Time taken for tests:   64.911 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      5928790 bytes
HTML transferred:       5786100 bytes
Requests per second:    1.54 [#/sec] (mean)
Time per request:       1298.214 [ms] (mean)
Time per request:       649.107 [ms] (mean, across all concurrent requests)
Transfer rate:          89.20 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   936 1279 172.5   1246    2011
Waiting:      936 1277 171.9   1245    2010
Total:        937 1279 172.5   1246    2011

Percentage of the requests served within a certain time (ms)
  50%   1246
  66%   1294
  75%   1368
  80%   1416
  90%   1486
  95%   1570
  98%   1840
  99%   2011
 100%   2011 (longest request)

Old way: with flushed cache:

Concurrency Level:      2
Time taken for tests:   68.926 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      5927896 bytes
HTML transferred:       5786100 bytes
Requests per second:    1.45 [#/sec] (mean)
Time per request:       1378.523 [ms] (mean)
Time per request:       689.262 [ms] (mean, across all concurrent requests)
Transfer rate:          83.99 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:  1132 1351 178.6   1284    2111
Waiting:     1131 1350 178.6   1284    2111
Total:       1132 1351 178.6   1284    2111

Percentage of the requests served within a certain time (ms)
  50%   1284
  66%   1316
  75%   1470
  80%   1565
  90%   1599
  95%   1611
  98%   2024
  99%   2111
 100%   2111 (longest request)

with cache:

Concurrency Level:      2
Time taken for tests:   70.938 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      5927892 bytes
HTML transferred:       5786100 bytes
Requests per second:    1.41 [#/sec] (mean)
Time per request:       1418.764 [ms] (mean)
Time per request:       709.382 [ms] (mean, across all concurrent requests)
Transfer rate:          81.61 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:  1115 1393 177.0   1331    2029
Waiting:     1115 1392 176.9   1331    2028
Total:       1115 1393 176.9   1332    2029

Percentage of the requests served within a certain time (ms)
  50%   1332
  66%   1385
  75%   1505
  80%   1600
  90%   1647
  95%   1669
  98%   1945
  99%   2029
 100%   2029 (longest request)

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team

Closes #220316 (closed)