Draft: Cache pipelines in merge requests
What does this MR do?
Introduce caching for pipelines in the MR Pipelines tab. This massively cuts the response time, particularly with large numbers of pipelines.
The caching is per-pipeline and the full set are multi-fetched from the cache in a single call. Each entity is cached with a context that should allow the sharing of the cache between different users based on permission levels, which will make this far more effective, at the expense of a slight maintenance overhead.
Our serializer system is a bit of a nightmare for caching and so I've created a new ancestor class for serializers we want to cache, which will return pre-rendered JSON rather than the usual Hash. This requires a slightly odd-looking heredoc in the controller to wrap the rendered JSON, but again I believe the performance benefits to outweigh the cost here.
Also generalises a few cache methods from the API caching helpers so they are used in both places.
It's hard to measure this locally but I expect it to be very effective in fixing a major pain point in larger MRs. On my machine the timing for pipelines.json for 5 pipelines drops from 2.5s to 300ms, YMMV.
Screenshots (strongly suggested)
Does this MR meet the acceptance criteria?
Conformity
-
📋 Does this MR need a changelog?-
I have included a changelog entry. -
I have not included a changelog entry because _____.
-
-
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec -
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team