[performance] Banzai reference filters execute too many queries, N+1
The Banzai reference filters, which transform user mentions, project references, etc, into actual links, must query the database into order to get information for those links. However, when there are many of the same type of references, many times we see an N+1 query problem.
A set of wiki pages attempt to demonstrate which type of references do and do not have an N+1 problem.
https://gitlab.com/digitalmoksha/bug-reproduction/-/wikis/home
Based on this, only Issues, Merge Requests, and Vulnerabilities do not have an obvious problem.
This issue was spun from #21112 for better search-ability and to track several smaller MRs to fix this issue. This problem is not wiki specific, but effects all areas which use the markdown/Banzai pipelines.
Filters to convert
These wiki pages are samples showing the increased DB queries for various filters.
-
AlertReferenceFilter
- originally done - verified !67073 (merged) -
CommitRangeReferenceFilter
- #376060 (closed) -
CommitReferenceFilter
- originally done - verified !67079 (merged) -
DesignReferenceFilter
- originally done - verify !67086 (closed) -
EpicReferenceFilter
- originally done - verified !61527 (merged) -
ExternalIssueReferenceFilter
- not needed - add spec - !69222 (merged) -
FeatureFlagReferenceFilter
- originally done - verify #376056 (closed) -
IssueReferenceFilter
- originally done - verify #376057 (closed) -
IterationReferenceFilter
- !67431 (merged) -
LabelReferenceFilter
- !61648 (merged) -
MergeRequestReferenceFilter
- originally done - verify #376058 (closed) -
MilestoneReferenceFilter
- !62742 (merged) -
ProjectReferenceFilter
- originally done - verified !67362 (merged) -
SnippetReferenceFilter
- !61076 (merged) -
UserReferenceFilter
- originally done - verified !61514 (merged) -
VulnerabilityReferenceFilter
- originally done - verify #376059 (closed)