Refactor project API helpers

What does this MR do?

Coming from #37919 (closed), this MR

1. Adds a regression spec for this type of problem
2. Refactors the projects API (helper) methods to make side effects more explicit.

In more detail, the side effect I wanted to make more explicit is the fact that we perform batch counting to refresh caches. This had been mixed into preparing the database query to retrieve projects (hence I think of it as a "side effect") and didn't have specs.

Closes #37919 (closed)

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• This is a refactoring only, but we've run into missing tests before.

added backstage [DEPRECATED] groupdatabase labels

Reviewer roulette

Changes that require review have been detected! A merge request is normally reviewed by both a reviewer and a maintainer in its primary category (e.g. frontend or backend), and by a maintainer in all other categories.

To spread load more evenly across eligible reviewers, Danger has randomly picked a candidate for each review slot. Feel free to override this selection if you think someone else would be better-suited, or the chosen person is unavailable.

Once you've decided who will review this merge request, mention them as you normally would! Danger does not (yet?) automatically notify them for you.

Category	Reviewer	Maintainer
backend	Alessio Caiazza (@nolith)	Jan Provaznik (@jprovaznik)

Generated by Danger

added 4 commits

• c590c9b8 - Split batch counting from relation builder
• bd04c577 - Refactor load/prepare/present pattern
• 9b4db6cf - Explicitly retrieve data after pagination
• 17dfb03e - Explicit counting in context of groups

Compare with previous version

added 1 commit

• b374b3c5 - Signal-boost side effect for prepare method

Compare with previous version

Setting devopsenablement based on groupdatabase.

added devopssystems label

added 186 commits

• b374b3c5...4c7be075 - 179 commits from branch master
• 2792aa1f - Add regression spec
• 8369ee50 - Extract executing batch counting
• fb751fe3 - Split batch counting from relation builder
• 7e3670e3 - Refactor load/prepare/present pattern
• 0cc1724f - Explicitly retrieve data after pagination
• 677c52a8 - Signal-boost side effect for prepare method
• 8617e9f6 - Move prepare! into concern

Compare with previous version

changed milestone to %12.6

unmarked as a Work In Progress

changed title from WIP: Ab/pagination batch counts to Refactor project API helpers

changed the description

added 1 commit

• 5e1f2d70 - Newlines for specs

Compare with previous version

mentioned in issue #37919 (closed)

@smcgivern Coming from #37919 (closed), you have the full context here already, would it be possible for you to review the refactoring too?

Failures are unrelated to this MR, see #38115 (closed).

added api backend labels

assigned to @mayra-cabrera

unassigned @mayra-cabrera

mentioned in commit 055bece0

mentioned in merge request !21194 (merged)

assigned to @smcgivern

added 28 commits

• 5e1f2d70...415868f0 - 20 commits from branch master
• 593d8283 - Add regression spec
• 02166e60 - Extract executing batch counting
• 8c4953a3 - Split batch counting from relation builder
• 44f7d093 - Refactor load/prepare/present pattern
• d4c3220b - Explicitly retrieve data after pagination
• 9203088c - Signal-boost side effect for prepare method
• 681e89c5 - Move prepare! into concern
• 52f91d3f - Newlines for specs

Compare with previous version

mentioned in commit e105662b

added 107 commits

• 52f91d3f...7a14c495 - 99 commits from branch master
• 1d10a286 - Add regression spec
• 289f03c2 - Extract executing batch counting
• bf81fd1f - Split batch counting from relation builder
• c4df33f1 - Refactor load/prepare/present pattern
• 81c4fe42 - Explicitly retrieve data after pagination
• dccd56f3 - Signal-boost side effect for prepare method
• 819c6b0c - Move prepare! into concern
• f3cf2e97 - Newlines for specs

Compare with previous version

@abrandl thanks! I have two very small comments which I'm happy to move to a future MR if you want to get this merged soon; neither are blockers.

unassigned @smcgivern

added 1 commit

• 03ab9886 - Apply suggestion to lib/api/projects.rb

Compare with previous version

added 31 commits

• 03ab9886...63534f6e - 21 commits from branch master
• 105ac2b6 - Add regression spec
• 2a915497 - Extract executing batch counting
• 5b27e0e0 - Split batch counting from relation builder
• 5566c021 - Refactor load/prepare/present pattern
• 50876d4c - Explicitly retrieve data after pagination
• 378e2345 - Signal-boost side effect for prepare method
• 1e0673f9 - Move prepare! into concern
• 400b979f - Newlines for specs
• d82fab0b - Fix wording in comment
• e50aaa6b - Rename method to more detailed name

Compare with previous version

added 1 commit

• 7e4649af - Rename method to more detailed name

Compare with previous version

mentioned in commit 1119876a

The package-and-qa-manual job from pipeline https://gitlab.com/gitlab-org/gitlab/pipelines/100987160 triggered https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/pipelines/101147479 downstream.

The Trigger:qa-test job from pipeline https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/pipelines/101147479 triggered https://gitlab.com/gitlab-org/gitlab-qa/pipelines/101168937 downstream.

The gitlab-qa downstream pipeline failed! .

You can download a report of the test results: combined-report.html

These failures are #38315 (closed), I commented there.

@smcgivern Thank you for the review. I think this is ready now, would you mind merging it please?

assigned to @smcgivern

mentioned in commit 88d33022

resolved all threads

approved this merge request

Thanks @abrandl!

merged

mentioned in commit 7a5656b7

mentioned in issue #32186 (closed)