Refactor project API helpers

What does this MR do?

Coming from #37919 (closed), this MR

1. Adds a regression spec for this type of problem
2. Refactors the projects API (helper) methods to make side effects more explicit.

In more detail, the side effect I wanted to make more explicit is the fact that we perform batch counting to refresh caches. This had been mixed into preparing the database query to retrieve projects (hence I think of it as a "side effect") and didn't have specs.

Closes #37919 (closed)

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• This is a refactoring only, but we've run into missing tests before.