Custom project templates unable to be created

Summary

I’ve found a group permission sharing bug for creating projects with project templates.

Steps to reproduce

1. groupA - all members guest
2. groupA/sub-groupA - contains project template group and projects created by them
3. groupA/sub-groupA/project-template-groupA/demo-project - simple project to test template
4. Set groupA/sub-groupA/project-template-groupA as custom project template for groupA/sub-groupA
5. groupA/teams-sub-groupB/foobar-team - has members, one is @fizzbuzzfoobar123 with owner role
6. Invite groupA/teams-sub-groupB/foobar-team group as Maintainer to groupA/sub-groupA
7. @fizzbuzzfoobar123 can create a project but not from template of project-template-groupA in groupA/sub-groupA
8. Manually add @fizzbuzzfoobar123 as direct member with Maintainer role to groupA/sub-groupA
9. @fizzbuzzfoobar123 can create with groupA/sub-groupA/project-template-groupA projects

Example Project

https://gitlab.com/test-gold-group1/filevine-project

What is the current bug behavior?

In the example, members of foobar-team shared group Maintainer permissions to sub-groupA unable to create project from custom project templates of sub-groupA.

What is the expected correct behavior?

In the example, members of foobar-team shared group Maintainer permissions to sub-groupA are able to create project from custom project templates of sub-groupA.

Possible fixes

@jwoodwardgl:

We have a few problems happening here. We tried to fix this issue by moving the permissions check from the database to rails but this caused a performance issue so we disabled the feature flag.

We investigated why this performance issue was happening and the query being generated used a huge union query linking every public and internal group and every group the user is a member of which performed very poorly when there are too many groups.

Our documentation is a little ambiguous but the wording suggests that project templates are supposed to be used within the group they are configured for, not any arbitrary namespace but historically the code has shown templates from every group you can access with templates unless you are creating the project within a group.

[Context] My current thinking is that we should remove the group tab when creating a project outside of a group. This will allow us to constrain the scope and simplify these complex queries.

Additionally we should create a dedicated finder as the GroupsFinder includes a lot of logic we don't need and isn't well suited for optimizing this query.

Currently we also have issues around pagination which could be solved with a new approach where we query for the projects directly and then preload the groups instead of loading the groups first and then querying for each groups projects.

The current approach means we paginate pages with different numbers of projects because we are paginating the groups. A group could have a large number or projects and be displayed on one page.