Backend: Group files by projects in config_file_project_validate_access
<!--
## Implementation Issue To-Do list
(_NOTE: This section can be removed when the issue is ready for creation_)
- [ ] Ensure that issue title is concise yet descriptive
- [ ] Add `Frontend :` or `Backend: ` per group [naming conventions](https://about.gitlab.com/handbook/engineering/development/ops/verify/pipeline-authoring/#splitting-issues)
- [ ] Ensure the issue containing the feature or change proposal and related discussions is linked as related to this implementation issue.
- [ ] Aside from default labeling, please make sure to include relevant labels for `type::`, `workflow::`, and `~frontend`/`~backend` labeling.
- [ ] Issues with user-facing changes should include the `~UX` label.
-->
## Summary
- [ ] Fix N+1 `project` call
- [ ] Fix N+1 `sha` call
For `config_file_project_validate_access`, each file is currently checked in a project which can lead to slow performance within the CI Lint API.
Tree Breakdown below:
```
LINT:
pipeline_creation_service ->
yaml_process ->
config_compose ->
config_build_context ->
config_build_variables
config_expand ->
config_yaml_load
config_external_process ->
config_mapper_process ->
config_mapper_normalize ->
config_mapper_variables
config_mapper_rules
config_mapper_wildcards
config_mapper_variables
config_mapper_select
!verify!
config_file_artifact_validate_context # not in ci lint
->>>>config_file_project_validate_access
config_file_fetch_content ->
config_file_fetch_local_content
config_file_fetch_project_content
config_file_fetch_remote_content
config_file_fetch_template_content
config_file_validate_content
config_file_fetch_content_hash
config_file_expand_content_includes ->
config_mapper_process...
config_external_verify
config_external_merge
config_yaml_extend
config_tags_resolve
config_stages_inject
config_compose ->
```
The code we track is this;
```ruby
def can_access_local_content?
strong_memoize(:can_access_local_content) do
context.logger.instrument(:config_file_project_validate_access) do
Ability.allowed?(context.user, :download_code, project)
end
end
end
```
## Proposal
Even if we use `strong_memoize` here, improvements can be made for `can_access_local_content?` by grouping files by projects and operating similarly to the work being described in https://gitlab.com/gitlab-org/gitlab/-/issues/382531 with batching.
## Additional details
<!--
_NOTE: If the issue has addressed all of these questions, this separate section can be removed._
-->
Some relevant technical details, if applicable, such as:
- Does this need a ~"feature flag"?
- Is there an example response showing the data structure that should be returned (new endpoints only)?
- What permissions should be used?
- Is this EE or CE?
- [ ] EE
- [ ] CE
- Additional comments:
## Implementation Table
<!--
_NOTE: If the issue is not part of an epic, the implementation table can be removed. If it is part of an epic, make sure that the implementation table below mirrors the corresponding epic's implementation table content._
-->
| Work Type | Description | Issue link |
| ------ | ------ | ------ |
| | **NOTE:** :rotating_light: All below issues can be done in parallel | |
| ~backend | Backend: The gitlab-ci.yml is limited to 100 includes | https://gitlab.com/gitlab-org/gitlab/-/issues/207270 |
| ~backend |Backend: Remove N+1 for Gitaly requests when fetching `includes`|https://gitlab.com/gitlab-org/gitlab/-/issues/344829 |
| ~backend ~frontend | Improve the error messaging when fetching remote includes are timing out | https://gitlab.com/gitlab-org/gitlab/-/issues/351168 |
| ~backend | Backend: Improve CI Linter performance through parallelizing HTTP calls | https://gitlab.com/gitlab-org/gitlab/-/issues/351250 |
| ~backend | Backend: Caching includes to improve performance when using remote includes | https://gitlab.com/gitlab-org/gitlab/-/issues/351252 |
| ~backend | Backend: Batch request calls to Gitaly when fetching `include` | https://gitlab.com/gitlab-org/gitlab/-/issues/382531 |
| ~backend | **Backend: Group files by projects in config_file_project_validate_access** | :point_left: You are here |
<!--
## Documentation
_NOTE: This section is optional, but can be used for easy access to any relevant documentation URLs._
-->
## Links/References
issue