Create endpoint for file tree browser (path-based lookups)

Problem

The current file tree browser implementation uses the paginatedTree GraphQL query, which requires multiple sequential requests to load deeply nested file paths. This creates performance bottlenecks when users navigate directly to deeply nested files.

Current Implementation

The frontend currently:

  • Uses paginatedTree query to fetch one directory at a time
  • Implements recursive fetching with frontend-imposed limits:
    • FTB_MAX_DEPTH = 20 (maximum directory depth to expand)
    • FTB_MAX_PAGES = 5 (maximum paginated pages to fetch per directory)
  • Must orchestrate multiple sequential requests to build the tree for deep paths

Example scenario:

Path: /mock_directories/dir_2000/nested_1500/test.js
Current approach:
- Fetch pages 1-5 of /mock_directories/ (limit reached, dir_2000 not found)
- User sees incomplete tree, and need t manually navigate to the path

With the current limits, if a directory appears beyond page 5 (item #500+), the file tree cannot display it.

Why This Matters

The paginatedTree endpoint works well for incremental navigation (clicking through directories), but it's not optimized for:

  • Initial page loads with deep file paths
  • Direct links to specific files
  • Page refreshes on nested files
  • Ensuring the complete tree hierarchy is visible

Proposal

Create a new dedicated backend endpoint optimized for path-based lookups that can efficiently return the tree structure needed to display a specific file path.

From the Original Discussion

As mentioned by @vyaklushin in #550196 (comment 2612046371):

I was thinking about building a project file tree index. That would remove the need to access Gitaly to traverse the project every time. This should also solve performance problems since the index would be pre-calculated, but maintenance and keeping it up-to-date could be challenging.

It can be also a good question to Gitaly team, maybe there are more effective ways to access project tree?

Requirements

The new endpoint should:

  1. Accept a file path as input and return the tree structure needed to display that path
  2. Return all parent directories and their immediate children in a single (or minimal number of) request(s)
  3. Handle pagination efficiently for directories with many items
  4. Support both files and directories as target paths
  5. Maintain performance even for deeply nested paths and large repositories

Expected Benefits

  • Reduce API requests from potentially dozens to 1 (or a small constant number)
  • Faster initial page load for linked files
  • Reduced Gitaly load through more efficient tree traversal
  • Remove frontend pagination limits - no more FTB_MAX_PAGES constraint
  • Simpler frontend logic - no need to orchestrate recursive fetching with artificial limits
  • Better caching opportunities based on commit SHA + path

Implementation Approach

The backend team should determine the best approach. Potential options include:

  • Building an efficient index structure
  • Optimizing Gitaly tree traversal
  • Implementing smart caching strategies
  • Consulting with the Gitaly team on best practices for tree access

Related Issues

  • #550196 (closed) - Ensure File Tree Navigation works with Paginated Results (original discussion)
  • #543753 (closed) - Implement search/filter functionality in file tree browser
  • Epic: &17781 - Navigate repo with a file tree

Labels

Category:Source Code Management backend groupsource code devopscreate sectiondev typefeature featureaddition

Edited by 🤖 GitLab Bot 🤖