Skip to content

Use root_ref to index commits

Terri Chu requested to merge 214601-refactor-indexer-root-ref-for-commit into master

What does this MR do and why?

Related to #214601 (closed)

This MR aims to solve issues with indexing projects where the default branch deviates from HEAD. This change will allow projects to be indexed that are in this state. The change is to fall back to root_ref instead of using HEAD to ask for the last commit to the project

Background from a slack discussion

This can happen because gitlab-rails doesn't always tell gitaly the default branch to use when calling CreateRepository (e.g. !119408 (merged)). When this happens repos use the gitaly default main, but the default branch in gitlab-rails depends on a feature flag.

FindDefaultBranchName will only return a name of a branch that exists with a crazy broken heuristic. The culmination of these two problems is that HEAD and FindDefaultBranch can be different. Usually it gets fixed on the first push, because of https://gitlab.com/gitlab-org/gitlab/-/blob/27f59b503b0938473cc8eced046435749f45c255/app/services/projects/protect_default_branch_service.rb#L23

Issue gitaly#1446

Screenshots or screen recordings

N/A

How to set up and validate locally

  1. setup gdk for elasticsearch and index everything
  2. checkout 15-11-ee stable branch (may need to restart your gdk): git checkout 15-11-stable-ee
  3. create a project using the API, initialize with a readme, and specify a default branch name that is not main 
    curl --request POST --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
     --header "Content-Type: application/json" --data '{
        "name": "new_project_toast_branch", "description": "New Project", "path": "new_project_toast_branch",
        "default_branch": "toast", "initialize_with_readme": "true"}' \
     --url 'http://gdk.test:3000/api/v4/projects/'
  4. check that project index_status: Project.last.index_status (should be nil)
  5. commit something new to that project/branch using the API, use project id from steps above
PAYLOAD=$(cat << 'JSON'
{
  "branch": "bread",
  "commit_message": "some commit message",
  "actions": [
    {
      "action": "create",
      "file_path": "foo/bar22",
      "content": "some content"
    }
  ]
}
JSON
)
curl --request POST --header "PRIVATE-TOKEN: $GITLAB_TOKEN" --header "Content-Type: application/json" \
     --data "$PAYLOAD" "http://gdk.test:3000/api/v4/projects/48/repository/commits"
  1. check project index_status: Project.last.index_status (should have a blank SHA 0000000000000000000000000000000000000000)
  2. checkout this branch: git checkout 214601-refactor-indexer-root-ref-for-commit (may need to restart gdk)
  3. commit something new to that project/branch using the API, use project id from steps above
PAYLOAD=$(cat << 'JSON'
{
  "branch": "bread",
  "commit_message": "some commit message",
  "actions": [
    {
      "action": "create",
      "file_path": "foo/bar55",
      "content": "some content"
    }
  ]
}
JSON
)
curl --request POST --header "PRIVATE-TOKEN: $GITLAB_TOKEN" --header "Content-Type: application/json" \
     --data "$PAYLOAD" "http://gdk.test:3000/api/v4/projects/48/repository/commits"
  1. check project index_status: Project.last.index_status (should match Project.last.commit(Project.last.default_branch) for the default branch) <-- this is the fix, previously the commit may have been unreachable according to HEAD being set to a branch that does not exist
  2. create a new project using the API
  3. check the new project index_status: Project.last.index_status (should not be nil, should point to a valid commit sha)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Terri Chu

Merge request reports