Skip to content

Draft: Allow branch to be specified for indexing

Terri Chu requested to merge 4-add-branches-param into main

Closes #4

Description

Indexing

  • Adds the ability to send indexing targets to the indexer.
  • Keeps backwards compatibility, if nothing is sent HEAD is used for the Zoekt branch name

Indexing status

  • return 404 when branch not found in Zoekt

How to test

Notes

  • Run indexer in gdk mode
  • I turned off the gdk zoekt indexer feature flag to run the tests manually: Feature.disable(:index_code_with_zoekt)

indexing

  1. checkout main branch: git checkout main
  2. truncate everything 
     curl --request POST --url http://127.0.0.1:6080/indexer/truncate \
 --header 'Content-Type: application/json' \
 --data '{}'
  3. index a repository (I used flightjs namespace, you will need to get the address and path for your gdk) 
    curl --request POST \
  --url http://127.0.0.1:6080/indexer/index \
  --header 'Content-Type: application/json' \
  --data '{
	"GitalyConnectionInfo": {
		"Address": "unix:/Users/terrichu/projects/gitlab-development-kit/praefect.socket",
		"Storage": "default",
		"Path": "@hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451.git"
	},
	"RepoId": 7,
	"FileSizeLimit": 2097152,
	"Timeout": "1h"
}'
  4. validate using the webserver UI: http://127.0.0.1:6090/search?q=test+file%3Amd&num=20 image
  5. switch to this branch: git checkout 4-add-branches-param
  6. push a new commit to the default branch of flightjs project, I pushed TEST.md file
  7. call the index api again 
    curl --request POST \
  --url http://127.0.0.1:6080/indexer/index \
  --header 'Content-Type: application/json' \
  --data '{
	"GitalyConnectionInfo": {
		"Address": "unix:/Users/terrichu/projects/gitlab-development-kit/praefect.socket",
		"Storage": "default",
		"Path": "@hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451.git"
	},
	"RepoId": 7,
	"FileSizeLimit": 2097152,
	"Timeout": "1h"
}'
  8. validate using the webserver UI: http://127.0.0.1:6090/search?q=test+file%3Amd&num=20
  9. push a new commit to the default branch of flightjs project, I pushed ANOTHER_TEST.md file
  10. call the index api, but add the new commit id 
    curl --request POST \
  --url http://127.0.0.1:6080/indexer/index \
  --header 'Content-Type: application/json' \
  --data '{
	"GitalyConnectionInfo": {
		"Address": "unix:/Users/terrichu/projects/gitlab-development-kit/praefect.socket",
		"Storage": "default",
		"Path": "@hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451.git"
	},
	"RepoId": 7,
	"FileSizeLimit": 2097152,
	"Timeout": "1h",
	"Targets": [
		{
		"TargetSHA": "7ad313a17b95ceb018727b1378b2f7047b4f4aec"
		}
	]
}'
  11. validate using the webserver UI: http://127.0.0.1:6090/search?q=test+file%3Amd&num=20 image
  12. call the index api, but add targets with branch names including HEAD. Note: if you push another branch that does not include HEAD, you get a 500 error. I'm not sure how to handle that yet (rails side or zoekt side) 
    curl --request POST \
  --url http://127.0.0.1:6080/indexer/index \
  --header 'Content-Type: application/json' \
  --data '{
	"GitalyConnectionInfo": {
		"Address": "unix:/Users/terrichu/projects/gitlab-development-kit/praefect.socket",
		"Storage": "default",
		"Path": "@hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451.git"
	},
	"RepoId": 7,
	"FileSizeLimit": 2097152,
	"Timeout": "1h",
	"Targets": [
		{
		"BranchName": "HEAD"
		}
	]
}'

indexing status

  1. call indexing status for flightjs 
    curl --request GET --url http://127.0.0.1:6080/indexer/index/7
  2. add branch to previous request 
    curl --request GET --url http://127.0.0.1:6080/indexer/index/7?branch=HEAD
  3. call it with branch name that does not exist, get 404 
    curl --request GET --url http://127.0.0.1:6080/indexer/index/7?branch=non_existent
  4. call indexing status for another repository, get 404 
    curl --request GET --url http://127.0.0.1:6080/indexer/index/11111
Edited by Terri Chu

Merge request reports