Skip to content

fix: add support for SHA256 repositories

Terri Chu requested to merge 144-support-sha256-repository into main

Related to #144 (closed)

Add support for SHA256 repositories

how to test

In GDK rails

  1. Enable feature flag
    Feature.enable(:support_sha256_repositories)
  2. Enable feature flag
     Feature.enable(:store_object_format)
  3. Visit create new project page (http://127.0.0.1:3000/projects/new)
  4. Select Create blank project
  5. Check Use SHA-256 for repository hashing algorithm checkbox
  6. Provide project name and group
  7. Click Create project button
  8. The project should be successfully created
  9. grab the following from the console
    p = Project.last
    p.id # project-id
    p.full_path # full-path
    p.repository_access_level # repository-access-level
    p.visibility_level # visibility-level
    p.namespace_ancestry # traversal-ids
    p.commit # to-sha
    "#{p.repository.disk_path}.git" # repository path (last argument)

In gitlab-elasticsearch-indexer project, this branch

  1. build a new version of the indexer binary
     make build
  2. set the following environment variables (needed to communicate with gdk gitaly and elasticsearch)
      export RAILS_ENV=development 
      export ELASTIC_CONNECTION_INFO="{\"url\":[\"http://localhost:9200\"],\"aws\":false,\"aws_access_key\":\"\",\"aws_region\":\"us-east-1\",\"max_bulk_size_bytes\":10485760,\"max_bulk_concurrency\":10,\"index_name\":\"gitlab-development\"}"
      export GITALY_CONNECTION_INFO=""{\"storage\":\"default\",\"limit_file_size\":1048576,\"address\":\"unix:/Users/terrichu/projects/gitlab-development-kit/praefect.socket\",\"token\":null}""
      export CORRELATION_ID=nil  
      export SSL_CERT_FILE="/usr/local/etc/openssl@1.1/cert.pem" 
      SSL_CERT_DIR="/usr/local/etc/openssl@1.1/certs"  
  3. test sending an empty string for from-sha run the following command using the info grabbed in rails above
      bin/gitlab-elasticsearch-indexer --blob-type="blob" --project-id=43 --full-path="Commit451/a-256-sha-project" --timeout=3600s --visibility-level=20 --repository-access-level=20 --archived=false --schema-version-blob=23_08 --from-sha="" --to-sha="5abd9db5f358c9477cbadcdfc12acc53831c3a41092c837cefad46072b84365c" --traversal-ids="29-"  "@hashed/44/cb/44cb730c420480a0477b505ae68af508fb90f96cf0ec54c6ad16949dd427f13a.git"
  4. verify in elasticsearch that the project was indexed
     curl --request GET \
       --url http://localhost:9200/gitlab-development/_search \
       --header 'Content-Type: application/json' \
       --data '{
     	"query": {
     		"bool": {
     			"must": [
     				{
     					"term": {
     						"type": {
     							"value": "blob"
     						}
     					}
     				},
     				{
     					"term": {
     						"project_id": {
     							"value": 43
     						}
     					}
     				}
     			]
     		}
     	}
     }'
  5. you can also test with a from-sha set to nothing (--from-sha=) or the Zero SHA (-from-sha="0000000000000000000000000000000000000000000000000000000000000000")
Edited by Terri Chu

Merge request reports