WIP: Add support for indexing multiple repositories
This MR adds the --input-file command line parameter to specify a file from which the repositories to be indexed will be read from, as a TSV (tab separated values)
gitlab-test.tsv
1 @hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904This new parameter will be used by to solve gitlab#205178, so that each worker can index many repositories in bulk.
The output has also been changed to as result set, whether there was an error for a specific repository, and the updated SHA to update the incremental status (Project.index_status).
Input file sample
1 @hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git ddd0f15ae83993f5cb66a927a28673882e99100b
2 @hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904
3 @hashed/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904
4 @hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904
5 @hashed/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904
6 @hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904
7 @hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451.git 588440f66559714280628a4f9799f0c4eb880a4a
8 @hashed/2c/62/2c624232cdd221771294dfbb310aca000a0df6ac8b66b696d90ef06fdefb64a3.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904Output sample
11 mass_insert_user_17/mass_insert_project_private_1.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904 0
16 mass_insert_user_36/mass_insert_project_private_1.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904 0
3 @hashed/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce.git cfdb75096443e8435cffb9c0f36fad92eaf048e7 1
5 @hashed/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git ca7d3f5cd27de32c1ed78581458e3dff485e48df 1
8 @hashed/2c/62/2c624232cdd221771294dfbb310aca000a0df6ac8b66b696d90ef06fdefb64a3.git ffd36de013ff00e2623c6ea35675b1c4763a4524 1
15 mass_insert_user_32/mass_insert_project_private_1.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904 0
17 mass_insert_user_43/mass_insert_project_private_1.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904 0
20 mass_insert_user_76/mass_insert_project_private_1.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904 0
1 @hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git ddd0f15ae83993f5cb66a927a28673882e99100b 1
10 mass_insert_user_15/mass_insert_project_private_1.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904 0
12 mass_insert_user_18/mass_insert_project_private_1.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904 0
19 mass_insert_user_75/mass_insert_project_private_1.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904 0
2 @hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git 6e4495c01509b20144b3b33d92a06f0a7a93e60a 1
7 @hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451.git 588440f66559714280628a4f9799f0c4eb880a4a 1
14 mass_insert_user_23/mass_insert_project_private_1.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904 0
13 mass_insert_user_22/mass_insert_project_private_1.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904 0
18 mass_insert_user_49/mass_insert_project_private_1.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904 0
4 @hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git 3844edcbdb6a14dc45976104ee6b88d6e2089623 1
6 @hashed/e7/f6/e7f6c011776e8db7cd330b54174fd76f7d0216b612387a5ffcfb81e6f0919683.git f15b32277d2c55c6c595845a87109b09c913c556 1
9 mass_insert_user_11/mass_insert_project_private_1.git 4b825dc642cb6eb9a060e54bf8d69288fbee4904 0Closes #58
Edited by Micaël Bergeron