Zoekt doesn't process missing repositories correctly
Summary
We've discovered that we have some stuck indices. The reason is that we mark tasks as orphaned when repo is missing, but we should mark repositories as ready.
Steps to reproduce
What is the current bug behavior?
What is the expected correct behavior?
Relevant logs and/or screenshots
[ gprd ] production> i
=>
#<Search::Zoekt::Index:0x00007efd409a8488
id: 1126954,
zoekt_enabled_namespace_id: 15615,
zoekt_node_id: 1000155,
namespace_id: 57636428,
created_at: Fri, 21 Feb 2025 13:46:29.985941000 UTC +00:00,
updated_at: Tue, 25 Feb 2025 14:24:05.003357000 UTC +00:00,
state: "initializing",
zoekt_replica_id: 1037013,
reserved_storage_bytes: 192318262164,
used_storage_bytes: 7504516175,
watermark_level: "overprovisioned",
metadata: {"project_namespace_id_from"=>50044758},
used_storage_bytes_updated_at: Tue, 25 Feb 2025 14:24:04.999367000 UTC +00:00,
last_indexed_at: Tue, 25 Feb 2025 14:23:56.670342000 UTC +00:00>
[ gprd ] production> i.zoekt_repositories.group(:state).count
=> {"initializing"=>1, "ready"=>1517}
[ gprd ] production> r = i.zoekt_repositories.initializing.first
=>
#<Search::Zoekt::Repository:0x00007efd405928a8
...
[ gprd ] production> r.tasks
=>
[#<Search::Zoekt::Task:0x00007efd405bd8f0
id: 319301689,
zoekt_node_id: 1000155,
zoekt_repository_id: 936108665,
project_identifier: 58193864,
perform_at: Fri, 21 Feb 2025 13:54:05.344708000 UTC +00:00,
created_at: Fri, 21 Feb 2025 13:54:05.495458000 UTC +00:00,
updated_at: Fri, 21 Feb 2025 13:54:05.495461000 UTC +00:00,
state: "orphaned",
task_type: "index_repo",
retries_left: 3>]
[ gprd ] production> Project.where(id: 58193864).exists?
=> true
[ gprd ] production> Project.where(id: 58193864).first.repository.exists?
=> false