Zoekt: Rolloutworker should skip the namespaces which can not be indexed due to storage and continue
Background
We found that during Zoekt rollout, if we get a storage error, the rollout gets blocked because we fetch the same namespaces in each iteration. And we get the same error again.
Proposal
We should skip the namespaces that can not be rolled out with the current available storage and continue with the other namespaces. We can add two new fields in the zoekt_enabled_namespaces.metadata: last_assignment_failed_at and estimated_storage_bytes. In the SelectionService we can filter out the namespaces which have a recent last_assignment_failed_at.
Edited by John Mason