Zoekt: Customers will be confused about last_rollout_failed_at

Summary

We've just merged Zoekt: Rollout should skip the namespaces which... (!186414 - merged) which will mark a namespace as failed to be assigned.

Steps to reproduce

What is the current bug behavior?

If SM customers have large namespaces and not enough storage, their namespaces will be marked as unassignable and they have no easy way to even see that.

What is the expected correct behavior?

I believe that we should at least add this counter to gitlab:zoekt:info so that we don't receive a lot of tickets about Zoekt not scheduling namespaces.

Relevant logs and/or screenshots

Possible fixes

  1. Add an index on zoekt_enabled_namespaces.metadata
    CREATE INDEX index_zoekt_enabled_namespaces_metadata_last_rollout_failed_at
    ON zoekt_enabled_namespaces (((metadata->>'last_rollout_failed_at')::timestamptz))
    WHERE metadata->>'last_rollout_failed_at' IS NOT NULL;
  2. Add a counter with the number of namespaces without indices and with the timestamp set to gitlab:zoekt:info
Edited by Dmitry Gruzd