Async index creation improvements
Failing async index creation can block the rest of the queue, as it happened with https://gitlab.com/gitlab-org/gitlab/-/merge_requests/156090#note_2127967577. To handle such cased better we can: - When picking index from the pending queue, skip entries with failed attempts above a predefined limit. - Update `prepare_async_index` to check if the target table is partitioned, and delegate to `prepare_partitioned_async_index` if needed. - Sort by `attempts DESC` (see https://gitlab.com/gitlab-org/gitlab/-/issues/494002#note_2129960177) - Log failures, and send alerts to #database channel. See example of watcher for Monitoring Locked Tables: https://gitlab.com/gitlab-com/runbooks/-/merge_requests/6802
epic