Instrumentation for Elasticsearch duration in monitoring
Monitoring dashboards like https://dashboards.gitlab.net/d/sidekiq-queue-detail/sidekiq-queue-detail?orgId=1&var-PROMETHEUS_DS=Global&var-environment=gprd&var-stage=main&var-queue=cronjob:elastic_index_bulk_cron have execution time broken up by database time, cpu time, gitaly time and some also have Redis time as well.
We should try to figure out how to instrument the time spent waiting for Elasticsearch API calls and add a graph for this as it will help understand a lot better where we are spending time in various different places and how to optimize workers and endpoints. This should be done for Rails controllers as well as Sidekiq.
Related work from before
Edited by Dylan Griffith