Replace sidekiq mtail metrics with direct instrumentation

Similar to https://gitlab.com/gitlab-org/gitlab-ce/issues/49215, we need additional metrics that we currently only get from mtail. See also #60303 (closed) for sidekiq process metrics.

  • Job metrics
    • sidekiq_jobs_completion_time... Histogram of job completions.
    • sidekiq_jobs_started_total Number of jobs started.
    • sidekiq_jobs_failed_total Number of jobs failed.
    • sidekiq_jobs_returned_to_redis_total Jobs kicked back to redis.
    • sidekiq_jobs_memory_allocated_bytes... Histogram of memory required for a job.
Edited by Logan King