Extend `rack` Prometheus metrics to measure `user` request duration
Following the: https://gitlab.com/gitlab-org/gitlab-ee/issues/31887 and https://gitlab.com/gitlab-org/gitlab-ee/issues/31888 and https://gitlab.com/gitlab-org/gitlab-ee/issues/30567.
We should extend lib/gitlab/metrics/rack_middleware.rb
(or don't know where it is) to measure user execution time of different requests in unicorn/puma and build a histogram of user_s
.
Currently we measure only wall clock time
. This is useful, but our duration are inflated by noisy environment, especially impact when running Puma and Multithreaded App Server.
Having user
execution time (time spend in thread) will allow us to see which requests cause severe performance impact (a lot of time spend in Ruby thread) and cause contention on Puma multi-threaded model.