"Transaction" metrics from workers are not collected

What does this MR do and why?

For #342578 (closed)

After !56005 (merged), we brought back BackgroundTransaction, which is a way for us to collect the metrics for background jobs. However, we didn't define a create a clear boundary between Gitlab::Transaction, Gitlab::Metrics, Gitlab::Metrics::WebTransaction and Gitlab::Metrics::BackgroundTransaciton. As a result, Gitlab::Metrics.add_event doesn't work with background transaction, leading to some events went missing.

The direct fix is simple. We just need to add the BackgroundTransaction.current to Gitlab::Metrics.current_transaction. When scanning through the source code, it turns out people access BackgroundTransaction frequently. The use cases are to export some metrics dedicated for background jobs:

Furthermore, Transaction and WebTransaction accesses are ambiguous. We didn't have BackgroundTransaction before. It means all calls to Transaction should be redirected to WebTransaction. So, I decided to move a further step to define a clear boundary spliting aforementioned entrypoints:

• Remove direct access to Gitlab::Metrics::Transaction. This class remains an abstract class for BackgroundTransaction and WebTransaciton instead.
• The the use of WebTransaction and BackgroundTransaction explicit. The users must declare the destinations for the metrics. If they want to export to both, they must state it out clear: Metrics::WebTransaction.current || Metrics::BackgroundTransaction.current.

Screenshots or screen recordings

After fixing the issues, the missing metrics, such as gitlab_transaction_event_stuck_export_jobs_total is not exported by Sidekiq:

How to set up and validate locally

1. Enable Sidekiq exporter in gitlab.yml:

1. Start rails background job process

2. Trigger a StuckExportJobsWorker job, or any equivalent jobs that call Gitlab::Metrics.add_event method, in Rails console:

[2] pry(main)> StuckExportJobsWorker.perform_async
=> "5bcd67cef16c7c89b589bbb7"

1. Inpsect the exported metrics from Sidekiq prometheus metrics. Run the following command in the shell:

curl http://localhost:3807/metrics 2> /dev/null | grep gitlab_transaction_event_stuck_export_jobs_total

1. gitlab_transaction_event_stuck_export_jobs_total should be available in the metrics after a while:

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.