Create an API to define & track SLI ratio metrics in the ruby Application
For &525 (closed) we want to figure out a way to easily define, initialize and use SLI (success-rate) type of metrics in the ruby parts of our codebase. This issue is to discuss what that should look like. Ideally this code ends up in Labkit to be abstracted away of the main application.
In this iteration, we want to define an API that allows developers to consistently initialize and increment 2 counters:
- operation rate: the counter that is incremented on every operation (for example a request)
- success rate: the counter that is incremented on a successful operation (for example a request with status code 2xx, or a request that was fast enough compared to the threshold)
The API should allow initializing these 2 counters with all the possible label combinations we expect. Though when tracking an unexpected combination, the code should not fail (it could perhaps emit a warning in the logs, now or in a future iteration?)
Later, we could implement the same kind of thing for our go-services, making sure that the metrics emitted look the same.
Ideas
Initialization (Idea 1)
Labkit::Metrics.initialize_sli(:rails_request_apdex, label_combinations:[{ feature_category: 'code_review', endpoint_id: 'Projects::MergeRequestsController#show' }]
This initializes these: 2 counters to 0 with all of the passed label combinations
labkit_sli:rails_request_apdex:total
labkit_sli:rails_request_apdex:success_total
Observation (Idea 1)
rails_request_apdex = Labkit::Metrics.sli(:rails_request_apdex)
observation = rails_request_apdex.observe({ feature_category: 'code_review', endpoint_id: 'Projects::MergeRequestsController#show' }) # increments the total
observation.success! if duration < 1# increments the success
Though this splits the observe
and success quite a bit
Observation (Idea 2)
Labkit::Metrics.sli(:rails_request_index).observe(
labels: {...}
success: duration < 1 # required argument
)
The :total
counter always incremented, the :success_total
is only incremented when success: true
is passed.