Skip to content

Draft: Opt 1 - Add spec helpers (composable assertions for internal events)

Sarah Yasonik requested to merge 450452-sy-metrics-instr-functions into master

What does this MR do and why?

This MR is a POC to help decide on a maintainable & usable framework for composing assertions about internal events & metrics.

Notable:

  • the pipeline is failing. that's ok. there're a few kinks to work out, but the approach is mostly functional.

Example usage:

RSpec.describe 'Composable assertions for events & metrics' do
  include InternalEventsHelpers

  it 'increments internal events metrics for compliance dashboard' do
    expect_internal_event('g_compliance_dashboard', user: user, namespace: group)
    expect_metric_to_increment('redis_hll_counters.compliance.g_compliance_dashboard_monthly', count: 1)
    expect_no_internal_event('web_ide_viewed')

    Gitlab::InternalEvents.track_event('g_compliance_dashboard', user: user, namespace: group)
  end

  it 'increments database-backed issue creation metrics' do
    expect_metric_to_increment('usage_activity_by_stage.plan.issues')
    expect_metric_to_increment('usage_activity_by_stage_monthly.plan.issues')

    Issues::CreateService.new(container: project, current_user: user)
  end
end
Composable how? Caveats? Example?
[THIS MR]
1: Add spec helpers
Assertions are made before calling subject. The relevant events/metrics are collected and expectations are run after the example body. The InternalEventsHelpers module must be included in any specs that want to use these methods.
expect_internal_event('g_compliance_dashboard', user: user, namespace: group)
expect_metric_to_increment('redis_hll_counters.compliance.g_compliance_dashboard_monthly', count: 1)
2: Add custom matchers with block Matcher mirrors the behavior of the .change matcher, so assertions are chained together and run alongside subject. Readability of the assertions may be a little trickier.
expect { request }
.to trigger_internal_event('g_compliance_dashboard').with(user: user, namespace: group, category: nil).times(1)
.and increment_usage_metric('redis_hll_counters.compliance.g_compliance_dashboard_monthly').by(1)
3: Add custom matcher with event/metric definition subject is called, then assertions are made afterwards. The service ping payload is generated ahead of the spec, so metric values can be compared before/after. Performance for database-backed metrics is quite slow, and occurs repeatedly. This also requires the use of metadata tags to include the relevant shared contexts.
expect(internal_event('mr_created')).to be_triggered_with(user: user, project: project, namespace: namepsace)
expect(usage_metric('counts.deployments')).to be_incremented_by(1)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #450452 (closed)

Edited by Sarah Yasonik

Merge request reports