Skip to content

Add recording finish times to usage data

What does this MR do?

Add usage ping recording finish times

We want the usage ping to be performant, we'd like to see how long it takes to calculate

  • the core edition data
  • the enterprise edition data (with a lot of slower counters

We already have - at the very beginning

"recorded_at": "2020-05-07T10:33:13.548+03:00",

We add two new fields to the usage data whenever calculations are finished

"recording_ce_finished_at": "2020-05-07T10:33:16.290+03:00",
"recording_ee_finished_at": "2020-05-07T10:33:17.331+03:00"

Details

We have two separate metrics for the duration of Core usage data calculations and Enterprise usage data calculations.

  • for CE we shall have only recording_ce_finished_at
  • for EE we have recording_ce_finished_at and recording_ee_finished_at

The reason is

  • EE edition has a lot more counters on top of the core counters
  • EE edition counters have a lot of distinct & time period calculations which are slow
  • CE edition calculations are less heavy
  • We may do an apples-to-apples comparison of durations of CE (very fast) and EE(slow) computations.

For an EE instances

{
      3:  "recorded_at": "2020-05-07T12:47:28.672+03:00",
.....
Core Edition Timing is calculated for both CE and EE
....
    324:  "recording_ce_finished_at": "2020-05-07T12:47:32.299+03:00",
....
Enterprise Edition timing is calculated for only EE
...
    523:  "recording_ee_finished_at": "2020-05-07T12:47:33.842+03:00"
}

For a CE instance (roughly)

{
      3:  "recorded_at": "2020-05-07T12:47:28.672+03:00",
.....
Core Edition Timing is calculated for both CE and EE
....
    324:  "recording_ce_finished_at": "2020-05-07T12:47:32.299+03:00",
}

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team

Closes #216157 (closed)

Edited by Alper Akgun

Merge request reports