Skip to content

Refactor how finished_at is merged into usage data

Alishan Ladhani requested to merge ali/refactor-usage-data-finished-at into master

What does this MR do?

In #219472 (closed), it was noted that we need a better way to ensure that recording_ce_finished_at is captured after the usage ping data is generated. This timestamp gives us an idea of how long it takes to run all the queries for usage ping.

Currently, it is unclear that recording_ce_finished_at should be captured last, and it is very easy for an MR to append some data after this timestamp is captured.

  license_usage_data
    # merging some metrics into the hash
    .merge(topology_usage_data)
    .merge(recording_ce_finish_data) # <- Timestamp captured here
    .merge(merge_requests_usage_data(default_time_period)) # <- This data is added after the timestamp is captured

This MR introduces with_finished_at, a helper that will record a timestamp after the block has finished execution. By structuring the code this way, it should be easier to enforce that recording_ce_finished_at is captured after all data is generated.

  with_finished_at(:recording_ce_finished_at) do
    license_usage_data
      # merging some metrics into the hash
      .merge(topology_usage_data)
      .merge(merge_requests_usage_data(default_time_period))
  end # <- Timestamp captured after block execution

The same change is also done on the EE side, with recording_ee_finished_at.

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team

Closes #219472 (closed)

Edited by Alishan Ladhani

Merge request reports