Skip to content

Extract events from metric definitions in CI to reduce startup time

Problem

Currently we load all metric definitions in the initializer config/initializers/internal_events.rb.

This is adding a bit more time to the startup than we would like due to the inefficiency of load_metric_definitions in lib/gitlab/internal_events/event_definitions.rb.

If the event definitions were accessable in a format that is better suited for the use case we wouldn't have to load and parse all metric definitions.

In addition the inherent problems of caching led to issues like #416627 (closed) where tests where getting flaky.

Desired Outcome

Performance of startup time not significantly affected by loading metric definitions

Proposed Solution

From a Slack conversation with @stanhu in #backend where the question about how to process the definitions without impacting performance:

could you load some pre-generated file that has all those keys parsed from every YAML file? for example, you could have a Rake task that extracts what you need into a single file, and then load that file upon initialization. There would be a CI task that would ensure that anytime someone added a metric that the Rake task would need to be updated

So, we would like to pre-compute and store a json file in CI that is structured so that we just need to load it to support lib/gitlab/internal_events/event_definitions.rb

How to verify

TBD - We should come up with a way to measure the impact on startup time we have right now, vs what a potential improvement should achieve

Further actions needed

None defined at this time.

Edited by Sebastian Rehm