Allow operations to be instrumented once in Pipeline creation logger
What does this MR do and why?
Related to Pipeline step log entries have too many fields (#371636 - closed)
The pipeline creation logger currently logs too many fields to Elasticsearch causing performance and stability issues to the stack. We need to reduce the number of fields and remove those that are not essential.
Some of the observations used in the logger are only occurring once per log instance. For example: pipeline_creation_duration_s
is observed only once at the end of the pipeline creation process. For such observations we don't need to have multiple values.
This would reduce further the number of fields indexed in Elasticsearch because pipeline_creation_duration_s.sum
and pipeline_creation_duration_s.count
simply become 1 field: pipeline_creation_duration_s
. In fact for these fields the .count
will always be 1.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.