New: User Timing API for monitoring performance health of the application
UPD: There was a workshop during the Virtual Contribute on the subject wit the follow-up discussion. Everything that has been discussed during the follow-up has been documented in the separate document. Also the video recording (in low quality, unfortunately) is available.
Main takeaways:
- it doesn't make a lot of sense to do this behind the feature flag as the metrics will be collected in the controlled environment of sitespeed's measurement, not on the client computers. However, the metrics will still be available on the clients' computers should we ever need to debug any performance issue happening on a particular machine of some customer.
- there's no performance penalty if we use sitespeed for monitoring
Original Discussion.
In the course of improving performance monitoring, I suggest implementing User Timing API marks for Web IDE.
New Pattern Proposal
Why measure/monitor at all?
We know, at least I hope we do, that there are computations-heavy parts of the application that need performance analysis/improvement. However, if we don't have a benchmarking mechanism based on numbers with predictable before/after change comparison, anything we do for performance is speculation.
FirstVisualChange,
SpeedIndex,
etc.?
Isn't it enough to monitor Currently, we run monitoring against several performance metrics using sitespeed.io with the output to a dedicated Grafana dashboard.
Even though these conventional parameters (like those mentioned in the title of the paragraph) are crucial for performance monitoring, they do not give a complete picture of the application's performance. They take the page as a whole into consideration while in large modern async applications, this is not necessarily what we want to monitor.
Every part of an application has core parts that constitute the essence for that or another view — for example, loading MR's description and a link to "Changes," loading diff for Diff view, loading navigation tree, and the editor for Web IDE. All of these are essential, however conventional metrics take the page as a whole, mixing in things like pipeline status, last commit, and other information that is not essential for that or another view and can be loaded asynchronously as the "second batch" of requests. Moreover, what if we want to measure the performance of one particular component? With the conventional metrics, we're out of luck.
User Timing API?
WhyUser Timing API allows one to measure anything that makes sense to measure in the application by setting distinctive marks and, at a later point, query those marks, measure duration between any two marks, etc. For example:
...
performance.mark('ide-editor-start');
export default {
// my Vue component
watch: {
file(newVal, oldVal) {
...
performance.measure('ide-editor', 'ide-editor-start');
}
}
}
Then later, we can query all the marks and measurements with:
performance.getEntriesByType('mark');
performance.getEntriesByType('measure')
The former one would return all timestamps for all mark
entries, while the latter would return the entries measuring durations between different marks.
One doesn't need to keep the marks in the same file, and those can be spread across the entire application to make cross-components measurements possible as well.
All the performance measurements will be output into the "Timings" panel of the "Performance" section of DevTools:
Proposal
- It is proposed to start to implement marks and measurements behind a Feature Flag for the time being.
- Introduce some way of measuring performance degradation on MR level (in CI) based on these new measurements
- Update Grafana monitoring dashboard with the new measurements to reflect the development of these over time.
Disadvantages of new pattern
~~This is a two way door decision~~ so no disadvantages so far, as I can see: ~~we control the effect with the Feature flag. At least at the beginning of the process.~~
What is the impact on our existing codebase?
The codebase will get several performance.mark
and performance.measure
that should be allowed on the linting level if it's not at the moment.