Picking performance metric: First Paint, First Visual, Speed Index
- SiteSpeed reports First Paint, First Visual Change, Speed Index, and a variety of other metrics.
- We currently have targets defined for First Paint on handbook/engineering/performance
- On SiteSpeed's default summary sheets, for one reason or another the default is to show First Visual Change, and not First Paint. See http://207.154.197.115/gl/sitespeed-result/gitlab.com/2017-08-16-09-03-26/pages.html and https://docs.google.com/spreadsheets/d/1CfIDdXfp5E5N6DNL-wx7J0hp1De9YEXIsXY7QeRiXOE/edit#gid=2128616118
- However, "First Visual Change" is not defined anywhere that I can find it.
- First Paint is defined, but can be ambiguous since people can interpret it as First Paint, First Contentful Paint, First Meaningful Paint. https://medium.com/@addyosmani/progressive-web-apps-with-react-js-part-2-page-load-performance-33b932d97cf2
So I think we have two options:
- Stick with First Paint, make it appear in the reporting and clarify which definition we are using, OR
- Pick a different alternative entirely, namely "Speed Index". It has the advantage that it is well-defined https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index, and also happens to be in part of the standard reports. @timzallmann first floated this idea in gitlab-com/www-gitlab-com!7019 (comment 37111249). I was not a fan of it then because it did not feel intuitive enough, but am coming around to it since it is well-defined.
@timzallmann @jschatz1 @JobV I would appreciate your opinions.