Transient failures: The headless browser doesn't start
We see it from time to time that the headless puppeteer browser is not started. We had the assumption that the babel 7 upgrade (!177 (merged)) made this worse. I have opened this test branch !188 (merged) and re-run the visual job 60 times each. (3 x 20 with the parallel keyword)
_ | Babel 6 | Babel 7 |
---|---|---|
Pipeline 1 | Pipeline #49632563: 12/20 failed | Pipeline #49633530: 13/20 failed |
Pipeline 2 | Pipeline #49640465: 9/20 failed | Pipeline #49643428: 9/20 failed |
Pipeline 3 | Pipeline #49651037: 10/20 failed | Pipeline #49647062: 12/20 failed |
Total | 31 / 60 ~ 51% failed | 34 / 60 ~ 56% failed |
We could run more tests, but around 50 % failure rate is too high. One thing we could try, is running on more beefy machines.
Another thing we probably could do, with the update of !177 (merged), we can start the browser ourselves and provide it to storybook/snapshot
so that we have more control and see what is happening.