Document performance testing for features at "gitlab.com scale"
In the CI long polling retrospective (gitlab-com/organization#44 (comment 27721838)), I noted that fixing a performance issue at GitLab.com scale is currently mostly guesswork and dead reckoning - we observe the problem, and rely on deep thought about it to come up with a solution that will work.
The current performance guides have good advice about how to write code that doesn't have s range of performance issues, e.g. http://docs.gitlab.com/ce/development/performance.html
I think they can be enhanced to include some more specific guides on:
- Calculating the current load caused by a particular feature
- Building a representative environment to test changes in (obviously, this can't be full scale)
- Validating changes in that environment
The goal is to help people reduce the feedback cycle between noticing a problem in large-scale environments, and seeing if a particular solution will make the problem go away. It's important to be able to validate changes without having to cut a release and deploy them to GitLab.com - and ad-hoc changes in production are verboten, of course!