Deploy feature branches in staging using review apps
Continuing https://gitlab.com/gitlab-com/infrastructure/issues/42 we will start by providing a simple way of testing a code branch in a staging environment with monitoring attached to it. We will use this as the testbed to start fiddling with containerizing the application to do a similar thing in production later on.
The idea is that we should be
- creating a package and docker image with the code of a branch.
- snapshotting the staging database to derive a new temporary server from it.
- snapshotting the git and shared filesystem to also derive it.
- spin up a container with this branch image wiring to the staging database copy.
- keep blackbox monitoring sample with the current version in a staging environment.
- wire prometheus white and blackbox monitoring to this branched pod (in a non production prometheus) at the same time to compare with the main one.
To get there we will need
- A solid release infrastructure that we track an monitor.
- A way of removing old builds of a branch (maybe every time we build a new version for a branch remove the packages, and then remove them when they are merged)
- Staging refreshing faster for the database and for the filesystem too.
- Something to trigger the build of the branch.
- A kubernetes cluster in which we can start spawning these pods.
- A simple way of adding more endpoints to the blackbox monitoring (as developers add features, they will surely need to add new urls)
- A prometheus server for the staging environment.
- Prometheus wired with kubernetes etcd service discovery to monitor these pods.
- An extremely simple way of triggering this process.
Review apps: https://docs.gitlab.com/ee/ci/review_apps/