Document Service Dependencies
Last week we moved our infrastructure to the new Azure ARM environment. One of the goals of the move was to split monolithic workers pool into more specific ones, which improves both the resource consumption and the scalability.
Sadly, it didn't take long to cause an outage because of hardcoded preconceptions in the application logic.
As we progress towards a more scalable and manageable infrastructure we need to decouple all the services and be more explicit in the configuration files about the location of each one: we can't just drop a configuration line because it will implicitly default to the local unix socket located at a default path.
This will be mandatory in a containerised environment, therefore it is a blocking issue for a number of projects in infrastructure. So we need to map all the assumptions that our architecture makes and make them explicitly configurable. If an application doesn't yet support this then we need to code the option.
So let's start with what we've already found out.
- Sidekiq requires a local API server (unicorn and nginx). Documented in https://gitlab.com/gitlab-com/infrastructure/issues/365, action issue not yet created.
- Sidekiq requires a local container registry configured properly. Documented in https://gitlab.com/gitlab-com/infrastructure/issues/1226, action issue not yet created.
Please take a look at the architecture and add all that you know.
cc/ @stanhu @pcarranza @ayufan @DouweM @marin @smcgivern @sytses