Distributed Systems / Microservices
From 12 Factor, if there are multiple codebases, it’s not an app – it’s a distributed system. We should support distributed systems, especially mindful of the rising trend in microservices and Docker adoption. We cover multi-project pipelines elsewhere, but we can go further with system-level views and coordinated deploys.
- New view of related apps in a distributed system
- Zoom out in the project pipeline
- Group in GitLab for team/issue/activity tracking (maybe using nested groups, but maybe allowing multiple inheritance)
- Block deploys of one component if it depends on a version of another component that has not been deployed yet to the same environment e.g.:
- Annotate relationship between merge requests so that we know if one MR blocks another.
- Track deploys of MRs to each environment so we know, for example, when a given MR is in production.
- We can block merges of one MR until upstream changes are merged.
- We can block deploys of one MR until upstream changes are deployed (to the same environment).
- This can work across projects so individual services get deployed in the right order.
- Coordinated deploy of all related apps to a new environment. e.g. autogenerate a "cloudformation" because we know how the projects relate to each other. This could, for example, be used by GitHost to spin up a new single-tenant instances of GitLab for a new customer.
Links / references
changed title from Distributed Systems to Distributed Systems / MicroservicesToggle commit list
@joshlambert You mentioned elsewhere that nested groups might map well to microservices. Nested groups may very well help here, and at least form a first iteration of proper support. e.g. a collection of services that need to be deployed together could be under the same nested group. You could even have multiple levels of hierarchy so one service could then depend on another group of services.
Unfortunately, you eventually hit limits. Sufficiently complex apps have service graphs that look more like directed acyclic graphs rather than stricter hierarchy trees. e.g. What do you do with a service that is used by two other services that are in different groups?
Maybe we can ignore that complexity for now and see how far the hierarchy gets us. It will solve a bunch of use-cases, even if it doesn't solve 100% of them. And perhaps even for the rest, it's good enough. Heck, operating systems have forced a hierarchy of organization for files for decades and people haven't complained much. :)
Another complication is how to make use of this hierarchy information. Specifically, projects may have different dependencies for production vs local development, CI test, and review apps. The idea of a single hierarchy to represent an app is kinda flawed. e.g. a production system might have 100+ services, but in order to develop on one specific service, I may be able to stub out or even just ignore the majority of them. Integration tests run during CI may only depend on 2 services. Most review apps might need only 2-3 services to provide enough fidelity for testing, but some changes might require additional services to test properly. Imagine a merge request to
gitlab-ce, for example, that only changes something in the UI. It wouldn't need the
gitalyservice or runners to be present.
But to flesh out more complex alternatives some options are:
- Use nested groups (trees) and force people to put all services in only one group.
- Create a separate concept outside of groups to represent collections of services.
- Define the dependencies explicitly by references, regardless of group ownership.
- Add tags to projects; reference systems by tag.
You could probably force people into a hierarchy model where all sub projects are built first, and then parent projects, and so on up the chain.. Then you'd need some pipeline definition concept at a group level, so you could run CI/CD tests on a group milestone/tag level to get a representative environment.
That would certainly be convention over configuration, but it seems overly burdensome than some services mapping overlay. The benefits though is it could integrate really well with group based tags/milestones/etc to reduce some of that potential complexity.