Feature Proposal: GitLab Service Registry
Release notes
In a microservices environment, organizations may be running dozens of different services (with new ones added on a continual basis), each serving a specific business function. Teams tend to develop these services independently and may have different standards with regard to documenting information about the service. When other team members need to interact with the service, it can be difficult to find information about the service. For example, if a service goes down, site operators need to retrieve information about the service in a timely fashion such as its purpose, owner, tier/SLA, dependencies, data classification, and log name to help with troubleshooting. They either have to rely on their own notes or read through several documentation pages to find the information they need.
To solve these challenges, we introduce GitLab Service Registry, a centralized catalog of all services running in an organization's environment providing users a single source of truth to find the most up to date metadata about their services. The Service Registry stores the key attributes of each service, so that users can quickly find the information they need in one place.
Problem to solve
As a Site Reliability Engineer, I want to access a single source of truth that shows the most up to date information about services running in the production environment so that I can quickly troubleshoot an outage.
As a DevOps engineer, when configuring my logging system, I want to automatically pull in the latest information about my services (e.g. logger prefix name) so that I don't have to manually enter it, which can lead to errors.
As a security operations engineer, I want to quickly look up a service and find its data classification so that I can properly understand the impact of a security breach.
As a developer new to my company, I want to see information about all the services we own to speed up my onboarding and help me determine whom to engage with.
As a platform engineer, I want to provide automation that serve all the requirements described above when a new service is created.
Intended users
- Delaney (Development Team Lead)
- Sasha (Software Developer)
- Devon (DevOps Engineer)
- Sidney (Systems Administrator)
- Sam (Security Analyst)
- Rachel (Release Manager)
- Simone (Software Engineer in Test)
User experience goal
Users should be able to populate a configuration file (e.g. yml) to catalog metadata for all of the services that make up an organization's infrastructure.
When configuring internal tooling (e.g. logging system or monitoring dashboards), users should be able to reference elements in the yml file so that if an attribute of a service is ever updated, changes will be automatically propagated to the tool.
Proposal
MVC
- Create a new project template enabling users to set up a Service Registry project that includes a yml file (to store the service metadata) and accompanying front-end app. E.g. https://gitlab.com/gitlab-com/gl-infra/service-catalog-app/-/tree/master
- Users with MR permissions for this project can then update service information in the metadata within the service registry and provide an audit trail of changes.
- Seed YML File with sample metadata fields ex:
- Service name
- Tier/ SLA
- Dependencies
- Data Classification
- Logging Name
- Alert Label
- Owner
- Documentation link
- Runbook link
- Infrastructure information (cloud provider for service, label names, tags etc)
- Documentation showing examples on how to read the yml file from code to configure other services (e.g. metrics/logging)
Second Iteration:
- Render contents of yml file into a webpage as described here: #13315
Later Iterations:
- UI Enhancements: let users browse, search, and update the Registry (with appropriate authentication and access control) similar to: https://us-central1-gitlab-infra-automation-stg.cloudfunctions.net/ui/services
- Visualization of all the different services and their dependencies as an interactive graph
- Make it easier to configure tools with information from the service registry by integrating GitLab managed apps and helm charts so that applications/tools can be properly configured with the most up to date service attributes.
Further details
Note, this proposal is not meant to be any of the following:
- A service discovery solution, which provides applications a way to find the DNS names of other services at runtime (although the service registry could be used to seed an initial configuration of the Service Discovery solution).
- A service mesh, which provides a communication mechanism between services (although the service registry could be used to seed an initial configuration of the service mesh)
- A service catalog, which allows users to provision IT services/software within their org