Sharding Headcount Request from Database Scalability Working Group
Background
Context
The Database Scalability Working Group is working on a long term solution to scale the GitLab application for the fast growth of SaaS. The working group has identified two categories of work that help achieve this goal: scale by data usage pattern and sharding the PostgreSQL database. There is also a prerequisite that we "clean up before scale up" where we need to address primary key overflow of identified tables and enable automated schema migration testing in CI to prevent migration failures in production.
The issue is designed to request additional headcount for the work Sharding the PostgreSQL database
.
Current Sharding Team Staffing
Team | BE | BEM | PM | SDET |
---|---|---|---|---|
Sharding | 0 | 0 | 0 | 0 |
Request From Priority Teams
Team | BE | BEM | PM | SDET | SRE | Total |
---|---|---|---|---|---|---|
Sharding | +5 | +0.5 | +0.5 | +0 | +0 | 6 |
Total | 5 | 0.5 | 0.5 | 0 | 0 | 6 |
What's Changing
Overall Adjustments
The total shall sum up to 0 in the table below, which means all the requested headcount is fully allocated. Team X/Y/Z are the teams who will give people out.
Team | BE | BEM | PM | SDET | SRE |
---|---|---|---|---|---|
Enablement:Sharding | +5 | +0.5 | +0.5 | 0 | 0 |
Enablement:Memory | -1 | 0 | 0 | 0 | 0 |
Enablement:Global Search | 0 | -0.5 | 0 | 0 | 0 |
Enablement:Geo | 0 | 0 | -0.5 | 0 | 0 |
Enablement:Database | -1 | ||||
Ops:Configure | -1 | ||||
Manage:Optimize | -1 | ||||
Delivery | -1 | ||||
Total | 0 | 0 | 0 | 0 | 0 |
- At least One BE must be a Backend Engineer, Database.
- The Sharding team is temporary and will disband when we achieve the business goals.
- The people who joins Sharding team will stay on the team for 6~12 months (subject to change though).
- The Sharding project will be staffed by 2 stages (see details below). The table above only reflects required headcount for stage 1.
- The team will start from 3 BEs at the beginning of stage 1.
- Two(2) other BEs will join the group above on 2021-05-19.
- The
Backend Engineer, Database
must join at the beginning.
- To avoid cascading impacts, suggest the following approach.
- Any impacted group can give one and only one engineer.
- Any impacted group shall have at least 3 engineers remaining on the team.
EMs of Impacted Teams
Team | EM |
---|---|
Enablement:Sharding | Craig Gomes |
Enablement:Database | Craig Gomes |
Enablement:Memory | Changzheng Liu |
Enablement:Global Search | Changzheng Liu |
Delivery | Marin Jankovski |
Ops:Configure | Nicholas Klick |
Manage:Optimize | Dan Jensen |
Timeline and Tasks
Week of 2021-03-29
-
Determine who join the Sharding team
Week of 2021-04-05
-
Activate at least one BE to join the Sharding team
Week of 2021-04-19
-
Activate another 2 BEs to join the Sharding team full time
2021-05-19
-
Activate the last 2 BEs to join stage 1 group
Additional Details
Assumptions
- There will be a distribution of work across various areas of backend engineering and not a singular focus in any one area (e.g. database)
- The other "single thread" items identified in the Database Scalability Working Group remain untouched (clean up before we can scale up)
Proposed Staffing Plan
- This is a fully staffed team with a singular focus on delivering an iterable sharding solution.
- There will be work required on the backend as well as the database.
- The work defined by the development team members will require rapid feedback from their Product Management and Infrastructure counterparts.
- With the unknowns and the complexity of sharding, this staffing plan is designed wotj 2 stages for efficiency -
- Stage 1: The initial team makeup is to work on research style PoC iterations to plot sharding path, make blueprint, articulate required work items (high level backlog), and produce building blocks (e.g. sort out as many unknowns as possible and decide the technical path).
- Stage 2: Activate more development capacity and have stage groups commit to the defined work articulated by the Sharding team once the sharding path is determined.
- Stage 1 requires depth and breadth of the GitLab codebase knowledge and certain specialty skills, therefore the participants may be subject to the working group's discretion.
Stage 1: Initial Team Makeup
DBBE | BE | BEM | PM |
---|---|---|---|
1 | 4 | 0.5 | 0.5 |
Stage 2: Additional Engineering Capacity
More backend engineering capacity will be necessary. The exact capacity requirement to accelerate the implementation will be assessed once stage 1 produces the blueprint and high level backlog. Engineering groups will take shares of work to their backlogs respectively and no engineers will be asked to change groups. Meanwhile, the stage 1 engineers remain on the Sharding group until the project meets exit criteria.
In addition to development capacity, there will be additional requests for QE and UX where appropriate.
Exit Criteria
The same exit criteria of the Database Scalability Working Group.