[Discussion] Design Management and Git
Hi all!
I’d like to open up a discussion on one aspect of the technical vision for design management, which is its use of Git for maintaining the versioning of designs. This is a complicated discussion to have with a new PM and BE engineers on opposite sides of the world, but it’s becoming an emergent issue that I’d like us to clarify in order for us to know whether the effort we're going to in order to support versioning with Git is going to have benefit in future.
Current state
Behind the scenes, designs are committed into a Git repository, that we call the "design repository". This repository is 100% a Git repository, but it's hidden from the user.
If a user uploads 3 designs at the same time, it creates a new "version" (what you would see in the drop-down versions selector in the design management page) and that version is currently tied to a particular Git commit of the three files into the design repository.
Where are the designs currently stored?
Technically, not in Git, because we've made Git LFS a requirement for design management. What is committed to Git is a very small plain text file that includes a particular string of characters (an OID
). When we serve the image for a design, we go through Git to find this OID
and use the OID
to find a record in PostgreSQL. We then either serve the file from a local file or if enabled, from object storage (like an S3 bucket). In Gitlab.com's case, the design file itself is stored in object storage (I believe, Google Cloud).
Problem
As far as I'm aware, in its current form, the underlying Git repository of design management doesn't add any benefits to our features.
Supporting the committing of designs to a Git repository has added extra complexity to some issues I've worked on, and am currently working on, and often surfaces as a limitation in the discussion of issues:
Alternative model
Designs wouldn't be uploaded to object storage, or local disk, via a commit to Git, but would instead be directly uploaded. We would stop supporting an underlying Git repository, and the backend of design management would only be concerned with data in PostgreSQL.
The benefits of this would be:
- I believe we could move quicker on a number of issues.
- The BE wouldn't need to translate features into paradigms that fit an underlying Git repository.
- The requirements of Git LFS and hashed storage could be removed.
The downsides of this would be:
- The migration away from Git would take engineering time.
- It would likely take a number of releases to achieve and in the meantime, we would need to support old and new.
- We would need to migrate existing designs out of LFS.
- Our annotations are highly-coupled to Git, and we would need to refactor this.
What I'd like to know
I'd like to understand if we have a technical vision for design management that is reliant on the continued support of maintaining a Git repository.
Part of this discussion is to identify all the potential future benefits of having a Git repository for design management.
I'll seed the discussion in this issue with the potential future benefits that I'm aware of, and invite others to do so too.
Hopefully, between Product and Engineering, we can help each other clarify the direction we should take.
Summary of Pros and Cons
Option 1: GIT (current) |
Option 2: DesignsUploader |
||
---|---|---|---|
Pro | Con | Pro | Con |
1) Ability to add commit messages when uploading new design versions 2) The semantics of the data model for git are well understood, meaning its objects and operations are defined clearly. This saves us having to make decisions about these things. 3) Ability to maintain a history, with ability to revert/review changes 4) We are currently using this so there is no migration required. |
1) Uses a multi step process. An OID is stored into a small text file which is then committed to GIT. This OID is used to retrieve a record from Postgre SQL. The file is then served from either a local file or from a object storage. 2) Supporting the committing of designs to a Git repository has added extra complexity when making code changes or adding functionality |
1)Designs wouldn't be uploaded to object storage, or local disk, via a commit to Git, but would instead be directly uploaded. 2)We would stop supporting an underlying Git repository, 3) the backend of design management would only be concerned with data in PostgreSQL. 4) The BE wouldn't need to translate features into paradigms that fit an underlying Git repository. 5) The requirements of Git LFS and hashed storage could be removed. |
1) The migration away from Git would take engineering time. We would need to migrate existing designs out of LFS. 2) It would likely take a number of releases to achieve and in the meantime, we would need to support old and new. 3) Our annotations are highly-coupled to Git, and we would need to refactor this. |