GDK Framework
Rationale
GDK started as a small bunch of scripts and Makefiles to help us configure and run the multiple pieces that form GitLab.
The approach we took so far was to make GDK a git repository itself that contains scripts that make possible to configure current state of the application.
It has always been painful to try to run an early version of GitLab with GDK, when you have to test things for backports or security releases.
While we don't officially support more than few versions, we have several enterprise customers that ask us for patches and fixes in more critical pieces of the software that deals with data integrity, replication, etc.
Sometimes those patches are needed so they can validate an intermediary upgrade, while moving to the goal of having the latest version.
When we started to migrate from Makefiles to Rakefiles and started to extract/build useful abstractions like GDK::Configuration
, it made possible to think about a future where GDK will behave more like a Framework and less like a glue-code to run a rolling release distro.
Proposal
Let's start to extract the useful abstractions that help us build "GDK" as a reusable piece:
- How to generate / read / validate and explain configuration
- How to define / check / verify / install required dependencies based current environment (OS version / architecture etc)
- How to define / control / enable / disable bundled "daemons" (our runit based implementation)
What should not be made part of the Gem:
- Any code that is specific to a single version of the product, or include hard-coded configuration etc
How to make the framework usable when we have to go back in time to run an early version of GitLab:
- We should consider everything that goes into the Gem as part of the Framework "Contract", and we should not break compatibility unless we release a major version.
- When we really have to breaking compatibility, we should consider if we can version the base class (like rails approach with migrations:
ActiveRecord::Migration[5.0]
), so we can still go back several versions when needed.
How not to loose the flexibility we have today while iterating:
- By moving the abstractions to the Gem, we can still keep building the "business code" on the main repository as if we were building a "Manifest".
- By making the changes always be backword compatible, we can keep building and improving our use of GDK, while making the library better
Long term vision
To really tie everything together, we should not have GDK be it's own repository. At some point in time in the future, we should move all the "business logic" back to the main gitlab repo, and have it live in some kind of structure that looks like this:
-
~/projects/gitlab/
(main repository) -
~/projects/gitlab/gdk
(this is part of the repository, and versioned along with gitlab itself, which means when a new feature is added, the required changes to run it is also submitted together) -
~/projects/gitlab/gdk/embedded
(folder that contains all required services to run gitlab today, likepostgres
,gitaly
etc) -
~/projects/gitlab/gdk/runit/services
(runit specific folder) -
~/projects/gitlab/gdk/app/config
(allGDK::Configuration
code) -
~/projects/gitlab/gdk/app/services
(allGDK::Services
code) -
~/projects/gitlab/gdk/app/dependencies
(allGDK::Dependencies
) -
~/projects/gitlab/gdk/gdk.yml
(like what we have today)
If we do this correct, running any new version of GitLab in the future would be as easy as :
git switch vX.Y.Z
gdk reconfigure
gdk restart