Draft: "PoC: Autogenerate JSON schema for CI Header"
What does this MR do and why?
This MR is a Proof of Concept to demonstrate that we can auto-generate a JSON schema from our CI Header and CI Entry classes. It generates a schema for the CI header spec
keyword because it is a single, achievable task and is needed for the upcoming CI Steps work.
Issue: #463328 (closed)
How does it work?
This MR defines a new Rake task gitlab:generate_ci_schema
that calls Gitlab::Config::Entry::Json::Schema.for
, which is the entry point to the new JSON schema generating logic. The Gitlab::Config::Entry::Json::Properties
classes generate JSON schema representations of the different CI config keywords. The classes in Gitlab::Config::Entry::Json::Validators
generate JSON schema representations of the different validations in Gitlab::Config::Entry::Validators
.
Pros
I see 3 things we get out of this:
- It's the first step towards auto-generating a JSON schema for the entire CI config. Writing the schema manually adds an additional step to any work on the config classes, and is prone to errors. For example, in this MR you can see that the auto-generated schema for the
spec:
keyword is more complete than the existing manually written one - As a byproduct of the work, we'll get config classes that are more consistent. The work in this MR revealed an inconsistent use of the
AllowNil
validation and the need for a more consistent way of referencing the attributes defined byAttributable
. More consistent config classes will also be more understandable for new team members and community contributors - A few times I've heard it mentioned that we should move away from the Entry classes to a way of defining the CI config that is easier to read and write. Truthfully, I don't imagine we'd do this any time soon (or ever), but, if we do ever try this, then having consistent config classes and a reliable JSON schema will make that work easier.
Cons
- It's hard! Finding a way to consistently map our CI config classes into a JSON schema is challenging and will require some refactoring of the config classes
- We'll need to update the JSON schema generating classes when we add new validations or new generic entry types to
Gitlab::Config::Entry
What comes next?
If we like this Proof of Concept and want to move forward with it, I'll create a few issues that I'll work on to officially auto-generate the CI schema for the spec:
keyword.
- Update
Attributable
to make attributes easily and reliably accessible - Update validators (and validations) so we can use
AllowNil
consistently - Add a fully tested version of the classes in this MR
- Then, update the CI Header classes for Steps and auto-generate the new schema: #460134
- Create an epic that I (or anyone) can work on in the background until we can auto-generate the entire CI JSON schema