Skip to content

Draft: "PoC: Autogenerate JSON schema for CI Header"

What does this MR do and why?

This MR is a Proof of Concept to demonstrate that we can auto-generate a JSON schema from our CI Header and CI Entry classes. It generates a schema for the CI header spec keyword because it is a single, achievable task and is needed for the upcoming CI Steps work.

Issue: #463328 (closed)

How does it work?

This MR defines a new Rake task gitlab:generate_ci_schema that calls Gitlab::Config::Entry::Json::Schema.for, which is the entry point to the new JSON schema generating logic. The Gitlab::Config::Entry::Json::Properties classes generate JSON schema representations of the different CI config keywords. The classes in Gitlab::Config::Entry::Json::Validators generate JSON schema representations of the different validations in Gitlab::Config::Entry::Validators.

Pros

I see 3 things we get out of this:

  1. It's the first step towards auto-generating a JSON schema for the entire CI config. Writing the schema manually adds an additional step to any work on the config classes, and is prone to errors. For example, in this MR you can see that the auto-generated schema for the spec: keyword is more complete than the existing manually written one
  2. As a byproduct of the work, we'll get config classes that are more consistent. The work in this MR revealed an inconsistent use of the AllowNil validation and the need for a more consistent way of referencing the attributes defined by Attributable. More consistent config classes will also be more understandable for new team members and community contributors
  3. A few times I've heard it mentioned that we should move away from the Entry classes to a way of defining the CI config that is easier to read and write. Truthfully, I don't imagine we'd do this any time soon (or ever), but, if we do ever try this, then having consistent config classes and a reliable JSON schema will make that work easier.

Cons

  1. It's hard! Finding a way to consistently map our CI config classes into a JSON schema is challenging and will require some refactoring of the config classes
  2. We'll need to update the JSON schema generating classes when we add new validations or new generic entry types to Gitlab::Config::Entry

What comes next?

If we like this Proof of Concept and want to move forward with it, I'll create a few issues that I'll work on to officially auto-generate the CI schema for the spec: keyword.

  1. Update Attributable to make attributes easily and reliably accessible
  2. Update validators (and validations) so we can use AllowNil consistently
  3. Add a fully tested version of the classes in this MR
  4. Then, update the CI Header classes for Steps and auto-generate the new schema: #460134
  5. Create an epic that I (or anyone) can work on in the background until we can auto-generate the entire CI JSON schema
Edited by Avielle Wolfe

Merge request reports

Loading