Technical Discovery: Standard/JSON Format for SAST Config UI

Problem to Solve

We need to decide a standard JSON format for our SAST config UI to render required fields in the config UI.

Details

We need to decide a standard that will be extensible to other types of scanners like secrets, and DAST. This will be the form that the parser uses when reading CI files to render the configuration UI.

There is a community-supported schema we might be able to use http://json.schemastore.org/gitlab-ci.

Field support:

  • scan type
  • version?
  • variable name
  • variable value
  • UI type (text field, dropdown, etc)
  • available variable values (in the case of an enum/dropdown)
  • variable description
  • variable doc link
  • variable default value
  • variable required?

There are a few other projects in flight that may also be looking for CI standards

GraphQL query

A GraphQL query approximately like the following will return the schema:

{
  ciConfig(scanType: "sast") {
    configuration
  }
}

To be defined in another issue (cc @ssarka).

Saikat: GraphQL work is addressed in this issue.

Form submission/GraphQL Mutation

Mutation payload will be something like:

{
  "SECURE_ANALYZERS_PREFIX": "foo",
  "SAST_ANALYZER_IMAGE_TAG": "bar"
  ...
}

Should either return new MR URL for frontend to redirect to (if using GraphQL), or a 302 response (if using normal Rails form) (cc @ssarka)

Saikat: this is already addressed in this issue.

JSON format

Based on the discussion below, the following is a possible JSON format. Possible values of type are string, text (i.e., multi-line strings), boolean, number, and list. Config UI components may be rendered using this JSON. We may change the JSON based on requirement in the future.

[
  {
    "scan_types": ["sast", "secret_detection", "dependency_scanning"],
    "analyzer": "all",
    "doc_link": "...",
    "variable_name" : "SECURE_ANALYZERS_PREFIX",
    "current_value" : null,
    "default_value" : "registry.gitlab.com/gitlab-org/security-products/analyzers",
    "short_name" : "Image prefix",
    "description" : "Overrides the name of the Docker registry providing the default images",
    "available_values": [],
    "type": "string"
  },
  {
    "scan_types": ["sast"],
    "analyzer": "all",
    "doc_link": "...",
    "variable_name" : "SAST_ANALYZER_IMAGE_TAG",
    "current_value" : null,
    "default_value" : "2",
    "short_name" : "Image tag",
    "description" : "Overrides the Docker tag of the default images",
    "available_values": [],
    "type": "string"
  },
  {
    "scan_types": ["sast"],
    "analyzer": "kubesec",
    "doc_link": "...",
    "variable_name" : "SCAN_KUBERNETES_MANIFESTS",
    "current_value" : null,
    "default_value" : "false",
    "short_name" : "",
    "description" : "",
    "type": "boolean"
  }
]
Written in (approximate) TypeScript notation
type SchemaItemGeneric<T> = {
  scan_types: "sast" | "secret_detection" | "dependency_scanning";
  analyzer: "all" | "kubesec" | "bandit" | "spotbugs" | "flawfinder" | "etc...";
  doc_link: string;
  variable_name : string;
  current_value : T;
  default_value : T;
  short_name : string;
  description : string;
  available_values: T[];
  type: T;
};

type SchemaItem = SchemaItemGeneric<string | text | number | boolean>

type Schema = Array<SchemaItem>;
Changes since the last version
  • scan_type: string -> scan_types: string[]
    • It may be that the frontend wants to indicate whether a particular field affects other scanners; if so, it would want to say which ones.
  • inline configuration field to top level object
    • The previous configuration array was a holdover from the initial iteration.
  • ui_type -> type
  • Add text as possible type, for multi-line strings. See #216635 (comment 366698167).

Unanswered questions

  • Can a variable be assigned to more than one analyzer, but not all? cc @ssarka

Saikat: Except for general settings variables(e.g. SECURE_ANALYZERS_PREFIX), each variable is associated with only one analyzer. This is the source of information.

Comments

  • In future, we might want to make the short_name/description fields translatable. Something for a later iteration (%13.3+).
  • current_value won't be used in %13.2, since it can only be populated with parsed CI value(s) (%13.3+).
Edited by Saikat Sarkar