Research and document the standards for Gitlab feature configuration files

Topics to cover

Standards and existing examples of config

We should also add this to standard gitlab docs to apply to other config files.

• Document existing standards used in other files (although we don't necessarily have to conform to them). Examples:
  • gitlab.rb (GitLab Omnibus Configuration) - Ruby:
    • Docs: https://docs.gitlab.com/omnibus/settings/configuration.html
    • Example: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template
  • .gitlab/.gitlab-webide.yml (GitLab Web IDE Configuration) - YAML:
    • Docs and Example: https://docs.gitlab.com/ee/user/project/web_ide/#web-ide-configuration-file
  • gitlab-runner/config.toml (GitLab Runner Configuration) - TOML:
    • Docs: https://docs.gitlab.com/runner/configuration/advanced-configuration.html
  • .gitlab-ci.yml (GitLab CI Configuration) - YAML:
    • Docs: https://docs.gitlab.com/ee/ci/yaml/
    • Example: https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml
  • .gitlab/route-map.yml (GitLab Review Apps Route Map) - YAML:
    • Docs and Example: https://docs.gitlab.com/ee/ci/review_apps/#route-maps-example
  • .gitlab/insights.yml (Gitlab Insights Configuration) - YAML:
    • Docs and Example: https://docs.gitlab.com/ee/user/project/insights/#configure-your-insights
  • .gitlab/dashboard/*.yml (GitLab Metrics Configuration) - YAML:
    • Docs: https://docs.gitlab.com/ee/operations/metrics/dashboards/#add-a-new-dashboard-to-your-project
• File Syntax:
  • Two main options currently used are TOML and YAML. YAML seems the obvious choice based on simplicity and wide usage.
• Template file, documentation, and defaults:
  • There will be a template file created by default, which contains documented entries and default values for all supported config options.
• Structure - for example, hierarchical vs flat values in the yaml.
  • Like .gitlab-ci.yml, the top level structure will be a mapping (AKA hash/object).
  • By default, top-level mapping keys will correspond to individual config items - no hierarchy or grouping (although we may add one in the future if it makes sense as the file grows).
  • Structure of individual mapping values will be determined by the needs of the config item - either a scalar value, or arrays, or more mappings.
  • Blank newlines will be left between top-level values
  • All top-level values should be documented in the template.
• Keyword naming convention - for example, dash vs. underscore vs. camel case
  • All lower-case snake_case is widely used as a standard and has precedent in other GitLab config files.
• Value conventions - for example quote strings or not
  • This is usually optional in YAML. In default/example/template files, for simplicity, we will not quote strings unless it is necessary.
• Supported value types
  • All standard YAML types.
• Formatting for documentation comment block sections for documenting auto-created default entries (Assuming we go with the route of auto-creating a config file from the project template, with default values defined.)
  • Since we are not initially going to have any sections or hierarchy, all comments in template files will be immediately above the value, with a standard one- or multi-line YAML comment starting with a hash, preceded by a newline, and limited to 80 character line length:

    previous_option: "value"      
    
    # Some documentation on the_option:
    # docs docs docs
    the_option: "value"

• Versioning / deprecation strategies
  • For .gitlab-ci.yml, deprecated parameters are documented: https://docs.gitlab.com/ee/ci/yaml/#deprecated-parameters
  • There doesn't seem to be any versioning strategy outside of the main gitlab version.

Example Config File

(This is just an example showing formatting, the exact keyword names and values may change in the future)

.gitlab/static_site_editor.yml

# images_upload_path (string): The path to which images will be uploaded.
images_upload_path: /images/

# static_site_generator (string): The tool/framework which is used to generate the static site.
static_site_generator: hugo

# website_base_path (string): The base path from which the website is served.
website_base_path: /

Technical notes on existing config file implementations

• .gitlab-ci.yml:
  • This is a more complex implementation.
  • See the lib/gitlab/config folder and subfolders. These contain some common logic which is used across multiple config file implementations (see below)
• .gitlab/dashboards/*.yml:
  • This leverages some common code under lib/gitlab/config.
  • See usage of ::Gitlab::Config::Loader::Yaml in #load_yaml, in app/services/metrics/dashboard/base_service.rb
• insights.yml:
  • See the insights_feature.rb for logic around reading this config.
  • insights_config in ee/app/models/concerns/insights_feature.rb also uses ::Gitlab::Config::Loader::Yaml.
• web_ide_config_service.rb:
  • This looks like a good example to follow, of a single configuration file, with access wrapped in a service, including validation.
  • https://gitlab.com/gitlab-org/gitlab/blob/23cd4a4dd0b2bc426e4e8a3876bcb90937b6b8c7/app/services/ci/web_ide_config_service.rb#L16-16