Backend: Testing components in CI

Problem to solve

As the author of a component I want to be able to test changes in CI and when tagging new releases.

Proposal

In order to test components in CI we use the .gitlab-ci.yml:

include:
  - component: $CI_PROJECT_PATH/my-component@$CI_COMMIT_SHA
    with:
      foo: bar

dummy-job:
  script: echo

The above configuration allows to test a component defined in the same repository using the CI_COMMIT_SHA version. This tests that the component can be consumed correctly and there are no syntax errors. It however doesn't test a lot of the details. In order to test a component we could do:

• shellcheck.
• check presence of README.md for publishing.
• lint the manifest/publishing YAML file for publishing process.
• check for recommendations/warnings.
• component specific tests (e.g. check that it adds the expected jobs)

In order to run all these tests GitLab could provide a component that abstracts that away making it easier for users to write their own CI configs for component projects.

Test script

include:
  - component: $CI_PROJECT_PATH/my-component@$CI_COMMIT_SHA
    with:
      foo: bar
  - component: gitlab-org/component-testing@1.0
    with:
      paths: [my-component] # or could read paths from a manifest/publishing YAML file in the repo.
      publish: true

my-component-test:
  script: ./my-component-test.sh # check that "my-component" does what's expected.

Imagine that gitlab-org/component-testing@1.0 (default component) adds several jobs to the pipeline by using more specific components in the project:

• Use gitlab-org/component-testing/lint@1.0 to add a job that lints the component path.
• Use gitlab-org/component-testing/best-practices@1.0 to add jobs that check for best practices and gives recommendations.
• Use gitlab-org/component-testing/publish-checks@1.0 only if: $CI_COMMIT_TAG to check if the components project is good to be released/published (e.g. README present, manifest/publishing file present, version doesn't exist already, etc.)
• Could also provide jobs that release and publish the component automatically after all checks pass. (e.g. in the .post stage).