Serverless GitLab Events

GitLab Serverless Events

TL;DR;

  1. Work on GitLab Event Source that is deployable from within GitLab (a managed application)
  2. Installing the event source from GitLab deploys it along with a secret / key to a cluster.
  3. Installing the event source configures GitLab Webhook and sets the event source sink address.
  4. Installing the event source connects the webhook secret / private key with the cluster secret / public key.
  5. Event source controller transform a webhook in a cloud event payload and propagates it within the cluster.
  6. A user configures events: in serverless.yaml and defines relationships between functions and events.
  7. gitlabktl creates all necessary resources when deploying functions to make sure that events are being routed through a valid broker to a valid function.

Description

In GitLab Serverless we currently do not support triggering functions when some kind of an event occurs. We also do not support configuring / defining relationships between events and functions.

The Event Source

In order to propagate events in a Knative cluster we need GitLab Event Source.

We do have two GitLab Event Sources that we might work on more:

  1. Triggermesh gitlabsource ➡ https://gitlab.com/triggermesh/gitlabsource
  2. Knative-source GitLab Event Source ➡ https://github.com/knative/eventing-sources/pull/382

We need to work on one of them, and make them installable from the GitLab UI.

Integration

Once someone clicks "Install" button, next to the even source app, we are going to:

  1. Create a secret token or public/private key pair and deploy one of these (token or a public key) to a cluster.
  2. Create a webhook in GitLab and connect it with a GitLab Event Source Controller / Manager endpoint in a cluster.
  3. Create a webhook token or transform webhook into a signed JWT
  4. Create a resource of GitLabSource and later (during deployment) Trigger / Broker to route events to a function / set of functions

Configuration

We need to make it possible for users to define relationship between events and functions. We do have a few issues mentioning events:

  1. https://gitlab.com/gitlab-org/gitlab-ce/issues/61709
  2. https://gitlab.com/gitlab-org/gitlab-ce/issues/61171

Current proposal for configuration:

functions:
  echo:
    handler: MyEcho.my_function
    source: echo-rb/
    runtime: runtimes/ruby
    description: "echo function using Ruby runtime"
    trigger:
      - new_merge_request_created

events:
  new_merge_request_created:
    cloud_event:
      type: knative.gitlab.merge_request.created
    # or, both ways correct
    gitlab:
      webhook: new_merge_request_created # or however we call that in a webhook payload

There are some technical details that I decided to skip when creating this issue, but perhaps this is a good start to foster discussion about it and eventually moving forward.

/cc @danielgruesso @DylanGriffith

Edited by Grzegorz Bizon