Skip to content

Serverless tab to list functions deployed in Knative

Problem to solve

functions that are defined in the user's repository need a dedicated place where we can show:

  1. functions currently active
  2. function details
  3. function invocation
  4. function scale

Further details

Related mocks https://projects.invisionapp.com/share/2HOT0SKUVM8#/screens/327989769

Proposal

Create "serverless" tab under operations that will show the functions that have been defined in the users' repo.

Possible flow could look like:

After user has deployed knative to their kubernetes cluster:

  1. User adds gitlab-ci.yml "serverless" template that knows how to invoke knative to do all necessary things
  2. User adds serverless.yml file to repo, which contains function definitions (pointer to files in repo, runtimes, etc).
  3. Once serverless.yml is updated, gitlab ci parses the file and passes the available functions to be presented on the "serverless" tab
  4. User can see all defined functions in the "serverless" tab along with general info
  5. User can drill down to a particular function to see its information (endpoint, etc) and some metrics.

We will use existing triggermesh runtimes.

image

Components
Components
  • Breadcrumbs: You can find breadcrumbs on every page
  • Label buttons: Found on project homepages https://gitlab.com/gitlab-org/gitlab-ce
  • Pods: Found on environments view - deployboard
  • Charts: Found under Prometheus metrics

Design Decisions

  • 1. serverless.yml parsing: .gitlab.yml will do a single command tm deploy -f serverless.yml and tm will parse the file and deploy the functions. Each "function" defined in the serverless.yml will become a knative service and the function description will be stored as an annotation of that object. For example:
for service in services:
    print service.metadata.annotation.description
    print service.metadata.name
    print service.spec.build.template

  • 2. running functions: for mvc users must run externally accessible clusters; we will interact with them via k8s api, hence knative. Future iterations to support non-externally accessible clusters.

  • 3. On Frontend/Backend we read all serverless functions directly from Knative

  • 4. The following runtimes will be supported:
    i. node.js
    ii. kaniko

  • 5. functions and multiple clusters: Function stages can be scoped to environments in gitlab-ci.yml as shown in this example. Since clusters can be scoped to environments this would allow it to work with multiple clusters.

  • 6. Viewing all functions from all clusters: for MVC, user will be able to view all functions defined in their serverless.yml file, regardless of which cluster they are executed in (env).

  • 7. In order to delete functions, users will simply remove them from serverless.yml. Changes to this file will result in a commit which will trigger ci and tm to parse the file again and populate the function list.

  • 8. Parsing high number of functions (1000+): we'll release this in beta with the understanding that is not complete.

  • 9. Metrics: invocations (prometheus or knative?): we will gather metrics through prometheus (related conversation).

  • 10. Security: users with write access to the repo will be able to add/delete/edit existing functions. Repo security will the model for function security for mvc.

  • 11. No functions will be stored in the db for mvc

  • 12. Usage ping: we'll start with tracking knative app deploys https://gitlab.com/meltano/looker/issues/292

  • 13. Functions security features (ex. security scanning of runtimes/functions) will be done post-MVC https://gitlab.com/gitlab-org/gitlab-ce/issues/54549

  • 14. serverless.yml validation in GitLab GUI: there is validation both on the tm cli side as well as on the knative side once tm parses and submits the serverless.yml file.

  • 15. Sourcing/function interactivity with GitLab API:

  • 16, 17. Build mechanism is kaniko

  • 18. Knative alpha status, when updated to stable how do we retain data on previously deployed functions/revision? we'll release functions as beta and will make sure to outline knative is still not production ready and there is risk of not retaining this data.

Implementation

Legacy mocks

image

image

What does success look like, and how can we measure that?

(If no way to measure success, link to an issue that will implement a way to measure this)

Links / references

Original issue content for posterity

This can be based on OpenFaaS (or Dispatch) and have:

  1. API endpoints
  2. Images and dependencies (based on container registry)
  3. Function code (based on repo)
  4. Events and subscriptions
  5. Secrets (based on our secret variables)
  6. Identity manager (based on GitLab user)

This will enable you to easily use OpenFaaS and other serverless framweorks.

Extensive notes:

Mocking events

Integration as a task runner for CI

Write functions against GitLab

API gateway Multi-tenancy

Open source

Lambda popular, open source frontrunner changes from openwisk to openfaas

features might be pulled into faas platforms too, but you don't know right know who will win the faas wars

image management would be a good integration

event driver for testing would be nice

lambda, 5 minutes max, takes time to pull dependencies

lambda + fargate maybe?

Dispatch could manage API gateway + lambda + fargate in order to integrate with AWS

suggest fargate to customer? can fargate do DinD?

interface for serverless with Dispatch:

  1. API endpoints
  2. Images and dependencies HAVE
  3. Function code (in repo, right now 1 file, will add drivers) HAVE
  4. Events and subscriptions
  5. Secrets HAVE
  6. Identity manager HAVE

Stateful services are excluded. But to test you might need this.

https://github.com/vmware/dispatch#architecture

Lambda, take all of your dependencies. zip them, 50MB max.

Dispatch, can build a base image via an API.

Serverless tab in GitLab

VMware has an enterprise class registry, Harbor, many chinese contribution

Edited by Daniel Gruesso