Persisted Serverless Functions
Description
Currently whenever we create a serverless function in a Knative cluster we need to poll the cluster in order to show functions to a user in Operations -> Serverless tab.
This behavior works, but there are some problems with it:
- It is terribly slow.
- It depends on Knative APIs, there is no anti-corruption layer between GitLab and Knative
- It does not persist data about functions in GitLab itself, we can't use this information
- It might be a performance problem when a lot of people want to view the Serverless tab
- Because we don't store information about functions we can't generate internal metrics to track Serverless usage
Proposal
Alternative solution is to design an internal Serverless API and let the cluster notify GitLab about its state. This way we could persist information about functions in the database, and the cluster mechanisms would ensure that functions are getting created, updated and destroyed on the GitLab side.
Similar mechanism exists already in the Distribution Registry, and we find it very useful (we are not using it yet, though) https://docs.docker.com/registry/notifications/
There are a few ways to implement that, Kubernetes operator is one way, this solution is described in https://gitlab.com/gitlab-org/gitlab-ee/issues/10729 /cc @marshall007.
Another solution is to use KubernetesEventSource with a ... Serverless Function that would notify GitLab about new serverless functions being created / updated / destroyed.