Investigate serverless' knative provider

Summary

What about moving our Knative provider to https://serverless.com/framework/docs/providers/knative/?

Improvements

By @mattkasa:

If it works out, I wonder if we could leverage this under the hood to reduce our dependence on tm

By @alexives:

It would let us use consistent tooling with both knative and aws, as well as the other serverless providers. Might reduce confusion over the documentation that talks about both options for serverless. Allows us to leverage the serverless framework ecosystem for testing.

Definition of Done

• have an example project set up, and the setup documented (done in comment)
• collect all developments needed to support our existing features. Write these up under a separate section here in the description. A non-exhaustive list of features to support:
  • openfaas runtimes
  • triggermesh js and ruby runtimes
  • serverless metrics, like pods used, number of requests per function
  • simple setup and installation (assuming that the user already has a cluster)

Risks

• can we still support openfaas runtimes?
• can we still support the triggermesh js and ruby runtimes?
• can we use the GitLab Docker registry?
• can we use GitLab namespaces instead of an sls- prefix?

Summary of findings:

To sum up and answer the DoD and Risks sections in the issue description:

• openfaas runtimes, triggermesh js and ruby runtimes do not seem to be related to this issue
• If we use namespace, that GitLab has access to and not the sls- namespace, then the serverless metrics, like pods used, number of requests per function seem to be working (sort of, the invocations graph not there). The PR towards library to allow for different namespace is pending. If not approved we might consider doing it more properly, with another PR - adding the possibility to configure the namespace from serverless.yml
• Prometheus can be successfully installed with that cluster
• We can use GitLab Docker registry (although the serverless.yml looks bit confusing, maybe another PR towards library or reporting a bug?)
• The setup is probably not the simplest, or quickest, but can be documented well (Important! - the steps to set up above are slightly outdated already, e.g the part about creating account on Docker Hub).
• We've opened an MR to support custom namespaces. The discussion closed our MR, but a nice design came up there that we could develop.

What hasn't been done yet:

• I didn't try scheduling any events
• No tests

Arguments in favour and against moving into this direction

Pro:

• We get knative eventing support out of the box
• Being compatible with Serverless framework allows better support for AWS and other providers too
• No need to maintain gitlabktl

Contra:

• Kubernetes secrets support should be added beside env var support
• We loose our current runtimes support
• Adds nodejs to the tech stack