Lambda based serverless runner
Description
Inspired by the description (below) of an existing GitLab CI user
create a fully serverless shared runner using AWS step functions to facilitate the stages of the pipeline with lambdas facilitating logic where necessary. The runner would give you your choice of lambda environments in which to execute your stages (e.g. Node 6.10.3, Java 1.8, golang, etc. based on what kinds of lambda environments AWS offers). Each stage is kicked off in its own lambda while S3 is used for caching and artifacts
problem, benefits, and/or goals
integration with least-privileged AWS IAM roles at build/deploy time. k8s runner is not sufficient because kube2iam is a clunky fit for this use case
all this can be done via webhooks & GitLab API rather than implementing a runner, but we lose artifact storage. artifacts can be retrieved via the API, but not stored. Thus we can't accommodate 100% of runner functionality through a webhook
Proposal
Arrange discovery call with requestor and CI team confirm use case and discuss collaboration opportunities.
Links to related issues and merge requests / references
https://gitlab.com/gitlab-org/gitlab-ce/issues/36175#note_56235678 #2995
CNCF serverless whitepaper https://github.com/cncf/wg-serverless/blob/master/whitepapers/serverless-overview/cncf_serverless_whitepaper_v1.0.pdf