Some thoughts on becoming a platform-as-a-service (PaaS).
What's a PaaS?
The basic components of a PaaS are:
- Ephemeral compute
We do not want to make money selling compute. From our strategy on pricing:
We charge for making people more effective and will charge per user, per application, or per instance. We do include free minutes with our subscriptions and trials to make it easier for users to get started. As we look towards more deployment related functionality on .com it's tempting to offer compute and charge a percent on top of for example Google Cloud Platform (GCP). We don't want to charge an ambiguous margin on top of another provider since this limits user choice and is not transparent. So we will always let you BYOK (bring your own Kubernetes) and never lock you into our infrastructure to charge you an opaque premium on those costs.
In addition, leveraging k8s should allow us to enable auto scaling by default.
We're not doing this now. We offer 2000 pipeline minutes, which is about 33 hours of compute. Heroku gives away 750+ hours, which is enough to run an app for a month. They used to give away a lot more. It's very expensive. Idling helps stretch hours into months. Without idling your review app is gone in a day.
We need to have 0 container autoscale that detects traffic, ingress to sponge to delay requests.
First iteration could be automatic shutdown after 1 hour or manual. You'd need to manually wake up your app. This is a significant burden and really affects adoption, but might be a reasonable position to take as being a PaaS is not our primary business and we likely can't afford to give away hundreds of hours of compute.
Log management is still complicated in large deployments, and Kubernetes doesn't magically make this better. There are lots of third parties that solve this. Perhaps AppSignal?
Heroku offers a large library of addons that are really easy to install, and offer centralized billing (which should not be underestimated). We would likely leverage Docker images instead, but this will have limitations. It'll get a big part of the way though, maybe 60-70%, so enough to cover a lot of real situations. We might need to offer other integrations. I hate to even suggest creating another marketplace, but marketplaces do seem to be all the rage these days. :)
With our integration with Kubernetes, we have already come really close to being a PaaS. We added Herokuish in Auto Deploy so we can even build a bunch of different languages/frameworks automatically. But Auto Deploy is still lacking in a LOT of ways. We only recently let you even scale your app. We still don't make it easy to customize your deployment, pick databases, pass variables, etc. But work in that area, and other onboarding areas, could get us just a stone's through away from being a full PaaS.
- I2P Onboarding/adoption: #32638
If we do, we'd like everything enabled by default (CI, Review apps, CD, Metrics). It's OK to have a button to spin up the review app instead of a link, to save compute.
changed the descriptionToggle commit list
Some additional thoughts:
- We should support all Heroku languages (plus let you specify your own buildpack or your own Dockerfile)
- Everyone gets a cluster (dedicated cluster or shared cluster?) => shared up to CPU/IO limit: #32731
- I really like Deis workflow. I wonder how to leverage it, if at all. There's some redundancy in their architecture. e.g. they have a registry, and we'd likely be better off using ours. But is it possible to other parts of their open source product to address some of the needs above? e.g. I assume they've got a logs solution.
@markpundsack very good issue.
I assume you know it already but most people seem to use http://www.fluentd.org/ to improve logging on Kubernetes. I agree that this doesn't magically make it better. We already allow you to attach an Elasticsearch (ES) cluster to GitLab for doing code search. Should we use FluentD to send the logs to the same cluster and to offer search? Search is hard to do fast, need a lot of tuning of the ES cluster to search big logs in seconds.
We can also do something crazy and do federation for logs. Just like Prometheus assumes you deploy a metrics server per application we can do the same for logs. And then have GitLab search all those log servers to get a company wide picture. I think all of this is very far in the future.
If they don't do compute they just have memory. Just need ephemeral compute first. That is just autoscaling. Let's do that.