Skip to content

Automation in the DevSecOps domain

Mental model

DevSecOps domain is:

  • domain objects, e.g.: a Deployment, an Artifact, a Release, an Issue, a Comment, a CI pipeline.
  • domain subjects, e.g.: User, Service Account (bot)
  • verbs, specific to the domain object they are applicable to, e.g.: to deploy a Release into an Environment.
  • events. When a change happens, an event is emitted, e.g.: a new User is created; a Release is deployed to an Environment; an issue is created.
  • time. Some things (should) happen when time passes, e.g. scheduled deployments, timeouts, due dates.
  • a way to encode business logic that operates on domain objects, reacts to events - AutoFlow.

Automation

Automation is key to the progress of civilization. Grand words but that's what it deserves. Think about some of the big inventions:

  • Steam engine
  • Electricity
  • Trains
  • AI

These allowed to increase efficiency, automate processes, reduce costs. Not a bit, but by orders of magnitude. Hence, these things are key to unlock the next steps of the civilization's development.

All of the above is to say automation is obviously a big deal for any company: increase efficiency of business processes by automating them to reduce costs.

GitLab operates in the DevSecOps domain, but we don't have a comprehensive, systematic way to automate things in the domain.

Automating deployments?

If automation in the DevSecOps domain is the pie, deployments are a tiny part of it. If it's an addressable market, deployments are a tiny part of it.

Why provide a solution for a small part of the problem if we can tackle the whole thing? As a comprehensive DevSecOps platform, we are already competing not only for the slice, not only for the pie, but for all the pies (the whole domain). It'd be inconsistent with our own messaging to tackle only deployments and not the whole problem space of "automation in the DevSecOps domain".

AutoFlow is a solution for the whole problem. It allows workflows to react to events and/or time, and to manipulate domain objects by making API calls. Please see these docs for a lot more info on AutoFlow. This issue focuses on the bigger picture, not AutoFlow.

Holistic high level view

  • Formalized domain entities.
  • Events for when things happen/change.
  • Events and/or timers trigger workflows.
  • Some events and actions are recorded. E.g. Deployment (id=123) was started for Release (id=234) in an Environment (name=prod) at 1:27pm by User (name=mike).
  • All the reporting needs can be satisfied by looking at:
    • Current state of domain object(s).
    • Records of actions taken and transitions that happened as the result.

We have lots of concrete use cases coming from the customers. The above would provide a high level framework / the structure / the scaffolding / a mental model for how to think about the problem space, how to look for solutions, how to "structure" features. It's a conceptual and technical foundation.

Role of CI functions

CI functions can be a way to execute resource-heavy (CPU/RAM) tasks - such as running binaries in containers.

Related:

Role of Duo

  • Perhaps AutoFlow can be used internally in Duo Agent Platform to orchestrate things?
  • Invoking Duo can be a step in a workflow.
  • Duo can author workflows.
  • ...

See Relation of GitLab AutoFlow to GitLab Duo Agent Platform.

Next steps

Re-design AutoFlow on top of existing infrastructure

AutoFlow on existing stack.

Formalize domain objects

We need to better formalize domain objects and verbs. Some examples here. Things need to be improved.

UI to make workflows interactive

Some workflows need to be interactive, require user input. GitLab needs to provide an API to create simple forms to input data, make choices (e.g. a form "Do you approve this and that?" with "Yes/no" buttons).

Perhaps Duo Agent Platform could use this API to create UI dynamically too.

....

Add more here.

Notes

Sales corner

As a sales person, imagine how you would be able to reframe any conversation:

  • Capabilities: "Competitor X only solves this and that. We solve the whole problem of automation". Just like today we talk about comprehensive platform vs point solutions.
  • Capabilities: "GitHub Actions only solves a narrow set of problems that can be solved with binaries/scripts running in a container. Can you automate rollback of a botched deployment when metrics breach a threshold, declare an incident, ping SREs in Slack, send an email? All while leaving audit traces of the whole timeline. You can with GitLab AutoFlow". See more examples in the docs.
  • Pricing: "It's included in GitLab Free, comes with per-second usage-based billing, like Runner. You literally spend your Runner minutes with it. Easy"
  • Cost: "Your workflows can run for months, not minutes or hours, while you pay only for the actual execution time, you don't pay for any blocking/waiting time. Example: for { poll_api_and_maybe_do_things(); sleep(1h); } - you don't pay for the sleep(). Yes, you could use scheduled pipelines for this, it's just a silly example to illustrate the point. Waiting for an event or sleeping is free, that's the point. Scheduled pipelines do not carry state across runs while a running workflow keeps all the variables as they were before the sleep so it continues where its left as if no sleep happened".
  • Reliability: "It doesn't matter if you/us/cloud restarts all the servers at an unfortunate moment - once the servers are up, workflows just keep working as if nothing happened. How? Durable execution".
  • Composability: "You/us/third parties can build composable libraries that anyone can reuse. It's a platform where anyone can provide building blocks that run on top".
  • Latency: "GitHub Actions needs to download and spin up a container before doing anything. GitLab AutoFlow is running your business logic in under a second. E.g. a typical use case - takes under 100 milliseconds to ensure labels on an issue are correct when you mess them up. It puts them back faster than you can remove them! Yes, give it a try!".
Edited by Mikhail Mazurskiy