GitLab managed terraform state

Ideation phase

Google Doc to brainstorm our ideas around this feature can be found here -> https://docs.google.com/document/d/11VTQFrgLKOGQjLeNW-Op9uPIUAHFxsH9y9DjP8oYZyY (GitLab internal for the time being). Once we came up with the complete proposal / MVP, we will extract the content to this issue/issues.

Problem to solve

As a DevOps engineer, in order to quickly get started with a Terraform project, I would like to use a GitLab managed Terraform state.

If GitLab CI is used to manage infrastructure using Terraform, it would be convenient if the Terraform state could be stored in GitLab itself instead of having to set up a traditional remote backend (eg. on Amazon S3).

Opportunity canvas

Intended users

Further details

To be a viable alternative to existing options, solution would need to provide:

  • Locking
  • State history (forever is fine)
  • Encryption at rest

We would like to support multiple state files in a single git project.

Proposal

As an MVC, we could wrap the S3 backend or the gcp backend.

Our bucket naming should support multiple terraform projects / states in a single project. We could always use the full GitLab path.

This should become an officially documented GitLab backend.

You should be able to run it from inside of GitLab as we need the current path (group/project/path/to/file) for the bucket name.

terraform {
  backend "gitlab" {
    endpoint = "https://www.gitlab.com"
  }
}

this actually translates into functionality of

terraform {
  backend "s3" {
    bucket = "<gitlab config>"
    key    = "group/project/path/to/file"
    region = "<gitlab config>"
  }
}

The instance level AWS account credentials are stored in the gitlab config too.

MVC steps:

  1. get it working on gitlab.com
  2. get it working for on-prem

Permissions and Security

  • Every project should be able to read only its own terraform state

  • We should provide a way to get the actual or previous versions of the terraform state (for recovery purposes). Do we really need this?

  • Can a project developer access the terraform state? Or only project maintainers?

  • We should get the security teams approval on this too. What AWS account should we use? Who will own the account?

Documentation

Availability & Testing

What does success look like, and how can we measure that?

What is the type of buyer?

Links / references

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.

Edited by 🤖 GitLab Bot 🤖