Skip to content

Update POST terraform state api to check max allowed size

Pam Artiaga requested to merge pam/post-terraform-state-api-introduce-limits into master

What does this MR do and why?

Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/352951

Currently, the Terraform State data sent to the POST /projects/:project_id/terraform/state/:name API has no size limits. A malicious actor can spam the API with very big files, which can cause a denial of service and an increase to storage cost.

We introduced a limit to the file/data accepted by the API in the application_settings table, with the new field called as max_terraform_state_size_bytes. (See !106257 (merged))

In this MR, the POST /projects/:project_id/terraform/state/:name API is updated to check the size of the request against the application_settings.max_terraform_state_size_bytes. The following conditions are followed:

  • If max_terraform_state_size_bytes - there is no limit the size of the request
  • If max_terraform_state_size_bytes is set, AND:
    • request body < max_terraform_state_size_bytes - request is allowed
    • request body == max_terraform_state_size_bytes - request is allowed
    • request body > max_terraform_state_size_bytes - request is not allowed, and a 413 Request Entity Too Large is returned

Documentation/changelog Considerations

As indicated here, this MR is part of several steps to implement the Terraform State file size limit. The default limit is 0/unlimited, which is the same as the previous behavior. Until we allow that limit to be configurable, this change will not result in a noticeable change for the user/client. Documentation or changelog updates are not yet necessary in this MR.

Screenshots or screen recordings

Payload size:

setup_basestatesize

State size has no limit

setup_initialmaxstatesize

HTTP status 200/OK

test_nolimit

State size has a limit

setup_updatedmaxstatesize

Payload size is equal to limit - HTTP status 200/OK

test_sizeequallimit

Payload size is less than limit - HTTP status 200/OK

test_sizelessthanlimit

Payload size is greater than limit - HTTP status 413/Request entity too large

test_sizegreaterthanlimit

How to set up and validate locally

  1. Create a project in your local GitLab instance or use an existing one

  2. Check the max_terraform_state_size_bytes value in the application settings

    Gitlab::CurrentSettings.max_terraform_state_size_bytes

  3. Update the setting according to what you need for your test

    ApplicationSetting.first.update(max_terraform_state_size_bytes: 42)

  4. Test the API

    curl \
"https://gdk.test:3443/api/v4/projects/21/terraform/state/teststate" \
-ki -X POST \
--header "Authorization: Bearer <admin access token>" \
--header "Content-Type: application/json" \
--data "{\"instance\":\"example-instance\",\"serial\":1}"

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Pam Artiaga

Merge request reports