Skip to content

Support the JWT workhorse parameter for file uploads

Summary

Follow up step of gitlab-workhorse#261 (closed).

In gitlab-workhorse!490 (merged), we added a JWT signed parameter that contains all the upload parameters. This parameter is available for multipart and body uploads.

Currently, on the rails side, we have the following:

  • Multipart uploads: in multipart.rb, we read the workhorse parameters in the Rack parameter and build a UploadedFile with UploadedFile.from_params.
  • Body uploads: there is no middleware at all. Each endpoint has to use UploadedFile.from_params and read the Rails params.

This is not ideal for several reasons:

  • Multipart uploads: rake params should not be trusted.
  • Body uploads: rails params should not be trusted and it's a developer burden to use UploadedFile.from_params.

Improvements

Create a middleware that replaces Gitlab::Middleware::Multipart and does what Gitlab::Middleware::Multipart is doing currently but:

  • Read the uploaded file params from a JWT param set by workhorse.
  • From these, build an UploadedFile and set it on the query params.
  • Support multipart and body uploads.

Risks

This issue will replace an existing feature that is used on all file uploads.

I'm considering a medium risk so we definitely need a safety net here. Here is the plan:

  • Have both middleware mounted so the both process upload requests.
  • Use a feature flag to decide which will actually process the upload request and which one will just do a no op.

Involved components

  • A new rails middleware
  • Gitlab::Middleware::Multipart

Optional: Intended side effects

  • This issue will unlock a cleanup:
    • Business code will not need to use UploadedFile.from_params. It's already done by the middleware and thus, those parts can read the UploadedFile that is present in the rails params directly.

Optional: Missing test coverage

⚠ For the specs, use a feature spec with an inline workhorse.

Cases for the spec:

  • Nuget upload (multipart)
  • Maven upload (body)
  • Graphql upload / add design to an issue (multipart)
  • CI artifacts upload (multipart with 2 files)
  • User avatar upload (multipart with a nested file, eg. parameter name is something like foo[bar])
  • Project Import (multipart where the API is already using the UploadedFile from the middleware)
  • Group import (multipart upload)
  • git LFS Upload (body)
Edited by David Fernandez