Add a new JWT signed field for uploads
What does the MR do
Essentially two things:
- On body uploads, add the
Gitlab-Workhorse-Multipart-Fields
header and set the appropriate value. This is similar to what it's done for multipart uploads (see https://gitlab.com/gitlab-org/gitlab-workhorse/-/blob/master/internal/upload/saved_file_tracker.go#L43-49) so that existing rails code (multipart.rb
) can process it. - For all uploads, add a new form field with the following name
X.gitlab-workhorse-upload
,X
being the file param name.- The value will be JWT signed and set to the following structure (in short all params relevant to the upload):
{
"upload": {
"md5": "...",
"name": "...",
"path": "...",
"remote_id": "",
"remote_url": "",
"sha1": "...",
"sha256": "...",
"sha512": "...",
"size": "..."
}
}
See #261 (closed)
Verification
I don't want to break the upload functionality
Object storage disabled | Object storage enabled, Direct upload enabled | Object storage enabled, Direct upload disabled | |
---|---|---|---|
NuGet pkg (multipart upload) | |||
Maven pkg (body upload) | |||
CI artifact (multipart upload, 2 files) | |||
User's avatar (multipart upload) | |||
Design in issue (upload through GraphQL) |
In addition, for each case, we checked that on the rails side, in the multipart.rb
file, we were able to read the X.gitlab-workhorse-upload
param and decode it's value. Here is an excerpt of such check:
>>> SIGNED WORKHORSE UPLOAD PARAM: package.gitlab-workhorse-upload
[{"upload"=>{"etag"=>"efbe8641a22b6702c535df497b3f63ab-1", "md5"=>"4906fa0e8eff7af4c9255e658e109ca2", "name"=>"package.nupkg", "path"=>"", "remote_id"=>"1587563640-65264-0011-1604-509a1ba1682d5c149a7ceaa5e332e0ce", "remote_url"=>"http://gitlab.local:9000/packages/tmp/uploads/1587563640-65264-0011-1604-509a1ba1682d5c149a7ceaa5e332e0ce?X-Amz-Expires=15300&X-Amz-Date=20200422T135400Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minio%2F20200422%2Fgdk%2Fs3%2Faws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=d214ebb22b47f9f703e649180ae14478d54b217799a04fe55d91b44162d4ffeb", "sha1"=>"eba8a2548658678fdda13d527032692e575492b3", "sha256"=>"1b6ffc5bb1fc2cf921733395a4bf236d6f8fc0446348e88c1e92ffd2927fc7f8", "sha512"=>"483c9232bcd82647306dd38fd8a4f308edd4a490b707b72a6c8c5c26a2e84f689368bdf52a0e94fb3a7914abc32c438ab855b3d6a871a2dcea7ec3fe6d82434d", "size"=>"3866"}, "iss"=>"gitlab-workhorse"}, {"alg"=>"HS256", "typ"=>"JWT"}]
Lastly, for body uploads, we verified that the JWT header Gitlab-Workhorse-Multipart-Fields
is properly set to something similar to (the important part is that we get rewritten_fields
keys that include a file
key):
[{"rewritten_fields"=>{"file"=>""}, "iss"=>"gitlab-workhorse"}, {"alg"=>"HS256", "typ"=>"JWT"}]
Edited by David Fernandez