Corpus Management - Corpus upload corrupt zip download
Follow-up to Corpus Management - Corpus Upload - Upload Corpus
Bug
ADDITIONAL DOCS
Controller endpoint:
https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/generic_packages.rb#L56
Package Creation Service:
Working cURL file upload
Request via command line
curl --verbose --header "PRIVATE-TOKEN: <ACCESS_TOKEN>" \
--upload-file corpus.zip \
"http://172.16.123.1:3000/api/v4/projects/fernando%2Fcorpus-management-demo/packages/generic/corpus1_package/0.0.1/bin.zip?status=default&select=package_file"
Response
* Trying 172.16.123.1...
* TCP_NODELAY set
* Connected to 172.16.123.1 (172.16.123.1) port 3000 (#0)
> PUT /api/v4/projects/fernando%2Fcorpus-management-demo/packages/generic/corpus1_package/0.0.1/bin.zip?status=default&select=package_file HTTP/1.1
> Host: 172.16.123.1:3000
> User-Agent: curl/7.64.1
> Accept: */*
> PRIVATE-TOKEN: glpat-4fzxxM7Nog4dSCsZ8rq-
> Content-Length: 2494
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Cache-Control: max-age=0, private, must-revalidate
< Content-Type: application/json
< Etag: W/"4e003e91a7c4c6896da55bf65bda6d90"
< Vary: Origin
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-Request-Id: 01FPDS51JVQTM6E26K6AP8XX99
< X-Runtime: 0.272144
< Date: Wed, 08 Dec 2021 19:43:31 GMT
< Content-Length: 600
<
* Connection #0 to host 172.16.123.1 left intact
{"id":159,"package_id":98,"created_at":"2021-12-08T19:43:31.489Z","updated_at":"2021-12-08T19:43:31.489Z","size":2494,"file_store":1,"file_md5":null,"file_sha1":null,"file_name":"bin.zip","file":{"url":"/67/06/670671cd97404156226e507973f2ab8330d3022ca96e0c93bdbdb320c41adcaf/packages/98/files/159/bin.zip"},"file_sha256":"31de279ba4fe1619740a16cdaa78ef1df851b08af1c40758d6c36becb779887b","verification_retry_at":null,"verified_at":null,"verification_failure":null,"verification_retry_count":null,"verification_checksum":null,"verification_state":0,"verification_started_at":null,"new_file_path":null}* Closing connection 0
File download after uploading via curl (response headers)
URL: http://gdk.test:3000/fernando/corpus-management-demo/-/package_files/159/download
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: private, no-store
Content-Disposition: attachment; filename="bin.zip"; filename*=UTF-8''bin.zip
Content-Length: 2494
Content-Transfer-Encoding: binary
Content-Type: application/zip
Last-Modified: Wed, 08 Dec 2021 19:43:31 GMT
Permissions-Policy: interest-cohort=()
Pragma: no-cache
Referrer-Policy: strict-origin-when-cross-origin
X-Accel-Buffering: no
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: DENY
X-Permitted-Cross-Domain-Policies: none
X-Request-Id: 01FPE2WXQSB9MS4DZ6WY5MAKWT
X-Runtime: 0.400853
X-Ua-Compatible: IE=edge
X-Xss-Protection: 1; mode=block
Date: Wed, 08 Dec 2021 22:33:51 GMT
Apollo Client Resolver wrapping AXIOS request (Returns 200, but downloading the artifact is corrupt)
Endpoint Docs: https://docs.gitlab.com/ee/user/packages/generic_packages/index.html#publish-a-package-file
Request Headers
PUT /api/v4/projects/fernando%2Fcorpus-management-demo/packages/generic/corpus/0/corpus.zip?status=default&select=package_file HTTP/1.1
Host: 172.16.123.1:3000
Connection: keep-alive
Content-Length: 2683
Accept: application/json, text/plain, */*
X-CSRF-Token: b/F994TJLrev7yfTY0Ewtg1eXIun6Ejuk8VC7gunoiCmwESPSlqHba5ZPvp8CX0tbsusw2L4OXmOyJ8pS8rhww==
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryAcGJbopkkLGRFGjy
Origin: http://172.16.123.1:3000
Referer: http://172.16.123.1:3000/fernando/corpus-management-demo/-/security/configuration/corpus_management
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Cookie: perf_bar_enabled=true; experimentation_subject_id=eyJfcmFpbHMiOnsibWVzc2FnZSI6IkltRTNPREl5WVdNeExUSmhaVFF0TkRCa1ppMDRaREk1TFdOaU5qTmtZemMzWTJOa01DST0iLCJleHAiOm51bGwsInB1ciI6ImNvb2tpZS5leHBlcmltZW50YXRpb25fc3ViamVjdF9pZCJ9fQ%3D%3D--02f4fafe3dba49c01595b359f2ead35e25a0860c; event_filter=all; sidebar_collapsed=false; hide_auto_devops_implicitly_enabled_banner_26=false; hide_no_ssh_message=false; BetterErrors-2.9.1-CSRF-Token=0e78a8ab-7c01-4c84-a748-21f36a1a0719; collapsed_gutter=true; visitor_id=0ba4d06a-9b97-4da6-b51e-d580aa9d6092; known_sign_in=VnVTb1RBZlZHdi9WdHRPa0xDaXovN2NNUmxTK09VSEZxUmcybnU3T3ZWOG5sQy92T3hmdDdWbTZUanBTeGxuZ0J0WXBMU3JVbk9WK1lxOFlsNlR0MDdqcjF2YlhVbis1M1VnNEEyTDl6MWlYWUFrVUJqSnlyclF1VTU4c1grTGYtLUplWU9vOXpxeXYvSlVodGZZVmZYb3c9PQ%3D%3D--c6e21fe2c9647991a1b98da2da12563d2b8a6f8d; _gitlab_session_64231db9a2b9dd5ad1690f772392f6945f6988fb7971c15bdbb424962c079c2e=d22dc213bb8e6e186374b020436b1c59
Response headers
HTTP/1.1 200 OK
Access-Control-Allow-Methods: GET, HEAD, POST, PUT, PATCH, DELETE, OPTIONS
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Link, X-Total, X-Total-Pages, X-Per-Page, X-Page, X-Next-Page, X-Prev-Page, X-Gitlab-Blob-Id, X-Gitlab-Commit-Id, X-Gitlab-Content-Sha256, X-Gitlab-Encoding, X-Gitlab-File-Name, X-Gitlab-File-Path, X-Gitlab-Last-Commit-Id, X-Gitlab-Ref, X-Gitlab-Size
Access-Control-Max-Age: 7200
Cache-Control: max-age=0, private, must-revalidate
Content-Type: application/json
Etag: W/"f780e831c506a7c425610a28f6828d25"
Vary: Origin
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: 01FPE2G6Y73T3CN4J1CQTNWY0E
X-Runtime: 0.629082
Date: Wed, 08 Dec 2021 22:26:55 GMT
Content-Length: 608
Request ("Copy as CURL from chrome debugger")
curl 'http://172.16.123.1:3000/api/v4/projects/fernando%2Fcorpus-management-demo/packages/generic/corpus/0/corpus.zip?status=default&select=package_file' \
-X 'PUT' \
-H 'Connection: keep-alive' \
-H 'Accept: application/json, text/plain, */*' \
-H 'X-CSRF-Token: b/F994TJLrev7yfTY0Ewtg1eXIun6Ejuk8VC7gunoiCmwESPSlqHba5ZPvp8CX0tbsusw2L4OXmOyJ8pS8rhww==' \
-H 'X-Requested-With: XMLHttpRequest' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36' \
-H 'Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryAcGJbopkkLGRFGjy' \
-H 'Origin: http://172.16.123.1:3000' \
-H 'Referer: http://172.16.123.1:3000/fernando/corpus-management-demo/-/security/configuration/corpus_management' \
-H 'Accept-Language: en-US,en;q=0.9' \
-H 'Cookie: perf_bar_enabled=true; experimentation_subject_id=eyJfcmFpbHMiOnsibWVzc2FnZSI6IkltRTNPREl5WVdNeExUSmhaVFF0TkRCa1ppMDRaREk1TFdOaU5qTmtZemMzWTJOa01DST0iLCJleHAiOm51bGwsInB1ciI6ImNvb2tpZS5leHBlcmltZW50YXRpb25fc3ViamVjdF9pZCJ9fQ%3D%3D--02f4fafe3dba49c01595b359f2ead35e25a0860c; event_filter=all; sidebar_collapsed=false; hide_auto_devops_implicitly_enabled_banner_26=false; hide_no_ssh_message=false; BetterErrors-2.9.1-CSRF-Token=0e78a8ab-7c01-4c84-a748-21f36a1a0719; collapsed_gutter=true; visitor_id=0ba4d06a-9b97-4da6-b51e-d580aa9d6092; known_sign_in=VnVTb1RBZlZHdi9WdHRPa0xDaXovN2NNUmxTK09VSEZxUmcybnU3T3ZWOG5sQy92T3hmdDdWbTZUanBTeGxuZ0J0WXBMU3JVbk9WK1lxOFlsNlR0MDdqcjF2YlhVbis1M1VnNEEyTDl6MWlYWUFrVUJqSnlyclF1VTU4c1grTGYtLUplWU9vOXpxeXYvSlVodGZZVmZYb3c9PQ%3D%3D--c6e21fe2c9647991a1b98da2da12563d2b8a6f8d; _gitlab_session_64231db9a2b9dd5ad1690f772392f6945f6988fb7971c15bdbb424962c079c2e=d22dc213bb8e6e186374b020436b1c59' \
--data-raw $'------WebKitFormBoundaryAcGJbopkkLGRFGjy\r\nContent-Disposition: form-data; name="file"; filename="corpus.zip"\r\nContent-Type: application/zip\r\n\r\n\r\n------WebKitFormBoundaryAcGJbopkkLGRFGjy--\r\n' \
--compressed \
--insecure
Copy as fetch from chrome debugger
fetch("http://172.16.123.1:3000/api/v4/projects/fernando%2Fcorpus-management-demo/packages/generic/corpus/0/corpus.zip?status=default&select=package_file", {
"headers": {
"accept": "application/json, text/plain, */*",
"accept-language": "en-US,en;q=0.9",
"content-type": "multipart/form-data; boundary=----WebKitFormBoundaryAcGJbopkkLGRFGjy",
"x-csrf-token": "b/F994TJLrev7yfTY0Ewtg1eXIun6Ejuk8VC7gunoiCmwESPSlqHba5ZPvp8CX0tbsusw2L4OXmOyJ8pS8rhww==",
"x-requested-with": "XMLHttpRequest"
},
"referrer": "http://172.16.123.1:3000/fernando/corpus-management-demo/-/security/configuration/corpus_management",
"referrerPolicy": "strict-origin-when-cross-origin",
"body": "------WebKitFormBoundaryAcGJbopkkLGRFGjy\r\nContent-Disposition: form-data; name=\"file\"; filename=\"corpus.zip\"\r\nContent-Type: application/zip\r\n\r\n\r\n------WebKitFormBoundaryAcGJbopkkLGRFGjy--\r\n",
"method": "PUT",
"mode": "cors",
"credentials": "include"
});
File download after uploading via Apollo Client resolver wrapping axios request
URL: http://172.16.123.1:3000/fernando/corpus-management-demo/-/package_files/161/download
Response headers
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: private, no-store
Content-Disposition: attachment; filename="whywontitwork.zip"; filename*=UTF-8''whywontitwork.zip
Content-Length: 2683
Content-Transfer-Encoding: binary
Content-Type: application/zip
Last-Modified: Wed, 08 Dec 2021 19:56:51 GMT
Permissions-Policy: interest-cohort=()
Pragma: no-cache
Referrer-Policy: strict-origin-when-cross-origin
X-Accel-Buffering: no
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: DENY
X-Permitted-Cross-Domain-Policies: none
X-Request-Id: 01FPE31SAA2BY8339QBMM5VV8A
X-Runtime: 0.349617
X-Ua-Compatible: IE=edge
X-Xss-Protection: 1; mode=block
Date: Wed, 08 Dec 2021 22:36:30 GMT
Edited by -