Include has_length parameter for conan packages to avoid 501 not implemented
-
Please check this box if this contribution uses AI-generated content (including content generated by GitLab Duo features) as outlined in the GitLab DCO & CLA. As a benefit of being a GitLab Community Contributor, you receive complimentary access to GitLab Duo.
Conan Object Storage Compatibility Fix
What does this MR do and why?
This MR addresses an issue with Conan package uploads where object storage (particularly AWS S3) was returning 501 NOT IMPLEMENTED errors due to unsupported Transfer-Encoding: chunked requests.
Changes Made
-
Object Storage Compatibility: Modified Conan authorize endpoints to use
has_length: falseparameter - Consistent Implementation: Applied the fix to both Conan V1 and V2 API endpoints
Why This Change?
-
AWS S3 Limitation: AWS S3 doesn't support
PutObjectrequests withTransfer-Encoding: chunked -
Multipart Upload Solution: Setting
has_length: falseforces Workhorse to use multipart uploads which are fully supported - Consistency: Brings Conan packages in line with other package types (NuGet, RubyGems, PyPI, Helm, LFS) that already use this pattern
- Production Impact: Prevents upload failures for GitLab instances using object storage
Technical Details
Files Modified
-
lib/api/concerns/packages/conan/v1_endpoints.rb:- Updated recipe file authorize endpoint:
/export/:file_name/authorize - Updated package file authorize endpoint:
/package/:conan_package_reference/:package_revision/:file_name/authorize
- Updated recipe file authorize endpoint:
-
lib/api/conan/v2/project_packages.rb:- Updated V2 API authorize endpoints for both recipe and package files
Code Changes
Before:
authorize_workhorse!(subject: project, maximum_size: project.actual_limits.conan_max_file_size)
After:
authorize_workhorse!(subject: project, has_length: false, maximum_size: project.actual_limits.conan_max_file_size)
References
- Issue #550925 - Conan package upload fails with object storage
-
LFS Storage Controller - Reference implementation using
has_length: false
Related Issues
Related to #550925 (closed)
Edited by 🤖 GitLab Bot 🤖