Failure in browser_ui/5_package/conan_repository_spec.rb | 'too many 500 error responses'
Summary
Job: https://gitlab.com/gitlab-org/gitlab-qa-mirror/-/jobs/1419251211
Conan Package manager running on MinIO (Object Storage)
Stack trace
170-- Build files have been written to: /builds/gitlab-qa-sandbox-group/qa-test-2021-07-13-08-02-42-f948362bccd75039/conan-package-project-7eaa7ed4515a1d64/test_package/build/5c141e8fbd3174d18b06584d69645e4da44006b2
171Scanning dependencies of target example
172[ 50%] Building CXX object CMakeFiles/example.dir/example.cpp.o
173[100%] Linking CXX executable bin/example
174[100%] Built target example
175conantest/0.1@mycompany/stable (test package): Running test()
176Hello World Release!
177$ CONAN_LOGIN_USERNAME=ci_user CONAN_PASSWORD=${CI_JOB_TOKEN} conan upload conantest/0.1@mycompany/stable --all --remote=gitlab
178Uploading to remote 'gitlab':
179Uploading conantest/0.1@mycompany/stable to remote 'gitlab'
180Please log in to "gitlab" to perform this action. Execute "conan user" command.
181Remote 'gitlab' username: Got username 'ci_user' from environment
182Please enter a password for "ci_user" account: Got password '******' from environment
183Uploading conanfile.py -> conantest/0.1@mycompany/stable
184Uploading conanmanifest.txt -> conantest/0.1@mycompany/stable
185ERROR:
186Error uploading file: conanfile.py, 'HTTPConnectionPool(host='gitlab-ee-0988240d.test', port=80): Max retries exceeded with url: /api/v4/projects/2/packages/conan/v1/files/conantest/0.1/mycompany/stable/0/export/conanfile.py (Caused by ResponseError('too many 500 error responses'))'
187ERROR:
188Error uploading file: conanmanifest.txt, 'HTTPConnectionPool(host='gitlab-ee-0988240d.test', port=80): Max retries exceeded with url: /api/v4/projects/2/packages/conan/v1/files/conantest/0.1/mycompany/stable/0/export/conanmanifest.txt (Caused by ResponseError('too many 500 error responses'))'
189ERROR: conantest/0.1@mycompany/stable: Upload recipe to 'gitlab' failed: Execute upload again to retry upload the failed files: conanfile.py, conanmanifest.txt. [Remote: gitlab]
190ERROR: Errors uploading some packages
191ERROR: Job failed: exit code 1
Screenshot / HTML page
Possible fixes
From workhorse logs:
{"content_type":"text/plain; charset=utf-8","correlation_id":"01FAFEB64174ZMS35CA1WXG0C2","duration_ms":134,"host":"gitlab-ee-0988240d.test","level":"info","method":"PUT","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/projects/[^/]+/packages/conan/","status":500,"system":"http","time":"2021-07-13T08:04:44Z","ttfb_ms":134,"uri":"/api/v4/projects/3/packages/conan/v1/files/conantest/0.1/mycompany/stable/0/export/conanmanifest.txt","user_agent":"Conan/1.38.0 (Linux 4.19.23-coreos-r1; Python 3.7.5; x86_64) python-requests/2.25.1","written_bytes":22}
{“correlation_id”:“01FAFE9RYD0JS6JP8ARYHZD7HZ”,“error”:“BodyUploader: upload failed: PUT request http://minio-fa27d594.test:9000/packages-bucket/tmp/uploads/1626163438-2114-0006-8940-e570caa35e1b999765ef6bff9ef7b213?X-Amz-Expires=15300\u0026X-Amz-Date=20210713T080358Z\u0026X-Amz-Algorithm=AWS4-HMAC-SHA256\u0026X-Amz-Credential=AKIAIOSFODNN7EXAMPLE%2F20210713%2Fus-east-1%2Fs3%2Faws4_request\u0026X-Amz-SignedHeaders=host\u0026X-Amz-Signature=[FILTERED] returned: 411 Length Required”,“level”:“error”,“method”:“PUT”,“msg”:“”,“time”:“2021-07-13T08:03:58Z”,“uri”:“/api/v4/projects/2/packages/conan/v1/files/conantest/0.1/mycompany/stable/0/export/conanmanifest.txt”}
It seems that when workhorse tried to upload the data to minio, the request got rejected with 411 Length Required