Upload options of export API don't work at all - Impossible to schedule external backup of projects
Summary
Upload options of export API don't work at all, making it impossible to schedule external backup of projects
Please see: https://docs.gitlab.com/ee/api/project_import_export.html#schedule-an-export
Steps to reproduce
You can make a simple php script and put it on a webserver, on the document root folder, called import.php
:
<?php
/* We put max error level to ensure no php errors during testing */
error_reporting(E_ALL);
/* PUT data comes in on the stdin stream */
$putdata = fopen("php://input", "r");
/* Open a file for writing */
$fp = fopen("myputfile.ext", "w");
/* Read the data 1 KB at a time
and write to the file */
while ($data = fread($putdata, 1024))
fwrite($fp, $data);
/* Close the streams */
fclose($fp);
fclose($putdata);
echo "Upload is done.";
Make a simple curl request on it (replace MYPRIVATETOKEN, PROJECTID and MYDOMAIN by the one you use for the test):
curl --request POST --header "PRIVATE-TOKEN: MYPRIVATETOKEN " https://gitlab.com/api/v4/projects/PROJECTID/export --data "upload[http_method]=PUT" --data-urlencode "upload[url]=http://MYDOMAIN/import.php"
You will get a response: {"message":"202 Accepted"}
And a few minutes later an email (on the email of the user of the private token):
Project {hidden} couldn't be exported.
The errors we encountered were:
• private method `open' called for #<CarrierWave::Storage::Fog::File:{hidden}>
—
No file is uploaded on the webserver.
If you upload directly with curl it works:
curl -T "/path/to/myfile.ext" http://MYDOMAIN/glkjr78.php
So Gitlab cloud upload option for export seems to not work at all.
For reference, i tested initally with an S3 bucket and get the same error. You can see this issue, i closed because it was less generic: https://gitlab.com/gitlab-org/gitlab-ce/issues/50970
What is the expected correct behavior?
It is espected to have the file uploaded on the server and not having an error email.
Output of checks
This bug happens on GitLab.com
Possible fixes
Fix the upload code on Gitlab side, it is just a curl to do from Gitlab to the server, so i think it will be pretty simple/fast to fix.
I think it should be classified pretty urgent, as automate backup on external server for now is broken, and there is no other way with the cloud version to export projects to external storage (appart from manually...). As it don't work it can break with some enterprise compliance which need to be able to make external backups too, so i think it is pretty important.
Thanks.