Connection reset by peer when uploading large file on S3

Hello,

I got an error when I try to push file larger than 2GB on S3 with LFS:

Fatal error: Server error: https://pr-sti-git.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 error: failed to push some refs to 'git@pr-sti-git01.tec.in.phm.education.gouv.fr:educnat.sic.crt-hebergement/test_lfs_s3.git'

Here is the full log with GIT_TRACE=1 GIT_CURL_VERBOSE=1

GIT_TRACE=1 GIT_CURL_VERBOSE=1 git push origin master trace: built-in: git 'push' 'origin' 'master' trace: run_command: 'ssh' 'git@pr-sti-git01.tec.in.phm.education.gouv.fr' 'git-receive-pack '''educnat.sic.crt-hebergement/test_lfs_s3.git'''' trace: run_command: '.git/hooks/pre-push' 'origin' 'git@pr-sti-git01.tec.in.phm.education.gouv.fr:educnat.sic.crt-hebergement/test_lfs_s3.git' trace: exec: 'git-lfs' 'pre-push' 'origin' 'git@pr-sti-git01.tec.in.phm.education.gouv.fr:educnat.sic.crt-hebergement/test_lfs_s3.git' trace: run_command: 'git-lfs' 'pre-push' 'origin' 'git@pr-sti-git01.tec.in.phm.education.gouv.fr:educnat.sic.crt-hebergement/test_lfs_s3.git' 11:28:40.145281 trace git-lfs: exec: git 'version' 11:28:40.156691 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=cat' '-c' 'filter.lfs.clean=cat' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' 'HEAD' '--symbolic-full-name' 'HEAD' 11:28:40.175321 trace git-lfs: exec: git 'config' '-l' 11:28:40.184117 trace git-lfs: pre-push: refs/heads/master 8ac72364b16f803b8ae6b85320b11bc0a69c4575 refs/heads/master ba6a39d27e944f7b6fe525ebd87eaf2334d4363a 11:28:40.487593 trace git-lfs: run_command: ssh -- git@pr-sti-git01.tec.in.phm.education.gouv.fr git-lfs-authenticate educnat.sic.crt-hebergement/test_lfs_s3.git upload 11:28:40.795766 trace git-lfs: HTTP: POST https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/locks/verify

POST /educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/locks/verify HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Accept: application/vnd.git-lfs+json; charset=utf-8 Authorization: Basic * * * * * Content-Length: 36 Content-Type: application/vnd.git-lfs+json; charset=utf-8 User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

{"ref":{"name":"refs/heads/master"}}11:28:40.837103 trace git-lfs: HTTP: 200

< HTTP/1.1 200 OK < Content-Length: 23 < Cache-Control: max-age=0, private, must-revalidate < Connection: keep-alive < Content-Type: application/vnd.git-lfs+json; charset=utf-8 < Date: Wed, 12 Jun 2019 09:28:40 GMT < Etag: W/"928ac95441b0331342f5e9d1e1150f67" < Page-Title: GitLab < Referrer-Policy: strict-origin-when-cross-origin < Server: nginx < Strict-Transport-Security: max-age=31536000 < X-Content-Type-Options: nosniff < X-Frame-Options: DENY < X-Request-Id: BhNqwOsNnCa < X-Runtime: 0.023837 < X-Ua-Compatible: IE=edge < X-Xss-Protection: 1; mode=block < 11:28:40.837298 trace git-lfs: HTTP: {"ours":[],"theirs":[]} {"ours":[],"theirs":[]}Locking support detected on remote "origin". Consider enabling it with: $ git config lfs.https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs.locksverify true 11:28:40.837390 trace git-lfs: tq: running as batched queue, batch size of 100 11:28:40.837504 trace git-lfs: run_command: git rev-list --stdin --objects --not --remotes=origin -- 11:28:40.874595 trace git-lfs: tq: sending batch of size 1 11:28:40.874741 trace git-lfs: ssh cache: git@pr-sti-git01.tec.in.phm.education.gouv.fr git-lfs-authenticate educnat.sic.crt-hebergement/test_lfs_s3.git upload 11:28:40.874889 trace git-lfs: api: batch 1 files 11:28:40.875027 trace git-lfs: HTTP: POST https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch

POST /educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Accept: application/vnd.git-lfs+json; charset=utf-8 Authorization: Basic * * * * * Content-Length: 162 Content-Type: application/vnd.git-lfs+json; charset=utf-8 User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

{"operation":"upload","objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000}],"ref":{"name":"refs/heads/master"}}11:28:40.904866 trace git-lfs: HTTP: 200

< HTTP/1.1 200 OK < Content-Length: 745 < Cache-Control: max-age=0, private, must-revalidate < Connection: keep-alive < Content-Type: application/json; charset=utf-8 < Date: Wed, 12 Jun 2019 09:28:40 GMT < Etag: W/"776a530dfdb4905f7af631948bc77cf4" < Page-Title: GitLab < Referrer-Policy: strict-origin-when-cross-origin < Server: nginx < Strict-Transport-Security: max-age=31536000 < X-Content-Type-Options: nosniff < X-Frame-Options: DENY < X-Request-Id: ZCfSfPuf6S5 < X-Runtime: 0.028158 < X-Ua-Compatible: IE=edge < X-Xss-Protection: 1; mode=block < 11:28:40.904952 trace git-lfs: HTTP: {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF0 {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF011:28:40.904994 trace git-lfs: HTTP: T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]} T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]}11:28:40.905092 trace git-lfs: tq: starting transfer adapter "basic" 11:28:40.905107 trace git-lfs: xfer: adapter "basic" Begin() with 8 workers 11:28:40.905124 trace git-lfs: xfer: adapter "basic" started 11:28:40.905169 trace git-lfs: xfer: adapter "basic" worker 0 starting 11:28:40.905194 trace git-lfs: xfer: adapter "basic" worker 0 processing job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:28:40.905201 trace git-lfs: xfer: adapter "basic" worker 1 starting 11:28:40.905214 trace git-lfs: xfer: adapter "basic" worker 1 waiting for Auth 11:28:40.905228 trace git-lfs: xfer: adapter "basic" worker 2 starting 11:28:40.905257 trace git-lfs: xfer: adapter "basic" worker 2 waiting for Auth 11:28:40.905266 trace git-lfs: xfer: adapter "basic" worker 3 starting 11:28:40.905282 trace git-lfs: xfer: adapter "basic" worker 3 waiting for Auth 11:28:40.905296 trace git-lfs: xfer: adapter "basic" worker 6 starting 11:28:40.905304 trace git-lfs: xfer: adapter "basic" worker 6 waiting for Auth 11:28:40.905308 trace git-lfs: xfer: adapter "basic" worker 5 starting 11:28:40.905321 trace git-lfs: xfer: adapter "basic" worker 5 waiting for Auth 11:28:40.905328 trace git-lfs: xfer: adapter "basic" worker 7 starting 11:28:40.905339 trace git-lfs: xfer: adapter "basic" worker 7 waiting for Auth 11:28:40.905185 trace git-lfs: xfer: adapter "basic" worker 4 starting 11:28:40.905349 trace git-lfs: xfer: adapter "basic" worker 4 waiting for Auth 11:28:40.906232 trace git-lfs: HTTP: PUT https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000

PUT /educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Authorization: Basic * * * * * Content-Length: 5120000000 Content-Type: application/octet-stream User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

11:28:40.906425 trace git-lfs: xfer: adapter "basic" worker 3 auth signal received 11:28:40.906436 trace git-lfs: xfer: adapter "basic" worker 1 auth signal received 11:28:40.906447 trace git-lfs: xfer: adapter "basic" worker 2 auth signal received 11:28:40.906452 trace git-lfs: xfer: adapter "basic" worker 5 auth signal received 11:28:40.906458 trace git-lfs: xfer: adapter "basic" worker 6 auth signal received 11:28:40.906465 trace git-lfs: xfer: adapter "basic" worker 7 auth signal received 11:28:40.906472 trace git-lfs: xfer: adapter "basic" worker 4 auth signal received 11:29:30.894980 trace git-lfs: HTTP: 500B | 38 MB/s

< HTTP/1.1 500 Internal Server Error < Content-Length: 22 < Connection: keep-alive < Content-Type: text/plain; charset=utf-8 < Date: Wed, 12 Jun 2019 09:29:30 GMT < Server: nginx < X-Content-Type-Options: nosniff < 11:29:30.896176 trace git-lfs: xfer: adapter "basic" worker 0 finished job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:29:30.896340 trace git-lfs: tq: retrying object c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e: Fatal error: Server error: https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 11:29:30.896393 trace git-lfs: tq: enqueue retry gitlab-workhorse#1 (closed) for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" (size: 5120000000) 11:29:30.896456 trace git-lfs: tq: sending batch of size 1 11:29:30.896708 trace git-lfs: ssh cache: git@pr-sti-git01.tec.in.phm.education.gouv.fr git-lfs-authenticate educnat.sic.crt-hebergement/test_lfs_s3.git upload 11:29:30.896781 trace git-lfs: api: batch 1 files 11:29:30.897062 trace git-lfs: HTTP: POST https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch

POST /educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Accept: application/vnd.git-lfs+json; charset=utf-8 Authorization: Basic * * * * * Content-Length: 162 Content-Type: application/vnd.git-lfs+json; charset=utf-8 User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

{"operation":"upload","objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000}],"ref":{"name":"refs/heads/master"}}11:29:30.938723 trace git-lfs: HTTP: 200

< HTTP/1.1 200 OK < Content-Length: 745 < Cache-Control: max-age=0, private, must-revalidate < Connection: keep-alive < Content-Type: application/json; charset=utf-8 < Date: Wed, 12 Jun 2019 09:29:30 GMT < Etag: W/"776a530dfdb4905f7af631948bc77cf4" < Page-Title: GitLab < Referrer-Policy: strict-origin-when-cross-origin < Server: nginx < Strict-Transport-Security: max-age=31536000 < X-Content-Type-Options: nosniff < X-Frame-Options: DENY < X-Request-Id: 9Pm0xKRGzy9 < X-Runtime: 0.034780 < X-Ua-Compatible: IE=edge < X-Xss-Protection: 1; mode=block < 11:29:30.938810 trace git-lfs: HTTP: {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF0 {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF011:29:30.938858 trace git-lfs: HTTP: T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]} T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]}11:29:30.938908 trace git-lfs: xfer: adapter "basic" End() 11:29:30.938917 trace git-lfs: xfer: adapter "basic" worker 3 stopping 11:29:30.938924 trace git-lfs: xfer: adapter "basic" worker 5 stopping 11:29:30.938929 trace git-lfs: xfer: adapter "basic" worker 2 stopping 11:29:30.938921 trace git-lfs: xfer: adapter "basic" worker 6 stopping 11:29:30.938932 trace git-lfs: xfer: adapter "basic" worker 0 stopping 11:29:30.938936 trace git-lfs: xfer: adapter "basic" worker 1 stopping 11:29:30.938939 trace git-lfs: xfer: adapter "basic" worker 4 stopping 11:29:30.938941 trace git-lfs: xfer: adapter "basic" worker 7 stopping 11:29:30.938965 trace git-lfs: xfer: adapter "basic" stopped 11:29:30.938986 trace git-lfs: tq: starting transfer adapter "basic" 11:29:30.938994 trace git-lfs: xfer: adapter "basic" Begin() with 8 workers 11:29:30.939001 trace git-lfs: xfer: adapter "basic" started 11:29:30.939062 trace git-lfs: xfer: adapter "basic" worker 2 starting 11:29:30.939079 trace git-lfs: xfer: adapter "basic" worker 2 waiting for Auth 11:29:30.939086 trace git-lfs: xfer: adapter "basic" worker 0 starting 11:29:30.939094 trace git-lfs: xfer: adapter "basic" worker 1 starting 11:29:30.939100 trace git-lfs: xfer: adapter "basic" worker 1 waiting for Auth 11:29:30.939104 trace git-lfs: xfer: adapter "basic" worker 4 starting 11:29:30.939110 trace git-lfs: xfer: adapter "basic" worker 4 waiting for Auth 11:29:30.939113 trace git-lfs: xfer: adapter "basic" worker 3 starting 11:29:30.939119 trace git-lfs: xfer: adapter "basic" worker 3 waiting for Auth 11:29:30.939123 trace git-lfs: xfer: adapter "basic" worker 5 starting 11:29:30.939139 trace git-lfs: xfer: adapter "basic" worker 5 waiting for Auth 11:29:30.939143 trace git-lfs: xfer: adapter "basic" worker 6 starting 11:29:30.939150 trace git-lfs: xfer: adapter "basic" worker 6 waiting for Auth 11:29:30.939153 trace git-lfs: xfer: adapter "basic" worker 7 starting 11:29:30.939161 trace git-lfs: xfer: adapter "basic" worker 7 waiting for Auth 11:29:30.939540 trace git-lfs: xfer: adapter "basic" worker 0 processing job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:29:30.951233 trace git-lfs: HTTP: PUT https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000

PUT /educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Authorization: Basic * * * * * Content-Length: 5120000000 Content-Type: application/octet-stream User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

11:29:30.951523 trace git-lfs: xfer: adapter "basic" worker 3 auth signal received 11:29:30.951540 trace git-lfs: xfer: adapter "basic" worker 2 auth signal received 11:29:30.951548 trace git-lfs: xfer: adapter "basic" worker 1 auth signal received 11:29:30.951555 trace git-lfs: xfer: adapter "basic" worker 4 auth signal received 11:29:30.951561 trace git-lfs: xfer: adapter "basic" worker 6 auth signal received 11:29:30.951568 trace git-lfs: xfer: adapter "basic" worker 5 auth signal received 11:29:30.951574 trace git-lfs: xfer: adapter "basic" worker 7 auth signal received 11:30:21.069673 trace git-lfs: HTTP: 500B | 37 MB/s

< HTTP/1.1 500 Internal Server Error < Content-Length: 22 < Connection: keep-alive < Content-Type: text/plain; charset=utf-8 < Date: Wed, 12 Jun 2019 09:30:21 GMT < Server: nginx < X-Content-Type-Options: nosniff < 11:30:21.071352 trace git-lfs: xfer: adapter "basic" worker 0 finished job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:30:21.071374 trace git-lfs: tq: retrying object c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e: Fatal error: Server error: https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 11:30:21.071432 trace git-lfs: tq: enqueue retry gitlab-workhorse#2 (moved) for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" (size: 5120000000) 11:30:21.071486 trace git-lfs: tq: sending batch of size 1 11:30:21.071620 trace git-lfs: ssh cache: git@pr-sti-git01.tec.in.phm.education.gouv.fr git-lfs-authenticate educnat.sic.crt-hebergement/test_lfs_s3.git upload 11:30:21.071682 trace git-lfs: api: batch 1 files 11:30:21.071790 trace git-lfs: HTTP: POST https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch

POST /educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Accept: application/vnd.git-lfs+json; charset=utf-8 Authorization: Basic * * * * * Content-Length: 162 Content-Type: application/vnd.git-lfs+json; charset=utf-8 User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

{"operation":"upload","objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000}],"ref":{"name":"refs/heads/master"}}11:30:21.108265 trace git-lfs: HTTP: 200

< HTTP/1.1 200 OK < Content-Length: 745 < Cache-Control: max-age=0, private, must-revalidate < Connection: keep-alive < Content-Type: application/json; charset=utf-8 < Date: Wed, 12 Jun 2019 09:30:21 GMT < Etag: W/"776a530dfdb4905f7af631948bc77cf4" < Page-Title: GitLab < Referrer-Policy: strict-origin-when-cross-origin < Server: nginx < Strict-Transport-Security: max-age=31536000 < X-Content-Type-Options: nosniff < X-Frame-Options: DENY < X-Request-Id: I1dbteSYcH9 < X-Runtime: 0.029798 < X-Ua-Compatible: IE=edge < X-Xss-Protection: 1; mode=block < 11:30:21.108364 trace git-lfs: HTTP: {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF0 {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF011:30:21.108420 trace git-lfs: HTTP: T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]} T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]}11:30:21.108527 trace git-lfs: xfer: adapter "basic" End() 11:30:21.108548 trace git-lfs: xfer: adapter "basic" worker 3 stopping 11:30:21.108557 trace git-lfs: xfer: adapter "basic" worker 4 stopping 11:30:21.108557 trace git-lfs: xfer: adapter "basic" worker 6 stopping 11:30:21.108562 trace git-lfs: xfer: adapter "basic" worker 1 stopping 11:30:21.108564 trace git-lfs: xfer: adapter "basic" worker 2 stopping 11:30:21.108571 trace git-lfs: xfer: adapter "basic" worker 7 stopping 11:30:21.108573 trace git-lfs: xfer: adapter "basic" worker 0 stopping 11:30:21.108575 trace git-lfs: xfer: adapter "basic" worker 5 stopping 11:30:21.108583 trace git-lfs: xfer: adapter "basic" stopped 11:30:21.108611 trace git-lfs: tq: starting transfer adapter "basic" 11:30:21.108622 trace git-lfs: xfer: adapter "basic" Begin() with 8 workers 11:30:21.108631 trace git-lfs: xfer: adapter "basic" started 11:30:21.108695 trace git-lfs: xfer: adapter "basic" worker 2 starting 11:30:21.108713 trace git-lfs: xfer: adapter "basic" worker 2 waiting for Auth 11:30:21.108737 trace git-lfs: xfer: adapter "basic" worker 0 starting 11:30:21.108746 trace git-lfs: xfer: adapter "basic" worker 1 starting 11:30:21.108755 trace git-lfs: xfer: adapter "basic" worker 1 waiting for Auth 11:30:21.108766 trace git-lfs: xfer: adapter "basic" worker 4 starting 11:30:21.108774 trace git-lfs: xfer: adapter "basic" worker 4 waiting for Auth 11:30:21.108778 trace git-lfs: xfer: adapter "basic" worker 3 starting 11:30:21.108784 trace git-lfs: xfer: adapter "basic" worker 3 waiting for Auth 11:30:21.108788 trace git-lfs: xfer: adapter "basic" worker 5 starting 11:30:21.108795 trace git-lfs: xfer: adapter "basic" worker 5 waiting for Auth 11:30:21.108799 trace git-lfs: xfer: adapter "basic" worker 6 starting 11:30:21.108806 trace git-lfs: xfer: adapter "basic" worker 6 waiting for Auth 11:30:21.108809 trace git-lfs: xfer: adapter "basic" worker 7 starting 11:30:21.108815 trace git-lfs: xfer: adapter "basic" worker 7 waiting for Auth 11:30:21.112463 trace git-lfs: xfer: adapter "basic" worker 0 processing job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:30:21.114089 trace git-lfs: HTTP: PUT https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000

PUT /educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Authorization: Basic * * * * * Content-Length: 5120000000 Content-Type: application/octet-stream User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

11:30:21.114251 trace git-lfs: xfer: adapter "basic" worker 4 auth signal received 11:30:21.114264 trace git-lfs: xfer: adapter "basic" worker 2 auth signal received 11:30:21.114271 trace git-lfs: xfer: adapter "basic" worker 1 auth signal received 11:30:21.114278 trace git-lfs: xfer: adapter "basic" worker 5 auth signal received 11:30:21.114282 trace git-lfs: xfer: adapter "basic" worker 3 auth signal received 11:30:21.114286 trace git-lfs: xfer: adapter "basic" worker 6 auth signal received 11:30:21.114290 trace git-lfs: xfer: adapter "basic" worker 7 auth signal received 11:35:30.951042 trace git-lfs: HTTP: 500B | 12 MB/s

< HTTP/1.1 500 Internal Server Error < Content-Length: 22 < Connection: keep-alive < Content-Type: text/plain; charset=utf-8 < Date: Wed, 12 Jun 2019 09:35:30 GMT < Server: nginx < X-Content-Type-Options: nosniff < 11:35:30.952073 trace git-lfs: xfer: adapter "basic" worker 0 finished job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:35:30.952096 trace git-lfs: tq: retrying object c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e: Fatal error: Server error: https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 11:35:30.952122 trace git-lfs: tq: enqueue retry gitlab-workhorse#3 (closed) for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" (size: 5120000000) 11:35:30.952146 trace git-lfs: tq: sending batch of size 1 11:35:30.952271 trace git-lfs: ssh cache: git@pr-sti-git01.tec.in.phm.education.gouv.fr git-lfs-authenticate educnat.sic.crt-hebergement/test_lfs_s3.git upload 11:35:30.952331 trace git-lfs: api: batch 1 files 11:35:30.952437 trace git-lfs: HTTP: POST https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch

POST /educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Accept: application/vnd.git-lfs+json; charset=utf-8 Authorization: Basic * * * * * Content-Length: 162 Content-Type: application/vnd.git-lfs+json; charset=utf-8 User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

{"operation":"upload","objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000}],"ref":{"name":"refs/heads/master"}}11:35:30.991177 trace git-lfs: HTTP: 200

< HTTP/1.1 200 OK < Content-Length: 745 < Cache-Control: max-age=0, private, must-revalidate < Connection: keep-alive < Content-Type: application/json; charset=utf-8 < Date: Wed, 12 Jun 2019 09:35:30 GMT < Etag: W/"776a530dfdb4905f7af631948bc77cf4" < Page-Title: GitLab < Referrer-Policy: strict-origin-when-cross-origin < Server: nginx < Strict-Transport-Security: max-age=31536000 < X-Content-Type-Options: nosniff < X-Frame-Options: DENY < X-Request-Id: NSQiHwc2jT5 < X-Runtime: 0.031296 < X-Ua-Compatible: IE=edge < X-Xss-Protection: 1; mode=block < 11:35:30.991269 trace git-lfs: HTTP: {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF0 {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF011:35:30.991316 trace git-lfs: HTTP: T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]} T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]}11:35:30.991368 trace git-lfs: xfer: adapter "basic" End() 11:35:30.991376 trace git-lfs: xfer: adapter "basic" worker 4 stopping 11:35:30.991383 trace git-lfs: xfer: adapter "basic" worker 0 stopping 11:35:30.991387 trace git-lfs: xfer: adapter "basic" worker 7 stopping 11:35:30.991391 trace git-lfs: xfer: adapter "basic" worker 6 stopping 11:35:30.991394 trace git-lfs: xfer: adapter "basic" worker 3 stopping 11:35:30.991398 trace git-lfs: xfer: adapter "basic" worker 5 stopping 11:35:30.991402 trace git-lfs: xfer: adapter "basic" worker 1 stopping 11:35:30.991406 trace git-lfs: xfer: adapter "basic" worker 2 stopping 11:35:30.991411 trace git-lfs: xfer: adapter "basic" stopped 11:35:30.991428 trace git-lfs: tq: starting transfer adapter "basic" 11:35:30.991436 trace git-lfs: xfer: adapter "basic" Begin() with 8 workers 11:35:30.991443 trace git-lfs: xfer: adapter "basic" started 11:35:30.991530 trace git-lfs: xfer: adapter "basic" worker 2 starting 11:35:30.991549 trace git-lfs: xfer: adapter "basic" worker 2 waiting for Auth 11:35:30.991557 trace git-lfs: xfer: adapter "basic" worker 0 starting 11:35:30.991565 trace git-lfs: xfer: adapter "basic" worker 1 starting 11:35:30.991572 trace git-lfs: xfer: adapter "basic" worker 1 waiting for Auth 11:35:30.991577 trace git-lfs: xfer: adapter "basic" worker 4 starting 11:35:30.991584 trace git-lfs: xfer: adapter "basic" worker 4 waiting for Auth 11:35:30.991588 trace git-lfs: xfer: adapter "basic" worker 3 starting 11:35:30.991594 trace git-lfs: xfer: adapter "basic" worker 3 waiting for Auth 11:35:30.991598 trace git-lfs: xfer: adapter "basic" worker 5 starting 11:35:30.991604 trace git-lfs: xfer: adapter "basic" worker 5 waiting for Auth 11:35:30.991608 trace git-lfs: xfer: adapter "basic" worker 6 starting 11:35:30.991615 trace git-lfs: xfer: adapter "basic" worker 6 waiting for Auth 11:35:30.991619 trace git-lfs: xfer: adapter "basic" worker 7 starting 11:35:30.991625 trace git-lfs: xfer: adapter "basic" worker 7 waiting for Auth 11:35:30.995028 trace git-lfs: xfer: adapter "basic" worker 0 processing job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:35:30.996528 trace git-lfs: HTTP: PUT https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000

PUT /educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Authorization: Basic * * * * * Content-Length: 5120000000 Content-Type: application/octet-stream User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

11:35:30.996776 trace git-lfs: xfer: adapter "basic" worker 4 auth signal received 11:35:30.996787 trace git-lfs: xfer: adapter "basic" worker 2 auth signal received 11:35:30.996791 trace git-lfs: xfer: adapter "basic" worker 1 auth signal received 11:35:30.996795 trace git-lfs: xfer: adapter "basic" worker 5 auth signal received 11:35:30.996799 trace git-lfs: xfer: adapter "basic" worker 3 auth signal received 11:35:30.996803 trace git-lfs: xfer: adapter "basic" worker 6 auth signal received 11:35:30.996871 trace git-lfs: xfer: adapter "basic" worker 7 auth signal received 11:36:17.395007 trace git-lfs: HTTP: 500B | 14 MB/s

< HTTP/1.1 500 Internal Server Error < Content-Length: 22 < Connection: keep-alive < Content-Type: text/plain; charset=utf-8 < Date: Wed, 12 Jun 2019 09:36:17 GMT < Server: nginx < X-Content-Type-Options: nosniff < 11:36:17.396006 trace git-lfs: xfer: adapter "basic" worker 0 finished job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:36:17.396078 trace git-lfs: tq: retrying object c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e: Fatal error: Server error: https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 11:36:17.396146 trace git-lfs: tq: enqueue retry gitlab-workhorse#4 (closed) for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" (size: 5120000000) 11:36:17.396247 trace git-lfs: tq: sending batch of size 1 11:36:17.396462 trace git-lfs: ssh cache: git@pr-sti-git01.tec.in.phm.education.gouv.fr git-lfs-authenticate educnat.sic.crt-hebergement/test_lfs_s3.git upload 11:36:17.396557 trace git-lfs: api: batch 1 files 11:36:17.396691 trace git-lfs: HTTP: POST https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch

POST /educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Accept: application/vnd.git-lfs+json; charset=utf-8 Authorization: Basic * * * * * Content-Length: 162 Content-Type: application/vnd.git-lfs+json; charset=utf-8 User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

{"operation":"upload","objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000}],"ref":{"name":"refs/heads/master"}}11:36:17.436111 trace git-lfs: HTTP: 200

< HTTP/1.1 200 OK < Content-Length: 745 < Cache-Control: max-age=0, private, must-revalidate < Connection: keep-alive < Content-Type: application/json; charset=utf-8 < Date: Wed, 12 Jun 2019 09:36:17 GMT < Etag: W/"776a530dfdb4905f7af631948bc77cf4" < Page-Title: GitLab < Referrer-Policy: strict-origin-when-cross-origin < Server: nginx < Strict-Transport-Security: max-age=31536000 < X-Content-Type-Options: nosniff < X-Frame-Options: DENY < X-Request-Id: bR2fXLzVgx9 < X-Runtime: 0.033396 < X-Ua-Compatible: IE=edge < X-Xss-Protection: 1; mode=block < 11:36:17.436200 trace git-lfs: HTTP: {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF0 {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF011:36:17.436238 trace git-lfs: HTTP: T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]} T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]}11:36:17.436289 trace git-lfs: xfer: adapter "basic" End() 11:36:17.436298 trace git-lfs: xfer: adapter "basic" worker 4 stopping 11:36:17.436305 trace git-lfs: xfer: adapter "basic" worker 0 stopping 11:36:17.436309 trace git-lfs: xfer: adapter "basic" worker 7 stopping 11:36:17.436313 trace git-lfs: xfer: adapter "basic" worker 6 stopping 11:36:17.436317 trace git-lfs: xfer: adapter "basic" worker 3 stopping 11:36:17.436321 trace git-lfs: xfer: adapter "basic" worker 5 stopping 11:36:17.436326 trace git-lfs: xfer: adapter "basic" worker 1 stopping 11:36:17.436330 trace git-lfs: xfer: adapter "basic" worker 2 stopping 11:36:17.436334 trace git-lfs: xfer: adapter "basic" stopped 11:36:17.436355 trace git-lfs: tq: starting transfer adapter "basic" 11:36:17.436363 trace git-lfs: xfer: adapter "basic" Begin() with 8 workers 11:36:17.436372 trace git-lfs: xfer: adapter "basic" started 11:36:17.436438 trace git-lfs: xfer: adapter "basic" worker 2 starting 11:36:17.436456 trace git-lfs: xfer: adapter "basic" worker 2 waiting for Auth 11:36:17.436464 trace git-lfs: xfer: adapter "basic" worker 0 starting 11:36:17.436472 trace git-lfs: xfer: adapter "basic" worker 1 starting 11:36:17.436501 trace git-lfs: xfer: adapter "basic" worker 1 waiting for Auth 11:36:17.436506 trace git-lfs: xfer: adapter "basic" worker 4 starting 11:36:17.436513 trace git-lfs: xfer: adapter "basic" worker 4 waiting for Auth 11:36:17.436517 trace git-lfs: xfer: adapter "basic" worker 3 starting 11:36:17.436524 trace git-lfs: xfer: adapter "basic" worker 3 waiting for Auth 11:36:17.436529 trace git-lfs: xfer: adapter "basic" worker 5 starting 11:36:17.436535 trace git-lfs: xfer: adapter "basic" worker 5 waiting for Auth 11:36:17.436539 trace git-lfs: xfer: adapter "basic" worker 6 starting 11:36:17.436545 trace git-lfs: xfer: adapter "basic" worker 6 waiting for Auth 11:36:17.436549 trace git-lfs: xfer: adapter "basic" worker 7 starting 11:36:17.436555 trace git-lfs: xfer: adapter "basic" worker 7 waiting for Auth 11:36:17.438092 trace git-lfs: xfer: adapter "basic" worker 0 processing job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:36:17.439878 trace git-lfs: HTTP: PUT https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000

PUT /educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Authorization: Basic * * * * * Content-Length: 5120000000 Content-Type: application/octet-stream User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

11:36:17.440056 trace git-lfs: xfer: adapter "basic" worker 4 auth signal received 11:36:17.440072 trace git-lfs: xfer: adapter "basic" worker 2 auth signal received 11:36:17.440079 trace git-lfs: xfer: adapter "basic" worker 1 auth signal received 11:36:17.440085 trace git-lfs: xfer: adapter "basic" worker 5 auth signal received 11:36:17.440091 trace git-lfs: xfer: adapter "basic" worker 3 auth signal received 11:36:17.440098 trace git-lfs: xfer: adapter "basic" worker 6 auth signal received 11:36:17.440105 trace git-lfs: xfer: adapter "basic" worker 7 auth signal received 11:48:58.830929 trace git-lfs: HTTP: 500B | 7.1 MB/s

< HTTP/1.1 500 Internal Server Error < Content-Length: 22 < Connection: keep-alive < Content-Type: text/plain; charset=utf-8 < Date: Wed, 12 Jun 2019 09:48:58 GMT < Server: nginx < X-Content-Type-Options: nosniff < 11:48:58.832605 trace git-lfs: xfer: adapter "basic" worker 0 finished job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:48:58.832640 trace git-lfs: tq: retrying object c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e: Fatal error: Server error: https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 11:48:58.832682 trace git-lfs: tq: enqueue retry gitlab-workhorse#5 (closed) for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" (size: 5120000000) 11:48:58.832713 trace git-lfs: tq: sending batch of size 1 11:48:58.832859 trace git-lfs: ssh cache: git@pr-sti-git01.tec.in.phm.education.gouv.fr git-lfs-authenticate educnat.sic.crt-hebergement/test_lfs_s3.git upload 11:48:58.832909 trace git-lfs: api: batch 1 files 11:48:58.833017 trace git-lfs: HTTP: POST https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch

POST /educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Accept: application/vnd.git-lfs+json; charset=utf-8 Authorization: Basic * * * * * Content-Length: 162 Content-Type: application/vnd.git-lfs+json; charset=utf-8 User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

{"operation":"upload","objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000}],"ref":{"name":"refs/heads/master"}}11:48:58.877006 trace git-lfs: HTTP: 200

< HTTP/1.1 200 OK < Content-Length: 745 < Cache-Control: max-age=0, private, must-revalidate < Connection: keep-alive < Content-Type: application/json; charset=utf-8 < Date: Wed, 12 Jun 2019 09:48:58 GMT < Etag: W/"776a530dfdb4905f7af631948bc77cf4" < Page-Title: GitLab < Referrer-Policy: strict-origin-when-cross-origin < Server: nginx < Strict-Transport-Security: max-age=31536000 < X-Content-Type-Options: nosniff < X-Frame-Options: DENY < X-Request-Id: xqvJamJW4X2 < X-Runtime: 0.032847 < X-Ua-Compatible: IE=edge < X-Xss-Protection: 1; mode=block < 11:48:58.877120 trace git-lfs: HTTP: {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF0 {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF011:48:58.877177 trace git-lfs: HTTP: T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]} T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]}11:48:58.877274 trace git-lfs: xfer: adapter "basic" End() 11:48:58.877295 trace git-lfs: xfer: adapter "basic" worker 4 stopping 11:48:58.877303 trace git-lfs: xfer: adapter "basic" worker 5 stopping 11:48:58.877307 trace git-lfs: xfer: adapter "basic" worker 1 stopping 11:48:58.877303 trace git-lfs: xfer: adapter "basic" worker 3 stopping 11:48:58.877310 trace git-lfs: xfer: adapter "basic" worker 0 stopping 11:48:58.877314 trace git-lfs: xfer: adapter "basic" worker 7 stopping 11:48:58.877316 trace git-lfs: xfer: adapter "basic" worker 6 stopping 11:48:58.877318 trace git-lfs: xfer: adapter "basic" worker 2 stopping 11:48:58.877332 trace git-lfs: xfer: adapter "basic" stopped 11:48:58.877357 trace git-lfs: tq: starting transfer adapter "basic" 11:48:58.877367 trace git-lfs: xfer: adapter "basic" Begin() with 8 workers 11:48:58.877375 trace git-lfs: xfer: adapter "basic" started 11:48:58.877397 trace git-lfs: xfer: adapter "basic" worker 2 starting 11:48:58.877414 trace git-lfs: xfer: adapter "basic" worker 2 waiting for Auth 11:48:58.877421 trace git-lfs: xfer: adapter "basic" worker 0 starting 11:48:58.877430 trace git-lfs: xfer: adapter "basic" worker 1 starting 11:48:58.877441 trace git-lfs: xfer: adapter "basic" worker 1 waiting for Auth 11:48:58.877446 trace git-lfs: xfer: adapter "basic" worker 4 starting 11:48:58.877456 trace git-lfs: xfer: adapter "basic" worker 4 waiting for Auth 11:48:58.877460 trace git-lfs: xfer: adapter "basic" worker 3 starting 11:48:58.877466 trace git-lfs: xfer: adapter "basic" worker 3 waiting for Auth 11:48:58.877470 trace git-lfs: xfer: adapter "basic" worker 5 starting 11:48:58.877490 trace git-lfs: xfer: adapter "basic" worker 5 waiting for Auth 11:48:58.877495 trace git-lfs: xfer: adapter "basic" worker 6 starting 11:48:58.877502 trace git-lfs: xfer: adapter "basic" worker 6 waiting for Auth 11:48:58.877506 trace git-lfs: xfer: adapter "basic" worker 7 starting 11:48:58.877513 trace git-lfs: xfer: adapter "basic" worker 7 waiting for Auth 11:48:58.885400 trace git-lfs: xfer: adapter "basic" worker 0 processing job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:48:58.934812 trace git-lfs: HTTP: PUT https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000

PUT /educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Authorization: Basic * * * * * Content-Length: 5120000000 Content-Type: application/octet-stream User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

11:48:58.935577 trace git-lfs: xfer: adapter "basic" worker 3 auth signal received 11:48:58.935592 trace git-lfs: xfer: adapter "basic" worker 2 auth signal received 11:48:58.935607 trace git-lfs: xfer: adapter "basic" worker 1 auth signal received 11:48:58.935613 trace git-lfs: xfer: adapter "basic" worker 4 auth signal received 11:48:58.935618 trace git-lfs: xfer: adapter "basic" worker 6 auth signal received 11:48:58.935632 trace git-lfs: xfer: adapter "basic" worker 5 auth signal received 11:48:58.935637 trace git-lfs: xfer: adapter "basic" worker 7 auth signal received 11:48:58.986900 trace git-lfs: HTTP: 500

< HTTP/1.1 500 Internal Server Error < Content-Length: 22 < Connection: keep-alive < Content-Type: text/plain; charset=utf-8 < Date: Wed, 12 Jun 2019 09:48:58 GMT < Server: nginx < X-Content-Type-Options: nosniff < 11:48:58.987522 trace git-lfs: xfer: adapter "basic" worker 0 finished job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:48:58.987543 trace git-lfs: tq: retrying object c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e: Fatal error: Server error: https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 11:48:58.987576 trace git-lfs: tq: enqueue retry gitlab-workhorse#6 (closed) for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" (size: 5120000000) 11:48:58.987598 trace git-lfs: tq: sending batch of size 1 11:48:58.987683 trace git-lfs: ssh cache: git@pr-sti-git01.tec.in.phm.education.gouv.fr git-lfs-authenticate educnat.sic.crt-hebergement/test_lfs_s3.git upload 11:48:58.987715 trace git-lfs: api: batch 1 files 11:48:58.987811 trace git-lfs: HTTP: POST https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch

POST /educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Accept: application/vnd.git-lfs+json; charset=utf-8 Authorization: Basic * * * * * Content-Length: 162 Content-Type: application/vnd.git-lfs+json; charset=utf-8 User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

{"operation":"upload","objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000}],"ref":{"name":"refs/heads/master"}}11:48:59.027321 trace git-lfs: HTTP: 200

< HTTP/1.1 200 OK < Content-Length: 745 < Cache-Control: max-age=0, private, must-revalidate < Connection: keep-alive < Content-Type: application/json; charset=utf-8 < Date: Wed, 12 Jun 2019 09:48:59 GMT < Etag: W/"776a530dfdb4905f7af631948bc77cf4" < Page-Title: GitLab < Referrer-Policy: strict-origin-when-cross-origin < Server: nginx < Strict-Transport-Security: max-age=31536000 < X-Content-Type-Options: nosniff < X-Frame-Options: DENY < X-Request-Id: 40mbyC5rnza < X-Runtime: 0.033292 < X-Ua-Compatible: IE=edge < X-Xss-Protection: 1; mode=block < 11:48:59.027422 trace git-lfs: HTTP: {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF0 {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF011:48:59.027466 trace git-lfs: HTTP: T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]} T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]}11:48:59.027528 trace git-lfs: xfer: adapter "basic" End() 11:48:59.027549 trace git-lfs: xfer: adapter "basic" worker 3 stopping 11:48:59.027560 trace git-lfs: xfer: adapter "basic" worker 4 stopping 11:48:59.027554 trace git-lfs: xfer: adapter "basic" worker 6 stopping 11:48:59.027573 trace git-lfs: xfer: adapter "basic" worker 2 stopping 11:48:59.027570 trace git-lfs: xfer: adapter "basic" worker 1 stopping 11:48:59.027578 trace git-lfs: xfer: adapter "basic" worker 0 stopping 11:48:59.027582 trace git-lfs: xfer: adapter "basic" worker 7 stopping 11:48:59.027585 trace git-lfs: xfer: adapter "basic" worker 5 stopping 11:48:59.027594 trace git-lfs: xfer: adapter "basic" stopped 11:48:59.027627 trace git-lfs: tq: starting transfer adapter "basic" 11:48:59.027638 trace git-lfs: xfer: adapter "basic" Begin() with 8 workers 11:48:59.027651 trace git-lfs: xfer: adapter "basic" started 11:48:59.027699 trace git-lfs: xfer: adapter "basic" worker 0 starting 11:48:59.027715 trace git-lfs: xfer: adapter "basic" worker 0 processing job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:48:59.027768 trace git-lfs: xfer: adapter "basic" worker 4 starting 11:48:59.027781 trace git-lfs: xfer: adapter "basic" worker 4 waiting for Auth 11:48:59.027786 trace git-lfs: xfer: adapter "basic" worker 1 starting 11:48:59.027793 trace git-lfs: xfer: adapter "basic" worker 1 waiting for Auth 11:48:59.027797 trace git-lfs: xfer: adapter "basic" worker 2 starting 11:48:59.027803 trace git-lfs: xfer: adapter "basic" worker 2 waiting for Auth 11:48:59.027807 trace git-lfs: xfer: adapter "basic" worker 3 starting 11:48:59.027813 trace git-lfs: xfer: adapter "basic" worker 3 waiting for Auth 11:48:59.027817 trace git-lfs: xfer: adapter "basic" worker 6 starting 11:48:59.027823 trace git-lfs: xfer: adapter "basic" worker 6 waiting for Auth 11:48:59.027827 trace git-lfs: xfer: adapter "basic" worker 5 starting 11:48:59.027843 trace git-lfs: xfer: adapter "basic" worker 5 waiting for Auth 11:48:59.027847 trace git-lfs: xfer: adapter "basic" worker 7 starting 11:48:59.027854 trace git-lfs: xfer: adapter "basic" worker 7 waiting for Auth 11:48:59.028992 trace git-lfs: HTTP: PUT https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000

PUT /educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Authorization: Basic * * * * * Content-Length: 5120000000 Content-Type: application/octet-stream User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

11:48:59.029527 trace git-lfs: xfer: adapter "basic" worker 3 auth signal received 11:48:59.029538 trace git-lfs: xfer: adapter "basic" worker 4 auth signal received 11:48:59.029543 trace git-lfs: xfer: adapter "basic" worker 1 auth signal received 11:48:59.029546 trace git-lfs: xfer: adapter "basic" worker 2 auth signal received 11:48:59.029556 trace git-lfs: xfer: adapter "basic" worker 5 auth signal received 11:48:59.029561 trace git-lfs: xfer: adapter "basic" worker 6 auth signal received 11:48:59.029564 trace git-lfs: xfer: adapter "basic" worker 7 auth signal received 11:48:59.062266 trace git-lfs: HTTP: 500B | 7.1 MB/s

< HTTP/1.1 500 Internal Server Error < Content-Length: 22 < Connection: keep-alive < Content-Type: text/plain; charset=utf-8 < Date: Wed, 12 Jun 2019 09:48:59 GMT < Server: nginx < X-Content-Type-Options: nosniff < 11:48:59.063262 trace git-lfs: xfer: adapter "basic" worker 0 finished job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:48:59.063272 trace git-lfs: tq: retrying object c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e: Fatal error: Server error: https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 11:48:59.063313 trace git-lfs: tq: enqueue retry gitlab-workhorse#7 (closed) for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" (size: 5120000000) 11:48:59.063349 trace git-lfs: tq: sending batch of size 1 11:48:59.063499 trace git-lfs: ssh cache: git@pr-sti-git01.tec.in.phm.education.gouv.fr git-lfs-authenticate educnat.sic.crt-hebergement/test_lfs_s3.git upload 11:48:59.063533 trace git-lfs: api: batch 1 files 11:48:59.063657 trace git-lfs: HTTP: POST https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch

POST /educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Accept: application/vnd.git-lfs+json; charset=utf-8 Authorization: Basic * * * * * Content-Length: 162 Content-Type: application/vnd.git-lfs+json; charset=utf-8 User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

{"operation":"upload","objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000}],"ref":{"name":"refs/heads/master"}}11:48:59.100130 trace git-lfs: HTTP: 200

< HTTP/1.1 200 OK < Content-Length: 745 < Cache-Control: max-age=0, private, must-revalidate < Connection: keep-alive < Content-Type: application/json; charset=utf-8 < Date: Wed, 12 Jun 2019 09:48:59 GMT < Etag: W/"776a530dfdb4905f7af631948bc77cf4" < Page-Title: GitLab < Referrer-Policy: strict-origin-when-cross-origin < Server: nginx < Strict-Transport-Security: max-age=31536000 < X-Content-Type-Options: nosniff < X-Frame-Options: DENY < X-Request-Id: CPfPrsQeVg9 < X-Runtime: 0.030125 < X-Ua-Compatible: IE=edge < X-Xss-Protection: 1; mode=block < 11:48:59.100220 trace git-lfs: HTTP: {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF0 {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF011:48:59.100265 trace git-lfs: HTTP: T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]} T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]}11:48:59.100315 trace git-lfs: xfer: adapter "basic" End() 11:48:59.100325 trace git-lfs: xfer: adapter "basic" worker 3 stopping 11:48:59.100331 trace git-lfs: xfer: adapter "basic" worker 0 stopping 11:48:59.100336 trace git-lfs: xfer: adapter "basic" worker 7 stopping 11:48:59.100340 trace git-lfs: xfer: adapter "basic" worker 6 stopping 11:48:59.100344 trace git-lfs: xfer: adapter "basic" worker 1 stopping 11:48:59.100349 trace git-lfs: xfer: adapter "basic" worker 4 stopping 11:48:59.100346 trace git-lfs: xfer: adapter "basic" worker 2 stopping 11:48:59.100353 trace git-lfs: xfer: adapter "basic" worker 5 stopping 11:48:59.100367 trace git-lfs: xfer: adapter "basic" stopped 11:48:59.100398 trace git-lfs: tq: starting transfer adapter "basic" 11:48:59.100407 trace git-lfs: xfer: adapter "basic" Begin() with 8 workers 11:48:59.100423 trace git-lfs: xfer: adapter "basic" started 11:48:59.100470 trace git-lfs: xfer: adapter "basic" worker 0 starting 11:48:59.100496 trace git-lfs: xfer: adapter "basic" worker 0 processing job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:48:59.100500 trace git-lfs: xfer: adapter "basic" worker 2 starting 11:48:59.100511 trace git-lfs: xfer: adapter "basic" worker 2 waiting for Auth 11:48:59.100516 trace git-lfs: xfer: adapter "basic" worker 3 starting 11:48:59.100522 trace git-lfs: xfer: adapter "basic" worker 3 waiting for Auth 11:48:59.100527 trace git-lfs: xfer: adapter "basic" worker 6 starting 11:48:59.100533 trace git-lfs: xfer: adapter "basic" worker 6 waiting for Auth 11:48:59.100537 trace git-lfs: xfer: adapter "basic" worker 5 starting 11:48:59.100548 trace git-lfs: xfer: adapter "basic" worker 5 waiting for Auth 11:48:59.100553 trace git-lfs: xfer: adapter "basic" worker 7 starting 11:48:59.100560 trace git-lfs: xfer: adapter "basic" worker 7 waiting for Auth 11:48:59.100473 trace git-lfs: xfer: adapter "basic" worker 4 starting 11:48:59.100565 trace git-lfs: xfer: adapter "basic" worker 4 waiting for Auth 11:48:59.100494 trace git-lfs: xfer: adapter "basic" worker 1 starting 11:48:59.100571 trace git-lfs: xfer: adapter "basic" worker 1 waiting for Auth 11:48:59.101642 trace git-lfs: HTTP: PUT https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000

PUT /educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Authorization: Basic * * * * * Content-Length: 5120000000 Content-Type: application/octet-stream User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

11:48:59.101805 trace git-lfs: xfer: adapter "basic" worker 2 auth signal received 11:48:59.101821 trace git-lfs: xfer: adapter "basic" worker 5 auth signal received 11:48:59.101829 trace git-lfs: xfer: adapter "basic" worker 3 auth signal received 11:48:59.101835 trace git-lfs: xfer: adapter "basic" worker 6 auth signal received 11:48:59.101842 trace git-lfs: xfer: adapter "basic" worker 7 auth signal received 11:48:59.101847 trace git-lfs: xfer: adapter "basic" worker 4 auth signal received 11:48:59.101850 trace git-lfs: xfer: adapter "basic" worker 1 auth signal received 11:48:59.137772 trace git-lfs: HTTP: 500

< HTTP/1.1 500 Internal Server Error < Content-Length: 22 < Connection: keep-alive < Content-Type: text/plain; charset=utf-8 < Date: Wed, 12 Jun 2019 09:48:59 GMT < Server: nginx < X-Content-Type-Options: nosniff < 11:48:59.138647 trace git-lfs: xfer: adapter "basic" worker 0 finished job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:48:59.138673 trace git-lfs: tq: retrying object c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e: Fatal error: Server error: https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 11:48:59.138696 trace git-lfs: tq: enqueue retry gitlab-workhorse#8 (closed) for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" (size: 5120000000) 11:48:59.138722 trace git-lfs: tq: sending batch of size 1 11:48:59.138813 trace git-lfs: ssh cache: git@pr-sti-git01.tec.in.phm.education.gouv.fr git-lfs-authenticate educnat.sic.crt-hebergement/test_lfs_s3.git upload 11:48:59.138849 trace git-lfs: api: batch 1 files 11:48:59.138963 trace git-lfs: HTTP: POST https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch

POST /educnat.sic.crt-hebergement/test_lfs_s3.git/info/lfs/objects/batch HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Accept: application/vnd.git-lfs+json; charset=utf-8 Authorization: Basic * * * * * Content-Length: 162 Content-Type: application/vnd.git-lfs+json; charset=utf-8 User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

{"operation":"upload","objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000}],"ref":{"name":"refs/heads/master"}}11:48:59.182697 trace git-lfs: HTTP: 200

< HTTP/1.1 200 OK < Content-Length: 745 < Cache-Control: max-age=0, private, must-revalidate < Connection: keep-alive < Content-Type: application/json; charset=utf-8 < Date: Wed, 12 Jun 2019 09:48:59 GMT < Etag: W/"776a530dfdb4905f7af631948bc77cf4" < Page-Title: GitLab < Referrer-Policy: strict-origin-when-cross-origin < Server: nginx < Strict-Transport-Security: max-age=31536000 < X-Content-Type-Options: nosniff < X-Frame-Options: DENY < X-Request-Id: GroONOurTv < X-Runtime: 0.037352 < X-Ua-Compatible: IE=edge < X-Xss-Protection: 1; mode=block < 11:48:59.182776 trace git-lfs: HTTP: {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF0 {"objects":[{"oid":"c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e","size":5120000000,"actions":{"upload":{"href":"https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000","header":{"Authorization":"Basic YW1hdHRpbzpleUpoYkdjaU9pSklVekkxTmlJc0luUjVjQ0k2SWtwWFZDSjkuZXlKa1lYUmhJanA3SW1GamRHOXlJam9pWVcxaGRIUnBieUo5TENKcWRHa2lPaUpsT0dFd01Ea3hPUzB6TWpNd0xUUXdPVFF011:48:59.182808 trace git-lfs: HTTP: T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]} T1RKaU9TMHdNREZoTkdRMU4yVmpPREFpTENKcFlYUWlPakUxTmpBek16RTNNakFzSW01aVppSTZNVFUyTURNek1UY3hOU3dpWlhod0lqb3hOVFl3TXpNek5USXdmUS56WVk3U01WX2hXZGJRdUVLeFdMRWt3ZE9DMFUxS3lMRHBITUZwTmtmYXlZ","Content-Type":"application/octet-stream"}}}}]}11:48:59.182852 trace git-lfs: xfer: adapter "basic" End() 11:48:59.182861 trace git-lfs: xfer: adapter "basic" worker 2 stopping 11:48:59.182867 trace git-lfs: xfer: adapter "basic" worker 0 stopping 11:48:59.182871 trace git-lfs: xfer: adapter "basic" worker 1 stopping 11:48:59.182875 trace git-lfs: xfer: adapter "basic" worker 4 stopping 11:48:59.182879 trace git-lfs: xfer: adapter "basic" worker 7 stopping 11:48:59.182883 trace git-lfs: xfer: adapter "basic" worker 6 stopping 11:48:59.182887 trace git-lfs: xfer: adapter "basic" worker 3 stopping 11:48:59.182892 trace git-lfs: xfer: adapter "basic" worker 5 stopping 11:48:59.182896 trace git-lfs: xfer: adapter "basic" stopped 11:48:59.182918 trace git-lfs: tq: starting transfer adapter "basic" 11:48:59.182925 trace git-lfs: xfer: adapter "basic" Begin() with 8 workers 11:48:59.182933 trace git-lfs: xfer: adapter "basic" started 11:48:59.182972 trace git-lfs: xfer: adapter "basic" worker 0 starting 11:48:59.182984 trace git-lfs: xfer: adapter "basic" worker 0 processing job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:48:59.183028 trace git-lfs: xfer: adapter "basic" worker 4 starting 11:48:59.183041 trace git-lfs: xfer: adapter "basic" worker 4 waiting for Auth 11:48:59.183049 trace git-lfs: xfer: adapter "basic" worker 1 starting 11:48:59.183057 trace git-lfs: xfer: adapter "basic" worker 1 waiting for Auth 11:48:59.183061 trace git-lfs: xfer: adapter "basic" worker 2 starting 11:48:59.183068 trace git-lfs: xfer: adapter "basic" worker 2 waiting for Auth 11:48:59.183071 trace git-lfs: xfer: adapter "basic" worker 3 starting 11:48:59.183078 trace git-lfs: xfer: adapter "basic" worker 3 waiting for Auth 11:48:59.183082 trace git-lfs: xfer: adapter "basic" worker 6 starting 11:48:59.183089 trace git-lfs: xfer: adapter "basic" worker 6 waiting for Auth 11:48:59.183100 trace git-lfs: xfer: adapter "basic" worker 5 starting 11:48:59.183108 trace git-lfs: xfer: adapter "basic" worker 5 waiting for Auth 11:48:59.183112 trace git-lfs: xfer: adapter "basic" worker 7 starting 11:48:59.183119 trace git-lfs: xfer: adapter "basic" worker 7 waiting for Auth 11:48:59.184279 trace git-lfs: HTTP: PUT https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000

PUT /educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 HTTP/1.1 Host: pr-sti-git01.tec.in.phm.education.gouv.fr Authorization: Basic * * * * * Content-Length: 5120000000 Content-Type: application/octet-stream User-Agent: git-lfs/2.7.2 (GitHub; linux amd64; go 1.12.4; git 08a08ae0)

11:48:59.184450 trace git-lfs: xfer: adapter "basic" worker 2 auth signal received 11:48:59.184463 trace git-lfs: xfer: adapter "basic" worker 4 auth signal received 11:48:59.184467 trace git-lfs: xfer: adapter "basic" worker 1 auth signal received 11:48:59.184471 trace git-lfs: xfer: adapter "basic" worker 6 auth signal received 11:48:59.184486 trace git-lfs: xfer: adapter "basic" worker 3 auth signal received 11:48:59.184494 trace git-lfs: xfer: adapter "basic" worker 5 auth signal received 11:48:59.184498 trace git-lfs: xfer: adapter "basic" worker 7 auth signal received 11:48:59.237932 trace git-lfs: HTTP: 500B | 4.3 MB/s

< HTTP/1.1 500 Internal Server Error < Content-Length: 22 < Connection: keep-alive < Content-Type: text/plain; charset=utf-8 < Date: Wed, 12 Jun 2019 09:48:59 GMT < Server: nginx < X-Content-Type-Options: nosniff < 11:48:59.238607 trace git-lfs: xfer: adapter "basic" worker 0 finished job for "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e" 11:48:59.238624 trace git-lfs: tq: refusing to retry "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e", too many retries (8) 11:48:59.238641 trace git-lfs: tq: refusing to retry "c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e", too many retries (8) 11:48:59.238657 trace git-lfs: xfer: adapter "basic" End() 11:48:59.238663 trace git-lfs: xfer: adapter "basic" worker 2 stopping 11:48:59.238667 trace git-lfs: xfer: adapter "basic" worker 0 stopping 11:48:59.238671 trace git-lfs: xfer: adapter "basic" worker 7 stopping 11:48:59.238676 trace git-lfs: xfer: adapter "basic" worker 5 stopping 11:48:59.238679 trace git-lfs: xfer: adapter "basic" worker 3 stopping 11:48:59.238683 trace git-lfs: xfer: adapter "basic" worker 6 stopping 11:48:59.238687 trace git-lfs: xfer: adapter "basic" worker 1 stopping 11:48:59.238691 trace git-lfs: xfer: adapter "basic" worker 4 stopping 11:48:59.238694 trace git-lfs: xfer: adapter "basic" stopped Fatal error: Server error: https://pr-sti-git01.tec.in.phm.education.gouv.fr/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/c888bd8e5cde9920033d59725d2ab5a23d693b6045c91d43dd13249fe4940f5e/5120000000 error: failed to push some refs to 'git@pr-sti-git01.tec.in.phm.education.gouv.fr:educnat.sic.crt-hebergement/test_lfs_s3.git'

And the error seen in workhorse:

2019-06-12_09:58:41.24614 time="2019-06-12T11:58:41+02:00" level=error msg=error correlation_id=2S9O8MfX3E error="BodyUploader: upload failed: PUT request "https://storagegrid.in.phm.education.gouv.fr/git/tmp/uploads/1560333459-18781-0012-1421-c45cd988e227c54f9fb653188ff508bf?X-Amz-Expires=15300&X-Amz-Date=20190612T095739Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=xxxxxxxxxx%xxxxxxx%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=[FILTERED]\": Put https://storagegrid.in.phm.education.gouv.fr/git/tmp/uploads/1560333459-18781-0012-1421-c45cd988e227c54f9fb653188ff508bf?X-Amz-Expires=15300&X-Amz-Date=20190612T095739Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=xxxxxxxxxxx%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=d72e5bc04e15fe1367c5055dce7d0f1a825ef1b2d2f975679121b0dd745b9f19: net/http: HTTP/1.x transport connection broken: write tcp xxx.xxx.xxx.xxx:42254->xxx.xxx.xxx.xxx:443: write: connection reset by peer" method=PUT uri=/educnat.sic.crt-hebergement/test_lfs_s3.git/gitlab-lfs/objects/e74663b08cd6dc5a28fe64a13841e4d7d515350e2388e5edc80a41d547ef0c30/2048000000

Do you have any clue on that error? Is there a timeout on workhorse to send object to S3?

We are on gitlab-ce version 11.11.2 and git-lfs version 2.7.2-1

Thank for your help.

Edited by 🤖 GitLab Bot 🤖