gitlab.com based project hangs on clone
I have a CI system that has been working fine for months using env usernames and passwords to clone over https.
Last week my agent stopped being able to git clone a remote repo
Here is the redacted output for where git clone is hanging:
[jack@centos7165118 tmp]$ GIT_TRACE=1 GIT_CURL_VERBOSE=1 git clone -v https://myuser:XXXXXX@gitlab.com/comp/api.git
trace: built-in: git 'clone' '-v' 'https://myuser:XXXXXX@gitlab.com/comp/api.git'
Cloning into 'api'...
trace: run_command: 'git-remote-https' 'origin' 'https://myuser:XXXXXX@gitlab.com/comp/api.git'
* Couldn't find host gitlab.com in the .netrc file; using defaults
* About to connect() to gitlab.com port 443 (#0)
* Trying 52.167.219.168...
* Connected to gitlab.com (52.167.219.168) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
* subject: CN=gitlab.com,OU=PositiveSSL,OU=Domain Control Validated
* start date: Nov 07 00:00:00 2016 GMT
* expire date: Nov 30 23:59:59 2017 GMT
* common name: gitlab.com
* issuer: CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB
> GET /comp/api.git/info/refs?service=git-upload-pack HTTP/1.1
User-Agent: git/1.8.3.1
Host: gitlab.com
Accept: */*
Accept-Encoding: gzip
Pragma: no-cache
< HTTP/1.1 401 Unauthorized
< Server: nginx
< Date: Mon, 25 Sep 2017 18:57:22 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 26
< Cache-Control: no-cache
< WWW-Authenticate: Basic realm="GitLab"
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-Request-Id: 89a3d9e6-ec13-4a3f-aa15-de16c402c127
< X-Runtime: 0.028451
< X-Ua-Compatible: IE=edge
< X-Xss-Protection: 1; mode=block
<
* Ignoring the response-body
* Connection #0 to host gitlab.com left intact
* Issue another request to this URL: 'https://myuser:XXXXXX@gitlab.com/comp/api.git/info/refs?service=git-upload-pack'
* Couldn't find host gitlab.com in the .netrc file; using defaults
* Found bundle for host gitlab.com: 0x7e7250
* Re-using existing connection! (#0) with host gitlab.com
* Connected to gitlab.com (52.167.219.168) port 443 (#0)
* Server auth using Basic with user 'myuser'
> GET /comp/api.git/info/refs?service=git-upload-pack HTTP/1.1
Authorization: Basic cGllbnNvX3BhZG1hOkpucTZwWktVVjJjbg==
User-Agent: git/1.8.3.1
Host: gitlab.com
Accept: */*
Accept-Encoding: gzip
Pragma: no-cache
< HTTP/1.1 200 OK
< Server: nginx
< Date: Mon, 25 Sep 2017 18:57:23 GMT
< Content-Type: application/x-git-upload-pack-advertisement
< Content-Length: 1282
< Cache-Control: no-cache
< Strict-Transport-Security: max-age=31536000
<
* Connection #0 to host gitlab.com left intact
trace: run_command: 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--thin' 'https://myuser:XXXXXX@gitlab.com/comp/api.git/'
trace: exec: 'git' 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--thin' 'https://myuser:XXXXXX@gitlab.com/comp/api.git/'
trace: built-in: git 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--thin' 'https://myuser:XXXXXX@gitlab.com/comp/api.git/'
POST git-upload-pack (792 bytes)
* Couldn't find host gitlab.com in the .netrc file; using defaults
* About to connect() to gitlab.com port 443 (#1)
* Trying 52.167.219.168...
* Connected to gitlab.com (52.167.219.168) port 443 (#1)
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
* subject: CN=gitlab.com,OU=PositiveSSL,OU=Domain Control Validated
* start date: Nov 07 00:00:00 2016 GMT
* expire date: Nov 30 23:59:59 2017 GMT
* common name: gitlab.com
* issuer: CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB
* Server auth using Basic with user 'myuser'
> POST /comp/api.git/git-upload-pack HTTP/1.1
Authorization: Basic cGllbnNvX3BhZG1hOkpucTZwWktVVjJjbg==
User-Agent: git/1.8.3.1
Host: gitlab.com
Accept-Encoding: gzip
Content-Type: application/x-git-upload-pack-request
Accept: application/x-git-upload-pack-result
Content-Length: 792
* upload completely sent off: 792 out of 792 bytesGit clone should not hang