GitLab(.com?) does not exploit If-Not-Modified-Since HTTP request header

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

GitLab.com appears to miss the same feature as GitHub: it does not determine whether content needs not be retransferred, which is the case when the client has the current content already. I think this isn't related to GitLab.com per se, but would also happen with the self-hosted products. Adding this feature should have the biggest impact for the Container Registry and Pipeline artifacts, since those are generally large blobs.

Example

curl -v --fail --location --show-error --silent --remote-name --tlsv1.2 --time-cond 'sys_provisioning-0.3.0-py3-none-any.whl' \
>     'https://gitlab.com/sander_maijers/sys-provisioning/builds/3059261/artifacts/file/dist/sys_provisioning-0.3.0-py3-none-any.whl'
*   Trying 104.210.2.228...
* Connected to gitlab.com (104.210.2.228) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [98 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [4315 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: OU=Domain Control Validated; OU=EssentialSSL; CN=gitlab.com
*  start date: Nov  3 00:00:00 2015 GMT
*  expire date: Nov 30 23:59:59 2016 GMT
*  subjectAltName: host "gitlab.com" matched cert's "gitlab.com"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=COMODO RSA Domain Validation Secure Server CA
*  SSL certificate verify ok.
} [5 bytes data]
> GET /sander_maijers/sys-provisioning/builds/3059261/artifacts/file/dist/sys_provisioning-0.3.0-py3-none-any.whl HTTP/1.1
> Host: gitlab.com
> User-Agent: curl/7.50.1
> Accept: */*
> If-Modified-Since: Fri, 12 Aug 2016 18:23:53 GMT
> 
{ [5 bytes data]
< HTTP/1.1 200 OK
< Server: nginx
< Date: Fri, 12 Aug 2016 18:25:48 GMT
< Content-Type: application/octet-stream
< Content-Length: 3805
< Cache-Control: no-cache
< Content-Disposition: attachment; filename="sys_provisioning-0.3.0-py3-none-any.whl"
< Status: 200 OK
< Strict-Transport-Security: max-age=31536000
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-Request-Id: 01af2a87-9bff-4647-9359-4fc071026555
< X-Runtime: 0.049006
< X-Ua-Compatible: IE=edge
< X-Xss-Protection: 1; mode=block
< 
{ [3805 bytes data]
* Connection #0 to host gitlab.com left intact

This file hadn't changed at the time of request, but was retransferred.

For one, improving this has the potential to significantly speed up GitLab CI pipelines. I think this issue is more pressing for GitLab.com than GitHub since GitHub does not offer a Container Registry and is presently noticably snappier and probably has more network resources available.

Edited by 🤖 GitLab Bot 🤖