GCP migration and Areas where google is blocked

At the moment I am optimistic on the access after GCP migration. It sounds to me only the resources by Google Inc. are blocked. I have some VMs on Google Cloud Platform at the moment for testing or demoing GitLab. No any trouble to access them from China by either IP or domain names.

Hello @xiaogang_gitlab, do you have any info on where (which datacenters) the infrastructure will be deployed in Asia? I had widely different results on my side when trying to access services located in HongKong vs Taiwan or Singapore for instance.

@dario.martini I don't have such information but @andrewn may have.

mentioned in issue #645 (closed)

As mentioned in the internal chat by @northrup We’re in the US East Google data center and will be using anycast termination for our web services.

mentioned in merge request www-gitlab-com!13097 (merged)

Many (very probably everything) Google Cloud Platform properties are server-side blocked in Cuba. This is what I get when trying to open Google's Cloud:

Thus, GitLab moving to GCP will effectively make it unavailable to us Cubans (I realize we're under the noise floor, but still it is kind of sad when myself and several of my friends have been using this great service for years).

Since we moved registry.gitlab.com to GCP can we actually ask people to check this endpoint?

So, please try open this: https://registry.gitlab.com/v2/. You should see:

{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":null}]}

Maybe http://www.gcping.com/ can help here, too. In case the Global HTTP Load Balancer is used, the network packages enter the google internal backbone very early.

I've raised a support ticket with Google to find out if this block is one their side, and if so, where we can find details of the policy.

https://enterprise.google.com/supportcenter/managecases#Case/0016000001JFbLO/U-16416321

Feedback from google

Please note the sanction is for the countries involved in the following countries: Crimea; Cuba; Iran; North Korea; Sudan and Syria. Please do take time to review this "U.S. Department of The Treasury" link [1]. As there are legal restrictions that were imposed for those mentioned countries

[1] http://www.treasury.gov/resource-center/sanctions/Programs/Pages/Programs.aspx

Just confirmed with @xiaogang_gitlab that access to the registry from China is working correctly.

$ curl -vi https://registry.gitlab.com/
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Cache-Control: no-cache
Cache-Control: no-cache
< Date: Fri, 20 Jul 2018 08:56:00 GMT
Date: Fri, 20 Jul 2018 08:56:00 GMT
< Content-Length: 0
Content-Length: 0

<
• Connection #0 to host registry.gitlab.com left intact

Hi @xiaogang_gitlab, @andrewn , I confirm that I was able to access the registry from China as well getting the same authentication error.

However I also tested to download a blob to check the speed (I think I am doing something wrong with the authentication, even tho I am able to correctly list blobs with the same token), and have noticed that the requests are still going trough AWS.

I am getting the token like this:

# curl --user 'USERNAME:MYPASSWORD' 'https://gitlab.com/jwt/auth?client_id=docker&offline_token=true&service=container_registry&scope=repository:gitlab
-org/gitlab-runner:pull'

Then I am getting a blob like this:

# curl --header "Authorization: Bearer ${TOKEN}" https://registry.gitlab.com/v2/gitlab-org/gitlab-runner/manifests/latest

Then, once I have a blob I try to download it but I get stuck.

# wget --output-document=/dev/null --header="Authorization: Bearer ${TOKEN}" https://registry.gitlab.com/v2/gitlab-org/gitlab-runner/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
--2018-07-21 01:31:17--  https://registry.gitlab.com/v2/gitlab-org/gitlab-runner/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Resolving registry.gitlab.com (registry.gitlab.com)... 35.231.145.151
Connecting to registry.gitlab.com (registry.gitlab.com)|35.231.145.151|:443... connected.
HTTP request sent, awaiting response... 307 Temporary Redirect
Location: https://registry-gitlab-com.s3.amazonaws.com/docker/registry/v2/blobs/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4/data?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=XXXXXXXXXXX%2F20180720%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180720T173118Z&X-Amz-Expires=1200&X-Amz-SignedHeaders=host&X-Amz-Signature=XXXXXXXXXXX [following]
--2018-07-21 01:31:19--  https://registry-gitlab-com.s3.amazonaws.com/docker/registry/v2/blobs/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4/data?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=XXXXXXXXXXX%2F20180720%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180720T173118Z&X-Amz-Expires=1200&X-Amz-SignedHeaders=host&X-Amz-Signature=XXXXXXXXXXX
Resolving registry-gitlab-com.s3.amazonaws.com (registry-gitlab-com.s3.amazonaws.com)... 52.216.20.99
Connecting to registry-gitlab-com.s3.amazonaws.com (registry-gitlab-com.s3.amazonaws.com)|52.216.20.99|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2018-07-21 01:31:20 ERROR 400: Bad Request.

Is the registry already migrated but the data still on AWS or am I misunderstanding something?

Is the registry already migrated but the data still on AWS or am I misunderstanding something?

@dario.martini - your conclusion is spot on, great investigation The data will - for a couple of weeks, at least - remain on AWS. Your issue appears to be unrelated to the GCP migration. When did this start happening to you? I don't think AWS would block requests with a HTTP 400 response, so I think something else must be wrong.

@andrewn not sure yet what the issue is with the 400 error, I have been able to do the same thing on registry.docker.io with no problem, and I have used this system to test the connectivity before. Do you have any clue that could point me in the right direction so I can better run tests?

Following is what I have been doing on docker.io:

I am getting the token like this:

curl --user 'USERNAME:PASSWORD' "https://auth.docker.io/token?service=registry.docker.io&scope=repository:library/ubuntu:pull"

Then I download a blob (from the list as above):

# wget --output-document=/dev/null --header="Authorization: Bearer ${TOKEN}" "https://registry-1.docker.io/v2/library/ubuntu/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb1642
2d00e8a7c22955b46d4"
--2018-07-22 02:41:45--  https://registry-1.docker.io/v2/library/ubuntu/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Resolving registry-1.docker.io (registry-1.docker.io)... 54.164.230.151, 52.22.181.254, 34.205.207.96, ...
Connecting to registry-1.docker.io (registry-1.docker.io)|54.164.230.151|:443... connected.
HTTP request sent, awaiting response... 307 Temporary Redirect
Location: https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4/data?verify=1532201506-%2F4oflF2sRaOgwxJkeKurrVk1miI%3D [following]
--2018-07-22 02:41:46--  https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4/data?verify=1532201506-%2F4oflF2sRaOgwxJkeKurrVk1miI%3D
Resolving production.cloudflare.docker.com (production.cloudflare.docker.com)... 104.18.123.25, 104.18.124.25, 104.18.125.25, ...
Connecting to production.cloudflare.docker.com (production.cloudflare.docker.com)|104.18.123.25|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 32 [application/octet-stream]
Saving to: '/dev/null'

100%[==================================================================================================================================================================>] 32          --.-K/s   in 0s      

2018-07-22 02:41:48 (1.09 MB/s) - '/dev/null' saved [32/32]

I can see that the initial authentication seems successful in both cases, however in the case of Gitlab's registry I get blocked by AWS after being correctly redirected to it.

GCP completely banned iran from accessing services hosted on GCP . Gitlab migration to GCP means no one from iran can access gitlab without proxy.

curl https://cloud.google.com/

<!DOCTYPE html>
<html lang=en>
  <meta charset=utf-8>
  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
  <title>Error 403 (Forbidden)!!1</title>
  <style>
    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
  </style>
  <a href=//www.google.com/><span id=logo aria-label=Google></span></a>
  <p><b>403.</b> <ins>That’s an error.</ins>
  <p>Your client does not have permission to get URL <code>/</code> from this server.  <ins>That’s all we know.</ins>

 curl www.gcping.com
<?xml version='1.0' encoding='UTF-8'?><Error><Code>AccessDenied</Code><Message>Access denied.</Message><Details>We're sorry, but this service is not available in your location</Details></Error>%   [reza:~]$

also see

https://groups.google.com/forum/#!topic/gce-discussion/NhrK8vQPp_o

is there any workaround for this issue ?

mentioned in merge request www-gitlab-com!13323 (merged)

GCP migration and Areas where google is blocked

Child items ...

Activity

GCP migration and Areas where google is blocked

Relates to

Activity