Container Registry - Images tags doesn't appears gitlab Ui and gitlab API

Summary

After a gitlab implementation, the tags of docker registry doesn't appear anymore, the implementation consists to disable NGINX of gitlab bundle and configure an external NGINX. After run a pipeline and build an image with new tag I noticed that the image with new tag doesn't appear on gitlab container registry, however the image with new tag exists on S3 bucket.

Steps to reproduce

Build and push an Image with the pretended
In the left side bar click in Packages & registries and next container registry
In this page I have noticed that the tag counter, of the image, is empty (=0)
Click in the image repository the page is presented with the following message:

    This image has no active tags: The last tag related to this image was recently removed. This empty image and any associated data will be automatically removed as part of the regular Garbage Collection process. If you have any questions, contact your administrator.

What is the current bug behavior?

Run the following pipeline, with new tag my gitlab-ci.yml:

image: docker:stable
variables:
  CONTAINER_TEST_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME-SNAPSHOT
  CONTAINER_RELEASE_BRANCH_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME
  CONTAINER_RELEASE_MASTER_IMAGE: $CI_REGISTRY_IMAGE:test
stages:
  - build
  - release
Build image:
  stage: build
  before_script:
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN dockerhub.example.com
  script:
    - docker build -t $CONTAINER_TEST_IMAGE .
    - docker push $CONTAINER_TEST_IMAGE
Release this version:
  stage: release
  before_script:
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN dockerhub.example.com
  script:
    - docker pull $CONTAINER_TEST_IMAGE
    - docker tag $CONTAINER_TEST_IMAGE $CONTAINER_RELEASE_BRANCH_IMAGE
    - docker push $CONTAINER_RELEASE_BRANCH_IMAGE
  except:
    - master
  when: manual

After build and push the image with a new tag we can see that this image is allocated in s3 bucket storage. However when I go to container registry we can see that the number of tags is empty, appearing 0 tags: And inside of the image the created tags is not visible on container registry, as you can see in the following image: However I can pull the image with the presented tag:

docker pull dockerhub.example.com/[user]/ansible_test:test
test: Pulling from [user]/ansible_test
5a3ea8efae5d: Already exists
46490a515f44: Pull complete
34161dfd4744: Pull complete
cc100e0b3a99: Pull complete
a3c26d3e31a6: Pull complete
Digest: sha256:00b73cf8f84d1ed1d209b2263957393d51c3cccd6d468f97fd79eac70a0debd6
Status: Downloaded newer image for dockerhub.example.com/[user]/ansible_test:test
dockerhub.example.com/[user]/ansible_test:test

In order to understand if this behavior could be caused by the external NGINX I did a request to return number of tags of the project via Gitlab api inside Gitlab container and the result was the same an empty array.

root@git:/# curl  -v --noproxy "*" --header "PRIVATE-TOKEN:--------" "http://127.0.0.1:8181/api/v4/projects/2603/registry/repositories/872/tags"
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8181 (#0)
> GET /api/v4/projects/2603/registry/repositories/872/tags HTTP/1.1
> Host: 127.0.0.1:8181
> User-Agent: curl/7.59.0-DEV
> Accept: */*
> PRIVATE-TOKEN:-----
>
< HTTP/1.1 200 OK
< Cache-Control: max-age=0, private, must-revalidate
< Content-Length: 2
< Content-Type: application/json
< Etag: W/"-----"
< Link: <http://git.example.net:443/api/v4/projects/2603/registry/repositories/872/tags?id=2603&page=1&per_page=20&repository_id=872>; rel="first", <http://git.example.net:443/api/v4/projects/2603/registry/repositories/872/tags?id=2603&page=1&per_page=20&repository_id=872>; rel="last"
< Vary: Origin
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-Next-Page:
< X-Page: 1
< X-Per-Page: 20
< X-Prev-Page:
< X-Request-Id: ------
< X-Runtime: 0.229169
< X-Total: 0
< X-Total-Pages: 1
< Date: Fri, 06 Nov 2020 15:54:47 GMT
<
* Connection #0 to host 127.0.0.1 left intact
[]

What is the expected correct behavior?

When I click in container registry I should be able to see the number total of tags and when I click in the image I should be able to see all the images.

Environment description

docker --version Docker version 18.09.6, build 481bc77156

My environment is the following; an external NGINX (v.1.19.3) that communicates with Gitlab (v.13.4.1) container and the gitlab.rb configuration file is:

external_url "https://git.example.net"
registry_external_url 'https://dockerhub.example.com'
nginx['enable'] = false
web_server['external_users'] = ['nginx']
gitlab_workhorse['enable'] = true
gitlab_workhorse['listen_network'] = "tcp"
gitlab_workhorse['listen_addr'] = "0.0.0.0:8181"
gitlab_rails['registry_enabled'] = true
registry['registry_http_addr'] = "0.0.0.0:5000"
registry['storage'] = {
  's3' => {
    'accesskey' => '----',
    'secretkey' => '----',
    'region' => '---',
    'bucket' => 'gitlab-docker-registry',
    'chunksize' => '5242880'
  },
 'redirect' => {
   'disable' => true
 }
}

Results of GitLab environment info

Expand for output related to GitLab environment info


root@git:/# gitlab-rake gitlab:env:info
System information
System:
Current User:	git
Using RVM:	no
Ruby Version:	2.6.6p146
Gem Version:	2.7.10
Bundler Version:1.17.3
Rake Version:	12.3.3
Redis Version:	5.0.9
Git Version:	2.28.0
Sidekiq Version:5.2.9
Go Version:	unknown

GitLab information
Version:	13.4.1
Revision:	c90be62bdef
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	11.9
URL:		https://git.example.net
HTTP Clone URL:	https://git.example.net/some-group/some-project.git
SSH Clone URL:	git@git.example.net:some-group/some-project.git
Using LDAP:	yes
Using Omniauth:	yes
Omniauth Providers: azure_oauth2

GitLab Shell
Version:	13.7.0
Repository storage paths:
- default: 	/var/opt/gitlab/git-data/repositories
GitLab Shell path:		/opt/gitlab/embedded/service/gitlab-shell
Git:		/opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check

```
root@git:/# gitlab-rake gitlab:check SANITIZE=true
Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 13.7.0 ? ... OK (13.7.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... Server: ldapmain
LDAP authentication... Success
LDAP users with access to your GitLab server (only showing the first 100 results)
        User output sanitized. Found 100 users of 100 limit.
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
all namespaces are yes, example
13/2571 ... yes
872/2573 ... yes
Redis version >= 4.0.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.6.6)
Git version >= 2.24.0 ? ... yes (2.28.0)
Git user has default SSH configuration? ... yes
Active users: ... 291
Is authorized keys file accessible? ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
```

Possible fixes

Can you please help me to find what is going on?

~bug severity3

Edited Nov 09, 2020 by NadiaGoncalves