GitLab Registry Corrupt / Broken
Summary
Gitlab registry gives 500 for new projects and shows "Something went wrong when fetching the registry list". in the ui when expanding list for existing projects. Permission denied when trying to pull from ci.
Steps to reproduce
Not sure but it seems everything started going wrong when I used import on a git repo and supplied http// instead of http:// and then tried to delete the repo right away. After that accessing the registry page was broken.
What is the current bug behaviour?
500 errors and error messages
What is the expected correct behaviour?
A working registry
Relevant logs
every time I fetch it I see this in the logs.
Relevant logs
==> /var/log/gitlab/gitlab-rails/production_json.log <== {"method":"GET","path":"/analytics/principles/container_registry","format":"html","controller":"Projects::Registry::RepositoriesController","action":"index","status":500,"error":"Errno::EADDRNOTAVAIL: Failed to open TCP connection to localhost:5000 (Cannot assign requested address - connect(2) for \"localhost\" port 5000)","duration":363.58,"view":0.0,"db":11.1,"time":"2018-07-25T20:42:51.083Z","params":[{"key":"namespace_id","value":"analytics"},{"key":"project_id","value":"principles"}],"remote_ip":"10.2.1.59","user_id":1,"username":"root"}==> /var/log/gitlab/gitlab-rails/production.log <==
Errno::EADDRNOTAVAIL (Failed to open TCP connection to localhost:5000 (Cannot assign requested address - connect(2) for "localhost" port 5000)): lib/container_registry/client.rb:19:in
repository_tags' app/models/container_repository.rb:36:in
manifest' app/models/container_repository.rb:41:intags' app/models/container_repository.rb:53:in
has_tags?' app/controllers/projects/registry/repositories_controller.rb:48:inblock (2 levels) in ensure_root_container_repository!' app/controllers/projects/registry/repositories_controller.rb:47:in
tap' app/controllers/projects/registry/repositories_controller.rb:47:inblock in ensure_root_container_repository!' app/controllers/projects/registry/repositories_controller.rb:44:in
tap' app/controllers/projects/registry/repositories_controller.rb:44:inensure_root_container_repository!' lib/gitlab/i18n.rb:51:in
with_locale' lib/gitlab/i18n.rb:57:inwith_user_locale' app/controllers/application_controller.rb:362:in
set_locale' lib/gitlab/middleware/multipart.rb:97:incall' lib/gitlab/request_profiler/middleware.rb:14:in
call' lib/gitlab/middleware/go.rb:17:incall' lib/gitlab/etag_caching/middleware.rb:11:in
call' lib/gitlab/middleware/read_only/controller.rb:38:incall' lib/gitlab/middleware/read_only.rb:16:in
call' lib/gitlab/request_context.rb:18:incall' lib/gitlab/metrics/requests_rack_middleware.rb:27:in
call' lib/gitlab/middleware/release_env.rb:10:in `call'
Details of package version
Provide the package version installation details
docker exec -it gitlab-ce dpkg-query -l "gitlab-*" Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==============-============-============-================================= ii gitlab-ce 11.1.1-ce.0 amd64 GitLab Community Edition (includi un gitlab-ee (no description available)
I'm using the release docker ce build 11.1.1-ce.0
Environment details
- Operating System:
ubuntu 16 (docker install though)
- Installation Target, remove incorrect values:
- VM:
vSphere
- VM:
- Installation Type, remove incorrect values:
- Upgrade from version
unknown, about 4 months ago latest.
- Upgrade from version
- Is there any other software running on the machine:
docker install, but no
- Is this a single or multiple node installation? single
- Resources
- CPU:
8cores
- Memory total:
10GB
- CPU:
Configuration details
docker run command
docker run -it \ --hostname gitlab.cugcr.ca \ --env GITLAB_OMNIBUS_CONFIG="external_url 'https://gitlab.hname.ca/';gitlab_rails['registry_enabled'] = true;gitlab_rails['registry_host'] = 'registry.hname.ca';" \ --publish 443:443 --publish 80:80 --publish 22:22 \ --name gitlab-ce \ --restart always \ --volume /srv/gitlab/config:/etc/gitlab \ --volume /srv/gitlab/logs:/var/log/gitlab \ --volume /srv/gitlab/data:/var/opt/gitlab \ gitlab/gitlab-ce:latest
I would also be happy with wiping out and resetting the registry if that's an option.