Error 500 when opening any project after updating to 10.5.2
Summary
I have a GitLab CE environment set up on my server for small personal projects. I recently updated it from 10.4.4 to 10.5.2 and only just realized that all projects except the one I currently use the most are now inaccessible and gives a "Something went wrong" 500 Internal Server Error when attempting to access them. I have 8 projects and only 1 of them doesn't give me the Error 500 right now.
Steps to reproduce
Step 1: Upgrade from 10.4.4 to 10.5.2
Step 2: Log in to self hosted GitLab.
Step 3: Open a project.
Step 4: Something went wrong (500) is shown.
What is the current bug behavior?
All projects except for one cannot be accessed and a Error 500 is returned to the user.
What is the expected correct behavior?
The projects should have been accessible properly as it should have.
Relevant logs and/or screenshots
I had no idea what log to look at, so I referred to other issues to get an idea of what to look for and where to find it. Once I found out that production.log is what I needed to find, I opened it up and immediately found the error after going to the bottom of the log. It looks like a database error however I have no idea how to fix it myself.
production.log as follows :
Completed 200 OK in 0ms (ActiveRecord: 0.0ms)
Processing by MetricsController#index as HTML
Completed 200 OK in 2ms (Views: 0.8ms | ActiveRecord: 0.0ms)
Started GET "/YandolsZX/Icarus-Imeji" for [redacted] at 2018-03-06 01:39:06 +0800
Processing by Gitlab::RequestForgeryProtection::Controller#index as HTML
Completed 200 OK in 0ms (ActiveRecord: 0.0ms)
Processing by Gitlab::RequestForgeryProtection::Controller#index as HTML
Completed 200 OK in 0ms (ActiveRecord: 0.0ms)
Processing by ProjectsController#show as HTML
Parameters: {"namespace_id"=>"YandolsZX", "id"=>"Icarus-Imeji"}
Read fragment views/YandolsZX/Icarus-Imeji/f0fac16f797d4f4527d9104ff209cb838e7acfb6/application_settings/1-20180128235014032382000/false/false/en/ae797fc72ed2d4cfe1aa7d0426430b72 (0.4ms)
Completed 500 Internal Server Error in 430ms (ActiveRecord: 37.1ms)
ActionView::Template::Error (PG::UndefinedTable: ERROR: relation "user_callouts" does not exist
LINE 5: WHERE a.attrelid = '"user_callouts"'::regclas...
^
: SELECT a.attname, format_type(a.atttypid, a.atttypmod),
pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod
FROM pg_attribute a LEFT JOIN pg_attrdef d
ON a.attrelid = d.adrelid AND a.attnum = d.adnum
WHERE a.attrelid = '"user_callouts"'::regclass
AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum
):
184: Environments
185:
186: - if project_nav_tab? :clusters
187: - show_cluster_hint = show_gke_cluster_integration_callout?(@project)
188: = nav_link(controller: [:clusters, :user, :gcp]) do
189: = link_to project_clusters_path(@project), title: _('Kubernetes'), class: 'shortcuts-cluster' do
190: %span
app/helpers/user_callouts_helper.rb:12:in `user_dismissed?'
app/helpers/user_callouts_helper.rb:6:in `show_gke_cluster_integration_callout?'
app/views/layouts/nav/sidebar/_project.html.haml:187:in `block in _app_views_layouts_nav_sidebar__project_html_haml___3288687455951704904_69928563667000'
app/helpers/tab_helper.rb:55:in `nav_link'
app/views/layouts/nav/sidebar/_project.html.haml:149:in `_app_views_layouts_nav_sidebar__project_html_haml___3288687455951704904_69928563667000'
app/views/layouts/_page.html.haml:3:in `_app_views_layouts__page_html_haml___1258951775622053518_69928416840680'
app/views/layouts/application.html.haml:8:in `_app_views_layouts_application_html_haml___910304661803239554_69928424122700'
app/views/layouts/project.html.haml:14:in `_app_views_layouts_project_html_haml___1391579587303322604_69928471288340'
lib/gitlab/i18n.rb:47:in `with_locale'
lib/gitlab/i18n.rb:53:in `with_user_locale'
app/controllers/application_controller.rb:325:in `set_locale'
lib/gitlab/middleware/multipart.rb:95:in `call'
lib/gitlab/request_profiler/middleware.rb:14:in `call'
lib/gitlab/middleware/go.rb:17:in `call'
lib/gitlab/etag_caching/middleware.rb:11:in `call'
lib/gitlab/middleware/read_only.rb:31:in `call'
lib/gitlab/request_context.rb:18:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:27:in `call'
Output of checks
Ran sudo gitlab-rake gitlab:check SANITIZE=true, sudo gitlab-rake gitlab:git:fsck and sudo gitlab-rake gitlab:uploads:check and none of which reported any problems.
Results of GitLab environment info
Expand for output related to GitLab environment info
System information System: Current User: git Using RVM: no Ruby Version: 2.3.6p384 Gem Version: 2.6.13 Bundler Version:1.13.7 Rake Version: 12.3.0 Redis Version: 3.2.11 Git Version: 2.14.3 Sidekiq Version:5.0.5 Go Version: unknown
GitLab information Version: 10.5.2 Revision: b951e0d Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: [redacted] HTTP Clone URL: [redacted]/some-group/some-project.git SSH Clone URL: [redacted]:some-group/some-project.git Using LDAP: no Using Omniauth: no
GitLab Shell Version: 6.0.3 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
Checking GitLab Shell ...GitLab Shell version >= 6.0.3 ? ... OK (6.0.3) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:root, or git:git? default... yes Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 1/1 ... repository is empty 2/2 ... ok 2/3 ... repository is empty 2/4 ... ok 3/5 ... repository is empty 2/6 ... ok 2/7 ... repository is empty 2/8 ... repository is empty Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Reply by email is disabled in config/gitlab.yml Checking LDAP ...
LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 1/1 ... yes 2/2 ... yes 2/3 ... yes 2/4 ... yes 3/5 ... yes 2/6 ... yes 2/7 ... yes 2/8 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.3.5 ? ... yes (2.3.6) Git version >= 2.9.5 ? ... yes (2.14.3) Git user has default SSH configuration? ... yes Active users: ... 3
Checking GitLab ... Finished