After Upgrade to 12.7.0-ee 500 Errors on repositories cause ActionView::Template::Error
Summary
After the upgrade from gitlab-ee from 12.6.4
to 12.7.0
some of our projects are not reachable using the GUI anymore. We get an 500 - Whoops, something went wrong on our end.
Steps to reproduce
Upgrade from gitlab-ee from 12.6.4
to 12.7.0
Example Project
None - sorry
What is the current bug behavior?
Some projects generate a 500 - Whoops, something went wrong on our end.
when accessed using the GUI.
What is the expected correct behavior?
The Project's overview page should appear.
Relevant logs and/or screenshots
The Upgrade logs dont contain any errors. But in production.log I see following, each time this error occurs:
Started GET "/group/my_project" for 93.234.28.191 at 2020-01-23 10:21:19 +0000
Processing by ProjectsController#show as HTML
Parameters: {"namespace_id"=>"group", "id"=>"my_project"}
Completed 500 Internal Server Error in 284ms (ActiveRecord: 42.2ms | Elasticsearch: 0.0ms)
ActionView::Template::Error (must specify an iv):
23: - if view_details && merge_request
24: = link_to commit.title, project_commit_path(project, commit.id, merge_request_iid: merge_request.iid), class: ["commit-row-message item-title js-onboarding-commit-item", ("font-italic" if commit.message.empty?)]
25: - else
26: = link_to_markdown_field(commit, :title, link, class: "commit-row-message item-title js-onboarding-commit-item #{"font-italic" if commit.message.empty?}")
27: %span.commit-row-message.d-inline.d-sm-none
28: ·
29: = commit.short_id
app/models/project_services/data_fields.rb:14:in `url'
app/models/project_services/jira_service.rb:130:in `url'
lib/gitlab/metrics/instrumentation.rb:161:in `block in url'
lib/gitlab/metrics/method_call.rb:36:in `measure'
lib/gitlab/metrics/instrumentation.rb:161:in `url'
app/models/project_services/jira_service.rb:121:in `issues_url'
app/models/project_services/issue_tracker_service.rb:88:in `issue_url'
app/models/project_services/issue_tracker_service.rb:100:in `issue_path'
app/helpers/issues_helper.rb:37:in `url_for_tracker_issue'
app/helpers/issues_helper.rb:26:in `url_for_issue'
lib/banzai/filter/external_issue_reference_filter.rb:86:in `url_for_issue'
lib/banzai/filter/external_issue_reference_filter.rb:71:in `block in issue_link_filter'
lib/banzai/filter/external_issue_reference_filter.rb:26:in `block in references_in'
lib/banzai/filter/external_issue_reference_filter.rb:25:in `gsub'
lib/banzai/filter/external_issue_reference_filter.rb:25:in `references_in'
lib/banzai/filter/external_issue_reference_filter.rb:68:in `issue_link_filter'
lib/banzai/filter/external_issue_reference_filter.rb:40:in `block (2 levels) in call'
lib/banzai/filter/reference_filter.rb:120:in `replace_text_when_pattern_matches'
lib/banzai/filter/external_issue_reference_filter.rb:39:in `block in call'
lib/banzai/filter/reference_filter.rb:97:in `block in each_node'
lib/banzai/filter/reference_filter.rb:96:in `each_node'
lib/banzai/filter/external_issue_reference_filter.rb:37:in `call'
lib/banzai/pipeline/base_pipeline.rb:23:in `block (2 levels) in singleton class'
lib/banzai/renderer.rb:130:in `render_result'
lib/gitlab/metrics/instrumentation.rb:161:in `block in render_result'
lib/gitlab/metrics/method_call.rb:36:in `measure'
lib/gitlab/metrics/instrumentation.rb:161:in `render_result'
lib/banzai/renderer.rb:161:in `block in cacheless_render'
lib/gitlab/metrics/influx_db.rb:103:in `measure'
lib/banzai/renderer.rb:160:in `cacheless_render'
lib/gitlab/metrics/instrumentation.rb:161:in `block in cacheless_render'
lib/gitlab/metrics/method_call.rb:36:in `measure'
lib/gitlab/metrics/instrumentation.rb:161:in `cacheless_render'
lib/banzai/renderer.rb:52:in `cacheless_render_field'
lib/gitlab/metrics/instrumentation.rb:161:in `block in cacheless_render_field'
lib/gitlab/metrics/method_call.rb:36:in `measure'
lib/gitlab/metrics/instrumentation.rb:161:in `cacheless_render_field'
app/models/concerns/cache_markdown_field.rb:49:in `block in refresh_markdown_cache'
app/models/concerns/cache_markdown_field.rb:46:in `map'
app/models/concerns/cache_markdown_field.rb:46:in `refresh_markdown_cache'
app/models/concerns/cache_markdown_field.rb:58:in `refresh_markdown_cache!'
lib/banzai/renderer.rb:42:in `render_field'
lib/gitlab/metrics/instrumentation.rb:161:in `block in render_field'
lib/gitlab/metrics/method_call.rb:36:in `measure'
lib/gitlab/metrics/instrumentation.rb:161:in `render_field'
lib/banzai.rb:16:in `render_field'
app/helpers/markup_helper.rb:115:in `markdown_field'
app/helpers/markup_helper.rb:33:in `link_to_markdown_field'
app/views/projects/commits/_commit.html.haml:26:in `_app_views_projects_commits__commit_html_haml___1412967740593786203_69947234125360'
app/views/shared/_commit_well.html.haml:4:in `_app_views_shared__commit_well_html_haml___4180285108834016973_69947233818320'
app/views/projects/_files.html.haml:15:in `_app_views_projects__files_html_haml__1849006244752247561_69947271734520'
app/views/projects/show.html.haml:30:in `_app_views_projects_show_html_haml__3025681234978118110_69947309490200'
app/controllers/application_controller.rb:121:in `render'
app/controllers/application_controller.rb:550:in `block in allow_gitaly_ref_name_caching'
lib/gitlab/gitaly_client.rb:330:in `allow_ref_name_caching'
app/controllers/application_controller.rb:549:in `allow_gitaly_ref_name_caching'
ee/lib/gitlab/ip_address_state.rb:10:in `with'
ee/app/controllers/ee/application_controller.rb:43:in `set_current_ip_address'
lib/gitlab/session.rb:11:in `with_session'
app/controllers/application_controller.rb:467:in `set_session_storage'
lib/gitlab/i18n.rb:55:in `with_locale'
lib/gitlab/i18n.rb:61:in `with_user_locale'
app/controllers/application_controller.rb:461:in `set_locale'
lib/gitlab/application_context.rb:18:in `with_context'
app/controllers/application_controller.rb:453:in `set_current_context'
lib/gitlab/error_tracking.rb:34:in `with_context'
app/controllers/application_controller.rb:545:in `sentry_context'
lib/gitlab/middleware/rails_queue_duration.rb:27:in `call'
lib/gitlab/metrics/rack_middleware.rb:17:in `block in call'
lib/gitlab/metrics/transaction.rb:62:in `run'
lib/gitlab/metrics/rack_middleware.rb:17:in `call'
lib/gitlab/request_profiler/middleware.rb:17:in `call'
ee/lib/gitlab/jira/middleware.rb:19:in `call'
lib/gitlab/middleware/go.rb:20:in `call'
lib/gitlab/etag_caching/middleware.rb:13:in `call'
lib/gitlab/middleware/multipart.rb:117:in `call'
lib/gitlab/middleware/read_only/controller.rb:48:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/middleware/request_context.rb:23:in `call'
config/initializers/fix_local_cache_middleware.rb:9:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:49:in `call'
lib/gitlab/middleware/release_env.rb:12:in `call'
Output of checks
$ sudo gitlab-rake gitlab:check
Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 11.0.0 ? ... OK (11.0.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
<--snip-->
all Projects indicate `yes` here
<--snip-->
Redis version >= 2.8.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.6.5)
Git version >= 2.22.0 ? ... yes (2.24.1)
Git user has default SSH configuration? ... yes
Active users: ... 32
Is authorized keys file accessible? ... yes
Elasticsearch version 5.6 - 6.x? ... skipped (elasticsearch is disabled)
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
Results of GitLab environment info
$ sudo gitlab-rake gitlab:env:info
System information
System: Debian 9.11
Proxy: no
Current User: git
Using RVM: no
Ruby Version: 2.6.5p114
Gem Version: 2.7.10
Bundler Version:1.17.3
Rake Version: 12.3.3
Redis Version: 5.0.7
Git Version: 2.24.1
Sidekiq Version:5.2.7
Go Version: unknown
GitLab information
Version: 12.7.0-ee
Revision: c6ca82f7978
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: PostgreSQL
DB Version: 9.6.11
URL: https://gitlab.example.com
HTTP Clone URL: https://gitlab.example.com/some-group/some-project.git
SSH Clone URL: git@gitlab.example.com:some-group/some-project.git
Elasticsearch: no
Geo: no
Using LDAP: no
Using Omniauth: no
GitLab Shell
Version: 11.0.0
Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories
GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell
Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
See above
Possible fixes
See error output above
Edited by Lukas 'ai-pi' Eipert