UI Fails to properly format/encode content merge request diff
Summary
When viewing a specific MR an error is returned due to encoding/templating errors. This is due to the fact that GitLab only supports UTF-8 for diffs. All other encodings (UTF-16, ASCII-8BIT, etc... ) are not supported.
Steps to reproduce
- Visit an MR that has this issue
- Observe error
Essentially, from what I can see its returned by /-/merge_requests/ID/diffs_batch.json
throwing an error.
Example Project
What is the current bug behavior?
The error "Error: Couldn't load some or all of the changes." is displayed when viewing an MR, most likely due to the format.
What is the expected correct behavior?
No errors displayed.
Relevant logs and/or screenshots
https://log.gprd.gitlab.net/goto/bb9c7730-f6c0-11ec-b86b-d963a1a6788e
Missing template errors/encoding with {:locale=>[:en], :formats=>[:json], :variants=>[], :handlers=>[:raw, :erb, :html, :builder, :ruby, :haml]}. Searched in:
* "/srv/gitlab/ee/app/views"
* "/srv/gitlab/app/views"
* "/srv/gitlab/vendor/bundle/ruby/2.7.0/gems/peek-1.1.0/app/views"
* "/srv/gitlab/vendor/bundle/ruby/2.7.0/gems/kaminari-core-1.2.1/app/views"
* "/srv/gitlab/vendor/bundle/ruby/2.7.0/gems/graphiql-rails-1.8.0/app/views"
* "/srv/gitlab/vendor/bundle/ruby/2.7.0/gems/doorkeeper-5.5.0.rc2/app/views"
* "/srv/gitlab/vendor/bundle/ruby/2.7.0/gems/devise-4.7.3/app/views"
* "/srv/gitlab/vendor/bundle/ruby/2.7.0/gems/view_component-2.50.0/app/views"
There is also another error:
incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string)
lib/rouge/formatters/html_gitlab.rb:38:in `gsub', lib/rouge/formatters/html_gitlab.rb:38:in `highlight_unicode_control_characters', lib/rouge/formatters/html_gitlab.rb:26:in `block (2 levels) in stream', lib/rouge/formatters/html_gitlab.rb:25:in `each', lib/rouge/formatters/html_gitlab.rb:25:in `block in stream', lib/rouge/formatters/html_gitlab.rb:19:in `stream', lib/gitlab/highlight.rb:66:in `highlight_plain', lib/gitlab/highlight.rb:77:in `rescue in highlight_rich', lib/gitlab/highlight.rb:69:in `highlight_rich', lib/gitlab/highlight.rb:61:in `highlight_text', lib/gitlab/highlight.rb:28:in `highlight', lib/gitlab/highlight.rb:7:in `highlight', app/presenters/blob_presenter.rb:15:in `highlight', lib/gitlab/diff/highlight.rb:156:in `highlighted_blob_lines', lib/gitlab/diff/highlight.rb:148:in `new_lines', lib/gitlab/diff/highlight.rb:124:in `blob_highlighting', lib/gitlab/diff/highlight.rb:89:in `highlight_line', lib/gitlab/diff/highlight.rb:60:in `apply_syntax_highlight', lib/gitlab/diff/highlight.rb:34:in `block in highlight', lib/gitlab/diff/highlight.rb:29:in `map', lib/gitlab/diff/highlight.rb:29:in `with_index', lib/gitlab/diff/highlight.rb:29:in `highlight', lib/gitlab/diff/file.rb:208:in `highlighted_diff_lines', lib/gitlab/diff/highlight_cache.rb:58:in `block in write_if_empty', lib/gitlab/diff/highlight_cache.rb:57:in `each', lib/gitlab/diff/highlight_cache.rb:57:in `write_if_empty', lib/gitlab/diff/file_collection/merge_request_diff_base.rb:40:in `write_cache', app/controllers/projects/merge_requests/diffs_controller.rb:40:in `diffs_batch', app/controllers/application_controller.rb:578:in `block in allow_gitaly_ref_name_caching', lib/gitlab/gitaly_client.rb:321:in `allow_ref_name_caching', app/controllers/application_controller.rb:577:in `allow_gitaly_ref_name_caching', ee/lib/gitlab/ip_address_state.rb:10:in `with', ee/app/controllers/ee/application_controller.rb:45:in `set_current_ip_address', app/controllers/application_controller.rb:527:in `set_current_admin', lib/gitlab/session.rb:11:in `with_session', app/controllers/application_controller.rb:518:in `set_session_storage', lib/gitlab/i18n.rb:105:in `with_locale', lib/gitlab/i18n.rb:111:in `with_user_locale', app/controllers/application_controller.rb:512:in `set_locale', app/controllers/application_controller.rb:506:in `set_current_context', ee/lib/omni_auth/strategies/group_saml.rb:41:in `other_phase', lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call', lib/gitlab/middleware/memory_report.rb:13:in `call', lib/gitlab/middleware/speedscope.rb:13:in `call', lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call', lib/gitlab/middleware/rails_queue_duration.rb:33:in `call', lib/gitlab/metrics/rack_middleware.rb:16:in `block in call', lib/gitlab/metrics/web_transaction.rb:46:in `run', lib/gitlab/metrics/rack_middleware.rb:16:in `call', lib/gitlab/jira/middleware.rb:19:in `call', lib/gitlab/middleware/go.rb:20:in `call', lib/gitlab/etag_caching/middleware.rb:21:in `call', lib/gitlab/middleware/query_analyzer.rb:11:in `block in call', lib/gitlab/database/query_analyzer.rb:37:in `within', lib/gitlab/middleware/query_analyzer.rb:11:in `call', lib/gitlab/middleware/multipart.rb:173:in `call', lib/gitlab/middleware/read_only/controller.rb:50:in `call', lib/gitlab/middleware/read_only.rb:18:in `call', lib/gitlab/middleware/same_site_cookies.rb:27:in `call', lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call', lib/gitlab/middleware/basic_health_check.rb:25:in `call', lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call', lib/gitlab/middleware/request_context.rb:21:in `call', lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call', config/initializers/fix_local_cache_middleware.rb:11:in `call', lib/gitlab/middleware/compressed_json.rb:26:in `call', lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call', lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call', lib/gitlab/metrics/requests_rack_middleware.rb:77:in `call', lib/gitlab/middleware/release_env.rb:13:in `call'
Output of checks
This bug happens on GitLab.com
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)