Skip to content

500 When viewing commit diffs if gitattributes in main branch uses utf16

Summary

When viewing commit diffs on repositories, where .gitattributes has utf-16 encoding diff gets 500.

Github works fine with this use-case.

Analyzing of this problem takes about 3 days and posponed our migration from Github to Gitlab.

Tested on 2 omnibus installations (Ubuntu 14.04 and 16.04), reproduced on gitlab.com

Steps to reproduce

  1. Create new repository
  2. Put .gitattributes in it
  3. Push changes to gitlab
  4. Add some other changes and push it

Repo with reproduced problem: https://gitlab.com/ravenox/gitattributes-bug

Reproduce link with 500: ravenox/gitattributes-bug@12626553

Expected behavior

Show diff commits and:

  • support .gitattributes with encodings other than utf-8

or

  • ignore .gitattributes with invalid encoding ( possible display some warning message about that )

Actual behavior

Getting 500 error with

invalid byte sequence in UTF-8

Relevant logs and/or screenshots

production.log

Processing by Projects::CommitController#show as HTML
  Parameters: {"namespace_id"=>"***", "project_id"=>"***", "id"=>"***"}
Completed 500 Internal Server Error in 138ms (ActiveRecord: 8.0ms)

ActionView::Template::Error (invalid byte sequence in UTF-8):
     6:   - elsif blob.only_display_raw?
     7:     .nothing-here-block This file is too large to display.
     8:   - elsif blob_text_viewable?(blob)
     9:     - if !project.repository.diffable?(blob)
    10:       .nothing-here-block This diff was suppressed by a .gitattributes entry.
    11:     - elsif diff_file.collapsed?
    12:       - url = url_for(params.merge(action: :diff_for_path, old_path: diff_file.old_path, new_path: diff_file.new_path))
  app/models/repository.rb:516:in `method_missing'
  app/views/projects/diffs/_content.html.haml:9:in `_app_views_projects_diffs__content_html_haml___1885977870775046243_57105780'
  app/views/projects/diffs/_file.html.haml:19:in `_app_views_projects_diffs__file_html_haml___551463579386867991_58905280'
  app/views/projects/diffs/_diffs.html.haml:32:in `block in _app_views_projects_diffs__diffs_html_haml___150019359799572218_62759620'
  app/views/projects/diffs/_diffs.html.haml:26:in `each_with_index'
  app/views/projects/diffs/_diffs.html.haml:26:in `_app_views_projects_diffs__diffs_html_haml___150019359799572218_62759620'
  app/views/projects/commit/show.html.haml:10:in `_app_views_projects_commit_show_html_haml__3491327330629881173_64225440'
  app/controllers/projects/commit_controller.rb:23:in `show'
  lib/gitlab/request_profiler/middleware.rb:15:in `call'
  lib/gitlab/middleware/go.rb:16:in `call'

Output of checks

Results of GitLab application Check

Checking GitLab Shell ...

GitLab Shell version >= 3.6.1 ? ... OK (3.6.1)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
3/2 ... ok
3/3 ... ok
3/4 ... ok
7/5 ... ok
8/7 ... ok
8/8 ... ok
8/9 ... ok
8/10 ... ok
8/16 ... ok
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK
Send ping to redis server: OK
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Sidekiq ...

Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking Reply by email ...

IMAP server credentials are correct? ... yes
Init.d configured correctly? ... skipped (omnibus-gitlab has no init script)
MailRoom running? ... can't check because of previous errors

Checking Reply by email ... Finished

Checking LDAP ...

LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab ...

Git configured with autocrlf=input? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory setup correctly? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
projects have namespace: ...
3/2 ... yes
3/3 ... yes
3/4 ... yes
7/5 ... yes
8/7 ... yes
8/8 ... yes
8/9 ... yes
8/10 ... yes
8/16 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.1.0 ? ... yes (2.3.1)
Your git bin path is "/opt/gitlab/embedded/bin/git"
Git version >= 2.7.3 ? ... yes (2.7.4)
Active users: 19

Checking GitLab ... Finished

Results of GitLab environment info

System information
System:         Ubuntu 16.04
Current User:   git
Using RVM:      no
Ruby Version:   2.3.1p112
Gem Version:    2.6.6
Bundler Version:1.13.1
Rake Version:   10.5.0
Sidekiq Version:4.1.4

GitLab information
Version:        8.12.4
Revision:       ba8aeb7
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     postgresql
URL:            https://gitlab.youdo.sg
HTTP Clone URL: https://gitlab.youdo.sg/some-group/some-project.git
SSH Clone URL:  git@gitlab.youdo.sg:some-group/some-project.git
Using LDAP:     no
Using Omniauth: no

GitLab Shell
Version:        3.6.1
Repository storage paths:
- default:      /data/repositories
Hooks:          /opt/gitlab/embedded/service/gitlab-shell/hooks/
Git:            /opt/gitlab/embedded/bin/git
System information
System:         Ubuntu 14.04
Current User:   git
Using RVM:      no
Ruby Version:   2.3.1p112
Gem Version:    2.6.6
Bundler Version:1.13.1
Rake Version:   10.5.0
Sidekiq Version:4.1.4

GitLab information
Version:        8.12.4
Revision:       ba8aeb7
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     postgresql
URL:            https://lab.raxei.net
HTTP Clone URL: https://lab.raxei.net/some-group/some-project.git
SSH Clone URL:  git@lab.raxei.net:some-group/some-project.git
Using LDAP:     no
Using Omniauth: no

GitLab Shell
Version:        3.6.1
Repository storage paths:
- default:      /mnt/data/git-repos/repositories
Hooks:          /opt/gitlab/embedded/service/gitlab-shell/hooks/
Git:            /opt/gitlab/embedded/bin/git