Commit message may be shown currupted if its encoding is not UTF-8
Summary
Commit message may be shown currupted if its encoding is not UTF-8
Steps to reproduce
Make a commit and write it's message with i18n.commitEncoding set to not-UTF-8 encoding (Windows-1251 in my case). Push the commit onto GitLab server. View the commit message in a commits list view, particular commit view, tags list view, particular tag view, branches view, particular branch view
Example Project
https://gitlab.com/ashumkin/test-encodings
What is the current bug behavior?
Commit message text looks corrupted. E.g. "Windows-1251 UTF-8" instead of "Добавить файлы в кодировках Windows-1251 и UTF-8" or "WANT: VS_FF_PRERELEASE VS_FF_DEBUG, ReleaseCandidate" instead of "изменены скрипты WANT: добавлены флаги VS_FF_PRERELEASE и VS_FF_DEBUG, переменная ReleaseCandidate"
What is the expected correct behavior?
Commit messages are shown converted to UTF-8 (as git -c i18n.logOutputEncoding=UTF-8 log
does).
Relevant logs and/or screenshots
Corrupted commit messages
Correct commit messages
Output of checks
GitLab CE docker image gitlab/gitlab-ce:10.1.1-ce.0
Results of GitLab environment info
Expand for output related to GitLab environment info
System information System: Current User: git Using RVM: no Ruby Version: 2.3.5p376 Gem Version: 2.6.13 Bundler Version:1.13.7 Rake Version: 12.1.0 Redis Version: 3.2.5 Git Version: 2.13.6 Sidekiq Version:5.0.4 Go Version: unknown
GitLab information Version: 10.1.1 Revision: cc27e5f Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: http://dr-gitlab.rarus.ru HTTP Clone URL: http://dr-gitlab.rarus.ru/some-group/some-project.git SSH Clone URL: git@dr-gitlab.rarus.ru:some-group/some-project.git Using LDAP: yes Using Omniauth: no
GitLab Shell Version: 5.9.3 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
Checking GitLab Shell ...
GitLab Shell version >= 5.9.3 ? ... OK (5.9.3) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:root, or git:git? default... yes Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 4/2 ... ok 4/4 ... ok 5/6 ... ok 5/7 ... ok 5/8 ... ok 5/9 ... ok 5/10 ... ok 5/11 ... ok 5/12 ... ok 5/13 ... ok 5/14 ... ok 5/15 ... ok 5/16 ... ok 5/17 ... ok 5/18 ... ok 5/19 ... ok 5/20 ... ok 5/21 ... ok 5/22 ... ok 5/23 ... ok 5/24 ... ok 5/25 ... ok 5/26 ... ok 5/27 ... ok 5/28 ... ok 5/29 ... ok 5/30 ... ok 5/31 ... ok 5/32 ... ok 5/33 ... ok 5/34 ... ok 5/35 ... ok 5/36 ... ok 5/37 ... ok 5/38 ... ok 5/39 ... ok 5/40 ... ok 5/41 ... ok 5/42 ... ok 5/43 ... ok 5/44 ... ok 5/45 ... ok 5/46 ... ok 5/47 ... ok 5/48 ... ok 5/49 ... ok 5/50 ... ok 5/52 ... ok 5/53 ... ok 5/54 ... ok 5/55 ... ok 5/56 ... ok 5/57 ... ok 5/58 ... ok 5/59 ... ok 5/60 ... ok 5/64 ... ok 5/65 ... ok 5/66 ... ok 5/67 ... ok 5/68 ... ok 5/69 ... ok 5/70 ... ok 5/71 ... ok 5/72 ... ok 5/73 ... ok 5/74 ... ok 5/75 ... ok 5/76 ... ok 5/77 ... ok 5/78 ... ok 5/79 ... ok 5/80 ... ok 5/81 ... ok 5/82 ... ok 5/83 ... ok 5/84 ... ok 5/85 ... ok 5/86 ... repository is empty 5/87 ... ok 5/88 ... ok 5/89 ... ok 9/91 ... ok 5/92 ... ok 5/93 ... ok 10/94 ... ok 10/95 ... ok 4/96 ... ok Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Reply by email is disabled in config/gitlab.yml Checking LDAP ...
...
Checking LDAP ... Finished
Checking GitLab ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 4/2 ... yes 4/4 ... yes 5/6 ... yes 5/7 ... yes 5/8 ... yes 5/9 ... yes 5/10 ... yes 5/11 ... yes 5/12 ... yes 5/13 ... yes 5/14 ... yes 5/15 ... yes 5/16 ... yes 5/17 ... yes 5/18 ... yes 5/19 ... yes 5/20 ... yes 5/21 ... yes 5/22 ... yes 5/23 ... yes 5/24 ... yes 5/25 ... yes 5/26 ... yes 5/27 ... yes 5/28 ... yes 5/29 ... yes 5/30 ... yes 5/31 ... yes 5/32 ... yes 5/33 ... yes 5/34 ... yes 5/35 ... yes 5/36 ... yes 5/37 ... yes 5/38 ... yes 5/39 ... yes 5/40 ... yes 5/41 ... yes 5/42 ... yes 5/43 ... yes 5/44 ... yes 5/45 ... yes 5/46 ... yes 5/47 ... yes 5/48 ... yes 5/49 ... yes 5/50 ... yes 5/52 ... yes 5/53 ... yes 5/54 ... yes 5/55 ... yes 5/56 ... yes 5/57 ... yes 5/58 ... yes 5/59 ... yes 5/60 ... yes 5/64 ... yes 5/65 ... yes 5/66 ... yes 5/67 ... yes 5/68 ... yes 5/69 ... yes 5/70 ... yes 5/71 ... yes 5/72 ... yes 5/73 ... yes 5/74 ... yes 5/75 ... yes 5/76 ... yes 5/77 ... yes 5/78 ... yes 5/79 ... yes 5/80 ... yes 5/81 ... yes 5/82 ... yes 5/83 ... yes 5/84 ... yes 5/85 ... yes 5/86 ... yes 5/87 ... yes 5/88 ... yes 5/89 ... yes 9/91 ... yes 5/92 ... yes 5/93 ... yes 10/94 ... yes 10/95 ... yes 4/96 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.3.3 ? ... yes (2.3.5) Git version >= 2.7.3 ? ... yes (2.13.6) Git user has default SSH configuration? ... yes Active users: ... 3
Checking GitLab ... Finished
Possible fixes
diff --git lib/gitlab/git/commit.rb lib/gitlab/git/commit.rb
index d551881..ad3183e 100644
--- lib/gitlab/git/commit.rb
+++ lib/gitlab/git/commit.rb
@@ -435,6 +435,10 @@ module Gitlab
@raw_commit = commit
@id = commit.oid
@message = commit.message
+ # commit messages from Rugged are always in original encoding
+ # (which is explicitly defined in a raw Git commit object)
+ # So, we convert it into UTF-8 to avoid unneccessary (and buggy) encoding detection
+ @message.encode!('UTF-8', @message.encoding)
@authored_date = author[:time]
@committed_date = committer[:time]
@author_name = author[:name]