WebIDE: some plain text files are uploaded in base64 encoding
Summary
Files that should be uploaded to WebIDE as plain text are uploaded in base64 encoding.
This issue is very similar to gitlab-foss#48746 (closed). It seems that problem still exists in version 11.6.2 CE
Steps to reproduce
Looks like there is a problem related to mime-type detection and encoding when files are uploaded to WebIDE. I've prepared four files to show the problem:
$ file -i *.{sql,txt} | sort
ascii.sql: text/plain; charset=us-ascii
ascii.txt: text/plain; charset=us-ascii
non-ascii.sql: text/plain; charset=utf-8
non-ascii.txt: text/plain; charset=utf-8
$ sha1sum *.{sql,txt} | sort -k2,2
22fe79a8ad71c02c63d5b2c10c349064dd762ce1 ascii.sql
22fe79a8ad71c02c63d5b2c10c349064dd762ce1 ascii.txt
0f7b4136fcbe71359340241e91a20f0ad4386552 non-ascii.sql
0f7b4136fcbe71359340241e91a20f0ad4386552 non-ascii.txt
ascii.*
files have different extensions only and contain only ASCII characters. All files are attached below.
What is the current bug behavior?
File name | After upload to GitLab |
---|---|
ascii.sql | |
ascii.txt | |
non-ascii.sql | |
non-ascii.txt |
What is the expected correct behavior?
Well, I think this is obvious - use mime-type to detect if given file is a plain text file.
First two files were uploaded correctly.
Third file should not be uploaded in base64 encoding, because it is plain text file (text/plain
). After upload it should look like this:
Fourth file seems to be interpreted as plain text file just because it has *.txt
extension, but encoding is wrong. After upload it should look like this:
Proposal
Implement mime-type checking on the frontend via JS: #26113 (comment 239792655)
Results of GitLab environment info
Expand for output related to GitLab environment info
$ sudo gitlab-rake gitlab:env:info System information System: Current User: git Using RVM: no Ruby Version: 2.5.3p105 Gem Version: 2.7.6 Bundler Version:1.16.6 Rake Version: 12.3.1 Redis Version: 3.2.12 Git Version: 2.18.1 Sidekiq Version:5.2.3 Go Version: unknown GitLab information Version: 11.6.2 Revision: 4d0c547 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: https://<REDACTED> HTTP Clone URL: https://<REDACTED>/some-group/some-project.git SSH Clone URL: git@<REDACTED>:some-group/some-project.git Using LDAP: yes Using Omniauth: yes Omniauth Providers: GitLab Shell Version: 8.4.3 Repository storage paths: - default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
$ sudo gitlab-rake gitlab:check SANITIZE=true Checking GitLab subtasks ... Checking GitLab Shell ... GitLab Shell: ... GitLab Shell version >= 8.4.3 ? ... OK (8.4.3) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful Checking GitLab Shell ... Finished Checking Gitaly ... Gitaly: ... default ... OK Checking Gitaly ... Finished Checking Sidekiq ... Sidekiq: ... Running? ... yes Number of Sidekiq processes ... 1 Checking Sidekiq ... Finished Checking Incoming Email ... Incoming Email: ... Reply by email is disabled in config/gitlab.yml Checking Incoming Email ... Finished Checking LDAP ... LDAP: ... Server: ldapmain LDAP authentication... Success LDAP users with access to your GitLab server (only showing the first 100 results) <REDACTED> Checking LDAP ... Finished Checking GitLab App ... Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 2/5 ... yes 3/6 ... yes 3/7 ... yes 3/8 ... yes 3/9 ... yes 3/10 ... yes 4/11 ... yes 5/12 ... yes 8/13 ... yes 8/14 ... yes 8/16 ... yes 8/17 ... yes 16/18 ... yes 8/19 ... yes 8/20 ... yes 8/21 ... yes 8/22 ... yes 7/23 ... yes 7/24 ... yes 7/25 ... yes 12/26 ... yes 7/27 ... yes 8/28 ... yes 28/30 ... yes 28/31 ... yes 28/33 ... yes 28/34 ... yes 30/35 ... yes 3/36 ... yes 7/37 ... yes 28/38 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.3.5 ? ... yes (2.5.3) Git version >= 2.18.0 ? ... yes (2.18.1) Git user has default SSH configuration? ... yes Active users: ... 22 Checking GitLab App ... Finished Checking GitLab subtasks ... Finished