DOS line endings on multi-line secret variables

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Summary

Secret variables use DOS line endings which causes ^M to show up for each newline of the variable on Unix systems.

Steps to reproduce

  1. Create a multi-line secret variable and have GitLab CI write it out to a file and keep it as an artifact
  2. Download the artifact and open the file in vim or emacs
  3. There should be a ^M for each newline in the variable

Example Project

https://gitlab.com/AshleyDumaine/gitlab-secret-variable-bug

What is the current bug behavior?

^M gets appended to each newline of the variable

What is the expected correct behavior?

There should be no ^M character for each newline

Relevant logs and/or screenshots

https://gitlab.com/AshleyDumaine/gitlab-secret-variable-bug/-/jobs/22575699

Output of checks

This bug happens on GitLab.com

Results of GitLab environment info

Expand for output related to GitLab environment info 
System information
System:		Ubuntu 16.04
Proxy:		no
Current User:	git
Using RVM:	no
Ruby Version:	2.3.3p222
Gem Version:	2.6.6
Bundler Version:1.13.7
Rake Version:	10.5.0
Redis Version:	3.2.5
Git Version:	2.13.0
Sidekiq Version:5.0.0
Go Version:	unknown


GitLab information
Version:	9.3.5-ee
Revision:	44d109d
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	postgresql
DB Version:	9.6.3
URL:		https://gitlab.datto.net
HTTP Clone URL:	https://gitlab.datto.net/some-group/some-project.git
SSH Clone URL:	git@gitlab.datto.net:some-group/some-project.git
Elasticsearch:	no
Geo:		no
Using LDAP:	yes
Using Omniauth:	no


GitLab Shell
Version:	5.1.1
Repository storage paths:
  • default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
Checking GitLab Shell ...


GitLab Shell version >= 5.1.1 ? ... OK (5.1.1)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:root, or git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
5/2 ... ok
8/4 ... ok
9/5 ... ok
15/7 ... ok
15/8 ... ok
12/9 ... ok
8/12 ... ok
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK
Send ping to redis server: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Sidekiq ...


Running? ... yes
Number of Sidekiq processes ... 1


Checking Sidekiq ... Finished


Checking Reply by email ...


Reply by email is disabled in config/gitlab.yml


Checking Reply by email ... Finished


Checking LDAP ...


Server: ldapmain
LDAP authentication... Success
LDAP users with access to your GitLab server (only showing the first 100 results)
[redacted]


Checking LDAP ... Finished


Checking GitLab ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
5/2 ... yes
8/4 ... yes
9/5 ... yes
15/7 ... yes
15/8 ... yes
12/9 ... yes
8/12 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.3.3 ? ... yes (2.3.3)
Git version >= 2.7.3 ? ... yes (2.13.0)
Active users: ... 16
Elasticsearch version 5.1 - 5.3? ... skipped (elasticsearch is disabled)


Checking GitLab ... Finished

Possible fixes

(If you can, link to the line of code that might be responsible for the problem)

Edited by 🤖 GitLab Bot 🤖