Virtualbox executor fails to prepare environment using bash shell and no dot files
Summary
Trying to run a virtualbox executor fails to prepare the environment. The shell is bash, so no POSIX-related env variables should be required as stated in issue threads related to launching a dash shell. The following dot files have been removed completely from the login user's home directory:
- .bash_logout
- .bashrc
- .profile
Steps to reproduce
.gitlab-ci.yml
.test-winmgrd:
stage: test
tags:
- sw-vbox
script:
- echo Hello world
test-winmgrd-buster:
extends: .test-winmgrd
image: vagrant_buster
Actual behavior
ERROR: Job failed: prepare environment: Process exited with status 1. Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information
Expected behavior
Virtualbox executor should spin up and print "Hello world"
Relevant logs and/or screenshots
job log
Apr 15 12:51:09 micro-02 gitlab-runner[682]: Preparing the "virtualbox" executor job=40627 project=5 runner=7wFgmKuyZ Apr 15 12:51:09 micro-02 gitlab-runner[682]: Shell configuration: command: bash
Apr 15 12:51:09 micro-02 gitlab-runner[682]: arguments: Apr 15 12:51:09 micro-02 gitlab-runner[682]: - -l
Apr 15 12:51:09 micro-02 gitlab-runner[682]: cmdline: bash -l Apr 15 12:51:09 micro-02 gitlab-runner[682]: dockercommand:
Apr 15 12:51:09 micro-02 gitlab-runner[682]: - sh Apr 15 12:51:09 micro-02 gitlab-runner[682]: - -c
Apr 15 12:51:09 micro-02 gitlab-runner[682]: - "if [ -x /usr/local/bin/bash ]; then\n\texec /usr/local/bin/bash -l\nelif [ -x /usr/bin/bash
Apr 15 12:51:09 micro-02 gitlab-runner[682]: ]; then\n\texec /usr/bin/bash -l\nelif [ -x /bin/bash ]; then\n\texec /bin/bash
Apr 15 12:51:09 micro-02 gitlab-runner[682]: -l\nelif [ -x /usr/local/bin/sh ]; then\n\texec /usr/local/bin/sh -l\nelif [ -x
Apr 15 12:51:09 micro-02 gitlab-runner[682]: /usr/bin/sh ]; then\n\texec /usr/bin/sh -l\nelif [ -x /bin/sh ]; then\n\texec /bin/sh
Apr 15 12:51:09 micro-02 gitlab-runner[682]: -l\nelif [ -x /busybox/sh ]; then\n\texec /busybox/sh -l\nelse\n\techo shell not
Apr 15 12:51:09 micro-02 gitlab-runner[682]: found\n\texit 1\nfi\n\n" Apr 15 12:51:09 micro-02 gitlab-runner[682]: passfile: false
Apr 15 12:51:09 micro-02 gitlab-runner[682]: extension: "" Apr 15 12:51:09 micro-02 gitlab-runner[682]: job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:09 micro-02 gitlab-runner[682]: Executing VBoxManageOutput: []string{"--version"} Apr 15 12:51:09 micro-02 gitlab-runner[682]: Using VirtualBox version 6.1.50_Ubuntur161033 executor... job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:09 micro-02 gitlab-runner[682]: Executing VBoxManageOutput: []string{"showvminfo", "vagrant_buster-runner-7wFgmKuyZ-concurrent-0"}
Apr 15 12:51:09 micro-02 gitlab-runner[682]: Restoring VM from snapshot... job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:09 micro-02 gitlab-runner[682]: Reverting VM to current snapshot... job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:09 micro-02 gitlab-runner[682]: Executing VBoxManageOutput: []string{"snapshot", "vagrant_buster-runner-7wFgmKuyZ-concurrent-0", "restorecurrent"}
Apr 15 12:51:09 micro-02 gitlab-runner[682]: Executing VBoxManageOutput: []string{"showvminfo", "vagrant_buster-runner-7wFgmKuyZ-concurrent-0"}
Apr 15 12:51:09 micro-02 gitlab-runner[682]: Checking VM status... job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:09 micro-02 gitlab-runner[682]: Executing VBoxManageOutput: []string{"showvminfo", "vagrant_buster-runner-7wFgmKuyZ-concurrent-0", "--machinereadable"}
Apr 15 12:51:10 micro-02 gitlab-runner[682]: Starting VM... job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:10 micro-02 gitlab-runner[682]: Executing VBoxManageOutput: []string{"startvm", "vagrant_buster-runner-7wFgmKuyZ-concurrent-0", "--type", "headless"}
Apr 15 12:51:10 micro-02 gitlab-runner[682]: Feeding runners to channel builds=1 max_builds=1
Apr 15 12:51:10 micro-02 gitlab-runner[682]: Feeding runner to channel builds=1 max_builds=1 runner=AgWZHdvMs
Apr 15 12:51:12 micro-02 gitlab-runner[682]: Appending trace to coordinator...ok code=202 job=40627 job-log=0-329 job-status=running runner=7wFgmKuyZ sent-log=0-328 status=202 Accepted update-interval=1m0s
Apr 15 12:51:13 micro-02 gitlab-runner[682]: Waiting for VM to run... job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:13 micro-02 gitlab-runner[682]: Executing VBoxManageOutput: []string{"showvminfo", "vagrant_buster-runner-7wFgmKuyZ-concurrent-0", "--machinereadable"}
Apr 15 12:51:13 micro-02 gitlab-runner[682]: Identify SSH Port... job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:13 micro-02 gitlab-runner[682]: Executing VBoxManageOutput: []string{"showvminfo", "vagrant_buster-runner-7wFgmKuyZ-concurrent-0"}
Apr 15 12:51:13 micro-02 gitlab-runner[682]: Waiting for VM to become responsive... job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:13 micro-02 gitlab-runner[682]: Connecting to SSH... job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:34 micro-02 gitlab-runner[682]: Starting SSH command... job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:34 micro-02 gitlab-runner[682]: Connecting to SSH server... job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:34 micro-02 gitlab-runner[682]: Waiting for signals... job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:34 micro-02 gitlab-runner[682]: No referees configured job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:34 micro-02 gitlab-runner[682]: Executing build stage build_stage=prepare_script job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:34 micro-02 gitlab-runner[682]: Preparing environment job=40627 project=5 runner=7wFgmKuyZ
Apr 15 12:51:34 micro-02 gitlab-runner[682]: Executing VBoxManageOutput: []string{"controlvm", "vagrant_buster-runner-7wFgmKuyZ-concurrent-0", "poweroff"}
Apr 15 12:51:35 micro-02 gitlab-runner[682]: WARNING: Job failed: prepare environment: Process exited with status 1. Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information
Apr 15 12:51:35 micro-02 gitlab-runner[682]: duration_s=25.982284216 job=40627 project=5 runner=7wFgmKuyZ
Environment description
config.toml contents
concurrent = 1
check_interval = 0
connection_max_age = "15m0s"
shutdown_timeout = 0
log_level = "debug"
[session_server]
session_timeout = 1800
[[runners]]
name = "micro-02-vbox"
url = "https://git.foobar.com"
id = 17
token = "foobar"
token_obtained_at = 2024-04-15T13:16:38Z
token_expires_at = 0001-01-01T00:00:00Z
executor = "virtualbox"
[runners.cache]
MaxUploadedArchiveSize = 0
[runners.ssh]
user = "vagrant"
password = "vagrant"
identity_file = "/home/gitlab-runner/.vagrant.d/insecure_private_key"
disable_strict_host_key_checking = true
[runners.virtualbox]
base_name = "vagrant_noble"
base_folder = ""
disable_snapshots = false
start_type = "headless"
allowed_images = [ "vagrant_*" ]
virtualbox version
# vboxmanage -v
6.1.50_Ubuntur161033
virtual machine environment
vagrant@testing-buster:~$ ls -Al
total 152
-rw------- 1 vagrant vagrant 1402 Apr 15 17:47 .bash_history
drwxr-xr-x 2 vagrant vagrant 4096 Apr 12 17:17 bin
drwx------ 5 vagrant vagrant 4096 Apr 15 14:17 .cache
drwxr-xr-x 7 vagrant vagrant 4096 Apr 15 14:27 .config
drwxr-xr-x 2 vagrant vagrant 4096 Apr 15 17:31 Desktop
drwxr-xr-x 2 vagrant vagrant 4096 Apr 15 14:16 Documents
drwxr-xr-x 2 vagrant vagrant 4096 Apr 15 14:16 Downloads
drwx------ 3 vagrant vagrant 4096 Apr 12 17:13 .gnupg
drwxr-xr-x 3 vagrant vagrant 4096 Apr 15 14:16 .local
drwxr-xr-x 2 vagrant vagrant 4096 Apr 15 14:16 Music
drwxr-xr-x 2 vagrant vagrant 4096 Apr 15 14:16 Pictures
drwxr-xr-x 2 vagrant vagrant 4096 Apr 15 14:16 Public
drwx------ 2 vagrant vagrant 4096 Apr 12 17:13 .ssh
drwxr-xr-x 2 vagrant vagrant 4096 Apr 15 14:16 Templates
-rw-r----- 1 vagrant vagrant 4 Apr 15 17:31 .vboxclient-clipboard.pid
-rw-r----- 1 vagrant vagrant 4 Apr 15 17:31 .vboxclient-display-svga-x11.pid
-rw-r----- 1 vagrant vagrant 4 Apr 15 17:31 .vboxclient-draganddrop.pid
-rw-r----- 1 vagrant vagrant 4 Apr 15 17:31 .vboxclient-seamless.pid
drwxr-xr-x 2 vagrant vagrant 4096 Apr 15 14:16 Videos
drwxr-xr-x 2 vagrant vagrant 4096 Apr 12 17:14 .vim
-rw------- 1 vagrant vagrant 8485 Apr 15 15:39 .viminfo
-rw-r--r-- 1 vagrant vagrant 13 Jan 10 03:53 .vimrc
-rw------- 1 vagrant vagrant 59 Apr 15 17:31 .Xauthority
-rw------- 1 vagrant vagrant 51649 Apr 15 17:31 .xsession-errors
vagrant@testing-buster:~$ uname -a
Linux testing-buster 4.19.0-26-amd64 #1 SMP Debian 4.19.304-1 (2024-01-09) x86_64 GNU/Linux
vagrant@testing-buster:~$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
vagrant@testing-buster:~$ ls -alh /home/gitlab-runner/
total 12K
drwxr-xr-x 2 gitlab-runner gitlab-runner 4.0K Apr 15 15:21 .
drwxr-xr-x 4 root root 4.0K Apr 12 17:16 ..
-rw------- 1 gitlab-runner gitlab-runner 22 Apr 15 15:21 .bash_history
Used GitLab Runner version
# gitlab-runner -v
Version: 16.10.0
Git revision: 81ab07f6
Git branch: 16-10-stable
GO version: go1.21.7
Built: 2024-03-21T19:43:25+0000
OS/Arch: linux/amd64
Possible fixes
I tried tracing through the code to find what's actually happening when the environment is prepared, but I wound up at what looks like a no-op that I assume gets overridden somewhere I haven't located: https://gitlab.com/gitlab-org/gitlab-runner/-/blob/main/shells/abstract.go?ref_type=heads#L262