"ssh: handshake failed: knownhosts: key is unknown" error for VirtualBox executor

Summary

#28192 (closed) introduces disable_strict_host_key_checking = false by default. The change is documented here:

https://docs.gitlab.com/runner/executors/ssh.html#security

The documentation is misleading at best:

  • That change does not impact only the SSH executor, but also the VirtualBox executor (since ssh is used to connect to the VM).
  • The impact is not documented for the VirtualBox executor.
  • Yet the change to disable_strict_host_key_checking = false by default causes the following error:
ssh Dial() error: ssh: handshake failed: knownhosts: key is unknown

While I fully understand this change for the SSH executor, it is both strange and unexpectedly complex to have strict host key checking enabled by default for the VirtualBox executor:

  1. If a man in the middle attack is possible between the host and the VM, doesn't it mean the attacker already has access to the host?
  2. How can we add VMs that do not already exist to the SSH known hosts? They are most likely solutions, but it sounds rather cumbersome considering the potential benefits (see 1.).

Steps to reproduce

Assuming the virtualbox tag is set on a runner with the VirtualBox executor:

.gitlab-ci.yml 
test:
    tags:
        - virtualbox
    script:
        - echo 'OK!'

Actual behavior

Expected behavior

The job is successful and prints OK!.

Relevant logs and/or screenshots

Job logs:

job log 
Running with gitlab-runner 15.0.0 (febb2a09)
  on build-linux.aerys.in (VirtualBox, Ubuntu 18.04) 63s6ncCB
Preparing the "virtualbox" executor
Using VirtualBox version 6.1.8r137981 executor...
Creating new VM...
ERROR: Preparation failed: ssh Dial() error: ssh: handshake failed: knownhosts: key is unknown
Will be retried in 3s ...
Using VirtualBox version 6.1.8r137981 executor...
Creating new VM...
ERROR: Preparation failed: ssh Dial() error: ssh: handshake failed: knownhosts: key is unknown
Will be retried in 3s ...
Using VirtualBox version 6.1.8r137981 executor...
Creating new VM...
ERROR: Preparation failed: ssh Dial() error: ssh: handshake failed: knownhosts: key is unknown
Will be retried in 3s ...
ERROR: Job failed (system failure): ssh Dial() error: ssh: handshake failed: knownhosts: key is unknown

Environment description

  • GitLab Runner 15.0
  • Self-hosted
  • VirtualBox (6.1.8r137981) executor (Ubuntu 18.04 guest and host)
config.toml contents

Used GitLab Runner version

Version:      15.0.0
Git revision: febb2a09
Git branch:   15-0-stable
GO version:   go1.17.7
Built:        2022-05-19T19:34:08+0000
OS/Arch:      linux/amd64

Possible fixes

Set disable_strict_host_key_checking = true on the runner using the VirtualBox executor.