When runner on Linux and using virtualbox for win，powershell.Absolute is wrong.
Summary
When host OS is linux, executor=virtualbox, VM geust OS is win, and use powershell,
the powershell.go:Absolute() handle the Windows style absolute path is wrong.
reference to this link for configuration
I use gitlab-runner on windows, using virtualbox for guest win.
set runners.builds_dir in config.toml, like so:
config.toml
[[runners]]
executor = "virtualbox"
builds_dir = "C:\\Users\\ci\\builds"
shell = "pwsh".gitlab-ci.yml
echo $CI_PROJECT_DIRis work fine, the job log:
$ echo $CI_PROJECT_DIR
C:/Users/ci/builds/aozima/test_ciSteps to reproduce
I want migrate the runner to linux.
Environment description
The environment
host system:
$ cat /etc/issue
Ubuntu 22.04.2 LTS
gitlab-runner vision:
$ gitlab-runner -v
Version: 16.1.0
Git revision: b72e108d
Git branch: 16-1-stable
GO version: go1.19.9
Built: 2023-06-21T21:52:30+0000
OS/Arch: linux/amd64
executor:
$ virtualbox --help
Oracle VM VirtualBox VM Selector v7.0.8
VM geust OS:
win10At first, I didn't set runners.builds_dir in config.toml.
Get CI_PROJECT_DIR not absolute
$ echo $CI_PROJECT_DIR
builds/aozima/test_ciNext, I set runners.builds_dir in config.toml, like before on win:
config.toml
[[runners]]
executor = "virtualbox"
builds_dir = "C:\\Users\\ci\\builds"
shell = "pwsh"but failed to run, when I set variables CI_DEBUG_TRACE=true, the job log:
DEBUG: 69+ >>>> New-Item -ItemType directory -Force -Path "C:/Users/ci/builds/aozima/test_ci.tmp" | out-null
DEBUG: 70+ >>>> [System.IO.File]::WriteAllText("$CurrentDirectory/C:/Users/ci/builds/aozima/test_ci.tmp/CI_SERVER_TLS_CA_FILE", "-----BEGIN CERTIF*******==`n-----END CERTIFICATE-----")
ParentContainsErrorRecordException:
Line |
70 | [System.IO.File]::WriteAllText("$CurrentDirectory/C:/Users/ci/builds/ …
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Exception calling "WriteAllText" with "2" argument(s): "文件名、目录名或卷标语法不正确。 : 'C:\Users\ci\C:\Users\ci\builds\aozima\test_ci.tmp\CI_SERVER_TLS_CA_FILE'"
Cleaning up project directory and file based variables
00:02
DEBUG: 3+ >>>> $ErrorActionPreference = "Stop"
DEBUG: 4+ >>>> $CurrentDirectory = (Resolve-Path ./).Path
DEBUG: 5+ if( >>>> (Get-Command -Name Remove-Item2 -Module NTFSSecurity -ErrorAction SilentlyContinue) -and (Test-Path "$CurrentDirectory/C:/Users/ci/builds/aozima/test_ci.tmp/CI_SERVER_TLS_CA_FILE" -PathType Leaf) ) {
DEBUG: 7+ } elseif( >>>> Test-Path "$CurrentDirectory/C:/Users/ci/builds/aozima/test_ci.tmp/CI_SERVER_TLS_CA_FILE") {
DEBUG: 10+ >>>> }
ERROR: Job failed: Process exited with status 1root case:
WriteAllText("$CurrentDirectory/C:/UsersRead the code, I think the bug happened at shells/powershell.go:Absolute(),
so I add some log for test
func (p *PsWriter) Absolute(dir string) string {
logrus.Infoln("shells/powershell.go Absolute: dir=[", dir, "].")
if p.resolvePaths {
logrus.Infoln("shells/powershell.go Absolute: [if p.resolvePaths == true]")
return dir
}
if filepath.IsAbs(dir) {
logrus.Infoln("shells/powershell.go Absolute: [if filepath.IsAbs(dir) == true]")
return dir
}
logrus.Infoln("shells/powershell.go Absolute: p.Linef=[", string(os.PathSeparator), "].")
logrus.Infoln("shells/powershell.go Absolute: p.Join=[", p.Join("$CurrentDirectory", dir), "].")
p.Linef("$CurrentDirectory = (Resolve-Path .%s).Path", string(os.PathSeparator))
return p.Join("$CurrentDirectory", dir)
}get the runner logs:
shells/powershell.go Absolute: dir=[ C:/Users/ci/builds/aozima/test_ci.tmp/CI_SERVER_TLS_CA_FILE ].
shells/powershell.go Absolute: p.Linef=[ / ].
shells/powershell.go Absolute: p.Join=[ $CurrentDirectory/C:/Users/ci/builds/aozima/test_ci.tmp/CI_SERVER_TLS_CA_FILE ]. oops, filepath.IsAbs("C:/Users...") return false.
I wrote a examples of test
main.go
package main
import (
"fmt"
"runtime"
"path/filepath"
)
func main() {
fmt.Println("Hello World")
path := "C:/Users/ci/builds/aozima/test_ci.tmp/CI_SERVER_TLS_CA_FILE"
println(runtime.GOOS)
println(filepath.IsAbs(path)) // windows=true, linux=false
}Test on windows and linux, get the diff result...
>.\test.exe
windows
true
$ ./test.elf
linux
falsePossible fixes
I read the go source code, the IsAbs implement for different platforms,
https://cs.opensource.google/go/go/+/refs/tags/go1.20.5:src/path/filepath/path_windows.go
My host OS is linux, but the guest OS is windows, problem happened... Because powershell only(confirm?) use for windows, so I patch for powershell.go
diff --git a/shells/powershell.go b/shells/powershell.go
index 12675d7d9..8645afebd 100644
--- a/shells/powershell.go
+++ b/shells/powershell.go
@@ -11,6 +11,7 @@ import (
"runtime"
"strings"
+ "github.com/sirupsen/logrus"
"gitlab.com/gitlab-org/gitlab-runner/common"
"gitlab.com/gitlab-org/gitlab-runner/helpers"
"gitlab.com/gitlab-org/gitlab-runner/helpers/featureflags"
@@ -446,15 +447,105 @@ func (p *PsWriter) EmptyLine() {
p.Line(`echo ""`)
}
+func isSlash(c uint8) bool {
+ return c == '\\' || c == '/'
+}
+
+func toUpper(c byte) byte {
+ if 'a' <= c && c <= 'z' {
+ return c - ('a' - 'A')
+ }
+ return c
+}
+
+// cutPath slices path around the first path separator.
+func cutPath(path string) (before, after string, found bool) {
+ for i := range path {
+ if isSlash(path[i]) {
+ return path[:i], path[i+1:], true
+ }
+ }
+ return path, "", false
+}
+
+// volumeNameLen returns length of the leading volume name on Windows.
+// It returns 0 elsewhere.
+//
+// See: https://learn.microsoft.com/en-us/dotnet/standard/io/file-path-formats
+func volumeNameLen(path string) int {
+ if len(path) < 2 {
+ return 0
+ }
+ // with drive letter
+ c := path[0]
+ if path[1] == ':' && ('a' <= c && c <= 'z' || 'A' <= c && c <= 'Z') {
+ return 2
+ }
+ // UNC and DOS device paths start with two slashes.
+ if !isSlash(path[0]) || !isSlash(path[1]) {
+ return 0
+ }
+ rest := path[2:]
+ p1, rest, _ := cutPath(rest)
+ p2, rest, ok := cutPath(rest)
+ if !ok {
+ return len(path)
+ }
+ if p1 != "." && p1 != "?" {
+ // This is a UNC path: \\${HOST}\${SHARE}\
+ return len(path) - len(rest) - 1
+ }
+ // This is a DOS device path.
+ if len(p2) == 3 && toUpper(p2[0]) == 'U' && toUpper(p2[1]) == 'N' && toUpper(p2[2]) == 'C' {
+ // This is a DOS device path that links to a UNC: \\.\UNC\${HOST}\${SHARE}\
+ _, rest, _ = cutPath(rest) // host
+ _, rest, ok = cutPath(rest) // share
+ if !ok {
+ return len(path)
+ }
+ }
+ return len(path) - len(rest) - 1
+}
+
+// IsAbs reports whether the path is absolute.
+func IsAbs(path string) (b bool) {
+ l := volumeNameLen(path)
+ if l == 0 {
+ return false
+ }
+ // If the volume name starts with a double slash, this is an absolute path.
+ if isSlash(path[0]) && isSlash(path[1]) {
+ return true
+ }
+ path = path[l:]
+ if path == "" {
+ return false
+ }
+ return isSlash(path[0])
+}
+
func (p *PsWriter) Absolute(dir string) string {
+ logrus.Infoln("shells/powershell.go Absolute: dir=[", dir, "].")
if p.resolvePaths {
+ logrus.Infoln("shells/powershell.go Absolute: [if p.resolvePaths == true]")
return dir
}
if filepath.IsAbs(dir) {
+ logrus.Infoln("shells/powershell.go Absolute: [if filepath.IsAbs(dir) == true]")
return dir
}
+ if runtime.GOOS != "windows" {
+ if IsAbs(dir) {
+ logrus.Infoln("shells/powershell.go Absolute: [not windows, but path IsAbs]")
+ return dir
+ }
+ }
+
+ logrus.Infoln("shells/powershell.go Absolute: p.Linef=[", string(os.PathSeparator), "].")
+ logrus.Infoln("shells/powershell.go Absolute: p.Join=[", p.Join("$CurrentDirectory", dir), "].")
+
p.Linef("$CurrentDirectory = (Resolve-Path .%s).Path", string(os.PathSeparator))
return p.Join("$CurrentDirectory", dir)
}It't works!! the new log
shells/powershell.go Absolute: dir=[ C:/Users/ci/builds/aozima/test_ci.tmp/CI_SERVER_TLS_CA_FILE ].
shells/powershell.go Absolute: [if filepath.IsAbs(dir) == true]
shells/powershell.go Absolute: dir=[ C:/Users/ci/builds/aozima/test_ci.tmp ]. Is here has better solution?
Edited by aozima