Pass Powershell scripts as a file to shell executor
What does this MR do?
This MR fixes the regressions introduced in !2715 (merged) by wrapping scripts passed through STDIN in a Powershell code block.
Why was this MR needed?
There is a bug in Powershell where passing a script through STDIN exhibits a different behavior than passing the same script through a file. Essentially, the script is executed line by line with each line executed in a new pipeline, like in a REPL. This causes errors to not cause the script to exit.
What's the best way to test this MR?
Note: This MR has been shown to be working here: #27830 (comment 568852363)
-
Run integration test in vagrant:
$ vagrant up && vagrant sshC:\> powershellTest that new code passes the test
✅ PS C:\GitLab-Runner> go test -v -run TestBuildPwshHandlesSyntaxErrors -count 1 .\executors\shell\shell_integration_test.go Compiling gitlab-runner binary for tests Executing: C:\Go\bin\go.exe build -o C:\Users\vagrant\AppData\Local\Temp\test_executor791640591\gitlab-runner-integration.exe ../.. === RUN TestBuildPwshHandlesSyntaxErrors --- PASS: TestBuildPwshHandlesSyntaxErrors (4.33s) shell_integration_test.go:104: Build directory: C:\Users\vagrant\AppData\Local\Temp\gitlab-runner-shell-executor-test012291106 test.go:24: Running with gitlab-runner development version (HEAD) Preparing the "shell-integration-test" executor Using Shell executor... Preparing environment Running on VAGRANT-2019... Getting source from Git repository Fetching changes... Initialized empty Git repository in C:/Users/vagrant/AppData/Local/Temp/gitlab-runner-shell-executor-test012291106/0/project-0/.git/ Created fresh repository. Checking out 91956efe as master... git-lfs/2.8.0 (GitHub; windows amd64; go 1.12.2; git 30af66bb) Skipping Git submodules setup Executing "step_script" stage of the job script $ some syntax error # collapsed multi-line command some: Line | 59 | some syntax error | ~~~~ | The term 'some' is not recognized as a name of a cmdlet, function, script file, or executable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. ERROR: Job failed: exit status 1 PASS ok command-line-arguments 15.152sTest that old code fails the test
✅ PS C:\GitLab-Runner> go test -v -run TestBuildPwshHandlesSyntaxErrors -count 1 .\executors\shell\shell_integration_test.go Compiling gitlab-runner binary for tests Executing: C:\Go\bin\go.exe build -o C:\Users\vagrant\AppData\Local\Temp\test_executor796351435\gitlab-runner-integration.exe ../.. === RUN TestBuildPwshHandlesSyntaxErrors --- FAIL: TestBuildPwshHandlesSyntaxErrors (5.11s) shell_integration_test.go:104: Build directory: C:\Users\vagrant\AppData\Local\Temp\gitlab-runner-shell-executor-test093651630 test.go:24: Running with gitlab-runner development version (HEAD) Preparing the "shell-integration-test" executor Using Shell executor... Preparing environment Running on VAGRANT-2019... Getting source from Git repository Fetching changes... Initialized empty Git repository in C:/Users/vagrant/AppData/Local/Temp/gitlab-runner-shell-executor-test093651630/0/project-0/.git/ Created fresh repository. Checking out 91956efe as master... git-lfs/2.8.0 (GitHub; windows amd64; go 1.12.2; git 30af66bb) Skipping Git submodules setup Executing "step_script" stage of the job script $ some syntax error # collapsed multi-line command some: The term 'some' is not recognized as a name of a cmdlet, function, script file, or executable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. Name Value ---- ----- PSVersion 7.1.1 PSEdition Core GitCommitId 7.1.1 OS Microsoft Windows 10.0.17763 Platform Win32NT PSCompatibleVersions {1.0, 2.0, 3.0, 4.0�} PSRemotingProtocolVersion 2.3 SerializationVersion 1.1.0.1 WSManStackVersion 3.0 Job succeeded shell_integration_test.go:1435: Error Trace: shell_integration_test.go:1435 Error: An error is expected but got nil. Test: TestBuildPwshHandlesSyntaxErrors shell_integration_test.go:1436: Error Trace: shell_integration_test.go:1436 Error: "Running with gitlab-runner development version (HEAD) Preparing the "shell-integration-test" executor Using Shell executor... Preparing environment Running on VAGRANT-2019... Getting source from Git repository Fetching changes... Initialized empty Git repository in C:/Users/vagrant/AppData/Local/Temp/gitlab-runner-shell-executor-test093651630/0/project-0/.git/ Created fresh repository. Checking out 91956efe as master... git-lfs/2.8.0 (GitHub; windows amd64; go 1.12.2; git 30af66bb) Skipping Git submodules setup Executing "step_script" stage of the job script $ some syntax error # collapsed multi-line command some: The term 'some' is not recognized as a name of a cmdlet, function, script file, or executable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. Name Value ---- ----- PSVersion 7.1.1 PSEdition Core GitCommitId 7.1.1 OS Microsoft Windows 10.0.17763 Platform Win32NT PSCompatibleVersions {1.0, 2.0, 3.0, 4.0�} PSRemotingProtocolVersion 2.3 SerializationVersion 1.1.0.1 WSManStackVersion 3.0 Job succeeded " should not contain "PSEdition" Test: TestBuildPwshHandlesSyntaxErrors FAIL FAIL command-line-arguments 18.476s FAIL -
Run build in cloud VM
-
Create a Windows VM in GCP from the
windows-server-1909-dc-core-for-containers-v20210413base image -
Download the runner executable to it
-
Register the
docker-windowsandshellexecutors as seen in theconfig.toml:config.toml
concurrent = 1 check_interval = 0 [session_server] session_timeout = 1800 [[runners]] name = "Windows pwsh Docker executor" url = "https://gitlab.com" executor = "docker-windows" shell = "pwsh" [runners.custom_build_dir] [runners.cache] [runners.cache.s3] [runners.cache.gcs] [runners.cache.azure] [runners.docker] tls_verify = false image = "mcr.microsoft.com/powershell" privileged = false disable_entrypoint_overwrite = false oom_kill_disable = false disable_cache = false volumes = ["c:\\cache"] shm_size = 0 [[runners]] name = "Windows powershell shell executor" url = "https://gitlab.com/" executor = "shell" shell = "pwsh" [runners.custom_build_dir] [runners.cache] [runners.cache.s3] [runners.cache.gcs] [runners.cache.azure] -
Run a job with a known error in it. It should result in an error and fail fast. The regression causes it to continue execution on the next line and ultimately succeed.
.gitlab-ci.yml
start_evaluation: script: | $ErrorActionPreference = 'Stop' just bogus Write-Host 'hello' -
Run the job against each of the registered executors
In
master, we can see that the build succeeds even though it shouldn't:
With the runner built from this branch, we can see the correct behavior:
-
What are the relevant issue numbers?
Closes #27830 (closed) #27842 (closed)
Edited by Pedro Pombeiro