Powershell runner ignores ParserErrors

When using windows runner in docker

If your build script contains commands with PowerShell operators like if or while and you put a mistake in the command line. PowerShell will print out the error, but return True

Steps to reproduce

create a job for windows runner with the following script in .gitlab-ci.yml

.gitlab-ci.yml

  script:
    - New-Item -ItemType Directory -Force -Path build
    - Set-Location -Path build
    - if ($TEST_VAR) { () }
    - echo "WE DID NOT FAIL"

Actual behavior

PS C:\docker> if ($TEST_VAR) { () }
At line:1 char:19
+ if ($TEST_VAR) { () }
+                   ~
An expression was expected after '('.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : ExpectedExpression

PS C:\docker> echo $?
True

And Job succeed.

Expected behavior

Expected job failure after ParserError.

Relevant logs and/or screenshots

job log

PS C:\builds\project1> echo "`$ New-Item -ItemType Directory -Force -Path build"
$ New-Item -ItemType Directory -Force -Path build
PS C:\builds\project1> New-Item -ItemType Directory -Force -Path build


    Directory: C:\builds\project1


Mode                LastWriteTime         Length Name                          
----                -------------         ------ ----                          
d-----        1/23/2020  12:45 AM                build                         


PS C:\builds\project1> if(!$?) { Exit &{if($LASTEXITCODE) {$LASTEXITCODE} else {1}} }
PS C:\builds\project1> 
PS C:\builds\project1> echo "`$ Set-Location -Path build"
$ Set-Location -Path build
PS C:\builds\project1> Set-Location -Path build
PS C:\builds\project1\build> if(!$?) { Exit &{if($LASTEXITCODE) {$LASTEXITCODE} else {1}} }
PS C:\builds\project1\build> 


PS C:\builds\project1\build> echo "`$ if ($TEST_VAR) { () }"
$ if ($TEST_VAR) { () }
PS C:\builds\project1\build> if ($TEST_VAR) { () }
At line:1 char:19
+ if ($TEST_VAR) { () }
+                   ~
An expression was expected after '('.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : ExpectedExpression
PS C:\builds\project1\build> if(!$?) { Exit &{if($LASTEXITCODE) {$LASTEXITCODE} else {1}} }
PS C:\builds\project1\build>
PS C:\builds\project1\build> echo "`$ echo `"WE DID NOT FAIL`""
$ echo "WE DID NOT FAIL"
PS C:\builds\project1\build> echo "WE DID NOT FAIL"
WE DID NOT FAIL

Environment description

We are using Runners with executor docker-windows on windows server 2019 with native containers

config.toml contents

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "Docker-WIN-XXXXXYYYZZZ-2"
  url = "https://gitlab.domain.tld/"
  token = "XXXXXXXXXXXXXXXXXXXX"
  executor = "docker-windows"
  shell = "powershell"
  [runners.custom_build_dir]
  [runners.docker]
    tls_verify = false
    image = "mcr.microsoft.com/windows/servercore:ltsc2019"
    cpus = "8.0"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["c:\\cache"]
    shm_size = 0
  [runners.cache]

Used GitLab Runner version

Running with gitlab-runner 12.6.0 (ac8e767a)
  on Docker-WIN-XXXXXYYYZZZ-2 Hz34LWND
Using Docker executor with image windows:2019.12.26 ...

Possible fixes

We have found the solution to catch ParserError by using $Error

$Error – collection (ArrayList to be specific) of errors that have occurred in the current session. Errors are always inserted at the beginning of the collection. As a result, the most recent error is always located at index 0.

Possible fix is to https://gitlab.com/gitlab-org/gitlab-runner/blob/master/shells/powershell.go#L71

func (b *PsWriter) checkErrorLevel() {
 b.Line("if(!$?) { Exit &{if($LASTEXITCODE) {$LASTEXITCODE} else {1}} }")
 b.Line("")
 b.Line("if($Error.Count -ne 0) { Echo $Error; Exit 1 }")
 b.Line("")
}