Skip to content

Enable PowerShell Core support in Shell Executor

Pedro Pombeiro requested to merge 13134-powershell-core-shell-executor into master

What does this MR do?

Why was this MR needed?

See #13134 (closed)

What's the best way to test this MR?

  1. Register and run a runner with the executable from this MR (Windows/Mac/Linux) with the following lines in the config.toml:
      executor = "shell"
      shell  = "pwsh"
  2. Run a job with a Powershell Core script, e.g.:
    stages:
      - setup
    
    start_evaluation:
      stage: setup
      script:
        - "\"Date and time is: $((Get-Date).ToString())\" | Out-File -FilePath .date"
        - echo Done
      artifacts:
        paths:
          - .date

Scripts on Linux/Mac are generated with \n line breaks and / path separators, while on Windows they are generated with \r\n line breaks and \ path separators.

What are the relevant issue numbers?

Closes #13134 (closed)

Notes for the Reviewer

  • Assuming that we're not supporting interactive terminal for pwsh on Linux/Mac, do we have to do something to disable support for it explicitly?
  • I've added Powershell Core to the CI image so that we can run tests that exercise pwsh.

Other notes

I had to change to require.NoError(t, err) in TestInteractiveTerminal since the tests passed when running on a local machine, but not when run inside a Docker container (e.g. CI). When on CI, I'd see the following panic (only for pwsh, not for bash):

Log
time="2020-06-09T16:46:41Z" level=error msg="Interactive terminal not set" uri=/session/406515c2e26280ca9443a316a51dd4739c1abf0ea8b526faa539df170dfa1fa9/exec
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0xc8 pc=0x863396]

goroutine 11 [running]:
testing.tRunner.func1(0xc0000cd600)
        /usr/local/go/src/testing/testing.go:874 +0x3a3
panic(0xd29fa0, 0x1549ad0)
        /usr/local/go/src/runtime/panic.go:679 +0x1b2
github.com/gorilla/websocket.(*Conn).NextReader(0x0, 0xc0003901b0, 0xc00035745b, 0xcdd5e0, 0xc0003901b0, 0xcdd5e0)
        /go/pkg/mod/github.com/gorilla/websocket@v1.4.0/conn.go:938 +0x26
github.com/gorilla/websocket.(*Conn).ReadMessage(0x0, 0xc0000cd600, 0xcdb860, 0xc000357440, 0xcdb860, 0xc000357448, 0x0)
        /go/pkg/mod/github.com/gorilla/websocket@v1.4.0/conn.go:1028 +0x2f
gitlab.com/gitlab-org/gitlab-runner/executors/shell_test.TestInteractiveTerminal.func1(0xc0000cd600)
        /go/src/executors/shell/shell_integration_test.go:1169 +0x742
testing.tRunner(0xc0000cd600, 0xc0002f9b30)
        /usr/local/go/src/testing/testing.go:909 +0xc9
created by testing.(*T).Run
        /usr/local/go/src/testing/testing.go:960 +0x350
FAIL    gitlab.com/gitlab-org/gitlab-runner/executors/shell     0.803s

If stopping on the err, the error scenario becomes much clearer:

        shell_integration_test.go:1158: 
                Error Trace:    shell_integration_test.go:1158
                Error:          Received unexpected error:
                                websocket: bad handshake
                Test:           TestInteractiveTerminal/pwsh
FAIL

So even though we're not supporting interactive terminal for this MR, I think that the slight changes in the test are valuable for future debugging sessions in this test.

Notes for future work on interactive terminal: After some digging, there are two problems:

  1. os.Getwd is hanging for some reason, only when running inside a Docker container.

  2. Terminal proxying is failing during copy, again only on the Docker container:

    time="2020-06-10T08:21:35Z" level=info msg="Terminal: started proxying" clientAddr="127.0.0.1:33804" serverAddr=shell
    time="2020-06-10T08:21:35Z" level=info msg="Terminal: error proxying" clientAddr="127.0.0.1:33804" error="copying from shell to 127.0.0.1:33804: read /dev/ptmx: input/output error" serverAddr=shell
    time="2020-06-10T08:21:35Z" level=info msg="Terminal: finished proxying" clientAddr="127.0.0.1:33804" serverAddr=shell

    source: Terminal: error proxying

    source: copying from shell to 127.0.0.1:33804: read /dev/ptmx: input/output error

Edited by Georgi N. Georgiev

Merge request reports