Skip to content

Add powershell support for Docker Executor

Steve Xuereb requested to merge 3896-update-docker-executor-default-shell into master

What does this MR do?

Add PowerShell support for Docker executor

Why was this MR needed?

&535 (closed) is the epic to track Windows Containers for docker executor.

How to test

It's quite hard to test it since windows volumes are not supported yet, so we need to hack around this so we can test this. There is update-docker-executor-default-shell-wrong-volumes that is identical to this branch just has an extra commit 4ce0a8d0 that just changes how the build dir is created so it's PowerShell can clone the repo inside of the container.

Also, notice all configuration below [runnres.docker].volumes is empty on purpose since windows volumes are not supported yet.

config.toml 
 [[runners]]
  name = "windows-docker-2019"
  url = "xxxx"
  token = "xxx"
   executor = "docker-windows"
   [runners.docker]
     tls_verify = false
     image = "alpine:3.9"
     privileged = false
     disable_entrypoint_overwrite = false
     oom_kill_disable = false
     disable_cache = false
     shm_size = 0
   [runners.cache]
     [runners.cache.s3]
     [runners.cache.gcs]
simple `.gitlab-ci.yml` 
image: mcr.microsoft.com/windows/servercore:1809-amd64

stages:
- test

test:
  stage: test
  variables:
    ENV: "Test23"
  script:
  - echo $Env:GITLAB_USER_NAME
  - echo $Env:ENV
  - dir
dotnet core proejct

Clone https://gitlab.com/steveazz/playground/tree/windows-docker-executor and run it using the config below:

[[runners]]
  name = "windows-docker-2019"
  url = "xxxx"
  token = "xxx"
   executor = "docker-windows"
   [runners.docker]
     tls_verify = false
     image = "alpine:3.9"
     privileged = false
     disable_entrypoint_overwrite = false
     oom_kill_disable = false
     disable_cache = false
     shm_size = 0
   [runners.cache]
     [runners.cache.s3]
     [runners.cache.gcs]

Manual QA

Running a pipeline for https://gitlab.com/steveazz/playground/tree/windows-docker-executor on a local GDK installation with this MR checked out:

build job 
Running with gitlab-runner 11.11.0~beta.1343.g302a7708 (302a7708)
  on windows-docker-2019 vGvfck1g
Using Docker executor with image mcr.microsoft.com/windows/servercore:1809 ...
Pulling docker image mcr.microsoft.com/windows/servercore:1809 ...
Using docker image sha256:29a2c2cb7e4d89c2dc7728e1d675414857b0fe2b009287fa737a39dac99a60d4 for mcr.microsoft.com/windows/servercore:1809 ...
Running on RUNNER-VGVFCK1G via 
vagrant-2019...
Reinitialized existing Git repository in C:/builds/root/ci-scratch-pad/.git/
Fetching changes...
From http://192.168.1.79:3000/root/ci-scratch-pad
   980bcb9..0caec4a  windows-docker-executor -> origin/windows-docker-executor
Checking out 0caec4a3 as windows-docker-executor...
Removing .nuget/
Removing bin/Debug/netcoreapp2.1/
Removing dotnet-install.ps1
Removing obj/Debug/netcoreapp2.1/
git-lfs/2.7.1 (GitHub; windows amd64; go 1.11.5; git 6b7fb6e3)
Version:      11.9.0~beta.1139.g08121fc8
Git revision: 08121fc8
Git branch:   3755-helper-image-for-windows-container-execution-publish-windows-images
GO version:   go1.12
Built:        2019-03-04T16:34:20+0000
OS/Arch:      windows/amd64
$ Invoke-Webrequest "https://dot.net/v1/dotnet-install.ps1" -OutFile dotnet-install.ps1 -UseBasicParsing
$ .\dotnet-install.ps1 -Version 2.2.203
dotnet-install: Downloading link: https://dotnetcli.azureedge.net/dotnet/Sdk/2.2.203/dotnet-sdk-2.2.203-win-x64.zip
dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/Sdk/2.2.203/dotnet-sdk-2.2.203-win-x64.zip
dotnet-install: Adding to current process PATH: "C:\Users\ContainerAdministrator\AppData\Local\Microsoft\dotnet\". Note: This change will not be visible if PowerShell was run as a child process.
dotnet-install: Installation finished
$ dotnet restore --packages $NUGET_PACKAGES_DIRECTORY

Welcome to .NET Core!
---------------------
Learn more about .NET Core: https://aka.ms/dotnet-docs
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli-docs

Telemetry
---------
The .NET Core tools collect usage data in order to help us improve your experience. The data is anonymous and doesn't include command-line arguments. The data is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.

Read more about .NET Core CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry

Configuring...
--------------
A command is running to populate your local package cache to improve restore speed and enable offline access. This command takes up to one minute to complete and only runs once.
Decompressing .......... 7358 ms
Expanding .......... 28307 ms

ASP.NET Core
------------
Successfully installed the ASP.NET Core HTTPS Development Certificate.
To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only). For establishing trust on other platforms refer to the platform specific documentation.
For more information on configuring HTTPS see https://go.microsoft.com/fwlink/?linkid=848054.
  Restore completed in 1.13 sec for C:\builds\root\ci-scratch-pad\app.csproj.
$ dotnet build --no-restore
Microsoft (R) Build Engine version 16.0.450+ga8dc7f1d34 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  app -> C:\builds\root\ci-scratch-pad\bin\Debug\netcoreapp2.2\app.dll
  app -> C:\builds\root\ci-scratch-pad\bin\Debug\netcoreapp2.2\app.Views.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:07.55
Version:      11.9.0~beta.1139.g08121fc8
Git revision: 08121fc8
Git branch:   3755-helper-image-for-windows-container-execution-publish-windows-images
GO version:   go1.12
Built:        2019-03-04T16:34:20+0000
OS/Arch:      windows/amd64
Job succeeded
test job 
Running with gitlab-runner 11.11.0~beta.1343.g302a7708 (302a7708)
  on windows-docker-2019 vGvfck1g
Using Docker executor with image mcr.microsoft.com/windows/servercore:1809 ...
Pulling docker image mcr.microsoft.com/windows/servercore:1809 ...
Using docker image sha256:29a2c2cb7e4d89c2dc7728e1d675414857b0fe2b009287fa737a39dac99a60d4 for mcr.microsoft.com/windows/servercore:1809 ...
Running on RUNNER-VGVFCK1G via 
vagrant-2019...
Reinitialized existing Git repository in C:/builds/root/ci-scratch-pad/.git/
Fetching changes...
Checking out 0caec4a3 as windows-docker-executor...
Removing dotnet-install.ps1
git-lfs/2.7.1 (GitHub; windows amd64; go 1.11.5; git 6b7fb6e3)
Version:      11.9.0~beta.1139.g08121fc8
Git revision: 08121fc8
Git branch:   3755-helper-image-for-windows-container-execution-publish-windows-images
GO version:   go1.12
Built:        2019-03-04T16:34:20+0000
OS/Arch:      windows/amd64
$ Invoke-Webrequest "https://dot.net/v1/dotnet-install.ps1" -OutFile dotnet-install.ps1 -UseBasicParsing
$ .\dotnet-install.ps1 -Version 2.2.203
dotnet-install: Downloading link: https://dotnetcli.azureedge.net/dotnet/Sdk/2.2.203/dotnet-sdk-2.2.203-win-x64.zip
dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/Sdk/2.2.203/dotnet-sdk-2.2.203-win-x64.zip
dotnet-install: Adding to current process PATH: "C:\Users\ContainerAdministrator\AppData\Local\Microsoft\dotnet\". Note: This change will not be visible if PowerShell was run as a child process.
dotnet-install: Installation finished
$ dotnet restore --packages $NUGET_PACKAGES_DIRECTORY

Welcome to .NET Core!
---------------------
Learn more about .NET Core: https://aka.ms/dotnet-docs
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli-docs

Telemetry
---------
The .NET Core tools collect usage data in order to help us improve your experience. The data is anonymous and doesn't include command-line arguments. The data is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.

Read more about .NET Core CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry

Configuring...
--------------
A command is running to populate your local package cache to improve restore speed and enable offline access. This command takes up to one minute to complete and only runs once.
Decompressing .......... 7310 ms
Expanding .......... 26894 ms

ASP.NET Core
------------
Successfully installed the ASP.NET Core HTTPS Development Certificate.
To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only). For establishing trust on other platforms refer to the platform specific documentation.
For more information on configuring HTTPS see https://go.microsoft.com/fwlink/?linkid=848054.
  Restore completed in 961.07 ms for C:\builds\root\ci-scratch-pad\app.csproj.
$ dotnet test --no-restore
Version:      11.9.0~beta.1139.g08121fc8
Git revision: 08121fc8
Git branch:   3755-helper-image-for-windows-container-execution-publish-windows-images
GO version:   go1.12
Built:        2019-03-04T16:34:20+0000
OS/Arch:      windows/amd64
Job succeeded

You can see some successful pipeline using windows docker executor https://gitlab.com/steveazz/playground/pipelines/57050034

To Do

  • Prerequesites MRs merged
  • Don't create powershell script if there is no user script to execute !1243 (comment 159911563): Fixed in 302a7708
  • Check how shell executor will work when PassFile is set to false for PowerShell: Fixed in 324a9ec4, will only set PassFile to false when docker-windows is used
  • Fix failing tests: This was related to dbda239e which shouldn't be included in this MR

Does this MR meet the acceptance criteria?

  • Documentation created/updated
  • Added tests for this feature/bug
  • In case of conflicts with master - branch was rebased

What are the relevant issue numbers?

Closes #3896 (closed)

Edited by Steve Xuereb

Merge request reports