Cache extraction fails on Windows during retrieve of cache file.
Summary
Gitlab runner fails when retrieving cache for second stage in a two stage pipeline when the runner has been configured with minio s3 cache settings.
Steps to reproduce
Setup a minio instance for caching on the local network. Configure caching for the runner so that it uses the url, access and secret key of minio server. Create a simple .gitlab-ci.yml
file that has two stages. Set caching for untracked files.
variables:
FILENAME: "test.txt"
cache:
untracked: true
stages:
- build
- test
suite:
stage: build
tags:
- powershell
script:
- Add-Type -AssemblyName System.Web
- $basePath = (Resolve-Path .\).Path
- $filepath = [System.IO.Path]::Combine($basePath, $env:FILENAME)
- Write-Host $filepath
- $fs = New-Object System.IO.FileStream($filepath,[System.IO.FileMode]::OpenOrCreate)
- $fs.SetLength(0)
- $sw = New-Object System.IO.StreamWriter($fs,[System.Text.Encoding]::ASCII,128)
- (1..(220mb/128)).foreach({ [char[]]$value = [System.Web.Security.Membership]::GeneratePassword(128,0) ; $sw.Write($value, 0, $value.Length) })
- $sw.Close()
- $fs.Close()
suite.test:
stage: test
tags:
- powershell
script:
- if (Test-Path $env:FILENAME) { Write-Host "Success" } else { Exit 1 }
Note this seems to work when the cache size is small. The problem is certainly present when the cache file size is in the region of 213,000 KB in size.
Actual behavior
First stage succeeds and the cached data can be seen in the minio server. However the second stage fails to retrieve the cached files.
Executing the C:\gitlab-runner\gitlab-ci-multi-runner-windows-amd64.exe cache-extractor --file <file> --url <url>
fails with
FATAL: rename ..\..\..\..\..\cache\company\project\default\cache611178439 ..\..\..\..\..\cache\company\project\default\cache.zip: The process cannot access the file because it is being used by another process
Expected behavior
Cached data is available to the second stage.
Environment description
Fails on both Windows 7 and Windows 10 64 bit.
Gitlab runner configuration
concurrent = 1
check_interval = 0
[[runners]]
name = "stz-W7"
url = "https://gitlab.com/ci"
token = "<gitlab token>"
executor = "shell"
[runners.cache]
Type = "s3"
ServerAddress = "192.168.1.138:9005"
AccessKey = "5JHXIYMY2ETYC9ZDTD8A"
SecretKey = "awPTeTod9GQNyfwji+pq0ksUPGCSsL6YvuzK/T6b"
BucketName = "runner"
Insecure = true
Shared = true
Used GitLab Runner version
Version: 9.3.0
Git revision: 3df822b
Git branch: 9-3-stable
GO version: go1.7.5
Built: Thu, 22 Jun 2017 10:57:49 +0000
OS/Arch: windows/amd64