Skip to content

Cache extraction fails on Windows during retrieve of cache file.

Summary

Gitlab runner fails when retrieving cache for second stage in a two stage pipeline when the runner has been configured with minio s3 cache settings.

Steps to reproduce

Setup a minio instance for caching on the local network. Configure caching for the runner so that it uses the url, access and secret key of minio server. Create a simple .gitlab-ci.yml file that has two stages. Set caching for untracked files.

variables:
    FILENAME: "test.txt"

cache:
    untracked: true

stages:
    - build
    - test

suite:
    stage: build
    tags:
        - powershell
    script:
        - Add-Type -AssemblyName System.Web
        - $basePath = (Resolve-Path .\).Path
        - $filepath = [System.IO.Path]::Combine($basePath, $env:FILENAME)
        - Write-Host $filepath
        - $fs = New-Object System.IO.FileStream($filepath,[System.IO.FileMode]::OpenOrCreate)
        - $fs.SetLength(0)
        - $sw = New-Object System.IO.StreamWriter($fs,[System.Text.Encoding]::ASCII,128)
        - (1..(220mb/128)).foreach({ [char[]]$value = [System.Web.Security.Membership]::GeneratePassword(128,0) ; $sw.Write($value, 0, $value.Length) })
        - $sw.Close()
        - $fs.Close()

suite.test:
    stage: test
    tags:
        - powershell
    script:
        - if (Test-Path $env:FILENAME) { Write-Host "Success" } else { Exit 1 }

Note this seems to work when the cache size is small. The problem is certainly present when the cache file size is in the region of 213,000 KB in size.

Actual behavior

First stage succeeds and the cached data can be seen in the minio server. However the second stage fails to retrieve the cached files.

Executing the C:\gitlab-runner\gitlab-ci-multi-runner-windows-amd64.exe cache-extractor --file <file> --url <url> fails with

FATAL: rename ..\..\..\..\..\cache\company\project\default\cache611178439 ..\..\..\..\..\cache\company\project\default\cache.zip: The process cannot access the file because it is being used by another process

Expected behavior

Cached data is available to the second stage.

Environment description

Fails on both Windows 7 and Windows 10 64 bit.

Gitlab runner configuration

concurrent = 1
check_interval = 0

[[runners]]
  name = "stz-W7"
  url = "https://gitlab.com/ci"
  token = "<gitlab token>"
  executor = "shell"
  [runners.cache]
    Type = "s3"
    ServerAddress = "192.168.1.138:9005"
    AccessKey = "5JHXIYMY2ETYC9ZDTD8A"
    SecretKey = "awPTeTod9GQNyfwji+pq0ksUPGCSsL6YvuzK/T6b"
    BucketName = "runner"
    Insecure = true
    Shared = true

Used GitLab Runner version

Version:      9.3.0
Git revision: 3df822b
Git branch:   9-3-stable
GO version:   go1.7.5
Built:        Thu, 22 Jun 2017 10:57:49 +0000
OS/Arch:      windows/amd64
Edited by Adam Sparrowhawk