Step Runner cache not updating on Windows

Summary

For a Shell Executor on Windows with step-runner version 0.21.0, the Step runner cache folder is not updated between pipeline runs causing Step code changes to not apply to subsequent pipeline runs. Manually deleting the contents of C:\Windows\SystemTemp\step-runner-cache corrects the problem. This only seems to be a problem when the pipeline and the Step are in different repositories. A locally referenced Step seems to behave as expected.

Steps to reproduce

  1. Register a Runner on Windows and add the step-runner version 0.21.0 executable to the filesystem and system path

  2. Create a Step in repository "a"

spec:
  inputs:
    project_name:
      type: string
---
env:
  STEP_PROJECT_NAME: ${{ inputs.project_name }}
exec:
  work_dir: ${{step_dir}}
  command:
    - 'powershell'
    - 'write-output "----- FIRST RUN -----"'
  1. Reference the Step from a pipeline in repository "b". Repository separation is important! If the step code is referenced locally (same repository) it works as expected.
.hello-build:
  run:
    - name: windows_test
      #step: https://gitlab.com/amfament/devops/sandbox/dewane/windows-step-step/-/tree/main/%20steps/windows-test@main
      step: gitlab-ci-token:${{ job.CI_JOB_TOKEN }}@gitlab.com/amfament/devops/sandbox/dewane/windows-step-step.git/-/windows-test/step.yml@main
      inputs:
        project_name: "HelloWindows"
  1. Run the pipeline
Executing "step_script" stage of the job script
00:03
$ step-runner ci
***** WARNING *****
As of GitLab Runner 17.11, the legacy step-runner image registry.gitlab.com/gitlab-org/step-runner:v0 is replaced by the image defined by the job.
The legacy step-runner image will be supported until GitLab 18.0.
See [https://docs.gitlab.com/ci/steps/](https://docs.gitlab.com/ci/steps/) for information on how to configure CI/CD Steps.
***** END WARNING *****
Running step "windows_test"
----- FIRST RUN -----

Cleaning up project directory and file based variables
00:01
Job succeeded
  1. Change the Step code
  command:
    - 'powershell'
    - 'write-output "----- SECOND RUN -----"'
  1. Run the pipeline - change not reflected
Executing "step_script" stage of the job script
00:03
$ step-runner ci
***** WARNING *****
As of GitLab Runner 17.11, the legacy step-runner image registry.gitlab.com/gitlab-org/step-runner:v0 is replaced by the image defined by the job.
The legacy step-runner image will be supported until GitLab 18.0.
See [https://docs.gitlab.com/ci/steps/](https://docs.gitlab.com/ci/steps/) for information on how to configure CI/CD Steps.
***** END WARNING *****
Running step "windows_test"
----- FIRST RUN -----

Cleaning up project directory and file based variables
00:01
Job succeeded

  1. Delete contents of C:\Windows\SystemTemp\step-runner-cache on the Runner

  2. Run the pipeline - output now correct

Executing "step_script" stage of the job script
00:03
$ step-runner ci
***** WARNING *****
As of GitLab Runner 17.11, the legacy step-runner image registry.gitlab.com/gitlab-org/step-runner:v0 is replaced by the image defined by the job.
The legacy step-runner image will be supported until GitLab 18.0.
See [https://docs.gitlab.com/ci/steps/](https://docs.gitlab.com/ci/steps/) for information on how to configure CI/CD Steps.
***** END WARNING *****
Running step "windows_test"
----- SECOND RUN -----

Cleaning up project directory and file based variables
00:01
Job succeeded

Example Project

What is the current bug behavior?

After updating the Step code, subsequent pipeline runs do not reflect the changes

What is the expected correct behavior?

After updating the Step code, subsequent pipeline should reflect the changes

Relevant logs and/or screenshots

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Possible fixes

Patch release information for backports

If the bug fix needs to be backported in a patch release to a version under the maintenance policy, please follow the steps on the patch release runbook for GitLab engineers.

Refer to the internal "Release Information" dashboard for information about the next patch release, including the targeted versions, expected release date, and current status.

High-severity bug remediation

To remediate high-severity issues requiring an internal release for single-tenant SaaS instances, refer to the internal release process for engineers.

Additional work remaining

Follow-up:

  • slog to be removed in the oci steps
  • oci steps logging to be reviewed
  • Decorate runner.StepResource so that all step resources have consistent log messages
  • Convert Running step "foo" into [green]Running step name=foo[reset]
  • Don't serve from the cache if there is an error fetching the reference from git
  • Use CI_FUNCS_LOG_LEVEL=debug instead of CI_FUNCS_DEBUG
Edited by Cameron Swords