unable to pass unix builds_dir path to docker-windows executor even though shell executor accepts it

Summary

Before I describe the issue, I would like to clarify that I am not sure this is a bug, only that it is blocking my company's particular use case (and also to pass to GitLab Support later as we are a paying customer)

I am trying to use C:\Program Files\Git\bin\bash.exe as the shell for gitlab-runner on our windows instances but am encountering an issue with passing unix paths to the builds_dir parameter in C:\GitLab-Runner\config.toml when using the docker-windows executor.

This is confusing me because we have no such issues when using the shell executor as seen below

Executor builds_dir Job result
shell C:\GitLab-Runner\builds\ Fail
shell /c/GitLab-Runner/builds/ Pass
docker-windows C:\GitLab-Runner\builds\ Fail
docker-windows /c/GitLab-Runner/builds/ Fail

I would like to know why we are unable to to pass unix paths to the docker-windows executor even though there are no issues with passing it to the shell executor, I assumed the behavior would be the same regardless of which executor was used as builds_dir is not specific to the [runners.docker] section

If there is no fix, is there some sort of workaround that can be used? We have a lot of include files using bash and we would like to not have to write copies of everything for powershell. I've already taken a look at the following relevant issues as well as MRs and tried all the suggestions within but nothing has worked so far

Do we have no choice but to create our own custom executors to get the above to work?

Steps to reproduce

.gitlab-ci.yml
Executor builds_dir .gitlab-ci.yml
shell C:\GitLab-Runner\builds\
---
stages:
  - test

test-job:
  stage: test
  variables:
    CI_DEBUG_TRACE: "true"
  script:
    - ls
shell /c/GitLab-Runner/builds/
---
stages:
  - test

test-job:
  stage: test
  variables:
    CI_DEBUG_TRACE: "true"
  script:
    - ls
docker-windows C:\GitLab-Runner\builds\
---
stages:
  - test

test-job:
  stage: test
  image: <image_containing_bash_exe>
  variables:
    CI_DEBUG_TRACE: "true"
  script:
    - ls
docker-windows /c/GitLab-Runner/builds/
---
stages:
  - test

test-job:
  stage: test
  image: <image_containing_bash_exe>
  variables:
    CI_DEBUG_TRACE: "true"
  script:
    - ls

Actual behavior

Executor builds_dir Error
shell C:\GitLab-Runner\builds\
...
printf '\''%s'\'' $'\'' <certificate_str> > "$PWD/C:/GitLab-Runner/builds/Csa-CC1a3/0/<organization>/<group>/<subgroup>/<project>.tmp/CI_SERVER_TLS_CA_FILE"
...
/usr/bin/bash: line 46: /c/GitLab-Runner/C:/GitLab-Runner/builds/Csa-CC1a3/0/<organization>/<group>/<subgroup>/<project>.tmp/CI_SERVER_TLS_CA_FILE: No such file or directory
...
shell /c/GitLab-Runner/builds/ N/A
docker-windows C:\GitLab-Runner\builds\
...
printf '\''%s'\'' $'\'' <certificate_str> > "$PWD/C:/GitLab-Runner/builds/<organization>/<group>/<subgroup>/<project>.tmp/CI_SERVER_TLS_CA_FILE"
...
bash: line 47: /c/C:/GitLab-Runner/builds/<organization>/<group>/<subgroup>/<project>.tmp/CI_SERVER_TLS_CA_FILE: No such file or directory
...
docker-windows /c/GitLab-Runner/builds/
ERROR: Failed to remove network for build
ERROR: Job failed: build directory needs to be an absolute path

One thing to note is that I had to create a custom helper_image containing C:\Program Files\Git\bin\bash.exe to prevent docker-windows executor from throwing

'bash' is not recognized as an internal or external comand,
operable program or batch file.
ERROR: Job failed: prepare environment: exit code 1. Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information

Expected behavior

I fully expected builds_dir = "/c/GitLab-Runner/builds/" to work for the docker-windows executor since it worked for the shell executor

Relevant logs and/or screenshots

job log
Relevant job logs can be found in "Actual Behavior" above

Environment description

Self-hosted runners on AWS

$ docker info
Client:
 Version:    24.0.7
 Context:    default
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 10
 Server Version: 24.0.7
 Storage Driver: windowsfilter
  Windows:
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics internal l2bridge l2tunnel nat null overlay private transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
 Swarm: inactive
 Default Isolation: process
 Kernel Version: 10.0 20348 (20348.859.amd64fre.fe_release_svc_prod2.220707-1832)
 Operating System: Microsoft Windows Server Version 21H2 (OS Build 20348.2159)
 OSType: windows
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.82GiB
 Name: EC2AMAZ-XXXXXXX
 ID: 2955d6d3-14bd-473e-87a0-7e65ccca75d3
 Docker Root Dir: C:\ProgramData\docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine
$ git --version
git version 2.43.0.windows.1
config.toml contents
Executor builds_dir config.toml
shell C:\GitLab-Runner\builds\
concurrent = 100
check_interval = 0
shutdown_timeout = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "EC2AMAZ-XXXXXXX"
  url = "https://gitlab.com/"
  id = 30825055
  token = "glrt-xxx-xxxxxxxxxxxxxxxx"
  token_obtained_at = 2024-01-16T17:50:13Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "shell"
  builds_dir = "C:\GitLab-Runner\builds\"
  shell = "bash"
  [runners.custom_build_dir]
    enabled = true
shell /c/GitLab-Runner/builds/
concurrent = 100
check_interval = 0
shutdown_timeout = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "EC2AMAZ-XXXXXXX"
  url = "https://gitlab.com/"
  id = 30825055
  token = "glrt-xxx-xxxxxxxxxxxxxxxx"
  token_obtained_at = 2024-01-16T17:50:13Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "shell"
  builds_dir = "/c/GitLab-Runner/builds/"
  shell = "bash"
  [runners.custom_build_dir]
    enabled = true
docker-windows C:\GitLab-Runner\builds\
concurrent = 100
check_interval = 0
shutdown_timeout = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "EC2AMAZ-XXXXXXX"
  url = "https://gitlab.com/"
  id = 30825055
  token = "glrt-xxx-xxxxxxxxxxxxxxxx"
  token_obtained_at = 2024-01-16T17:50:13Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "docker-windows"
  builds_dir = "C:\GitLab-Runner\builds\"
  shell = "bash"
  [runners.cache]
    MaxUploadedArchiveSize = 0
  [runners.docker]
    tls_verify = false
    image = "<image_containing_bash_exe>"
    helper_image = "<gitlab_runner_helper_image_containing_bash_exe>"
    dns = ["172.21.64.1", "100.100.100.100", "8.8.8.8"]
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["c:\\cache"]
    shm_size = 0
    network_mtu = 0
docker-windows /c/GitLab-Runner/builds/
concurrent = 100
check_interval = 0
shutdown_timeout = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "EC2AMAZ-XXXXXXX"
  url = "https://gitlab.com/"
  id = 30825055
  token = "glrt-xxx-xxxxxxxxxxxxxxxx"
  token_obtained_at = 2024-01-16T17:50:13Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "docker-windows"
  builds_dir = "/c/GitLab-Runner/builds/"
  shell = "bash"
  [runners.cache]
    MaxUploadedArchiveSize = 0
  [runners.docker]
    tls_verify = false
    image = "<image_containing_bash_exe>"
    helper_image = "<gitlab_runner_helper_image_containing_bash_exe>"
    dns = ["172.21.64.1", "100.100.100.100", "8.8.8.8"]
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["c:\\cache"]
    shm_size = 0
    network_mtu = 0

Used GitLab Runner version

$ gitlab-runner --version
Version:      16.7.0
Git revision: 102c81ba
Git branch:   16-7-stable
GO version:   go1.20.10
Built:        2023-12-21T17:01:29+0000
OS/Arch:      windows/amd64

Possible fixes