## Summary <!-- Briefly summarize the bug --> Our FIPS runners will fail with any GitLab Runner version higher than 18.0.5 and throws a go panic with the latest 18.2.1 version ## Steps to reproduce <!-- What do you need to do to reproduce the bug? Please include job definitions or git repository structure if relevant --> Note, this configuration works fine with 18.0.5 GitLab runner version. Have a FIPS runner at version 18.2.1 with the configuration as follows: ``` concurrent = 256 check_interval = 5 log_format = "json" log_level = "info" [session_server] session_timeout = 1800 [[runners]] name = "ip-1-1-1-185" url = "REDACTED" id = 4 token = "glrt-REDACTED" executor = "docker-autoscaler" [runners.docker] #host = "unix:///var/run/docker.sock" image = "ubuntu:latest" helper_image_flavor = "ubi-fips" pull_policy = "if-not-present" network_mode = "host" [runners.autoscaler] plugin = "aws:latest" instance_ready_command = "sudo cloud-init status --wait" [runners.autoscaler.plugin_config] name = "worker" [runners.autoscaler.connector_config] username = "username" [[runners.autoscaler.policy]] idle_count = 0 idle_time = "5m0s" ``` If we use version 18.1.0 we hit this bug which is solved but I believe it is related [https://gitlab.com/gitlab-org/gitlab-runner/-/issues/38890](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/38890) If you update to 18.2.1 (currently the latest available) and run `gitlab-runner verify` you get this error: ``` # gitlab-runner verifyINFO[0000] FIPS mode is enabled. Using an external SSL library. Runtime platform arch=amd64 os=linux pid=4434 revision=cc489270 version=18.2.1 Running in system-mode. There might be a problem with your config based on jsonschema annotations in common/config.go (experimental feature): jsonschema: '/runners/0/Autoscaler/Policy/0/PreemptiveMode' does not validate with https://gitlab.com/gitlab-org/gitlab-runner/common/config#/$ref/properties/runners/items/$ref/properties/Autoscaler/$ref/properties/Policy/items/$ref/properties/PreemptiveMode/type: expected boolean, but got null Verifying runner... is valid correlation_id=01K239Q6JG2RT1T2FGD20JDZEF runner=yFuKOFdTi ``` Adding `preemptive_mode = false` to the configuration will get you past this error and `gitlab-runner verify` will pass. However trying to run a job the GitLab runner logs will give the following error: ``` Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: panic: curve25519: internal error: scalarBaseMult was not 32 bytes Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: goroutine 348 [running]: Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: golang.org/x/crypto/curve25519.ScalarBaseMult(0xc0018c0660, 0x40012a0?) Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: /go/pkg/mod/golang.org/x/crypto@v0.39.0/curve25519/curve25519.go:39 +0x13b Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: golang.org/x/crypto/ssh.(*curve25519KeyPair).generate(0xc0018c0640, {0x400c8a0?, 0x40012a0?}) Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: /go/pkg/mod/golang.org/x/crypto@v0.39.0/ssh/kex.go:457 +0x65 Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: golang.org/x/crypto/ssh.(*curve25519sha256).Client(0x63376a0?, {0x7faa3d271d48, 0xc001903b00}, {0x400c8a0, 0x40012a0}, 0xc001703ec0) Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: /go/pkg/mod/golang.org/x/crypto@v0.39.0/ssh/kex.go:468 +0x6e Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: golang.org/x/crypto/ssh.(*handshakeTransport).client(0xc000e40820, {0x402aaa0?, 0x64f8c20?}, 0xc001703ec0?) Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: /go/pkg/mod/golang.org/x/crypto@v0.39.0/ssh/handshake.go:826 +0x66 Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: golang.org/x/crypto/ssh.(*handshakeTransport).enterKeyExchange(0xc000e40820, {0xc000f81680, 0x405, 0x405}) Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: /go/pkg/mod/golang.org/x/crypto@v0.39.0/ssh/handshake.go:716 +0x49a Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: golang.org/x/crypto/ssh.(*handshakeTransport).kexLoop(0xc000e40820) Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: /go/pkg/mod/golang.org/x/crypto@v0.39.0/ssh/handshake.go:345 +0x98 Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: created by golang.org/x/crypto/ssh.newClientTransport in goroutine 313 Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS gitlab-runner[4466]: /go/pkg/mod/golang.org/x/crypto@v0.39.0/ssh/handshake.go:170 +0x256 Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS systemd[1]: gitlab-runner.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Aug 07 22:20:38 ip-REDACTED_IP_ADDRESS systemd[1]: gitlab-runner.service: Failed with result 'exit-code'. ``` <!-- Please add the definition of the job from `.gitlab-ci.yml` that is failing inside of the code blocks (```) below. --> <details> <summary> .gitlab-ci.yml </summary> ```yml stages: - helloWorld helloWorld: stage: helloWorld script: - echo "Hello world." ``` </details> ## Actual behavior The job fails to start and the GitLab runner throws a go panic in the logs <!-- What actually happens --> ## Expected behavior I expect the job to start and run <!-- What you should see instead --> ## Relevant logs and/or screenshots <!-- Paste the job logs inside of the code blocks (```) below so it would be easier to read. --> <details> <summary> job log </summary> The job fails to start at all and hangs with this output: ```sh Running with gitlab-runner 18.2.1 (cc489270) on worker yFuKOFdTi, system ID: s_7cca2598381a feature flags: FF_TIMESTAMPS:true Resolving secrets Preparing the "docker-autoscaler" executor ... ``` </details> ## Environment description This is a FIPS compliant self hosted ommnibus GitLab on version 18.1.1 The runners are also self hosted <!-- Are you using shared Runners on GitLab.com? Or is it a custom installation? Which executors are used? Please also provide the versions of related tools like `docker info` if you are using the Docker executor. --> <!-- Please add the contents of `config.toml` inside of the code blocks (```) below, remember to remove any secret tokens! --> <summary></summary> <details> ```sh concurrent = 256 check_interval = 5 log_format = "json" log_level = "info" [session_server] session_timeout = 1800 [[runners]] name = "ip-1-1-1-185" url = "REDACTED" id = 4 token = "glrt-REDACTED" executor = "docker-autoscaler" [runners.docker] #host = "unix:///var/run/docker.sock" image = "ubuntu:latest" helper_image_flavor = "ubi-fips" pull_policy = "if-not-present" network_mode = "host" [runners.autoscaler] plugin = "aws:latest" instance_ready_command = "sudo cloud-init status --wait" [runners.autoscaler.plugin_config] name = "worker" [runners.autoscaler.connector_config] username = "username" [[runners.autoscaler.policy]] idle_count = 0 idle_time = "5m0s" ``` </details> ### Used GitLab Runner version <!-- Please run and paste the output of `gitlab-runner --version`. If you are using a Runner where you don't have access to, please paste at least the first lines the from build log, like: ``` INFO[0000] FIPS mode is enabled. Using an external SSL library. Version: 18.2.1 Git revision: cc489270 Git branch: 18-2-stable GO version: go1.24.4 X:boringcrypto Built: 2025-07-28T12:43:39Z OS/Arch: linux/amd64 ``` --> ## Possible fixes <!-- (If you can, link to the line of code that might be responsible for the problem) --->