Per Lundberg (7c3fcce4) at 12 Mar 19:31
The line gets a bit long with this change, so we could reformat the paragraph a bit (to keep it more in line with the existing lines, I think it's 100 characters that I've kept as a ~limit)
(Sorry, couldn't resist myself from getting involved.
Thank you for merging @slovdahl, I haven't (for slightly understandable reasons) given this so much focus lately.
Per Lundberg (252dba89) at 12 Mar 10:50
Per Lundberg (252dba89) at 08 Mar 14:55
(README) Add missing name
setting in docs
@jooola Here's the MR with a README section you suggested elsewhere. Does this look reasonable to you, or is there anything else that we should mention there also?
Per Lundberg (12219ae3) at 08 Mar 14:20
(README) Add Project history section
v0.1.0 has now been released (https://gitlab.com/fleeting-plugin-hetzner/fleeting-plugin-hetzner/-/releases/v0.1.0), and we are testing this in our environment as we speak. Let's consider this epic closed now, we'll create new issues for any remaining work that we find moving forward.
This issue serves to track the work remaining until the Hetzner Fleeting plugin is ready to reach the "public beta" stage. "Public beta" doesn't mean that it has to perfect and completely free from bugs, but it must mean that any serious shortcomings have been addressed.
See also gitlab-org/gitlab-runner#37370 for the upstream GitLab Inc issue regarding this plugin.
config.toml
: !9
hcloud-go
v2 API: !19
ConnectInfo
: !20
With absolutely no jobs running or anything, I have seen a case of the number of runners going up to 2.
2024-02-20T13:42:56.452+0200 [DEBUG] plugin: starting plugin: path=/home/plundberg/git/fleeting-plugin-hetzner/cmd/fleeting-plugin-hetzner/fleeting-plugin-hetzner args=["/home/plundberg/git/fleeting-plugin-hetzner/cmd/fleeting-plugin-hetzner/fleeting-plugin-hetzner"]
2024-02-20T13:42:56.452+0200 [DEBUG] plugin: plugin started: path=/home/plundberg/git/fleeting-plugin-hetzner/cmd/fleeting-plugin-hetzner/fleeting-plugin-hetzner pid=223426
2024-02-20T13:42:56.452+0200 [DEBUG] plugin: waiting for RPC address: plugin=/home/plundberg/git/fleeting-plugin-hetzner/cmd/fleeting-plugin-hetzner/fleeting-plugin-hetzner
2024-02-20T13:42:56.455+0200 [DEBUG] plugin.fleeting-plugin-hetzner: plugin address: address=/tmp/plugin800304042 network=unix timestamp="2024-02-20T13:42:56.455+0200"
2024-02-20T13:42:56.455+0200 [DEBUG] plugin: using plugin: version=0
2024-02-20T13:42:56.455+0200 [TRACE] plugin.stdio: waiting for stdio data
2024-02-20T13:42:56.696+0200 [INFO] plugin initialized: version=dev build info="sha=HEAD; ref=HEAD; go=go1.19.8; built_at=now; os_arch=linux/amd64"
2024-02-20T13:42:57.876+0200 [INFO] required scaling change: required=1 capacity-info="{\"InstanceCount\":0,\"MaxInstanceCount\":10,\"Acquired\":0,\"UnavailableCapacity\":0,\"Pending\":0,\"Reserved\":0,\"IdleCount\":1,\"ScaleFactor\":0,\"ScaleFactorLimit\":0,\"CapacityPerInstance\":1}"
2024-02-20T13:42:58.147+0200 [INFO] increasing instances: amount=1 group=hetzner/hel1/fleeting-plugin-hetzner-test
2024-02-20T13:42:58.877+0200 [INFO] required scaling change: required=0 capacity-info="{\"InstanceCount\":1,\"MaxInstanceCount\":10,\"Acquired\":0,\"UnavailableCapacity\":0,\"Pending\":0,\"Reserved\":0,\"IdleCount\":1,\"ScaleFactor\":0,\"ScaleFactorLimit\":0,\"CapacityPerInstance\":1}"
2024-02-20T13:42:59.263+0200 [INFO] increasing instances response: group=hetzner/hel1/fleeting-plugin-hetzner-test num_requested=1 num_successful=1
2024-02-20T13:42:59.263+0200 [INFO] increase update: group=hetzner/hel1/fleeting-plugin-hetzner-test pending=1 total_pending=1 requesting=0
2024-02-20T13:42:59.742+0200 [INFO] instance discovery: group=hetzner/hel1/fleeting-plugin-hetzner-test id=43656487 state=creating cause=requested
2024-02-20T13:43:03.788+0200 [INFO] instance update: group=hetzner/hel1/fleeting-plugin-hetzner-test id=43656487 state=deleting
2024-02-20T13:43:03.880+0200 [INFO] required scaling change: required=1 capacity-info="{\"InstanceCount\":0,\"MaxInstanceCount\":10,\"Acquired\":0,\"UnavailableCapacity\":0,\"Pending\":0,\"Reserved\":0,\"IdleCount\":1,\"ScaleFactor\":0,\"ScaleFactorLimit\":0,\"CapacityPerInstance\":1}"
2024-02-20T13:43:04.881+0200 [INFO] required scaling change: required=0 capacity-info="{\"InstanceCount\":1,\"MaxInstanceCount\":10,\"Acquired\":0,\"UnavailableCapacity\":0,\"Pending\":0,\"Reserved\":0,\"IdleCount\":1,\"ScaleFactor\":0,\"ScaleFactorLimit\":0,\"CapacityPerInstance\":1}"
2024-02-20T13:43:05.172+0200 [INFO] increasing instances: amount=1 group=hetzner/hel1/fleeting-plugin-hetzner-test
2024-02-20T13:43:06.195+0200 [INFO] increasing instances response: group=hetzner/hel1/fleeting-plugin-hetzner-test num_requested=1 num_successful=1
2024-02-20T13:43:06.195+0200 [INFO] increase update: group=hetzner/hel1/fleeting-plugin-hetzner-test pending=1 total_pending=1 requesting=0
2024-02-20T13:43:06.606+0200 [INFO] instance update: group=hetzner/hel1/fleeting-plugin-hetzner-test id=43656487 state=creating
2024-02-20T13:43:06.606+0200 [INFO] instance discovery: group=hetzner/hel1/fleeting-plugin-hetzner-test id=43656495 state=creating cause=requested
2024-02-20T13:43:06.881+0200 [INFO] required scaling change: required=-1 capacity-info="{\"InstanceCount\":2,\"MaxInstanceCount\":10,\"Acquired\":0,\"UnavailableCapacity\":0,\"Pending\":0,\"Reserved\":0,\"IdleCount\":1,\"ScaleFactor\":0,\"ScaleFactorLimit\":0,\"CapacityPerInstance\":1}"
2024-02-20T13:43:11.643+0200 [INFO] instance update: group=hetzner/hel1/fleeting-plugin-hetzner-test id=43656487 state=running
2024-02-20T13:43:11.643+0200 [INFO] instance update: group=hetzner/hel1/fleeting-plugin-hetzner-test id=43656495 state=deleting
2024-02-20T13:43:11.885+0200 [INFO] required scaling change: required=0 capacity-info="{\"InstanceCount\":1,\"MaxInstanceCount\":10,\"Acquired\":0,\"UnavailableCapacity\":0,\"Pending\":0,\"Reserved\":0,\"IdleCount\":1,\"ScaleFactor\":0,\"ScaleFactorLimit\":0,\"CapacityPerInstance\":1}"
2024-02-20T13:43:12.341+0200 [INFO] ready: instance=43656487 took=697.303532ms
2024-02-20T13:43:12.955+0200 [INFO] instance update: group=hetzner/hel1/fleeting-plugin-hetzner-test id=43656495 state=creating
2024-02-20T13:43:13.887+0200 [INFO] required scaling change: required=-1 capacity-info="{\"InstanceCount\":2,\"MaxInstanceCount\":10,\"Acquired\":0,\"UnavailableCapacity\":0,\"Pending\":0,\"Reserved\":0,\"IdleCount\":1,\"ScaleFactor\":0,\"ScaleFactorLimit\":0,\"CapacityPerInstance\":1}"
2024-02-20T13:43:17.878+0200 [INFO] instance update: group=hetzner/hel1/fleeting-plugin-hetzner-test id=43656495 state=running
2024-02-20T13:43:18.132+0200 [INFO] ready: instance=43656495 took=253.682114ms
Configuration loaded builds=0 max_builds=1
2024-02-20T14:03:00.523+0200 [INFO] instance marked for removal: instance=43656487 reason="instance exceeded max idle time"
2024-02-20T14:03:01.442+0200 [INFO] decreasing instances: amount=1 group=hetzner/hel1/fleeting-plugin-hetzner-test
2024-02-20T14:03:03.525+0200 [INFO] required scaling change: required=0 capacity-info="{\"InstanceCount\":1,\"MaxInstanceCount\":10,\"Acquired\":0,\"UnavailableCapacity\":0,\"Pending\":0,\"Reserved\":0,\"IdleCount\":1,\"ScaleFactor\":0,\"ScaleFactorLimit\":0,\"CapacityPerInstance\":1}"
2024-02-20T14:03:03.721+0200 [INFO] instance pruned: group=hetzner/hel1/fleeting-plugin-hetzner-test id=43656487 lifetime=20m3.97873909s
When we've tested this plugin at Hibox, we have seen that the Preparing the "docker-autoscaler" executor
stage can take a significant amount of time - a few minutes, up to almost 6 minutes in some extreme case.
I believe that this is at least partially caused by the fact that we report the instance as "ready" at a too early stage, right when the Hetzner API marks it as running
. What does that mean, is cloud-init
even guaranteed to be complete at that stage?
We could either:
[runners.autoscaler.plugin_config] # plugin specific configuration (see plugin documentation)
healthcheck_command = ["/path/to/something", "param1", "param2"]
The given command would then be executed and zero exit code means "ready", and non-zero means "not yet ready". The healthcheck would be retried at given intervals (perhaps 1s by default, but configurable up to a maximim healtcheck_timeout
).
An interesting observation is that if you run sudo gitlab-runner restart
, the plugin exits with a quite ugly stack dump of all goroutines (full example below).
I believe this could be related to one of the following issues:
The underlying problem is probably what is described in gitlab-cookbooks/cookbook-wrapper-gitlab-runner!47 (diffs):
# As per https://www.freedesktop.org/software/systemd/man/systemd.kill.html#KillMode= # the default mode - if not configured explicitly - is `control-group`. In that case # all processes in the process group originated at Runner process will get the configured # kill signal immediately when stop or restart of the service is requested.
The SIGQUIT
should only be sent to gitlab-runner
in this case, which can then gracefully terminate the plugins. So technically, this is probably not a problem in our plugin per se, but creating this issue here anyway since it's likely that we or others will be running into it.
Mar 08 11:20:42 some-host gitlab-runner[1864277]: WARNING: [runWait] received stop signal builds=0 max_builds=10 stop-signal=quit
Mar 08 11:20:42 some-host systemd[1]: Stopping GitLab Runner...
Mar 08 11:20:42 some-host gitlab-runner[1864277]: WARNING: Starting graceful shutdown, waiting for builds to finish StopSignal=quit builds=0 max_builds=10
Mar 08 11:20:42 some-host gitlab-runner[1864277]: All workers stopped. builds=0 max_builds=10
Mar 08 11:20:42 some-host gitlab-runner[1864277]: Shutting down executor providers builds=0 max_builds=10 shutdown-timeout=30s
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.120+0200 [INFO] received shutdown signal; stopping taskscaler's loop and triggering provisioner shutdown
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.121+0200 [INFO] stopping main loop
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.121+0200 [INFO] waiting for acquired instances to be released
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.121+0200 [INFO] shutting down provisioner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.121+0200 [INFO] Received shutdown signal; stopping Provisioner: group=hetzner/hel1/auto-scaling-group-name
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.121+0200 [DEBUG] plugin.fleeting-plugin-hetzner: SIGQUIT: quit
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.121+0200 [DEBUG] plugin.fleeting-plugin-hetzner: PC=0x46b2c1 m=0 sigcode=0
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.121+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.121+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 0 [idle]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.121+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.futex()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.121+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/sys_linux_amd64.s:559 +0x21 fp=0x7ffce115c980 sp=0x7ffce115c978 pc=0x46b2c1
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.121+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.futexsleep(0x8?, 0xf45d00?, 0xc000044500?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/os_linux.go:69 +0x36 fp=0x7ffce115c9d0 sp=0x7ffce115c980 pc=0x434f56
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.notesleep(0xf45e48)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/lock_futex.go:160 +0x87 fp=0x7ffce115ca08 sp=0x7ffce115c9d0 pc=0x40bf27
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.mPark()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:1457 +0x25 fp=0x7ffce115ca28 sp=0x7ffce115ca08 pc=0x43e105
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.stoplockedm()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:2448 +0x65 fp=0x7ffce115ca78 sp=0x7ffce115ca28 pc=0x43fd05
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.schedule()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:3193 +0x3d fp=0x7ffce115cab0 sp=0x7ffce115ca78 pc=0x4419dd
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.park_m(0xc000185a00?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:3363 +0x14d fp=0x7ffce115cae0 sp=0x7ffce115cab0 pc=0x441f8d
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.mcall()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:448 +0x43 fp=0x7ffce115caf0 sp=0x7ffce115cae0 pc=0x467303
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 1 [select, 2739 minutes]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0xc0003dfbd8?, 0x2?, 0x8a?, 0xc0?, 0xc0003dfa5c?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0005657e8 sp=0xc0005657c8 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.selectgo(0xc000565bd8, 0xc0003dfa58, 0x1?, 0x0, 0xc0003dfc18?, 0x1)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000565928 sp=0xc0005657e8 pc=0x44abdc
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.Serve(0xc0003dfe50)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/server.go:469 +0x1497 fp=0xc000565e08 sp=0xc000565928 pc=0x8c44b7
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: gitlab.com/gitlab-org/fleeting/fleeting/plugin.Serve({0xb7e388?, 0xc00013c600})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/gitlab.com/gitlab-org/fleeting/fleeting@v0.0.0-20240214164802-bfdbedb134e3/plugin/plugin.go:22 +0x23c fp=0xc000565f38 sp=0xc000565e08 pc=0x8cd83c
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: main.main()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /builds/fleeting-plugin-hetzner/fleeting-plugin-hetzner/cmd/fleeting-plugin-hetzner/main.go:25 +0xd8 fp=0xc000565f80 sp=0xc000565f38 pc=0x93d698
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.main()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:250 +0x212 fp=0xc000565fe0 sp=0xc000565f80 pc=0x43b3b2
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000565fe8 sp=0xc000565fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 2 [force gc (idle), 2 minutes]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0x32a3708b68db97?, 0x0?, 0x0?, 0x0?, 0x0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000054fb0 sp=0xc000054f90 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goparkunlock(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:369
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.forcegchelper()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:302 +0xad fp=0xc000054fe0 sp=0xc000054fb0 pc=0x43b60d
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000054fe8 sp=0xc000054fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by runtime.init.6
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:290 +0x25
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 3 [GC sweep wait]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000055790 sp=0xc000055770 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goparkunlock(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:369
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.bgsweep(0x0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mgcsweep.go:297 +0xd7 fp=0xc0000557c8 sp=0xc000055790 pc=0x426f97
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gcenable.func1()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mgc.go:178 +0x26 fp=0xc0000557e0 sp=0xc0000557c8 pc=0x41bc06
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000557e8 sp=0xc0000557e0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by runtime.gcenable
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mgc.go:178 +0x6b
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 4 [GC scavenge wait]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0x32a2abf9100df9?, 0x1c00ec4?, 0x0?, 0x0?, 0x0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000055f70 sp=0xc000055f50 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goparkunlock(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:369
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.(*scavengerState).park(0xf45460)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mgcscavenge.go:389 +0x53 fp=0xc000055fa0 sp=0xc000055f70 pc=0x424ff3
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.bgscavenge(0x0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mgcscavenge.go:622 +0x65 fp=0xc000055fc8 sp=0xc000055fa0 pc=0x4255e5
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gcenable.func2()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mgc.go:179 +0x26 fp=0xc000055fe0 sp=0xc000055fc8 pc=0x41bba6
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000055fe8 sp=0xc000055fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by runtime.gcenable
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mgc.go:179 +0xaa
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 5 [finalizer wait, 2739 minutes]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0x0?, 0xc000054670?, 0x6b?, 0x72?, 0xc000054770?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000054628 sp=0xc000054608 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goparkunlock(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:369
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.runfinq()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mfinal.go:180 +0x10f fp=0xc0000547e0 sp=0xc000054628 pc=0x41ad0f
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000547e8 sp=0xc0000547e0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by runtime.createfing
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mfinal.go:157 +0x45
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 8 [select, 2739 minutes]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0xc000056758?, 0x2?, 0x60?, 0xb8?, 0xc00005674c?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0000565d8 sp=0xc0000565b8 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.selectgo(0xc000056758, 0xc000056748, 0xc000056790?, 0x0, 0xab3b61?, 0x1)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000056718 sp=0xc0000565d8 pc=0x44abdc
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.(*gRPCBrokerServer).Recv(0x0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_broker.go:121 +0x67 fp=0xc000056788 sp=0xc000056718 pc=0x8bd1a7
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0xc000100f50)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_broker.go:411 +0x44 fp=0xc0000567c8 sp=0xc000056788 pc=0x8bdf24
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.(*GRPCServer).Init.func2()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_server.go:85 +0x26 fp=0xc0000567e0 sp=0xc0000567c8 pc=0x8bf7c6
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000567e8 sp=0xc0000567e0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by github.com/hashicorp/go-plugin.(*GRPCServer).Init
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_server.go:85 +0x4ea
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 9 [IO wait, 2739 minutes]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0xc000056d38?, 0xb?, 0x0?, 0x0?, 0x7?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000056ce0 sp=0xc000056cc0 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.netpollblock(0x4af1c5?, 0x0?, 0x0?)
Mar 08 11:20:42 some-host systemd[1]: gitlab-runner.service: Deactivated successfully.
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/netpoll.go:526 +0xf7 fp=0xc000056d18 sp=0xc000056ce0 pc=0x434337
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.runtime_pollWait(0x7f4d4415bef0, 0x72)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/netpoll.go:305 +0x89 fp=0xc000056d38 sp=0xc000056d18 pc=0x4640a9
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.(*pollDesc).wait(0xc000076ae0?, 0xc0000aa000?, 0x1)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000056d60 sp=0xc000056d38 pc=0x4c7df2
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.(*pollDesc).waitRead(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.(*FD).Read(0xc000076ae0, {0xc0000aa000, 0x1000, 0x1000})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a fp=0xc000056de0 sp=0xc000056d60 pc=0x4c915a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: os.(*File).read(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/os/file_posix.go:31
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: os.(*File).Read(0xc000012890, {0xc0000aa000?, 0x400?, 0x99c840?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/os/file.go:119 +0x5e fp=0xc000056e38 sp=0xc000056de0 pc=0x4d31de
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: bufio.(*Reader).Read(0xc000056f40, {0xc0000ac000, 0x400, 0x0?})
Mar 08 11:20:42 some-host systemd[1]: Stopped GitLab Runner.
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/bufio/bufio.go:237 +0x1bb fp=0xc000056e70 sp=0xc000056e38 pc=0x53ec1b
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.copyChan({0xb82728, 0xc000172870}, 0x0?, {0xb78740?, 0xc000012890?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:181 +0x1f6 fp=0xc000056fa8 sp=0xc000056e70 pc=0x8c02d6
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.newGRPCStdioServer.func1()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:37 +0x36 fp=0xc000056fe0 sp=0xc000056fa8 pc=0x8bfe76
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000056fe8 sp=0xc000056fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by github.com/hashicorp/go-plugin.newGRPCStdioServer
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:37 +0xf5
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 10 [IO wait, 2739 minutes]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0x0?, 0xb?, 0x0?, 0x0?, 0x9?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0000574e0 sp=0xc0000574c0 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.netpollblock(0x4af1c5?, 0x0?, 0x0?)
Mar 08 11:20:42 some-host systemd[1]: gitlab-runner.service: Consumed 5min 20.061s CPU time.
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/netpoll.go:526 +0xf7 fp=0xc000057518 sp=0xc0000574e0 pc=0x434337
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.runtime_pollWait(0x7f4d4415bd10, 0x72)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/netpoll.go:305 +0x89 fp=0xc000057538 sp=0xc000057518 pc=0x4640a9
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.(*pollDesc).wait(0xc000076ba0?, 0xc0000ab000?, 0x1)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000057560 sp=0xc000057538 pc=0x4c7df2
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.(*pollDesc).waitRead(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.(*FD).Read(0xc000076ba0, {0xc0000ab000, 0x1000, 0x1000})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a fp=0xc0000575e0 sp=0xc000057560 pc=0x4c915a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: os.(*File).read(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/os/file_posix.go:31
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: os.(*File).Read(0xc0000128a0, {0xc0000ab000?, 0x400?, 0x99c840?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/os/file.go:119 +0x5e fp=0xc000057638 sp=0xc0000575e0 pc=0x4d31de
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: bufio.(*Reader).Read(0xc000057740, {0xc0000ac400, 0x400, 0x0?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/bufio/bufio.go:237 +0x1bb fp=0xc000057670 sp=0xc000057638 pc=0x53ec1b
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.copyChan({0xb82728, 0xc000172870}, 0x0?, {0xb78740?, 0xc0000128a0?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:181 +0x1f6 fp=0xc0000577a8 sp=0xc000057670 pc=0x8c02d6
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.newGRPCStdioServer.func2()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:38 +0x36 fp=0xc0000577e0 sp=0xc0000577a8 pc=0x8bfe16
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000577e8 sp=0xc0000577e0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by github.com/hashicorp/go-plugin.newGRPCStdioServer
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:38 +0x185
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 11 [select, 2739 minutes, locked to thread]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0xc000057fa8?, 0x2?, 0x6e?, 0x73?, 0xc000057fa4?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000057e18 sp=0xc000057df8 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.selectgo(0xc000057fa8, 0xc000057fa0, 0x0?, 0x0, 0x0?, 0x1)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000057f58 sp=0xc000057e18 pc=0x44abdc
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.ensureSigM.func1()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/signal_unix.go:991 +0x1b0 fp=0xc000057fe0 sp=0xc000057f58 pc=0x44ec50
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000057fe8 sp=0xc000057fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by runtime.ensureSigM
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/signal_unix.go:974 +0xbd
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 12 [syscall, 2739 minutes]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.notetsleepg(0x0?, 0x0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/lock_futex.go:236 +0x34 fp=0xc0000507a0 sp=0xc000050768 pc=0x40c234
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: os/signal.signal_recv()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/sigqueue.go:152 +0x2f fp=0xc0000507c0 sp=0xc0000507a0 pc=0x465f2f
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: os/signal.loop()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/os/signal/signal_unix.go:23 +0x19 fp=0xc0000507e0 sp=0xc0000507c0 pc=0x8bc819
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000507e8 sp=0xc0000507e0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by os/signal.Notify.func1.1
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/os/signal/signal.go:151 +0x2a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 13 [chan receive, 2739 minutes]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000050ec0 sp=0xc000050ea0 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.chanrecv(0xc000076c60, 0x0, 0x1)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/chan.go:583 +0x49b fp=0xc000050f50 sp=0xc000050ec0 pc=0x406cdb
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.chanrecv1(0x0?, 0x0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/chan.go:442 +0x18 fp=0xc000050f78 sp=0xc000050f50 pc=0x4067d8
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.Serve.func3()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/server.go:434 +0x75 fp=0xc000050fe0 sp=0xc000050f78 pc=0x8c4ad5
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000050fe8 sp=0xc000050fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by github.com/hashicorp/go-plugin.Serve
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/server.go:431 +0x126a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 14 [IO wait, 2739 minutes]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0x203000?, 0x203000?, 0xe0?, 0x20?, 0xc000108c60?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000108bf0 sp=0xc000108bd0 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.netpollblock(0x41b448?, 0x108cc8?, 0xc0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/netpoll.go:526 +0xf7 fp=0xc000108c28 sp=0xc000108bf0 pc=0x434337
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.runtime_pollWait(0x7f4d4415bfe0, 0x72)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/netpoll.go:305 +0x89 fp=0xc000108c48 sp=0xc000108c28 pc=0x4640a9
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.(*pollDesc).wait(0xc000121380?, 0x18?, 0x0)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000108c70 sp=0xc000108c48 pc=0x4c7df2
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.(*pollDesc).waitRead(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.(*FD).Accept(0xc000121380)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_unix.go:614 +0x234 fp=0xc000108d08 sp=0xc000108c70 pc=0x4cd074
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: net.(*netFD).accept(0xc000121380)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/net/fd_unix.go:172 +0x35 fp=0xc000108dc0 sp=0xc000108d08 pc=0x601155
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: net.(*UnixListener).accept(0x46736e?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/net/unixsock_posix.go:166 +0x1c fp=0xc000108de8 sp=0xc000108dc0 pc=0x61fc3c
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: net.(*UnixListener).Accept(0xc0001f86f0)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/net/unixsock.go:260 +0x3d fp=0xc000108e18 sp=0xc000108de8 pc=0x61e65d
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.(*rmListener).Accept(0x9c64a0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: <autogenerated>:1 +0x2a fp=0xc000108e30 sp=0xc000108e18 pc=0x8c548a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.(*Server).Serve(0xc0001cac40, {0xb7b780, 0xc000075640})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:779 +0x362 fp=0xc000108f48 sp=0xc000108e30 pc=0x86f022
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.(*GRPCServer).Serve(0xc000172990, {0xb7b780?, 0xc000075640?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_server.go:138 +0x71 fp=0xc000108fb8 sp=0xc000108f48 pc=0x8bfa71
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.Serve.func6()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/server.go:463 +0x33 fp=0xc000108fe0 sp=0xc000108fb8 pc=0x8c49d3
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000108fe8 sp=0xc000108fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by github.com/hashicorp/go-plugin.Serve
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/server.go:463 +0x140f
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.122+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 34 [runnable]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0xc0000e9f00?, 0x2?, 0xd?, 0x0?, 0xc0000e9ed4?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0000e9d60 sp=0xc0000e9d40 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.selectgo(0xc0000e9f00, 0xc0000e9ed0, 0x610e85?, 0x0, 0xc0000c8000?, 0x1)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc0000e9ea0 sp=0xc0000e9d60 pc=0x44abdc
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0000d8050, 0x1)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/internal/transport/controlbuf.go:407 +0x115 fp=0xc0000e9f30 sp=0xc0000e9ea0 pc=0x7e0815
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0000943c0)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/internal/transport/controlbuf.go:534 +0x85 fp=0xc0000e9f88 sp=0xc0000e9f30 pc=0x7e0f25
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc/internal/transport.NewServerTransport.func2()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/internal/transport/http2_server.go:326 +0xce fp=0xc0000e9fe0 sp=0xc0000e9f88 pc=0x7f78ce
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000e9fe8 sp=0xc0000e9fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by google.golang.org/grpc/internal/transport.NewServerTransport
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/internal/transport/http2_server.go:323 +0x1873
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 35 [select, 107 minutes]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0xc000105f70?, 0x4?, 0x0?, 0x30?, 0xc000105ed0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000105d18 sp=0xc000105cf8 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.selectgo(0xc000105f70, 0xc000105ec8, 0x0?, 0x0, 0x0?, 0x1)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000105e58 sp=0xc000105d18 pc=0x44abdc
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc000082820)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/internal/transport/http2_server.go:1125 +0x233 fp=0xc000105fc8 sp=0xc000105e58 pc=0x7fea93
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc/internal/transport.NewServerTransport.func4()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/internal/transport/http2_server.go:335 +0x26 fp=0xc000105fe0 sp=0xc000105fc8 pc=0x7f77c6
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000105fe8 sp=0xc000105fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by google.golang.org/grpc/internal/transport.NewServerTransport
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/internal/transport/http2_server.go:335 +0x18b8
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 36 [runnable]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: syscall.Syscall(0x100000000000047?, 0x1f2700000001?, 0x4c7cb2?, 0x7ffff800000?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/syscall/syscall_linux.go:68 +0x27 fp=0xc000104b28 sp=0xc000104ab8 pc=0x4b1027
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: syscall.read(0xc0000a8100?, {0xc0000c0000?, 0x100000000000000?, 0x4?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/syscall/zsyscall_linux_amd64.go:696 +0x45 fp=0xc000104b68 sp=0xc000104b28 pc=0x4af1c5
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: syscall.Read(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/syscall/syscall_unix.go:183
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.ignoringEINTRIO(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_unix.go:794
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.(*FD).Read(0xc0000a8100?, {0xc0000c0000?, 0x8000?, 0x8000?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_unix.go:163 +0x285 fp=0xc000104be8 sp=0xc000104b68 pc=0x4c9185
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: net.(*netFD).Read(0xc0000a8100, {0xc0000c0000?, 0x7f4d6b628108?, 0x18?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/net/fd_posix.go:55 +0x29 fp=0xc000104c30 sp=0xc000104be8 pc=0x5ff049
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: net.(*conn).Read(0xc00009e030, {0xc0000c0000?, 0x9fd660?, 0x7f4d44117801?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/net/net.go:183 +0x45 fp=0xc000104c78 sp=0xc000104c30 pc=0x610ca5
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: bufio.(*Reader).Read(0xc000094360, {0xc0000da040, 0x9, 0xc0004aa0f0?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/bufio/bufio.go:237 +0x1bb fp=0xc000104cb0 sp=0xc000104c78 pc=0x53ec1b
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: io.ReadAtLeast({0xb76c40, 0xc000094360}, {0xc0000da040, 0x9, 0x9}, 0x9)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/io/io.go:332 +0x9a fp=0xc000104cf8 sp=0xc000104cb0 pc=0x4a7d9a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: io.ReadFull(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/io/io.go:351
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: golang.org/x/net/http2.readFrameHeader({0xc0000da040?, 0x9?, 0xc0000340a0?}, {0xb76c40?, 0xc000094360?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/golang.org/x/net@v0.20.0/http2/frame.go:237 +0x6e fp=0xc000104d48 sp=0xc000104cf8 pc=0x7caace
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: golang.org/x/net/http2.(*Framer).ReadFrame(0xc0000da000)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/golang.org/x/net@v0.20.0/http2/frame.go:498 +0x95 fp=0xc000104df8 sp=0xc000104d48 pc=0x7cb335
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc000082820, 0x0?, 0x0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/internal/transport/http2_server.go:605 +0x174 fp=0xc000104f18 sp=0xc000104df8 pc=0x7fa6d4
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.(*Server).serveStreams(0xc0001cac40, {0xb80238?, 0xc000082820})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:905 +0x142 fp=0xc000104f80 sp=0xc000104f18 pc=0x870222
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.(*Server).handleRawConn.func1()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:847 +0x46 fp=0xc000104fe0 sp=0xc000104f80 pc=0x86f9e6
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000104fe8 sp=0xc000104fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by google.golang.org/grpc.(*Server).handleRawConn
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:846 +0x185
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 18 [select, 2739 minutes]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0xc00023fb00?, 0x3?, 0x0?, 0x0?, 0xc00023fab2?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc00023f938 sp=0xc00023f918 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.selectgo(0xc00023fb00, 0xc00023faac, 0x203000?, 0x0, 0x40d71f?, 0x1)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc00023fa78 sp=0xc00023f938 pc=0x44abdc
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.(*grpcStdioServer).StreamStdio(0xc000290c40, 0xa05120?, {0xb7f410, 0xc000290f00})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:58 +0x11d fp=0xc00023fb40 sp=0xc00023fa78 pc=0x8bffbd
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin/internal/plugin._GRPCStdio_StreamStdio_Handler({0x9bae40?, 0xc000290c40}, {0xb7e4a8, 0xc000174180})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/internal/plugin/grpc_stdio.pb.go:185 +0xd0 fp=0xc00023fb80 sp=0xc00023fb40 pc=0x886cb0
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.(*Server).processStreamingRPC(0xc0001cac40, {0xb80238, 0xc000082820}, 0xc000228120, 0xc0001f8ba0, 0xf30160, 0x0)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1540 +0xf3c fp=0xc00023fe48 sp=0xc00023fb80 pc=0x87591c
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.(*Server).handleStream(0xc0001cac40, {0xb80238, 0xc000082820}, 0xc000228120, 0x0)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1620 +0x9ea fp=0xc00023ff68 sp=0xc00023fe48 pc=0x87718a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.(*Server).serveStreams.func1.2()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:921 +0x98 fp=0xc00023ffe0 sp=0xc00023ff68 pc=0x8705d8
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00023ffe8 sp=0xc00023ffe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by google.golang.org/grpc.(*Server).serveStreams.func1
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:919 +0x28a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 19 [select, 2739 minutes]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0xc000104660?, 0x2?, 0x0?, 0x0?, 0xc000104644?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0002454c8 sp=0xc0002454a8 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.selectgo(0xc000245660, 0xc000104640, 0x0?, 0x0, 0x0?, 0x1)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc000245608 sp=0xc0002454c8 pc=0x44abdc
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc/internal/transport.(*recvBufferReader).read(0xc000232190, {0xc000220910, 0x5, 0x5})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/internal/transport/transport.go:178 +0x90 fp=0xc000245690 sp=0xc000245608 pc=0x8033f0
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc/internal/transport.(*recvBufferReader).Read(0xc000232190, {0xc000220910?, 0xc00021c0c0?, 0xc0001046d8?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/internal/transport/transport.go:172 +0x178 fp=0xc0002456d0 sp=0xc000245690 pc=0x8032b8
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc/internal/transport.(*transportReader).Read(0xc0002005a0, {0xc000220910?, 0xc000104750?, 0x7fa4a7?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/internal/transport/transport.go:484 +0x32 fp=0xc000245718 sp=0xc0002456d0 pc=0x8048f2
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: io.ReadAtLeast({0xb77bc0, 0xc0002005a0}, {0xc000220910, 0x5, 0x5}, 0x5)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/io/io.go:332 +0x9a fp=0xc000245760 sp=0xc000245718 pc=0x4a7d9a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: io.ReadFull(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/io/io.go:351
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc/internal/transport.(*Stream).Read(0xc000228240, {0xc000220910, 0x5, 0x5})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/internal/transport/transport.go:468 +0xa5 fp=0xc0002457a0 sp=0xc000245760 pc=0x804865
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.(*parser).recvMsg(0xc000220900, 0x7fffffff)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/rpc_util.go:559 +0x47 fp=0xc000245820 sp=0xc0002457a0 pc=0x86baa7
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.recvAndDecompress(0x0?, 0xc000228240, {0x0, 0x0}, 0x7fffffff, 0x0, {0x0, 0x0})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/rpc_util.go:690 +0x66 fp=0xc0002458b8 sp=0xc000245820 pc=0x86c3a6
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.recv(0x8?, {0x7f4d4411c7d8, 0xf757c0}, 0x203000?, {0x0?, 0x0?}, {0xa51340, 0xc0002321e0}, 0xc0?, 0x0, ...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/rpc_util.go:756 +0x6e fp=0xc000245930 sp=0xc0002458b8 pc=0x86ca8e
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.(*serverStream).RecvMsg(0xc00025c000, {0xa51340?, 0xc0002321e0})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/stream.go:1563 +0x178 fp=0xc000245a60 sp=0xc000245930 pc=0x8827d8
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin/internal/plugin.(*gRPCBrokerStartStreamServer).Recv(0xc000204420)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/internal/plugin/grpc_broker.pb.go:167 +0x4c fp=0xc000245a90 sp=0xc000245a60 pc=0x885eec
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.(*gRPCBrokerServer).StartStream(0xc0001f8960, {0xb7fc30, 0xc000204420})
Mar 08 11:20:42 some-host systemd[1]: Started GitLab Runner.
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_broker.go:84 +0x152 fp=0xc000245b48 sp=0xc000245a90 pc=0x8bcb52
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin/internal/plugin._GRPCBroker_StartStream_Handler({0x9f1e60?, 0xc0001f8960}, {0xb7e4a8?, 0xc00025c000})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/internal/plugin/grpc_broker.pb.go:148 +0x9f fp=0xc000245b80 sp=0xc000245b48 pc=0x885dff
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.(*Server).processStreamingRPC(0xc0001cac40, {0xb80238, 0xc000082820}, 0xc000228240, 0xc0001f89f0, 0xf30140, 0x0)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1540 +0xf3c fp=0xc000245e48 sp=0xc000245b80 pc=0x87591c
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.(*Server).handleStream(0xc0001cac40, {0xb80238, 0xc000082820}, 0xc000228240, 0x0)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1620 +0x9ea fp=0xc000245f68 sp=0xc000245e48 pc=0x87718a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.(*Server).serveStreams.func1.2()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:921 +0x98 fp=0xc000245fe0 sp=0xc000245f68 pc=0x8705d8
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000245fe8 sp=0xc000245fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by google.golang.org/grpc.(*Server).serveStreams.func1
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:919 +0x28a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 20 [select, 2739 minutes]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0xc0000b8fa0?, 0x3?, 0x0?, 0x0?, 0xc0000b8f62?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0000b8de0 sp=0xc0000b8dc0 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.selectgo(0xc0000b8fa0, 0xc0000b8f5c, 0x0?, 0x0, 0x0?, 0x1)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc0000b8f20 sp=0xc0000b8de0 pc=0x44abdc
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: github.com/hashicorp/go-plugin.(*gRPCBrokerServer).StartStream.func1()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_broker.go:70 +0x105 fp=0xc0000b8fe0 sp=0xc0000b8f20 pc=0x8bcde5
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000b8fe8 sp=0xc0000b8fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by github.com/hashicorp/go-plugin.(*gRPCBrokerServer).StartStream
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.3/grpc_broker.go:68 +0x131
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 54 [GC worker (idle)]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0x32a3708b755956?, 0x2?, 0x5f?, 0xcd?, 0x7b2391?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000051750 sp=0xc000051730 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gcBgMarkWorker()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0000517e0 sp=0xc000051750 pc=0x41dd51
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000517e8 sp=0xc0000517e0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by runtime.gcBgMarkStartWorkers
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mgc.go:1159 +0x25
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 23 [GC worker (idle)]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0x32a3708b75668f?, 0x3?, 0xf5?, 0xcf?, 0x0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0000b9f50 sp=0xc0000b9f30 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gcBgMarkWorker()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc0000b9fe0 sp=0xc0000b9f50 pc=0x41dd51
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc0000b9fe8 sp=0xc0000b9fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by runtime.gcBgMarkStartWorkers
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mgc.go:1159 +0x25
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 55 [GC worker (idle)]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0x32a3708b754737?, 0x3?, 0x92?, 0xf1?, 0x0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc000051f50 sp=0xc000051f30 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gcBgMarkWorker()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mgc.go:1235 +0xf1 fp=0xc000051fe0 sp=0xc000051f50 pc=0x41dd51
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000051fe8 sp=0xc000051fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by runtime.gcBgMarkStartWorkers
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/mgc.go:1159 +0x25
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 8723 [runnable]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: google.golang.org/grpc.(*Server).serveStreams.func1.2()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:919 fp=0xc000193fe0 sp=0xc000193fd8 pc=0x870540
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000193fe8 sp=0xc000193fe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by google.golang.org/grpc.(*Server).serveStreams.func1
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:919 +0x28a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: goroutine 8672 [IO wait]:
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.gopark(0xd1c?, 0xb?, 0x0?, 0x0?, 0xc?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc00026f618 sp=0xc00026f5f8 pc=0x43b776
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.netpollblock(0x4af1c5?, 0xa?, 0x0?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/netpoll.go:526 +0xf7 fp=0xc00026f650 sp=0xc00026f618 pc=0x434337
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.runtime_pollWait(0x7f4d4415b950, 0x72)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/netpoll.go:305 +0x89 fp=0xc00026f670 sp=0xc00026f650 pc=0x4640a9
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.(*pollDesc).wait(0xc000120400?, 0xc0001ab000?, 0x0)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc00026f698 sp=0xc00026f670 pc=0x4c7df2
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.(*pollDesc).waitRead(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: internal/poll.(*FD).Read(0xc000120400, {0xc0001ab000, 0x1000, 0x1000})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a fp=0xc00026f718 sp=0xc00026f698 pc=0x4c915a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: net.(*netFD).Read(0xc000120400, {0xc0001ab000?, 0xc0000a21c0?, 0xc0001ab2e4?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/net/fd_posix.go:55 +0x29 fp=0xc00026f760 sp=0xc00026f718 pc=0x5ff049
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: net.(*conn).Read(0xc000012028, {0xc0001ab000?, 0x5?, 0xc00026f800?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/net/net.go:183 +0x45 fp=0xc00026f7a8 sp=0xc00026f760 pc=0x610ca5
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: crypto/tls.(*atLeastReader).Read(0xc00050c270, {0xc0001ab000?, 0x0?, 0x0?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/crypto/tls/conn.go:787 +0x3d fp=0xc00026f7f0 sp=0xc00026f7a8 pc=0x6526bd
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: bytes.(*Buffer).ReadFrom(0xc0001d2278, {0xb76da0, 0xc00050c270})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/bytes/buffer.go:202 +0x98 fp=0xc00026f848 sp=0xc00026f7f0 pc=0x4f37b8
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: crypto/tls.(*Conn).readFromUntil(0xc0001d2000, {0xb78460?, 0xc000012028}, 0xd21?)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/crypto/tls/conn.go:809 +0xe5 fp=0xc00026f888 sp=0xc00026f848 pc=0x6528a5
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: crypto/tls.(*Conn).readRecordOrCCS(0xc0001d2000, 0x0)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/crypto/tls/conn.go:616 +0x116 fp=0xc00026fc10 sp=0xc00026f888 pc=0x64fcf6
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: crypto/tls.(*Conn).readRecord(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/crypto/tls/conn.go:582
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: crypto/tls.(*Conn).Read(0xc0001d2000, {0xc0001c3000, 0x1000, 0x7881c0?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/crypto/tls/conn.go:1315 +0x16f fp=0xc00026fc80 sp=0xc00026fc10 pc=0x655c6f
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: bufio.(*Reader).Read(0xc0004b9380, {0xc0000da2e0, 0x9, 0x796605?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.123+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/bufio/bufio.go:237 +0x1bb fp=0xc00026fcb8 sp=0xc00026fc80 pc=0x53ec1b
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: io.ReadAtLeast({0xb76c40, 0xc0004b9380}, {0xc0000da2e0, 0x9, 0x9}, 0x9)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/io/io.go:332 +0x9a fp=0xc00026fd00 sp=0xc00026fcb8 pc=0x4a7d9a
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: io.ReadFull(...)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/io/io.go:351
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: net/http.http2readFrameHeader({0xc0000da2e0?, 0x9?, 0xc0001f9080?}, {0xb76c40?, 0xc0004b9380?})
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/net/http/h2_bundle.go:1565 +0x6e fp=0xc00026fd50 sp=0xc00026fd00 pc=0x78016e
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: net/http.(*http2Framer).ReadFrame(0xc0000da2a0)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/net/http/h2_bundle.go:1829 +0x95 fp=0xc00026fe00 sp=0xc00026fd50 pc=0x7809d5
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: net/http.(*http2clientConnReadLoop).run(0xc00026ff98)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/net/http/h2_bundle.go:8874 +0x130 fp=0xc00026ff60 sp=0xc00026fe00 pc=0x7934b0
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: net/http.(*http2ClientConn).readLoop(0xc000316600)
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/net/http/h2_bundle.go:8770 +0x6f fp=0xc00026ffc8 sp=0xc00026ff60 pc=0x7929cf
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: net/http.(*http2Transport).newClientConn.func1()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/net/http/h2_bundle.go:7477 +0x26 fp=0xc00026ffe0 sp=0xc00026ffc8 pc=0x78b6a6
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: runtime.goexit()
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00026ffe8 sp=0xc00026ffe0 pc=0x469401
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: created by net/http.(*http2Transport).newClientConn
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: /usr/local/go/src/net/http/h2_bundle.go:7477 +0xaaa
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: rax 0xca
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: rbx 0x0
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: rcx 0x46b2c3
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: rdx 0x0
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: rdi 0xf45e48
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: rsi 0x80
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: rbp 0x7ffce115c9c0
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: rsp 0x7ffce115c978
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: r8 0x0
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: r9 0x0
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: r10 0x0
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: r11 0x286
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: r12 0x203000
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: r13 0x178
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: r14 0xf45760
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: r15 0x7f4d442fb134
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: rip 0x46b2c1
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: rflags 0x286
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: cs 0x33
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: fs 0x0
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [DEBUG] plugin.fleeting-plugin-hetzner: gs 0x0
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.124+0200 [ERROR] plugin: plugin process exited: plugin=/usr/local/bin/fleeting-plugin-hetzner id=1864285 error="exit status 2"
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.125+0200 [INFO] provisioner shutdown completed: err="rpc error: code = Unavailable desc = error reading from server: EOF" group=hetzner/hel1/auto-scaling-group-name
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.125+0200 [INFO] taskscaler's shutdown completed
Mar 08 11:20:42 some-host gitlab-runner[1864277]: 2024-03-08T11:20:42.125+0200 [DEBUG] plugin: plugin exited
Mar 08 11:20:42 some-host gitlab-runner[1864277]: All executor providers shut down. builds=0 max_builds=10
Mar 08 11:20:42 some-host gitlab-runner[1864277]: Can exit now! builds=0 max_builds=10
For reference, see also this: gitlab-org/ci-cd/shared-runners/infrastructure#48 (comment 1801560351)
Thank you very much for the reply @ajwalker, and thanks for the details.
[runners.machine]
# Number of machines that need to be created and waiting in Idle state
IdleCount = 0
# Time (in seconds) for machine to be in Idle state before it is removed.
IdleTime = 900
[[runners.machine.autoscaling]]
Periods = ["* * 8-15 * * mon-fri *"]
# Number of machines that need to be created and waiting in Idle state
IdleCount = 1
# Time (in seconds) for machine to be in Idle state before it is removed.
IdleTime = 3600
# Defaults to the locale system setting of the host if omitted or empty.
Timezone = "Europe/Helsinki"
In other words:
IdleCount
and IdleTime
for the "default" fallback values in runners.machine
.Periods
setting in runners.machine.autoscaling
.In other words, we don't use multiple periods in the previous setup. Now that we are migrating the above to a Fleeting-based setup, I transformed the above to the setting in the issue description => wrong, will use the latter period always.
I'll get this confirmed in our env very soon (by flipping the order of these runners.autoscaler.policy
) entries. However, I think we should perhaps keep this issue open but mark it as documentation? It would be nice to get the details you mention above explicitly noted in e.g. https://docs.gitlab.com/runner/executors/docker_autoscaler.html.
Per Lundberg (12449bef) at 07 Mar 14:08
chore: add note about project status
... and 24 more commits
Per Lundberg (ca06124b) at 07 Mar 14:07
chore: add note about project status
... and 24 more commits
Per Lundberg (0a134da2) at 07 Mar 14:06
chore: add note about project status
... and 24 more commits
Per Lundberg (cafc531b) at 07 Mar 14:04
chore: add note about project status
... and 24 more commits