deleteInstance already recovers from the empty-zone state a failed
bulkInsert leaves behind, but stop, start, and instance lookups still
called the API with an empty zone and failed with "Invalid value for
field 'zone': ''" (400). The runner hits this when draining a machine
whose create never resolved a zone.

Factor the recovery into ensureZone and use it in all four paths. As
with delete, a machine that was never placed yields a not-found error
so callers treat it as gone.

Each driver method builds a fresh ComputeUtil, so an in-memory recovery
lasted one call: a single stop discovered the zone three times (inspect,
stop, inspect). Writing it back to ResolvedZone makes one AggregatedList
serve the whole invocation, and persists to the machine config when
libmachine saves the host.

No zone is resolved before placement, so the check never found
anything in bulk mode: it always got a 400 on the empty zone and
passed. With zone recovery it would instead cost an AggregatedList on
every create. UseExisting keeps the lookup.

Only a successful AggregatedList that finds no instance means the
instance is absent. A failed lookup (403, 5xx, transport error) said
nothing, yet was also turned into a synthetic 404, letting callers reap
local state while the VM may still be running. Lookup failures now
propagate, for delete as well.

The pre-create existence check also returns for bulk-mode retries that
already carry a persisted ResolvedZone, keeping duplicate protection
via a cheap zonal lookup. Only the no-zone first attempt skips it.

The stop/start/inspect recovery tests now assert the exact method and
path, so a wrong operation against the right zone no longer passes.

GetIP was the remaining zone consumer reachable with an unresolved
zone: called before any recovered operation, it still hit the raw
empty-zone 400. deleteDisk needs nothing: Remove reuses the ComputeUtil
whose zone deleteInstance already recovered.

The UseExisting clause in the pre-create gate was dead: UseExisting and
BulkInsert are mutually exclusive at flag parsing, so effectiveZone is
never empty when UseExisting is set.

unwrapGoogleError flattened the synthetic 404 into a plain error,
which defeated callers' isNotFound checks.

GetState swallowed the instance and disk lookup errors, so a transient
API failure (403, 5xx, transport) reported an existing machine as
absent, and state.None is what tells callers to reap local state. Only
a genuine not-found proceeds to the disk check or reports absence now.

Recover the instance zone for stop, start, and inspect in bulk mode

See merge request !187

Merged-by: Igor <iwiedler@gitlab.com>
Approved-by: Kam Kyrala <kkyrala@gitlab.com>
Approved-by: Tomasz Maczukin <tomasz@maczukin.pl>
Reviewed-by: GitLab Duo <gitlab-duo@gitlab.com>

Managers need to refresh a liveness label on the instances they track
so an external reaper can tell orphaned VMs from tracked ones
(production-engineering#29652). The google driver merges labels into
the instance via SetLabels with the current fingerprint. Drivers that
don't implement LabelUpdater report label updates as unsupported.

Review feedback: the label-merge and fingerprint handling in the google
driver and the RPC delegation had no coverage, only the command layer.

An older plugin passes the API version handshake and then fails the
label call with a raw rpc method-lookup error. Map that to the same
error a non-implementing driver returns. Bumping the RPC API version
instead would reject every external driver plugin built against v1,
which is disproportionate for an optional feature.

Review feedback: zero arguments previously resolved the docker-machine
default host, which is meaningless for label updates. Also clarify the
invalid-label error message.

Add update-labels command backed by a LabelUpdater driver interface

See merge request !190

Merged-by: Igor <iwiedler@gitlab.com>
Approved-by: Tomasz Maczukin <tomasz@maczukin.pl>
Reviewed-by: Tomasz Maczukin <tomasz@maczukin.pl>
Reviewed-by: GitLab Duo <gitlab-duo@gitlab.com>

Split the Google COS readiness gate into two independent opt-ins. The
existing --google-cos-docker-network-readiness-gate now only waits for
cloud-init. A new --google-cos-docker-network-readiness-url runs the
container egress probe against the given URL, and only when it is set.

The previous probe fetched the GCE metadata server on :80 from a probe
container. On a correctly-firewalled worker the DOCKER-USER rule drops
container traffic to the metadata server, so the probe could never pass
there and every machine creation failed. Fetching an operator-supplied
URL exercises the same bridge NAT/FORWARD/MASQUERADE path against a
destination the worker firewall permits. Success means any HTTP response
came back; a redirect or an error status still proves egress works.

The URL is validated driver-side (http/https, host present, no shell
metacharacters) before it reaches the worker shell command.

The URL reached the worker shell interpolated unquoted, guarded by a
character blocklist. The blocklist rejected legitimate URLs (query
strings) and was the wrong tool for injection safety.

Pass the URL as a positional argument to the inner sh -c and single-quote
it (' -> '\''), so any value reaches wget as one literal word. The
driver-side check keeps only the http/https scheme and host validation
for a fast failure at flag time.

TestShellQuoteSurvivesTwoShellLayers runs hostile inputs (embedded
quotes, command substitution, backticks, query strings) through both
shell layers and asserts the URL arrives verbatim with no side effect.

Add a separate COS readiness URL egress check

See merge request !191

Merged-by: Igor <iwiedler@gitlab.com>
Approved-by: Kam Kyrala <kkyrala@gitlab.com>
Reviewed-by: GitLab Duo <gitlab-duo@gitlab.com>

Bump version to 0.16.2-gitlab.54

See merge request !192

Merged-by: Igor <iwiedler@gitlab.com>
Approved-by: Axel von Bertoldi <avonbertoldi@gitlab.com>