In the past, the runner needed to resolve a full TLS certificate
chain, including the self-signed root, in order for Git clones to work
over HTTPS. Go 1.9 changed the behavior to present a partial
certificate chain if a trusted intermediate certificate were placed in
the system certificate directory
(https://github.com/golang/go/issues/24685).
!1581
worked around that change by restoring the Go 1.8 behavior of
presenting the full chain in `CI_SERVER_TLS_CA_FILE`.

libcurl v7.68 has since fixed the behavior to trust a certificate
authority that is not self-signed
(https://github.com/curl/curl/commit/94f1f771586913addf5c68f9219e176036c50115).
As a result, the need to resolve the full chain is no longer
necessary. As long as there is a trusted certificate authority in the
chain, the TLS connection can proceed.

Go 1.18 modified `Certificate.Verify` to use the macOS and
Windows-specific platform APIs. As a result, a root certificate signed
with a SHA-1 certificate will be rejected, which prevents the runner
from generating `CI_SERVER_TLS_CA_FILE`. This may cause Git clones to
fail.

This commit adds a feature flag, `FF_RESOLVE_FULL_TLS_CHAIN`, that is
enabled by default. This flag makes it possible to disable this
resolving of the full certificate chain. On most platforms, this can
be disabled safely, assuming Git and other clients are compiled with
an updated libcurl version.

Relates to #29373

Capture helper service logs into job/tasks main trace log

See merge request !3680



Merged-by: Romuald Atchadé <ratchade@gitlab.com>
Approved-by: Romuald Atchadé <ratchade@gitlab.com>
Co-authored-by: Axel von Bertoldi <avonbertoldi@gitlab.com>

Three tests: one to make sure the feature is not enabled when
CI_DEBUG_SERVICES is not set, one to make sure the variable is handled
correctly, one to make sure the feature works (i.e. service container
logs are captured and written to the main trace log, with prefix).

This method will, for each configured (helper) service, call
`captureContainerLogs()` to set up log capture. In this case, logs are
redirected to the current task's trace log (i.e. `e.Trace`) via an
`InlineServiceLogWriter` instance (to prefix logs with the container's
name).

Again, this is analogous (and as similar as possible) to what is being
done in the docker executor to capture service logs there.

There are already two places where `svc-` is used directly, and I'm
about to add a third. May as well make it a const.

Given the name of a kubernetes-managed container (and a reference to the
pod), this method will hook into the container's log stream and copy all
logs to the specified sink, for the duration of the container's life.

This is analogous (and as similar as possible) to what is being done in
the docker executor to capture service logs there.

Three tests: one to make sure the feature is not enabled when
CI_DEBUG_SERVICES is not set, one to make sure the variable is handled
correctly, one to make sure the feature works (i.e. service container
logs are captured and written to the main tracelog, with prefix).

This method will, for each configured (helper) service, call
`captureContainerLogs()` to set up log capture. In this case, logs are
redirected to the current task's trace log (i.e. `e.Trace`) via an
`InlineServiceLogWriter` instance (to prefix logs with the container's
name).

In the future, if we want to redirect the logs elsewhere (e.g. a file to
be subsequently captured as a CI artifact), we just have to change the
`sink` passed to `captureContainerLogs()`.

Given a container's ID, this method will hook into the container's log
stream and copy all logs to the specified sink, for the duration of the
container's life. `stdcopy.StdCopy` handles the actual reading from the
container and writing to the sink, and will block until either the
connection to the container is closed or a read or write error occur.

Note that this approach uses the docker SDK's `ContainerLogs()` API; we
could have also used the `ContainerAttach()` API with basically the same
code here. Both APIs return an `io.Reader` which can be copied via
`StdCopy()`. I opted for `ContainerLogs()` because it's a slightly
simpler API.

Since `StdCopy()` blocks for the life of the service container, we have
to run that call in a goroutine.

Code provided by @ajwalker, with slight modification by me.

I'll use this in the next commit to test the `InlineServiceLogWriter`
type introduced in the previous commit. Specifically, in the success
case the number of bytes written returned by `Write()` must be exactly
the length of the message, and if not all bytes were written, `Write()`
must return an error.

This simple `io.Writer` prefixes log messages/lines with the service
container's name, and adds some colour to the logs. We'll use it later
on to stream service container logs into the tasks main trace log.

The prefix and colour will help to differentiate the service container's
logs from the main task's trace log, and from other service container
logs.

In truth the name of this type could be a little more generic, since
it's actually a property of the supplied sink that makes the final
`io.Writer` an "inline" writer (whatever that means). But int he context
of these changes, this writer is intended to (and will be) used to write
service container logs inline into the main trace logs, hence the name.

These files are quite large, and a good chuck of the code is related
specifically to handling helper services. This is basically a straight
move. The only differences are imports etc.

Update Runner Helm chart docs to include list of deprecated fields  (docs)

See merge request !3686



Merged-by: Romuald Atchadé <ratchade@gitlab.com>
Approved-by: Fiona Neill <fneill@gitlab.com>
Approved-by: Romuald Atchadé <ratchade@gitlab.com>
Co-authored-by: Darren Eastman <deastman@gitlab.com>

Improve development setup docs

See merge request !3661



Merged-by: Joe Burnett <jburnett@gitlab.com>
Approved-by: Joe Burnett <jburnett@gitlab.com>
Approved-by: Elliot Rushton <erushton@gitlab.com>
Co-authored-by: Lee Tickett <ltickett@gitlab.com>

[GitLab issue: x]

[GitLab issue: x]

Update redirected links in the runner docs

See merge request !3690



Merged-by: Suzanne Selhorn <sselhorn@gitlab.com>
Approved-by: Suzanne Selhorn <sselhorn@gitlab.com>
Co-authored-by: Marcel Amirault <mamirault@gitlab.com>

Allow multiple paths in GIT_SUBMODULE_PATHS

See merge request !3675



Merged-by: Axel von Bertoldi <avonbertoldi@gitlab.com>
Approved-by: Axel von Bertoldi <avonbertoldi@gitlab.com>
Co-authored-by: Nico Bollen <nico.bollen@gmail.com>

Add maximum size to uploaded cache

See merge request !3552



Merged-by: Axel von Bertoldi <avonbertoldi@gitlab.com>
Approved-by: Suzanne Selhorn <sselhorn@gitlab.com>
Approved-by: Axel von Bertoldi <avonbertoldi@gitlab.com>
Co-authored-by: Joseph Burnett <jburnett@gitlab.com>
Co-authored-by: Adrien Kohlbecker <adrien.kohlbecker@gmail.com>
Co-authored-by: Suzanne Selhorn <sselhorn@gitlab.com>
Co-authored-by: Arran Walker <ajwalker@gitlab.com>

Ensure `go-fips` container is rebuilt when the version of Go is updated

See merge request !3685



Merged-by: Axel von Bertoldi <avonbertoldi@gitlab.com>
Approved-by: Pedro Pombeiro <noreply@pedro.pombei.ro>

Handle job execution interruption for the new autoscaler executor provider

See merge request !3672



Merged-by: Tomasz Maczukin <tomasz@maczukin.pl>
Approved-by: Joe Burnett <jburnett@gitlab.com>

More specifically, ensure a build of the `go-fips` image is triggered
when the `GO_FIPS_VERSION` variable is changed.

The rules to rebuild the `go-fips` image are when any of the following
files are changed:
```
- dockerfiles/ci/go.fips.Dockerfile
- dockerfiles/ci/go.fips.Dockerfile.rebuild
- .gitlab/ci/prepare.gitlab-ci.yml
```

However, the `GO_FIPS_VERSION` variable is defined in
`.gitlab/ci/_common.gitlab-ci.yml`, which is imported and used in
`.gitlab/ci/prepare.gitlab-ci.yml`. Because `_common.gitlab-ci.yml` is
not in the above list, changing the value of that variable won't trigger
a rebuild of the image, which is wrong.

So, add `.gitlab/ci/_common.gitlab-ci.yml` to the build-rules for the
`go-fips` to ensure it's built when `GO_FIPS_VERSION` changes.

This should fix 29369.

If we only specify the image name (w/o the image registry path), the
security scan that runs agains this image in
https://gitlab.com/gitlab-com/gl-security/appsec/container-scanners will
fail because it tries to `docker pull` `go-fips:<tag>`.

Yes, this here is only a default value, and the correct full name is
specified elsewhere, but may as well fix the default value if we know
it's broken.

Fix docs review app script and domain

See merge request !3682



Merged-by: Evan Read <eread@gitlab.com>
Approved-by: Evan Read <eread@gitlab.com>
Co-authored-by: Marcel Amirault <mamirault@gitlab.com>