Commit 9b6c015a authored by Suzanne Selhorn's avatar Suzanne Selhorn Committed by Marcel Amirault

Edited runner to be lowercase

Related to: gitlab#233529
parent 46f0a217
---
stage: Verify
group: Runner
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
comments: false
---
......
---
stage: Verify
group: Runner
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# The system services of GitLab Runner
GitLab Runner uses the [Go `service` library](https://github.com/kardianos/service)
......
......@@ -19,7 +19,7 @@ as a local proxy and has 2 major advantages compared to adding the proxy details
everywhere manually:
- One single source where you need to change your credentials
- The credentials can not be accessed from the Docker Runners
- The credentials can not be accessed from the Docker runners
Assuming you [have installed CNTLM](https://www.howtoforge.com/linux-ntlm-authentication-proxy-isa-server-with-cntlm),
you need to first configure it.
......@@ -75,9 +75,9 @@ Environment="HTTP_PROXY=http://docker0_interface_ip:3128/"
Environment="HTTPS_PROXY=http://docker0_interface_ip:3128/"
```
## Adding Proxy variables to the Runner configuration
## Adding Proxy variables to the GitLab Runner configuration
The proxy variables need to also be added to the Runner's configuration, so that it can
The proxy variables need to also be added to the GitLab Runner configuration, so that it can
get builds assigned from GitLab behind the proxy.
This is basically the same as adding the proxy to the Docker service above:
......@@ -123,7 +123,7 @@ This is basically the same as adding the proxy to the Docker service above:
## Adding the proxy to the Docker containers
After you [registered your Runner](../register/index.md), you might want to
After you [register your runner](../register/index.md), you might want to
propagate your proxy settings to the Docker containers (for `git clone` and other
stuff).
......@@ -226,7 +226,7 @@ rate limits. As a result, GitLab Runner handles rate limited scenarios with the
1. A response code of **429 - TooManyRequests** is received.
1. The response headers are checked for a `RateLimit-ResetTime` header. The `RateLimit-ResetTime` header should have a value which is a valid **HTTP Date (RFC1123)**, like `Wed, 21 Oct 2015 07:28:00 GMT`.
- If the header is present and has a valid value the Runner waits until the specified time and issues another request.
- If the header is present and has a valid value the runner waits until the specified time and issues another request.
- If the header is present, but isn't a valid date, a fallback of **1 minute** is used.
- If the header is not present, no additional actions are taken, the response error is returned.
1. The process above is repeated 5 times, then a `gave up due to rate limit` error is returned.
......
---
stage: Verify
group: Runner
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Autoscaling GitLab Runner on AWS EC2
......@@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
One of the biggest advantages of GitLab Runner is its ability to automatically
spin up and down VMs to make sure your builds get processed immediately. It's a
great feature, and if used correctly, it can be extremely useful in situations
where you don't use your Runners 24/7 and want to have a cost-effective and
where you don't use your runners 24/7 and want to have a cost-effective and
scalable solution.
## Introduction
......@@ -20,7 +20,7 @@ demand. The runners on these instances are automatically created. They use the p
covered in this guide and do not require manual configuration after creation.
In addition, we'll make use of [Amazon's EC2 Spot instances](https://aws.amazon.com/ec2/spot/) which will
greatly reduce the costs of the Runner instances while still using quite
greatly reduce the costs of the GitLab Runner instances while still using quite
powerful autoscaling machines.
## Prerequisites
......@@ -32,7 +32,7 @@ We suggest a quick read through Docker machine [`amazonec2` driver
documentation](https://docs.docker.com/machine/drivers/aws/) to familiarize
yourself with the parameters we will set later in this article.
Your GitLab instance is going to need to talk to the Runners over the network,
Your GitLab instance is going to need to talk to the runners over the network,
and that is something you need think about when configuring any AWS security
groups or when setting up your DNS configuration.
......@@ -46,7 +46,7 @@ Docker Machine will attempt to use a
[default security group](https://docs.docker.com/machine/drivers/aws/#security-group)
with rules for port `2376` and SSH `22`, which is required for communication with the Docker
daemon. Instead of relying on Docker, you can create a security group with the
rules you need and provide that in the Runner options as we will
rules you need and provide that in the GitLab Runner options as we will
[see below](#the-runnersmachine-section). This way, you can customize it to your
liking ahead of time based on your networking environment.
You have to make sure that ports `2376` and `22` are accessible by the [Runner Manager instance](#prepare-the-runner-manager-instance).
......@@ -59,7 +59,7 @@ Create a new user with [policies](https://docs.aws.amazon.com/AWSEC2/latest/User
for EC2 (AmazonEC2FullAccess) and S3 (AmazonS3FullAccess). To be more secure,
you can disable console login for that user. Keep the tab open or copy paste the
security credentials in an editor as we'll use them later during the
[Runner configuration](#the-runnersmachine-section).
[GitLab Runner configuration](#the-runnersmachine-section).
## Prepare the Runner Manager instance
......@@ -86,19 +86,19 @@ Now that the Runner is installed, it's time to register it.
Before configuring the GitLab Runner, you need to first register it, so that
it connects with your GitLab instance:
1. [Obtain a Runner token](https://docs.gitlab.com/ee/ci/runners/)
1. [Register the Runner](../../register/index.md#linux)
1. [Obtain a runner token](https://docs.gitlab.com/ee/ci/runners/)
1. [Register the runner](../../register/index.md#linux)
1. When asked the executor type, enter `docker+machine`
You can now move on to the most important part, configuring the GitLab Runner.
NOTE:
If you want every user in your instance to be able to use the autoscaled Runners,
register the Runner as a shared one.
If you want every user in your instance to be able to use the autoscaled runners,
register the runner as a shared one.
## Configuring the GitLab Runner
## Configuring the runner
Now that the Runner is registered, you need to edit its configuration file and
Now that the runner is registered, you need to edit its configuration file and
add the required options for the AWS machine driver.
Let's first break it down to pieces.
......@@ -106,12 +106,12 @@ Let's first break it down to pieces.
### The global section
In the global section, you can define the limit of the jobs that can be run
concurrently across all Runners (`concurrent`). This heavily depends on your
needs, like how many users your Runners will accommodate, how much time your
concurrently across all runners (`concurrent`). This heavily depends on your
needs, like how many users GitLab Runner will accommodate, how much time your
builds take, etc. You can start with something low like `10`, and increase or
decrease its value going forward.
The `check_interval` option defines how often the Runner should check GitLab
The `check_interval` option defines how often the runner should check GitLab
for new jobs, in seconds.
Example:
......@@ -128,9 +128,9 @@ are also available.
From the `[[runners]]` section, the most important part is the `executor` which
must be set to `docker+machine`. Most of those settings are taken care of when
you register the Runner for the first time.
you register the runner for the first time.
`limit` sets the maximum number of machines (running and idle) that this Runner
`limit` sets the maximum number of machines (running and idle) that this runner
will spawn. For more information, check the [relationship between `limit`, `concurrent`
and `IdleCount`](../autoscale.md#how-concurrent-limit-and-idlecount-generate-the-upper-limit-of-running-machines).
......@@ -151,8 +151,8 @@ under `[[runners]]` are also available.
### The `runners.docker` section
In the `[runners.docker]` section you can define the default Docker image to
be used by the child Runners if it's not defined in [`.gitlab-ci.yml`](https://docs.gitlab.com/ee/ci/yaml/).
By using `privileged = true`, all Runners will be able to run
be used by the child runners if it's not defined in [`.gitlab-ci.yml`](https://docs.gitlab.com/ee/ci/yaml/).
By using `privileged = true`, all runners will be able to run
[Docker in Docker](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker-executor)
which is useful if you plan to build your own Docker images via GitLab CI/CD.
......@@ -248,7 +248,7 @@ Here's an example of the `runners.machine` section:
The Docker Machine driver is set to `amazonec2` and the machine name has a
standard prefix followed by `%s` (required) that is replaced by the ID of the
child Runner: `gitlab-docker-machine-%s`.
child runner: `gitlab-docker-machine-%s`.
Now, depending on your AWS infrastructure, there are many options you can set up
under `MachineOptions`. Below you can see the most common ones.
......@@ -262,9 +262,9 @@ under `MachineOptions`. Below you can see the most common ones.
| `amazonec2-subnet-id=subnet-xxxx` | The AWS VPC subnet ID. |
| `amazonec2-zone=x` | If not specified, the [availability zone is `a`](https://docs.docker.com/machine/drivers/aws/#environment-variables-and-default-values), it needs to be set to the same availability zone as the specified subnet, for example when the zone is `eu-west-1b` it has to be `amazonec2-zone=b` |
| `amazonec2-use-private-address=true` | Use the private IP address of Docker Machines, but still create a public IP address. Useful to keep the traffic internal and avoid extra costs.|
| `amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true` | AWS extra tag key-value pairs, useful to identify the instances on the AWS console. The "Name" [tag](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html) is set to the machine name by default. We set the "runner-manager-name" to match the Runner name set in `[[runners]]`, so that we can filter all the EC2 instances created by a specific manager setup. |
| `amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true` | AWS extra tag key-value pairs, useful to identify the instances on the AWS console. The "Name" [tag](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html) is set to the machine name by default. We set the "runner-manager-name" to match the runner name set in `[[runners]]`, so that we can filter all the EC2 instances created by a specific manager setup. |
| `amazonec2-security-group=xxxx` | AWS VPC security group name, not the security group ID. See [AWS security groups](#aws-security-groups). |
| `amazonec2-instance-type=m4.2xlarge` | The instance type that the child Runners will run on. |
| `amazonec2-instance-type=m4.2xlarge` | The instance type that the child runners will run on. |
Notes:
......@@ -295,7 +295,7 @@ check_interval = 0
[[runners]]
name = "gitlab-aws-autoscaler"
url = "<URL of your GitLab instance>"
token = "<Runner's token>"
token = "<runner's token>"
executor = "docker+machine"
limit = 20
[runners.docker]
......@@ -435,4 +435,4 @@ You can read the following use cases from which this tutorial was (heavily)
influenced:
- [HumanGeo switched from Jenkins to GitLab](https://about.gitlab.com/blog/2017/11/14/humangeo-switches-jenkins-gitlab-ci/)
- [Substrakt Health - Autoscale GitLab CI Runners and save 90% on EC2 costs](https://about.gitlab.com/blog/2017/11/23/autoscale-ci-runners/)
- [Substrakt Health - Autoscale GitLab CI/CD runners and save 90% on EC2 costs](https://about.gitlab.com/blog/2017/11/23/autoscale-ci-runners/)
......@@ -126,7 +126,7 @@ Instead of using AWS S3, you can create your own cache storage.
```
1. Your cache server will be available at `MY_CACHE_IP:9005`.
1. Create a bucket that will be used by the Runner:
1. Create a bucket that will be used by the runner:
```shell
sudo mkdir /export/runner
......@@ -136,7 +136,7 @@ Instead of using AWS S3, you can create your own cache storage.
bucket, then it will be different. All caches will be stored in the
`/export` directory.
1. Read the Access and Secret Key of MinIO and use it to configure the Runner:
1. Read the Access and Secret Key of MinIO and use it to configure the runner:
```shell
sudo cat /export/.minio.sys/config/config.json | grep Key
......
---
stage: Verify
group: Runner
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Development environment
## 1. Install dependencies and Go runtime
......@@ -34,7 +40,7 @@ pkg install go-1.13.8 gmake git mercurial
## 2. Install Docker Engine
The Docker Engine is required to create pre-built image that is embedded into runner and loaded when using Docker executor.
The Docker Engine is required to create pre-built image that is embedded into GitLab Runner and loaded when using Docker executor.
To install Docker, follow the Docker [installation
instructions](https://docs.docker.com/install/) for your OS.
......@@ -67,13 +73,13 @@ This is required for building ARM images that are embedded into the GitLab Runne
{ echo ':aarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-aarch64-static:CF' > /proc/sys/fs/binfmt_misc/register; } 2>/dev/null
```
## 3. Download runner sources
## 3. Download GitLab Runner sources
```shell
go get gitlab.com/gitlab-org/gitlab-runner
```
## 4. Install runner dependencies
## 4. Install GitLab Runner dependencies
After you clone GitLab Runner, `cd` into the `gitlab-runner` directory and download and restore all build dependencies:
......@@ -85,7 +91,7 @@ make deps
**For FreeBSD use `gmake deps`**
## 5. Run runner
## 5. Run GitLab Runner
Normally you would use `gitlab-runner`. In order to compile and run the Go sources, use the Go toolchain:
......@@ -94,7 +100,7 @@ make runner-and-helper-bin-host
./out/binaries/gitlab-runner run
```
You can run runner in debug-mode:
You can run GitLab Runner in debug-mode:
```shell
make runner-and-helper-bin-host
......@@ -109,8 +115,8 @@ takes care of building the Runner Helper Docker archive dependencies.
If you want to build the Docker images, run `make runner-and-helper-docker-host`, which will:
1. Build `gitlab-runner-helper` and create a helper Docker image from it.
1. Compile Runner for `linux/amd64`.
1. Build a DEB package for Runner. The official Runner images are based on Alpine and Ubuntu,
1. Compile GitLab Runner for `linux/amd64`.
1. Build a DEB package for Runner. The official GitLab Runner images are based on Alpine and Ubuntu,
and the Ubuntu image build uses the DEB package.
1. Build the Alpine and Ubuntu versions of the `gitlab/gitlab-runner` image.
......@@ -148,7 +154,7 @@ If you are developing functionality inside a helper, you'll most likely want to
the version of the Docker image that contains the newest changes.
If you run tests without passing `-ldflags`, the default version in `version.go` is `development`.
This means that the Runner defaults to pulling a [helper image](../configuration/advanced-configuration.md#helper-image)
This means that the runner defaults to pulling a [helper image](../configuration/advanced-configuration.md#helper-image)
with the `latest` tag.
### Make targets
......@@ -211,7 +217,7 @@ if a tool is missing.
## 9. Contribute
You can start hacking GitLab-Runner code.
You can start hacking `gitlab-runner` code.
If you need an IDE to edit and debug code, there are a few free suggestions you can use:
- [JetBrains GoLand IDE](https://www.jetbrains.com/go/).
......@@ -241,7 +247,7 @@ The following are required:
Which virtual machine to use depends on your use case:
- The Windows Server machine has Docker pre-installed and should always
be used when you are developing on Runner for Windows.
be used when you are developing on GitLab Runner for Windows.
- The Windows 10 machine is there for you to have a windows environment
with a GUI which sometimes can help you debugging some Windows
features. Note that you cannot have Docker running inside of Windows
......
---
stage: Verify
group: Runner
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Add Docker executor support for a Windows version
GitLab supports [specific versions of Windows](../install/windows.md#windows-version-support-policy).
......@@ -58,7 +64,7 @@ the [install part](#install).
At this point we should have a base image ready in our production
environment, so we can use it inside the CI pipeline for the GitLab Runner
project. The only thing that is left is to set up the runner managers.
project. The only thing that is left is to set up the Runner Managers.
#### Register
......@@ -69,8 +75,8 @@ settings](https://gitlab.com/gitlab-org/gitlab-runner/-/settings/ci_cd).
The name of the runner should follow the same naming convention as the
existing ones.
For example, for `Windows Server Core 2004` we should name the runner
managers the following:
For example, for `Windows Server Core 2004` we should name the Runner
Managers the following:
1. `windows-2004-private-runner-manager-1`
1. `windows-2004-private-runner-manager-2`
......
---
stage: Verify
group: Runner
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Reviewing GitLab Runner
This document contains rules and suggestions for GitLab Runner project reviewers.
## Reviewing tests coverage reports
In GitLab Runner project, we have a lot of code. Unfortunately, the code coverage is not comprehensive.
In the GitLab Runner project, we have a lot of code. Unfortunately, the code coverage is not comprehensive.
Currently (early 2019), the coverage is on the level of ~55%.
While adding tests to a legacy code is a hard task, we should ensure that new code that is being
......@@ -18,7 +24,7 @@ realistic and will ensure that the most important things are being tested. Dear
Getting back to the technical details...
Runner's CI Pipeline helps us here and provides the coverage reports in HTML format, for tests
The GitLab Runner CI/CD pipeline helps us here and provides the coverage reports in HTML format, for tests
executed in regular (`count`) and race (`atomic`) modes.
There are two places where test coverage reports can be seen. For:
......@@ -37,7 +43,7 @@ To open the report:
1. Find the Pipeline related to the change that we want to review. It may be the latest Pipeline for the
Merge Requests or a Pipeline for the tag. For example, we can look at this one:
<https://gitlab.com/gitlab-org/gitlab-runner/pipelines/48686952>, which released the `v11.8.0` version of Runner.
<https://gitlab.com/gitlab-org/gitlab-runner/pipelines/48686952>, which released the `v11.8.0` version of GitLab Runner.
1. In the pipeline, find the `stable S3` (for tagged releases), `bleeding edge S3` (for `master` and RC tagged releases),
or `development S3` (for regular commits) job which should be present at the `release` stage. In our example
......@@ -142,7 +148,7 @@ As an example, look at <https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requ
- `yml to yaml` - which is the original title and was added to changelog with our script,
- `Fix values.yaml file name in documentation` - which is what I've updated it to in the changelog.
What will `yml to yaml` tell a Runner administrator if they review the changelog before updating
What will `yml to yaml` tell a GitLab Runner administrator if they review the changelog before updating
to a newer version? Does it show the risks behind the update, the implemented behavior changes, a new
behavior/features that were added? Keep these questions in mind when reviewing the merge request and its title.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment