Commit 0b3eb180 authored by Achilleas Pipinellis's avatar Achilleas Pipinellis 🚀

Documentation clean-up.

parent 92b01edc
## GitLab CI Multi-purpose Runner
This is GitLab CI Multi-purpose Runner repository an **unofficial GitLab CI runner written in Go**, this application run tests and sends the results to GitLab CI.
[GitLab CI](https://about.gitlab.com/gitlab-ci) is the open-source continuous integration server that coordinates the testing.
This is the GitLab CI Multi-purpose Runner repository, the official GitLab CI
runner written in Go. It runs tests and sends the results to GitLab CI.
[GitLab CI](https://about.gitlab.com/gitlab-ci) is the open-source
continuous integration server that coordinates the testing.
[![Build Status](https://ci.gitlab.com/projects/1885/status.png?ref=master)](https://ci.gitlab.com/projects/1885?ref=master)
......@@ -14,7 +16,9 @@ The official repository for this project is on [GitLab.com](https://gitlab.com/g
### Requirements
**None. This project is designed for the Linux, OS X and Windows operating systems.**
**None:** gitlab-ci-multi-runner is run as a single binary.
This project is designed for the Linux, OS X and Windows operating systems.
### Features
......@@ -38,26 +42,27 @@ The official repository for this project is on [GitLab.com](https://gitlab.com/g
### Installation
* [Install using Debian/Ubuntu/CentOS/RedHat package (preferred)](docs/install-on-linux.md)
* [Install on OSX (preffered)](docs/install-on-osx.md)
* [Install on Windows (preffered)](docs/install-on-windows.md)
* [Install as Docker Service](docs/install-on-docker.md)
* [Manual installation (advanced)](docs/install-manually.md)
* [Bleeding edge (development)](docs/install-bleeding-edge.md)
* [Install using GitLab's repository for Debian/Ubuntu/CentOS/RedHat (preferred)](docs/install/linux-repository.md)
* [Install on OSX (preffered)](docs/install/osx.md)
* [Install on Windows (preffered)](docs/install/windows.md)
* [Install as Docker Service](docs/install/docker.md)
* [Manual installation (advanced)](docs/install/linux-manually.md)
* [Bleeding edge (development)](docs/install/bleeding-edge.md)
### Advanced Configuration
* [See advanced configuration options](docs/advanced-configuration.md)
* [See advanced configuration options](docs/configuration/advanced-configuration.md)
* [See example configuration file](config.toml.example)
### Example integrations
* [Integrate GitLab CE](docs/example-integration-gitlab.md)
* [Integrate GitLab CI](docs/example-integration-gitlab-ci.md)
* [Integrate GitLab CE](docs/examples/gitlab.md)
* [Integrate GitLab CI](docs/examples/gitlab-ci.md)
### Extra projects?
If you want to add another project, token or image simply RE-RUN SETUP. *You don't have to re-run the runner. He will automatically reload configuration once it changes.*
If you want to add another project, token or image simply RE-RUN SETUP.
*You don't have to re-run the runner. It will automatically reload configuration once it changes.*
### Changelog
......
### Config file
Configuration uses TOML format described here: https://github.com/toml-lang/toml
1. The global section:
```
concurrent = 4
root_dir = ""
```
This defines global settings of multi-runner:
* `concurrent` - limits how many jobs globally can be run concurrently. The most upper limit of jobs using all defined runners
* `root_dir` - allows to change relative dir where all builds, caches, etc. are stored. By default is current working directory
1. The [[runners]] section:
```
[[runners]]
name = "ruby-2.1-docker"
url = "https://CI/"
token = "TOKEN"
limit = 0
executor = "docker"
builds_dir = ""
shell = ""
clean_environment = false
environment = ["ENV=value", "LC_ALL=en_US.UTF-8"]
disable_verbose = false
```
This defines one runner entry:
* `name` - not used, just informatory
* `url` - CI URL
* `token` - runner token
* `limit` - limit how many jobs can be handled concurrently by this token. 0 simply means don't limit.
* `executor` - select how project should be built. See below.
* `builds_dir` - directory where builds will be stored in context of selected executor (Locally, Docker, SSH)
* `clean_environment` - do not inherit any environment variables from the multi-runner process
* `environment` - append or overwrite environment variables
* `shell` - the name of shell to generate the script (default value is platform dependent)
1. The EXECUTORS:
There are a couple of available executors currently:
* **shell** - run build locally, default
* **docker** - run build using Docker container - this requires the presence of *[runners.docker]*
* **docker-ssh** - run build using Docker container, but connect to it with SSH - this requires the presence of *[runners.docker]* and *[runners.ssh]*
* **ssh** - run build remotely with SSH - this requires the presence of *[runners.ssh]*
* **parallels** - run build using Parallels VM, but connect to it with SSH - this requires the presence of *[runners.parallels]* and *[runners.ssh]*
1. The SHELLS:
There are a couple of available shells that can be run on different platforms:
* **bash** - generate Bash (Bourne-shell) script. All commands executed in Bash context (default for all Unix systems)
* **cmd** - generate Windows Batch script. All commands are executed in Batch context (default for Windows)
* **powershell** - generate Windows PowerShell script. All commands are executed in PowerShell context
1. The [runners.docker] section:
```
[runners.docker]
host = ""
hostname = ""
tls_cert_path = "/Users/ayufan/.boot2docker/certs"
image = "ruby:2.1"
privileged = false
disable_cache = false
disable_pull = false
wait_for_services_timeout = 30
cache_dir = ""
registry = ""
volumes = ["/data", "/home/project/cache"]
extra_hosts = ["other-host:127.0.0.1"]
links = ["mysql_container:mysql"]
services = ["mysql", "redis:2.8", "postgres:9"]
```
This defines the Docker Container parameters:
* `host` - specify custom Docker endpoint, by default *DOCKER_HOST* environment is used or *"unix:///var/run/docker.sock"*
* `hostname` - specify custom hostname for Docker container
* `tls_cert_path` - when set it will use ca.pem, cert.pem and key.pem from that folder to make secure TLS connection to Docker (useful in boot2docker)
* `image` - use this image to run builds
* `privileged` - make container run in Privileged mode (insecure)
* `disable_cache` - disable automatic
* `disable_pull` - disable automatic image pulling if not found
* `wait_for_services_timeout` - specify how long to wait for docker services, set to 0 to disable, default: 30
* `cache_dir` - specify where Docker caches should be stored (this can be absolute or relative to current working directory)
* `registry` - specify custom Docker registry to be used
* `volumes` - specify additional volumes that should be cached
* `extra_hosts` - specify hosts that should be defined in container environment
* `links` - specify containers which should be linked with building container
* `services` - specify additional services that should be run with build. Please visit [Docker Registry](https://registry.hub.docker.com/) for list of available applications. Each service will be run in separate container and linked to the build.
1. The [runners.parallels] section:
```
[runners.parallels]
base_name = "my-parallels-image"
template_name = ""
disable_snapshots = false
```
This defines the Parallels parameters:
* `base_name` - name of Parallels VM which will be cloned
* `template_name` - custom name of Parallels VM linked template (optional)
* `disable_snapshots` - if disabled the VMs will be destroyed after build
1. The [runners.ssh] section:
```
[runners.ssh]
host = "my-production-server"
port = "22"
user = "root"
password = "production-server-password"
```
This defines the SSH connection parameters:
* `host` - where to connect (it's override when using *docker-ssh*)
* `port` - specify port, default: 22
* `user` - specify user
* `password` - specify password
Configuration uses the TOML format as described here: <https://github.com/toml-lang/toml>.
The file to be edited can be found in `/home/gitlab_ci_multi_runner/config.toml`.
### The global section
This defines global settings of multi-runner.
| Setting | Explanation |
| ------- | ----------- |
| `concurrent` | limits how many jobs globally can be run concurrently. The most upper limit of jobs using all defined runners |
| `root_dir` | allows to change relative dir where all builds, caches, etc. are stored. By default is current working directory |
Example:
```bash
concurrent = 4
root_dir = ""
```
### The [[runners]] section
This defines one runner entry.
| Setting | Explanation |
| ------- | ----------- |
| `name` | not used, just informatory |
| `url` | CI URL |
| `token` | runner token |
| `limit` | limit how many jobs can be handled concurrently by this token. 0 simply means don't limit |
| `executor` | select how a project should be built, see next section |
| `builds_dir` | directory where builds will be stored in context of selected executor (Locally, Docker, SSH) |
| `clean_environment` | do not inherit any environment variables from the multi-runner process |
| `environment` | append or overwrite environment variables |
| `shell` | the name of shell to generate the script (default value is platform dependent) |
Example:
```bash
[[runners]]
name = "ruby-2.1-docker"
url = "https://CI/"
token = "TOKEN"
limit = 0
executor = "docker"
builds_dir = ""
shell = ""
clean_environment = false
environment = ["ENV=value", "LC_ALL=en_US.UTF-8"]
disable_verbose = false
```
### The EXECUTORS
There are a couple of available executors currently.
| Executor | Explanation |
| -------- | ----------- |
| `shell` | run build locally, default |
| `docker` | run build using Docker container - this requires the presence of `[runners.docker]` |
| `docker-ssh` | run build using Docker container, but connect to it with SSH - this requires the presence of `[runners.docker]` and `[runners.ssh]` |
| `ssh` | run build remotely with SSH - this requires the presence of `[runners.ssh]` |
| `parallels` | run build using Parallels VM, but connect to it with SSH - this requires the presence of `[runners.parallels]` and `[runners.ssh]` |
### The SHELLS
There are a couple of available shells that can be run on different platforms.
| Shell | Explanation |
| ----- | ----------- |
| `bash` | generate Bash (Bourne-shell) script. All commands executed in Bash context (default for all Unix systems) |
| `cmd` | generate Windows Batch script. All commands are executed in Batch context (default for Windows) |
| `powershell` | generate Windows PowerShell script. All commands are executed in PowerShell context |
### The [runners.docker] section
This defines the Docker Container parameters.
| Parameter | Explanation |
| --------- | ----------- |
| `host` | specify custom Docker endpoint, by default `DOCKER_HOST` environment is used or `unix:///var/run/docker.sock` |
| `hostname` | specify custom hostname for Docker container |
| `tls_cert_path` | when set it will use `ca.pem`, `cert.pem` and `key.pem` from that folder to make secure TLS connection to Docker (useful in boot2docker) |
| `image` | use this image to run builds |
| `privileged` | make container run in Privileged mode (insecure) |
| `disable_cache` | disable automatic |
| `disable_pull` | disable automatic image pulling if not found |
| `wait_for_services_timeout` | specify how long to wait for docker services, set to 0 to disable, default: 30 |
| `cache_dir` | specify where Docker caches should be stored (this can be absolute or relative to current working directory) |
| `registry` | specify custom Docker registry to be used |
| `volumes` | specify additional volumes that should be cached |
| `extra_hosts` | specify hosts that should be defined in container environment |
| `links` | specify containers which should be linked with building container |
| `services` | specify additional services that should be run with build. Please visit [Docker Registry](https://registry.hub.docker.com/) for list of available applications. Each service will be run in separate container and linked to the build. |
Example:
```bash
[runners.docker]
host = ""
hostname = ""
tls_cert_path = "/Users/ayufan/.boot2docker/certs"
image = "ruby:2.1"
privileged = false
disable_cache = false
disable_pull = false
wait_for_services_timeout = 30
cache_dir = ""
registry = ""
volumes = ["/data", "/home/project/cache"]
extra_hosts = ["other-host:127.0.0.1"]
links = ["mysql_container:mysql"]
services = ["mysql", "redis:2.8", "postgres:9"]
```
### The [runners.parallels] section
This defines the Parallels parameters.
| Parameter | Explanation |
| --------- | ----------- |
| `base_name` | name of Parallels VM which will be cloned |
| `template_name` | custom name of Parallels VM linked template (optional) |
| `disable_snapshots` | if disabled the VMs will be destroyed after build |
Example:
```bash
[runners.parallels]
base_name = "my-parallels-image"
template_name = ""
disable_snapshots = false
```
### The [runners.ssh] section
This defines the SSH connection parameters.
| Parameter | Explanation |
| ---------- | ----------- |
| `host` | where to connect (overriden when using `docker-ssh`) |
| `port` | specify port, default: 22 |
| `user` | specify user |
| `password` | specify password |
Example:
```
[runners.ssh]
host = "my-production-server"
port = "22"
user = "root"
password = "production-server-password"
```
### How to configure runner for GitLab CI integration tests? (uses confined Docker executor)
### How to configure runner for GitLab CI integration tests (uses confined Docker executor)
1. Run setup
```bash
......@@ -62,6 +62,8 @@
bundle exec rake spec
```
1. Voila! You now have GitLab CI integration testing instance with bundle caching. Push some commits to test it.
1. You now have GitLab CI integration testing instance with bundle caching.
Push some commits to test it.
1. Look into `config.toml` and tune it.
1. For [advanced setup](../configuration/advanced_setup.md), look into
`/home/gitlab_ci_multi_runner/config.toml` and tune it.
### How to configure runner for GitLab CE integration tests? (uses confined Docker executor)
### How to configure runner for GitLab CE integration tests (uses confined Docker executor)
1. Run setup
```bash
......@@ -13,7 +12,7 @@
--docker-postgres latest --docker-redis latest
```
1. Add job to test with MySQL
1. Add a job to test with MySQL
```bash
wget -q http://ftp.de.debian.org/debian/pool/main/p/phantomjs/phantomjs_1.9.0-1+b1_amd64.deb
dpkg -i phantomjs_1.9.0-1+b1_amd64.deb
......@@ -23,7 +22,7 @@
bundle install --deployment --path /cache
cp config/application.yml.example config/application.yml
cp config/gitlab.yml.example config/gitlab.yml
cp config/database.yml.mysql config/database.yml
sed -i 's/username:.*/username: root/g' config/database.yml
......@@ -38,7 +37,7 @@
bundle exec rake test_ci
```
1. Add job to test with PostgreSQL
1. Add a job to test with PostgreSQL
```bash
wget -q http://ftp.de.debian.org/debian/pool/main/p/phantomjs/phantomjs_1.9.0-1+b1_amd64.deb
dpkg -i phantomjs_1.9.0-1+b1_amd64.deb
......@@ -48,7 +47,7 @@
bundle install --deployment --path /cache
cp config/application.yml.example config/application.yml
cp config/gitlab.yml.example config/gitlab.yml
cp config/database.yml.postgresql config/database.yml
sed -i 's/username:.*/username: postgres/g' config/database.yml
......@@ -63,6 +62,8 @@
bundle exec rake test_ci
```
1. Voila! You now have GitLab CE integration testing instance with bundle caching. Push some commits to test it.
1. You now have a GitLab CE integration testing instance with bundle caching.
Push some commits to test it.
1. Look into `config.toml` and tune it.
1. For [advanced setup](../configuration/advanced_setup.md), look into
`/home/gitlab_ci_multi_runner/config.toml` and tune it.
......@@ -39,12 +39,16 @@ rpm -i gitlab-ci-multi-runner_386.rpm
1. Download any other tagged release:
Simple replace the `master` with either `tag` (v0.2.0) or `latest` (the latest stable).
Simply replace `master` with either `tag` (v0.2.0) or `latest` (the latest
stable). For a list of tags see <https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/tags>.
For example:
* https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/master/binaries/gitlab-ci-multi-runner-linux-386
* https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-ci-multi-runner-linux-386
* https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/v0.2.0/binaries/gitlab-ci-multi-runner-linux-386
If you have problem downloading fallback to http://:
If you have problem downloading through `https`, fallback to plain `http`:
* http://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/master/binaries/gitlab-ci-multi-runner-linux-386
* http://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-ci-multi-runner-linux-386
* http://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/v0.2.0/binaries/gitlab-ci-multi-runner-linux-386
## Run gitlab-ci-multi-runner in a container
### Docker image installation and configuration (run gitlab-ci-multi-runner in a container)
### Docker image installation and configuration
1. Install Docker first:
```bash
......@@ -8,7 +9,8 @@
1. Start the container:
We need to mount a data volume into our gitlab-ci-multi-runner container to be used for configs and other resources:
We need to mount a data volume into our gitlab-ci-multi-runner container to
be used for configs and other resources:
```bash
$ docker run -d --name multi-runner --restart always \
-v /PATH/TO/DATA/FOLDER:/data \
......@@ -25,7 +27,8 @@
ayufan/gitlab-ci-multi-runner:latest
```
If you are planning on using Docker as the method of spawing runners you'll need to mount your docker socket like so:
If you are planning on using Docker as the method of spawing runners,
you will need to mount your docker socket like so:
```bash
$ docker run -d --name multi-runner --restart always \
-v /var/run/docker.sock:/var/run/docker.sock \
......@@ -71,13 +74,21 @@
-v /var/run/docker.sock:/var/run/docker.sock \
ayufan/gitlab-ci-multi-runner:latest
```
**note**: you need to use the same method for mounting you data volume as you did originally (`-v /PATH/TO/DATA/FOLDER:/data` or `--volumes-from multi-runner-data`)
**Note**: you need to use the same method for mounting you data volume as you
did originally (`-v /PATH/TO/DATA/FOLDER:/data` or `--volumes-from multi-runner-data`)
#### Installing Trusted SSL Server Certificates
If your GitLab CI server is using self-signed SSL certificates then you should make sure the GitLab CI server certificate is trusted by the gitlab-ci-multi-runner container for them to be able to talk to each other.
If your GitLab CI server is using self-signed SSL certificates then you should
make sure the GitLab CI server certificate is trusted by the gitlab-ci-multi-runner
container for them to be able to talk to each other.
The gitlab-ci-multi-runner image is configured to look for the trusted SSL certificates at `/data/certs/ca.crt`, this can however be changed using the `-e "CA_CERTIFICATES_PATH=/DIR/CERT"` configuration option.
The gitlab-ci-multi-runner image is configured to look for the trusted SSL
certificates at `/data/certs/ca.crt`, this can however be changed using the
`-e "CA_CERTIFICATES_PATH=/DIR/CERT"` configuration option.
Copy the `ca.crt` file into the `certs` directory on the data volume (or container). The `ca.crt` file should contain the root certificates of all the servers you want gitlab-ci-multi-runner to trust.
The gitlab-ci-multi-runner container will import the `ca.crt` file on startup so if your container is already running you may need to restart it for the changes to take place.
Copy the `ca.crt` file into the `certs` directory on the data volume (or container).
The `ca.crt` file should contain the root certificates of all the servers you
want gitlab-ci-multi-runner to trust. The gitlab-ci-multi-runner container will
import the `ca.crt` file on startup so if your container is already running you
may need to restart it for the changes to take effect.
......@@ -26,7 +26,7 @@
1. Setup the runner
```bash
cd ~gitlab_ci_multi_runner
sudo -u gitlab_ci_multi_runner -H gitlab-ci-multi-runner-linux setup
sudo -u gitlab_ci_multi_runner -H gitlab-ci-multi-runner setup
```
1. Secure config.toml
......
### Install and initial configuration (For Debian, Ubuntu and CentOS)
1. If you want to use Docker runnner install it before:
1. If you want to use Docker runnner, install it before using the multi runner:
```bash
curl -sSL https://get.docker.com/ | sh
```
1. Add package to apt-get or yum
1. Add GitLab's official repository via apt-get or yum
```bash
# For Debian/Ubuntu
curl https://packages.gitlab.com/install/repositories/runner/gitlab-ci-multi-runner/script.deb | sudo bash
......@@ -47,10 +47,11 @@
1. Simply execute to install latest version
```bash
# For Debian/Ubuntu
# For Debian/Ubuntu
apt-get update
apt-get install gitlab-ci-multi-runner
# For CentOS
yum update
yum install gitlab-ci-multi-runner
```
### Install on OSX
(In the future there will be brew package)
(In the future there will be a brew package)
1. Download binary for your system:
1. Download the binary for your system:
```bash
sudo wget -O /usr/local/bin/gitlab-ci-multi-runner https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-ci-multi-runner-darwin-amd64
```
......@@ -12,7 +12,7 @@
sudo chmod +x /usr/local/bin/gitlab-ci-multi-runner
```
1. The rest of commands execute as user who will run the runner
1. The rest of commands execute as the user who will run the runner.
1. Setup the runner
```bash
......
### Install on Windows
1. Create some folder somewhere in your system, ex.: `C:\Multi-Runner`.
1. Create a folder somewhere in your system, ex.: `C:\Multi-Runner`.
1. Download binary for [x86](https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-ci-multi-runner-windows-386.exe) or [amd64](https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-ci-multi-runner-windows-amd64.exe) and put it into previously saved folder.
1. Download the binary for [x86][] or [amd64][] and put it into the folder you
created.
1. Run `Administrator` command prompt. How to do that is described here: http://pcsupport.about.com/od/windows-8/a/elevated-command-prompt-windows-8.htm. The simplest is to write `Command Prompt` in Windows search field and right click and select `Run as administrator`. You will be asked to confirm that you want to execute elevated command prompt.
1. Run an `Administrator` command prompt ([How to][prompt]). The simplest is to
write `Command Prompt` in Windows search field, right click and select
`Run as administrator`. You will be asked to confirm that you want to execute
the elevated command prompt.
1. Go to created folder: `cd C:\Multi-Runner`.
1. Go to the folder you created before: `cd C:\Multi-Runner`.
1. Setup the runner
```batch
$ C:\
$ cd C:\Multi-Runner
$ gitlab-ci-multi-runner-linux setup
Please enter the gitlab-ci coordinator URL (e.g. http://gitlab-ci.org:3000/ )
https://ci.gitlab.org/
https://ci.gitlab.com
Please enter the gitlab-ci token for this runner
xxx
Please enter the gitlab-ci description for this runner
......@@ -27,7 +30,8 @@
INFO[0037] Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
```
1. Install runner as service and start it. You have to enter valid password for current user account, because it's required to start the service by Windows.
1. Install runner as a service and start it. You have to enter a valid password
for the current user account, because it's required to start the service by Windows.
```bash
$ gitlab-ci-multi-runner install --password ENTER-YOUR-PASSWORD
$ gitlab-ci-multi-runner start
......@@ -41,14 +45,17 @@
1. Stop service (you need elevated command prompt as before):
```batch
$ C:\
$ cd C:\Multi-Runner
$ gitlab-ci-multi-runner stop
```
1. Download binary for [x86](https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-ci-multi-runner-windows-386.exe) or [amd64](https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-ci-multi-runner-windows-amd64.exe) and replace runner's executable.
1. Download the binary for [x86][] or [amd64][] and replace runner's executable.
1. Start service:
```batch
$ gitlab-ci-multi-runner start
```
[x86]: https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-ci-multi-runner-windows-386.exe
[amd64]: https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-ci-multi-runner-windows-amd64.exe
[prompt]: http://pcsupport.about.com/od/windows-8/a/elevated-command-prompt-windows-8.htm
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment