Commit 03bbd847 authored by Marcia Ramos's avatar Marcia Ramos 🏃

Merge branch 'docs/ci-caching' into 'master'

Add docs on CI caching

Closes #32515, #18945, and gitlab-runner#1621

See merge request gitlab-org/gitlab-ce!12183
parents 9027d023 d24ee39e
Pipeline #19290376 passed with stages
in 36 minutes and 55 seconds
......@@ -65,7 +65,8 @@ learn how to leverage its potential even more.
environments and use them for different purposes like testing, building and
deploying
- [Job artifacts](../user/project/pipelines/job_artifacts.md)
- [Git submodules](git_submodules.md): How to run your CI jobs when Git
- [Caching dependencies](caching/index.md)
- [Git submodules](git_submodules.md) - How to run your CI jobs when Git
submodules are involved
- [Use SSH keys in your build environment](ssh_keys/README.md)
- [Trigger pipelines through the GitLab API](triggers/README.md)
......
This diff is collapsed.
......@@ -146,24 +146,7 @@ To protect/unprotect Runners:
## Manually clearing the Runners cache
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/41249) in GitLab 10.4.
GitLab Runners use [cache](../yaml/README.md#cache) to speed up the execution
of your jobs by reusing existing data. This however, can sometimes lead to an
inconsistent behavior.
To start with a fresh copy of the cache, you can easily do it via GitLab's UI:
1. Navigate to your project's **CI/CD > Pipelines** page.
1. Click on the **Clear Runner caches** to clean up the cache.
1. On the next push, your CI/CD job will use a new cache.
That way, you don't have to change the [cache key](../yaml/README.md#cache-key)
in your `.gitlab-ci.yml`.
Behind the scenes, this works by increasing a counter in the database, and the
value of that counter is used to create the key for the cache. After a push, a
new key is generated and the old cache is not valid anymore.
Read [clearing the cache](../caching/index.md#clearing-the-cache).
## How shared Runners pick jobs
......@@ -227,15 +210,16 @@ that it may encounter on the projects it's shared over. This would be
problematic for large amounts of projects, if it wasn't for tags.
By tagging a Runner for the types of jobs it can handle, you can make sure
shared Runners will only run the jobs they are equipped to run.
shared Runners will [only run the jobs they are equipped to run](../yaml/README.md#tags).
For instance, at GitLab we have Runners tagged with "rails" if they contain
the appropriate dependencies to run Rails test suites.
### Preventing Runners with tags from picking jobs without tags
You can configure a Runner to prevent it from picking jobs with tags when
the Runner does not have tags assigned. This setting can be enabled the first
You can configure a Runner to prevent it from picking
[jobs with tags](../yaml/README.md#tags) when the Runner does not have tags
assigned. This setting can be enabled the first
time you [register a Runner][register] and can be changed afterwards under
each Runner's settings.
......
......@@ -674,6 +674,10 @@ as Review Apps. You can see a simple example using Review Apps at
by default.
- From GitLab 9.2, caches are restored before [artifacts](#artifacts).
TIP: **Learn more:**
Read how caching works and find out some good practices in the
[caching dependencies documentation](../caching/index.md).
`cache` is used to specify a list of files and directories which should be
cached between jobs. You can only use paths that are within the project
workspace.
......@@ -681,35 +685,20 @@ workspace.
If `cache` is defined outside the scope of jobs, it means it is set
globally and all jobs will use that definition.
Cache all files in `binaries` and `.config`:
```yaml
rspec:
script: test
cache:
paths:
- binaries/
- .config
```
Cache all Git untracked files:
### `cache:paths`
```yaml
rspec:
script: test
cache:
untracked: true
```
Use the `paths` directive to choose which files or directories will be cached.
Wildcards can be used as well.
Cache all Git untracked files and files in `binaries`:
Cache all files in `binaries` that end in `.apk` and the `.config` file:
```yaml
rspec:
script: test
cache:
untracked: true
paths:
- binaries/
- binaries/*.apk
- .config
```
Locally defined cache overrides globally defined options. The following `rspec`
......@@ -723,33 +712,26 @@ cache:
rspec:
script: test
cache:
key: rspec
paths:
- binaries/
```
Note that since cache is shared between jobs, if you're using different
paths for different jobs, you should also set a different **cache:key**
otherwise cache content can be overwritten.
NOTE: **Note:**
The cache is provided on a best-effort basis, so don't expect that the cache
will be always present.
### `cache:key`
> Introduced in GitLab Runner v1.0.0.
The `key` directive allows you to define the affinity of caching
between jobs, allowing to have a single cache for all jobs,
cache per-job, cache per-branch or any other way that fits your needs.
Since the cache is shared between jobs, if you're using different
paths for different jobs, you should also set a different `cache:key`
otherwise cache content can be overwritten.
This way, you can fine tune caching, allowing you to cache data between
different jobs or even different branches.
The `key` directive allows you to define the affinity of caching between jobs,
allowing to have a single cache for all jobs, cache per-job, cache per-branch
or any other way that fits your workflow. This way, you can fine tune caching,
allowing you to cache data between different jobs or even different branches.
The `cache:key` variable can use any of the
[predefined variables](../variables/README.md), and the default key, if not set,
is set as `$CI_JOB_NAME-$CI_COMMIT_REF_NAME` which translates as "per-job and
is `$CI_JOB_NAME-$CI_COMMIT_REF_NAME` which translates as "per-job and
per-branch". It is the default across the project, therefore everything is
shared between pipelines and jobs running on the same branch by default.
......@@ -757,56 +739,56 @@ NOTE: **Note:**
The `cache:key` variable cannot contain the `/` character, or the equivalent
URI-encoded `%2F`; a value made only of dots (`.`, `%2E`) is also forbidden.
**Example configurations**
To enable per-job caching:
```yaml
cache:
key: "$CI_JOB_NAME"
untracked: true
```
To enable per-branch caching:
For example, to enable per-branch caching:
```yaml
cache:
key: "$CI_COMMIT_REF_SLUG"
untracked: true
paths:
- binaries/
```
To enable per-job and per-branch caching:
If you use **Windows Batch** to run your shell scripts you need to replace
`$` with `%`:
```yaml
cache:
key: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
untracked: true
key: "%CI_JOB_STAGE%-%CI_COMMIT_REF_SLUG%"
paths:
- binaries/
```
To enable per-branch and per-stage caching:
If you use **Windows PowerShell** to run your shell scripts you need to replace
`$` with `$env:`:
```yaml
cache:
key: "$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG"
untracked: true
key: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_SLUG"
paths:
- binaries/
```
If you use **Windows Batch** to run your shell scripts you need to replace
`$` with `%`:
### `cache:untracked`
Set `untracked: true` to cache all files that are untracked in your Git
repository:
```yaml
cache:
key: "%CI_JOB_STAGE%-%CI_COMMIT_REF_SLUG%"
untracked: true
rspec:
script: test
cache:
untracked: true
```
If you use **Windows PowerShell** to run your shell scripts you need to replace
`$` with `$env:`:
Cache all Git untracked files and files in `binaries`:
```yaml
cache:
key: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_SLUG"
untracked: true
rspec:
script: test
cache:
untracked: true
paths:
- binaries/
```
### `cache:policy`
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment