Add option to specify clone path
What does this MR do?
Replaces !876 (closed).
It adds support for specifying GIT_CLONE_PATH
, enabled by default for
docker/kubernetes executor.
Please read: !876 (closed)
Why was this MR needed?
Closes #2211 (closed)
Are there points in the code the reviewer needs to double check?
Does this MR meet the acceptance criteria?
-
Documentation created/updated -
Added tests for this feature/bug -
In case of conflicts with master
- branch was rebased
What are the relevant issue numbers?
-
blocked by !1282 (merged)
Merge request reports
Activity
mentioned in merge request !876 (closed)
changed milestone to %11.5
added Community contribution devopsverify workflowblocked + 1 deleted label
removed workflowblocked label
changed milestone to %11.10
@steveazz I did not yet revisit tests. Please take a look and lets aim it for %11.10. Please pass it to @tmaczukin next.
added 2 commits
added 17 commits
-
a14e0127...6995f661 - 10 commits from branch
master
- c41e6931 - Add custom clone path for the build
- 90057a76 - Set default value for CustomBuildsDir.Enabled per executor
- 9f64a6db - Improve documentation of custom build directories configuration
- 5d2da7f3 - Improve `CI_PROJECT_DIR` logic
- 38c6094a - Remove invalid tests
- 6e1f1dd3 - Expose BuildDir from all variables
- c4898988 - Fix test failures
Toggle commit list-
a14e0127...6995f661 - 10 commits from branch
added 2 commits
@steveazz @tmaczukin
I start to think that the variable that configures that should maybe be written differently, otherwise if we reject
CI_PROJECT_DIR
the user-defined will over overlay the actual PROJECT_DIR.Maybe, naming that:
USER_PROJECT_DIR
orCLONE_PATH
would be a better pick and less ambigious, as we never overlay it?Also, I don't think that we should fail job that has
CLONE_PATH
but this does not get accepted.Edited by Kamil TrzcińskiGood point. I'd chose the
GIT_CLONE_PATH
from the both proposed. It's simple and tells what it does.USER_PROJECT_DIR
is not so obvious.Also, I don't think that we should fail job that has
CLONE_PATH
but this does not get accepted.I'm not fully sure about this. The job specification may require that it will be executed in the arbitrary path, not the automatic one that Runner will create if the feature will be disabled by Runner's administrator. It may end without side effects (the job will work or will just fail - e.g. a Go based job that didn't get started from the GOPATH). But it may have some negative side effects that we will don't know about until they will happen.
If we allow user to set his own directory for the job, we should ensure that it's respected or fail the job in other cases.
If we allow user to set his own directory for the job, we should ensure that it's respected or fail the job in other cases.
I'm not sure of hard fail, but this is safer approach for now. Lets stick with that. It is easier to relax it than other way around.
I will change the code to use
GIT_CLONE_PATH
.Edited by Kamil Trzcińskiassigned to @tmaczukin
So, the revisited version uses
GIT_CLONE_PATH
to indicate the desire to changeCI_PROJECT_DIR
. It will be disallowed unless enabled. We do not kill ifconcurrent > 1
it is up to the user/administrator of the system to figure out that.Edited by Kamil Trzciński- Resolved by Kamil Trzciński
I have not investigated this yet, but using the following
.gitlab-ci.yml
results into a failed buidld using the docker executor..gitlab-ci.yml
image: golang:1.12 variables: GIT_CLONE_PATH: "/go/src/gitlab-org/giltab-runner" test: stage: test script: - pwd - ls -la - ls -la /go/src/gitlab-org/ - echo $CI_PROJECT_DIR
job log
Running with gitlab-runner development version (HEAD) on docker-local c7zJEHBv Using Docker executor with image golang:1.12 ... Pulling docker image golang:1.12 ... Using docker image sha256:213fe73a385296ac1fbd34d563a3041859d13fe72e2a8a53590f00784489c516 for golang:1.12 ... Running on runner-c7zJEHBv-project-28-concurrent-0 via steve-mbp-gitlab... Initialized empty Git repository in /go/src/gitlab-org/giltab-runner/.git/ <---- Notice how it's getting cloned properly Fetching changes... Created fresh repository. From http://192.168.1.79:3000/root/ci-scratch-pad * [new branch] cache -> origin/cache * [new branch] custom-project-dir -> origin/custom-project-dir * [new branch] dirty-repo -> origin/dirty-repo * [new branch] forever-build -> origin/forever-build * [new branch] master -> origin/master * [new branch] scripts -> origin/scripts * [new branch] services -> origin/services * [new branch] windows-docker-executor -> origin/windows-docker-executor Checking out 920d1747 as custom-project-dir... Skipping Git submodules setup /bin/bash: line 72: cd: /go/src/gitlab-org/giltab-runner: No such file or directory <---- Some how it doesn't find the direcotry /bin/bash: line 72: cd: /go/src/gitlab-org/giltab-runner: No such file or directory ERROR: Job failed: exit code 1
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Steve Xuereb
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
assigned to @ayufan
assigned to @tmaczukin
mentioned in merge request !1228 (closed)
added 26 commits
-
22d10e0f...b3070119 - 25 commits from branch
master
- b3248452 - Add custom clone path for the build
-
22d10e0f...b3070119 - 25 commits from branch
mentioned in issue #2900
@steveazz This incorporates the !1282 (merged).
- Resolved by Kamil Trzciński
@steveazz
I made a change.
This implementation currently requires that
GIT_CLONE_PATH
is within thebuilds_dir
. This makes this feature still vaiable, but it will not make it spread across every unsafe path :) For easy discovery ofbuilds_dir
the Runner variable is introduced$CI_BUILDS_DIR
.We can relax this later.
added 1 commit
- 4ae6a063 - Strong validate that `GIT_CLONE_PATH` is used within `CI_BUILDS_DIR`.
mentioned in merge request !1282 (merged)
added workflowblocked label
removed workflowblocked label
added workflowblocked label
- Resolved by Steve Xuereb
- Resolved by Steve Xuereb
- Resolved by Kamil Trzciński
743 743 Runner will try to download the proper image. This of course means that the image should be uploaded to the registry 744 744 before upgrading the Runner, otherwise the jobs will start failing with a "No such image" error. 745 745 746 ## The `[runners.custom_build_dir]` section @eread would it possible to quickly review this block of documentation from your perspective, please?
created #4113 (closed) to continue this discussion
- Resolved by Steve Xuereb
- Resolved by Kamil Trzciński
- Resolved by Steve Xuereb
- Resolved by Kamil Trzciński
- Resolved by Steve Xuereb
assigned to @ayufan
Manual QA
I have the
.gitlab-ci.yml
pipeline specified below for testing:.giltab-ci.yml
image: golang:1.12 test: stage: test script: - pwd - ls -la - ls -la go/src/gitlab-org/ - echo $CI_PROJECT_DIR
And having
builds_dir
inside of runner configuration to/go
No
GIT_CLONE_PATH
specified (default behavior)GIT_CLONE_PATH
specified inside of builds_dirGIT_CLONE_PATH
specified outside of builds_dirassigned to @ayufan
assigned to @ayufan
removed workflowblocked label
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
added 14 commits
-
4ae6a063...5140f2e7 - 11 commits from branch
master
- c9bd231f - Add custom clone path for the build
- f4645bfb - Strong validate that `GIT_CLONE_PATH` is used within `CI_BUILDS_DIR`.
- f11f3097 - Remove commands/register.go changes as they are unneeded
Toggle commit list-
4ae6a063...5140f2e7 - 11 commits from branch
mentioned in issue #4113 (closed)
assigned to @tmaczukin
I can't seem to approve this because there is a commit of mine. But LGTM, the only thing we need to check now is documentation
Thank you @ayufan for pushing this forward
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
added 1 commit
- 46c7aea8 - Remove commands/register.go changes as they are unneeded
added 1 commit
- 6d124010 - Remove commands/register.go changes as they are unneeded
marked the checklist item blocked by !1282 (merged) as completed
marked the checklist item blocked by !1282 (merged) as incomplete
It looks good now and the tests are passing. The documentation is being handled separately by gitlab-org/gitlab-ce!20427, so let's merge this one.
Thanks @ayufan for finalizing the work on this feature!
mentioned in commit 302c5edb
mentioned in issue #2211 (closed)
mentioned in commit 83b390ee
mentioned in merge request !1301 (merged)
mentioned in commit b9d932d9
mentioned in issue #4777
mentioned in commit container-manager/gitlab-runner@d8fb32f5
mentioned in issue gitlab-com/www-gitlab-com#6358 (closed)
mentioned in merge request gitlab-com/www-gitlab-com!37948 (closed)
mentioned in issue #25717
mentioned in issue #26414