Add support for credentials store

What does this MR do?

With this MR runner will correctly fetch credentials store from credsStore, if it's configured.

Why was this MR needed?

In some environments, specifically in AWS, credentials are provided via credentials store, as documented here: https://docs.docker.com/engine/reference/commandline/login/#credentials-store. Currently this auth method doesn't work with gitlab runner. In case of AWS the only solution would be to re-trigger aws ecr get-login in a cron job every 12 hours to re-fetch credentials. See this Issue for more details: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1583

With new implementation, in addition to all previous ways to provide credentials, Gitlab Runner will also use credsStore, in a similar way Docker CLI does it.

Please notice, that this MR still doesn't cover all possible use cases, specifically it doesn't cover usage of "credsHelpers" key of Docker config - I can add it in next MR, if I get more time to work on this.

In any case, even without credsHelper support, this code change simplifies usage of Gitlab Runner combined with AWS ECR or GCE GCR, making authentication simple, transparent and automated. Also see:

• https://github.com/awslabs/amazon-ecr-credential-helper
• https://github.com/GoogleCloudPlatform/docker-credential-gcr

What are the relevant issue numbers?

https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1583

mentioned in issue #1583 (closed)

mentioned in issue #1655 (closed)

added 1 commit

• b1c6490c - Add missing deps

Compare with previous version

added 1 commit

• 59900856 - Vendor all missing deps

Compare with previous version

I need some help here with dependency resolution. For this new code I just needed one extra dep, defined here https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/501/diffs#56a65ef5280f0b59ab67318c519856e9b36c16bd_16_16

After that I run "godep save" as in documentation, and as you see it added quite a lot of different packages to vendor. But still, it seems to be not enough: https://gitlab.com/Fodoj/gitlab-ci-multi-runner/builds/11545286

I spent more time debugging godep and how to get vendor/ folder right than writting the actual feature (and honestly at this point I believe that godep is total shit and it should be replaced with some other tool). So any help on getting correct stuff to Godeps.json and vendor/ folders is appreciated and much needed. :\

Extra info: if I run godep restore on master branch I get this:

➜  gitlab-ci-multi-runner git:(4d8a0a4) godep restore
# cd /home/kshirinkin/work/go/src/google.golang.org/cloud; git checkout 70d0953732b7f6b4efcb672f3044676c6581339c
godep: error restoring dep (google.golang.org/cloud/compute/metadata): chdir /home/kshirinkin/work/go/src/google.golang.org/cloud: no such file or directory
# cd /home/kshirinkin/work/go/src/google.golang.org/cloud; git checkout 70d0953732b7f6b4efcb672f3044676c6581339c
godep: error restoring dep (google.golang.org/cloud/internal): chdir /home/kshirinkin/work/go/src/google.golang.org/cloud: no such file or directory
godep: error restoring dep (github.com/opencontainers/runc/libcontainer/utils): Wanted to restore rev ce450bcc6c135cae93ee2a99d41a308c179ff6dc, already restored rev 94dc520a5732126985fec249f80c91b9e0601815 for another package in the repo
godep: Error restoring some deps. Aborting check.

Any feedback here, anyone?

added 10 commits

• 59900856...ca7707b0 - 8 commits from branch gitlab-org:master
• 312c196e - resolve conflicts
• dbcf9817 - more deps resolved

Compare with previous version

added 1 commit

• 649f6a98 - more deps resolved

Compare with previous version

@tmaczukin may be you can take a look?

assigned to @tmaczukin

@Fodoj I am not going to review this, but a couple of notes: You're missing a changelog entry, and you have a failed pipeline.

@Fodoj

It is interesting. You are missing tests for this storage.

@nick.thomas Could you jump-in and help with godep issue?

@tmaczukin how close are we to merging https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/505 ? It might be easier to do that then use govendor to add the new dependency here...

@Fodoj I purposefully avoided introducing support for these extra auth config methods in https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/301 because it pulled in so many extra packages. It doesn't feel like the docker code is built to allow sensible re-use of a subset of this code.

Can we produce a full dependency graph? It's possible there's an alternative way to get these auth mechanisms working without pulling in quite so many extra packages.

@nick.thomas I can try, just need to get the build running here in CI.. So I guess the fix to godeps isssue is to wait for !505 (merged), and try it with govendor, right?

You're missing a changelog entry, and you have a failed pipeline.

@zj Changelog entries are generated with script from merge requests titles at version release. There is no need to add them in the MR :)

@nick.thomas @Fodoj I hope I will find time and merge !505 (merged) in next week.

@Fodoj !505 (merged) was just merged into master branch

added executordocker ~58728 labels