Docs: buildkit docs do not work

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

• Start this issue's title with Docs: or Docs feedback:.

Problem to solve

Testing the buildkit docs the jobs do not work

As an example - https://docs.gitlab.com/ci/docker/using_buildkit/#build-images-in-rootless-mode the job just hangs for an hour until it is killed - https://gitlab.com/faleksic_ultimate_group/ci-testing/-/jobs/10371038332

Further details

We should make sure the job works as expected before merging it into the docs, perhaps something changed in the last version or so that made it break.

Migration insights - feedback

Greetings,

Just wanted to let you know what I've figured out after a day of trying to migrate from Kaniko to BuildKit in the most secure fashion. Feel free to forward it or use it for reference. No guarantee that it's the right way to do things, but to me it seems like a better solution than unconfined seccomp, although the giant inline json is very annoying to maintain and it doesn't look like file paths are supported on the runner yet for seccomp json files.

• What is written did not work for me out of the box. The image default entrypoint seems to start a server for long-lived build systems and hangs indefinitely. I ended up adding entrypoint: ["sh", "-c"] which overrides that default behavior and allowed the build command to run.
• There are privilege issues preventing proper operation when running the rootless with runner privileged set to false. Many people in the GitLab issue thread are talking about adding security_opt = ["seccomp:unconfined"] as a fix, and others worried about the implications of adding this in the runner options. Rightfully so, I think this removes all of the default seccomp configuration you normally get with Docker.

Instead you can add security_opt = ['seccomp={}'] and you can feed in inline json, and hopefully GitLab can add path support for json seccomp files instead in the future. Moby has the default seccomp profile here (I assume this is the default that Docker would use as well?)... https://github.com/moby/moby/blob/master/profiles/seccomp/default.json and I was able to modify this default profile and whitelist the things that BuildKit required to run and leave everything else as it was. Then you can get BuildKit functioning without setting the seccomp to unconfined and you can preserve least privilege access by only adding what BuildKit is using rather than everything. I think you need to move several things to the SCMP_ACT_ALLOW such as umount, sethostname, mount, and a few more. Unfortunately, I could not find a way quite as clean as how Kaniko operated, but I've read they may all tend to have a caveat or two compared to Kaniko.

Unclear how this solution plays in to App Armor as I am not using it at the moment for those on Ubuntu.

GitLab Slightly Incorrect Version for quick reference (I think):

build:
  image: moby/buildkit:rootless
  variables:
    BUILDKITD_FLAGS: --oci-worker-no-process-sandbox
  before_script:
    - mkdir -p ~/.docker
    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > ~/.docker/config.json
  script:
    - |
      buildctl-daemonless.sh build \
        --frontend dockerfile.v0 \
        --local context=. \
        --local dockerfile=. \
        --output type=image,name=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA,push=true

Proposal

Update the docs

Who can address the issue

Technical writing team

Other links/references

• #503827