Make Duo CLI (DAP Remote Execution binary) available for air-gapped customers
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
Problem
See background in gitlab-com/gl-infra/delivery#21190 (comment 2548935011)
At present there is no way for air-gapped customers to use Duo Workflow Remote Execution for the following reasons:
- The default default docker image is downloaded from registry.gitlab.com (https://gitlab.com/gitlab-org/gitlab/-/blob/e6eef317aa240df17017ff79a0020350d1ecc90a/ee/app/services/ai/duo_workflows/start_workflow_service.rb#L6) . This is possible to workaround on a per-project basis as any project can declare a different image and thus the self-managed customers could go to the effort of building their own image and configuring it individually for every project.
- The duo CLI is not included in the image but downloaded from npm when a flow runs in CI (https://gitlab.com/gitlab-org/gitlab/-/blob/e6eef317aa240df17017ff79a0020350d1ecc90a/ee/app/services/ai/duo_workflows/start_workflow_service.rb#L167). This has no workaround today. This would likely need to be configurable at the instance level or we could go with an approach that bundles this package into our GitLab releases.
Possible Solutions
- Bundle the duo CLI into Omnibus and CNG. This was started at omnibus-gitlab!8457 (closed) and gitlab-org/build/CNG!2410 (closed) for the old executor. But as can be seen from the discussion on the Omnibus MR there is a significant downside in that it makes our releases ~10% larger due to the amount of different OS/arch pairs we need to compile for. Even if we only need this binary in CI we'll still eventually need to support different runner architectures (ie. we have customers that will want to run these on Windows and Mac CI runners). Now with the executor being rewritten in node these binaries are around 5x the size as back then when we were discussing it. That will make this option even less attractive today.
- Bundle it somehow with
gitlab-runneror steps runner. Right now this is already released as a separate binary for air-gapped customers to download. Additionally having this as part of the runner might solve issues like #548494 and #548496 . Also see gitlab-com/content-sites/handbook!13888 (comment 2549676365) for discussion about steps distributing binaries. - From my understanding of discussions there are already several manual steps involved in setting up GitLab to work in an air-gapped environment so it may be sufficient as an initial step to just document how to download the assets and place them in the GitLab instance and configure the URLs in
config/gitlab.ymlsuch that their instance uses their local versions.
Edited by Dylan Griffith (ex GitLab)