Podman runmechanism

Important: This is in active development and subject to modification based upon testing and feedback.

Introduces a new user namespace RunMechanism that can be configured to execute both runner and user defined scripts using a target container image via the Podman container runtime.

[general]
# RunMechanism defines a proposed run_mechanism to executor user/runner scripts (default: undefined).
run_mechanism = "podman"

[general.podman]
# ApplicationPath full path to the Podman application, used in constructing all commands.
application_path = "/usr/bin/podman"
# RunnerImage helper image that are used by all jobs for gitlab-runner/git/tar standard interactions.
runner_image = "registry.gitlab.com/ecp-ci/jacamar-ci/alpine-runner:latest"
# RunnerOptions any 'podman run ...' options that are used for runner defined job steps.
runner_options = []
# RunnerEntryPoint override the default ENTRYPOINT/COMMAND (default: bash -l -c) use for runner defined job steps.
runner_entry_point = []
# DefaultImage is used when no user defined 'Image' is present for the job. There is no default and
# jobs will fail if a usr does not provide their own.
default_image = "docker.io/library/debian:latest"
# CustomOptions any 'podman run ...' options that are used for user defined job steps.
custom_options = []
# CustomEntryPoint override the default ENTRYPOINT/COMMAND (default: bash -l -c) used for user defined job steps.
user_entry_point = []
# StepScriptOnly limits the use of containers to the step_script/build_script. This is useful in cases
# where a deployment may limit containers to compute environment (i.e., Slurm/LSF executors).
step_script_only = false
# DisableContainerRemoval prevents the '--rm' options from being used on generated commands.
disable_container_removal = false
# ImagePullPolicy define policy (https://docs.podman.io/en/latest/markdown/options/pull.html) for
# all container images.
image_pull_policy = "always"
# ImageAllowlist when defined only images that match this list of regular expressions will be observed.
# Errors in the provided expressions will result in failed jobs.
image_allow_list = []
# UserVolumeVariable defines the prefix for a CI variable users can leverage to mount volumes.
# Leaving this undefined means no user defined volumes will be mounted.
user_volume_variable = "USER_VOLUMES"

Test Deployment

If you want to test the deployment I advise adding the following to your configuration:

[general]
run_mechanism = "podman"

[general.podman]
application_path = "/usr/bin/podman"
# I've setup this image as a functional test; however, you can create your own so long
# as bash/gitlab-runner/git applications are available.
runner_image = "registry.gitlab.com/ecp-ci/jacamar-ci/alpine-runner:latest"
# This can be any image you want, so long as the user has the ability to pull it 
# without additional authentication.
default_image = "docker.io/library/debian:latest"

With this we can observe some simple examples where a downscoped user executes a job (all all related components) in containers using the Podman runtime:

Another option is to limit the use of podman to the job script. With schedulers such as Slurm this means a sbatch ... podman run ... /jobscript.bash command will be generated by adding:

[general.podman]
step_script_only = true

ToDo

• Finalize strategy for Access via private regsitry. At the moment we automatically create an auth file using the CI_JOB_TOKEN, this allow the GitLab registry to be easily accessed. But we need to account for other registries and the fact user may want to use a default auth ($HOME/.config/container/auth.json).
• Better handle (or identify) difference in the host Bash shell and one in the container.
• Establish unit tests and address any remaining issues
• Add comprehensive documentation and tutorials to ecp-ci.gitlab.io
• Add option that allow limited images based upon regular expression