shell-scripts: add cki-dev.sh: fast local GitLab CI execution with tox caching

Simone Tollardorequested to merge
tollsimy/cki-lib:linter into main

Add cki-dev.sh: fast local GitLab CI execution with Tox caching ⚡

This MR introduces cki-dev.sh, a wrapper script around gitlab-ci-local that enables fast local execution of GitLab CI jobs with persistent tox environment caching.

Problem Solved

  • Inconsistent environments: Local testing differs from actual CI execution (e.g. different ruff versions show inconsistencies in preview rules, commands installed in host machine override tox ones etc.)
  • Development friction: Long feedback loops when iterating on code changes. Repeated tox environment rebuilds and package downloads can take ~2-5 minutes per run.

Solution

New cki-dev.sh script that:

  • Mounts local .tox cache into containers for persistent environments
  • Supports any GitLab CI job directly (no predefined wrappers needed)

Performances

After cache is populated, all the preparation steps takes less than 20 seconds, practically removing any overhead.

Usage Examples

# Run any GitLab CI job
cki-dev.sh linting
cki-dev.sh linting-root-ruff
cki-dev.sh tests

# Override variables
cki-dev.sh linting -- --variable CKI_DISABLED_LINTERS="markdownlint pylint"

# List available jobs
cki-dev.sh --list

Advantages over Alternative Approaches

vs. Makefile-based Wrappers

  • No maintenance overhead: No need to update wrapper commands when CI jobs change
  • Complete job coverage: Access to all GitLab CI jobs, not just predefined ones
  • Direct mapping: Job names match exactly what's in .gitlab-ci.yml
  • Variable flexibility: Full access to all CI variables and overrides

vs. Direct tox Execution

  • True CI environment: Exact same container, dependencies, and configuration as CI
  • Complete pipeline testing: Access to all CI jobs (yamllint, shellcheck, etc.), not just tox environments
  • CI-specific variables: Full support for CKI_* environment variables and CI context without annoying env var passing

Dependencies

This requires installation and usage of https://github.com/firecow/gitlab-ci-local .

This MR results in a significant improvement of the local development experience while maintaining full compatibility with our CI pipeline.

Closes: #84 (closed)

Closes: #85 (closed)

Edited by Simone Tollardo

Merge request reports