<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Work on this issue](https://contributors.gitlab.com/manage-issue?action=work&projectId=278964&issueIid=301161) - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=301161) </details> <!--IssueSummary end--> ## Problem Statement Currently, `cache:key:files` is limited to a maximum of 2 files. This restriction is problematic for projects with multiple dependency files, such as: - Multi-module .NET projects with multiple lock files - Monorepos with multiple package managers - Python projects with multiple requirements files Quoting from https://docs.gitlab.com/ee/ci/yaml/README.html#cachekeyfiles: > When you include cache:key:files, you must also list the project files that are used to generate the key, **up to a maximum of two files**. ## Proposed Solutions ### Option 1: Increase File Limit (Recommended - MVP) Increase the limit from 2 to 10 files. This is a minimal change that addresses the immediate need. ### Option 2: Multiple Cache Blocks Per Job Allow defining several caches with different keys in a single job: ```yaml variables: MOLECULE_EPHEMERAL_DIRECTORY: $CI_PROJECT_DIR/.cache/molecule POETRY_VIRTUALENVS_IN_PROJECT: "true" PRE_COMMIT_HOME: $CI_PROJECT_DIR/.cache/pre-commit cache: - paths: - $PRE_COMMIT_HOME key: prefix: pre-commit files: - .pre-commit-config.yaml - paths: - .venv key: prefix: poetry files: - pyproject.toml - poetry.lock - paths: - $MOLECULE_EPHEMERAL_DIRECTORY key: prefix: molecule files: - tests/molecule/default/collections.yml - tests/molecule/default/requirements.yml ``` ### Option 3: Use Glob Patterns Users can already use glob patterns like `**/package.json` to match multiple files, though this is undocumented. --- ## Implementation Proposal: Increase Limit to 10 Files (July 2 2026, GitLab Duo) ### Overview This proposal outlines a minimal, phased approach to increase the `cache:key:files` limit from 2 to 10 files, unblocking [#554248](https://gitlab.com/gitlab-org/gitlab/-/work_items/554248) and addressing high community demand. ### Phase 1: Increase Limit to 10 Files (Minimal Change) #### Changes Required **1. Update validation in `lib/gitlab/ci/config/entry/files.rb`** ```ruby # Current (line 14-18): validates :config, length: { minimum: 1, maximum: 2, # ← Change to 10 too_short: 'requires at least %{count} item', too_long: 'has too many items (maximum is %{count})' } ``` **2. Update JSON schema in `app/assets/javascripts/editor/schema/ci.json`** - Locate the `cache:key:files` definition (around line 1937) - Add `maxItems: 10` constraint to the array definition **3. Update documentation in `doc/ci/yaml/_index.md`** - Change "up to two file paths" to "up to ten file paths" - Add note about glob pattern support (already works but undocumented) - Update example to show 3+ files **4. Update CI schema documentation in `doc/ci/caching/_index.md`** - Reference the increased limit #### Test Coverage Add/update tests in `spec/lib/gitlab/ci/config/entry/files_spec.rb`: - Test with 3, 5, and 10 files (should pass) - Test with 11 files (should fail with appropriate error message) - Verify glob patterns still work #### Estimated Effort - **Code changes**: ~30 minutes (4 files, mostly validation updates) - **Testing**: ~20 minutes (add 3-4 test cases) - **Documentation**: ~15 minutes - **Total**: ~1 hour ### Phase 2: Future Enhancements (Post-MVP) Once this is merged and validated in production, consider: 1. **Increase to 20-50 files** — if no performance issues observed 2. **Multiple cache blocks per job** — allow defining separate caches with different keys (as proposed in original issue) 3. **Glob pattern documentation** — formally document that `**/package.json` and similar patterns work ### Implementation Checklist - [ ] Update `lib/gitlab/ci/config/entry/files.rb` (change `maximum: 2` to `maximum: 10`) - [ ] Update `app/assets/javascripts/editor/schema/ci.json` (add `maxItems: 10`) - [ ] Update `doc/ci/yaml/_index.md` (change limit text, add glob pattern note) - [ ] Update `doc/ci/caching/_index.md` (reference new limit) - [ ] Add test cases in `spec/lib/gitlab/ci/config/entry/files_spec.rb` - [ ] Run full CI/CD pipeline - [ ] Update CHANGELOG entry - [ ] Link to gitaly#2068 resolution in commit message ### Risk Assessment **Low Risk**: - Gitaly blocker (gitaly#2068) is resolved - No architectural changes needed - Backward compatible (existing 2-file configs continue to work) - Validation is purely numeric (no complex logic) **Testing Strategy**: - Unit tests for validation - Integration tests via `spec/lib/gitlab/ci/yaml_processor_spec.rb` - Manual testing with multi-file cache keys ### Success Criteria 1. ✅ Validation accepts 3-10 files, rejects 11+ 2. ✅ Documentation updated with new limit 3. ✅ All existing tests pass 4. ✅ New tests cover edge cases (3, 10, 11 files) 5. ✅ Unblocks [#554248](https://gitlab.com/gitlab-org/gitlab/-/work_items/554248) ### Related Issues - Blocks: [#554248](https://gitlab.com/gitlab-org/gitlab/-/work_items/554248) (Automate cache invalidation when patches/ directory changes) - Blocked by: [gitaly#2068](https://gitlab.com/gitlab-org/gitaly/-/issues/2068) ✅ RESOLVED - Duplicate of: [#352841](https://gitlab.com/gitlab-org/gitlab/-/work_items/352841) (Allow more file paths for `cache:key:files`)