Backport !241962 to 19.2: Make Ci::JobDefinition config size limit configurable - Fix failing pipelines when job config over 1 MiB

Leaminn Marequested to merge
ci-job-definition-config-size-limit-19-2 into 19-2-stable-ee

What does this MR do and why?

Backport of !241962 (merged) to 19.2.

Problem

!218219 (merged) (18.11) switched Ci::JobDefinition#config to the declarative json_schema validator with a hardcoded size_limit: 1.megabyte. From 18.11 the limit is enforced, and pipeline creation fails on GitLab Self-Managed instances whose compiled per-job configuration exceeds 1 MiB.

The 1 MiB value came along as part of adopting the validator. It was not a considered limit, and 18.11 through 19.2 give administrators no way to raise it without patching code.

This caused a customer incident: gitlab-com/support/support-pairing#24373.

Fix

Resolve the limit from the existing ci_max_total_yaml_size_bytes application setting instead of hardcoding it. JsonSchemaValidator now accepts a callable size_limit so the value is read from application settings at validation time.

Backport notes

Clean cherry-pick of 29133095fd5c5ea7af8fded4802306a374cc9f2e, no conflicts. The diff is byte-identical to the default-branch commit.

References

  • Customer incident: gitlab-com/support/support-pairing#24373
  • !218219 (merged) — introduced the hardcoded 1 MiB limit (18.11)
  • !241962 (merged) — the fix being backported (19.3)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

  • This MR is backporting a bug fix, documentation update, or spec fix, previously merged in the default branch.
  • The MR that fixed the bug on the default branch has been deployed to GitLab.com (not applicable for documentation or spec changes).
  • The MR title is descriptive (e.g. "Backport of 'title of default branch MR'"). This is important, since the title will be copied to the patch blog post.
  • Required labels have been applied to this merge request
  • This MR has been approved by a maintainer (only one approval is required).
  • Ensure the e2e:test-on-omnibus-ee job has succeeded, or if it has failed, investigate the failures. If you determine the failures are unrelated, you may proceed. If you need assistance investigating, request help in the #s_developer_experience Slack channel to confirm the failures are unrelated to the merge request.

Note to the merge request author and maintainer

If you have questions about the patch release process, please:

Merge request reports