PoC: Project-level Pipeline job sempahore

Shinya Maedarequested to merge
poc-pipeline-job-semaphore into master

What does this MR do?

PoC for #15536 (closed), therefore it's not for merge

Technical proposal

  • Introduce new tables ci_project_semaphores and ci_job_locks to track CI semaphores status.
  • Ci::ProjectSemaphore model is for managing all semaphores under the project.
  • Ci::JobLock model is for tracking the lock status of the job under a semaphore.
  • job_lock is created when a pipeline is created. Lockable jobs are executed by the order of job_lock.id. (nearly equal to pipeline.id order)
  • If a job is lockable, the system delegates the operation to job_lock for controlling its status with taking account of semaphore.
  • When a job finished, it released a obtained lock. The system finds a next blocked job, updates the status to locking.
  • The system supports predefined variables for lock key. For example, lock: $CI_COMMIT_REF_NAME is translated to lock: master.
  • We preserve the history of locked jobs. This is useful for investigating bottle neck of CI.

Concerns

  • We should use compound status - blocked. Currently, Ci::Status is super inefficient.
  • Any race conditions?
  • Any edge cases?
  • Potential deadlock?
  • Performance concern?

Sample gitlab-ci.yml

job:
  script: echo 'a'
  lock: test

TODO

  • YAML parse && persist
  • Present the extended status
  • Update description
  • Use compound status
Edited by Shinya Maeda

Merge request reports