Jenkins jobs do not get cleared up by StuckCIJobsWorker
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
Summary
Jenkins jobs do not get cleared up by StuckCIJobsWorker.
UPDATE: 2024-09-06 (@fabiopitino): StuckCiJobsWorker does not consider Ci::Bridge type either.
Steps to reproduce
Create jobs using Jenkins CI integration. Note that, in the ci_builds table, their type is marked as "GenericCommitStatus" and not as "Ci::Build".
What is the current bug behavior?
Worker is querying only for "Ci::Build" type of jobs and clearing them up, while those of type "GenericCommitStatus" created by Jenkins CI remain.
What is the expected correct behavior?
Worker querying for type "GenericCommitStatus" as well (or maybe all types).
Relevant logs and/or screenshots
Customer reporting this (internal): https://gitlab.zendesk.com/agent/tickets/103332
One example from self-managed instance:
gitlabhq_production=# select status, created_at, name, stage, type from ci_builds where created_at>'2018-09-15';
status | created_at | name | stage | type
---------+----------------------------+---------+----------+---------------------
running | 2018-09-15 06:33:39.980971 | jenkins | external | GenericCommitStatus
running | 2018-09-15 06:33:38.920743 | test | test | Ci::Build
(2 rows)
# sudo gitlab-rails console
-------------------------------------------------------------------------------------
GitLab: 11.2.3-ee (aadca994104)
GitLab Shell: 8.1.1
postgresql: 9.6.8
-------------------------------------------------------------------------------------
Loading production environment (Rails 4.2.10)
irb(main):001:0> Rails.logger.level = Logger::DEBUG
=> 0
irb(main):002:0> s = StuckCiJobsWorker.new
=> #<StuckCiJobsWorker:0x00007f24459eeee8>
irb(main):003:0> s.perform
StuckCiJobsWorker: Cleaning stuck builds
Ci::Build Load (2.0ms) SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."status" = 'running' AND (ci_builds.updated_at < '2018-09-14 20:34:37.777978') LIMIT 100 [["status", :running]]
Ci::Build Load (0.6ms) SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."status" = 'pending' AND (ci_builds.updated_at < '2018-09-13 21:34:37.782229') LIMIT 100 [["status", :pending]]
Ci::Build Load (0.6ms) SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."status" = 'pending' AND (ci_builds.updated_at < '2018-09-14 20:34:37.784146') LIMIT 100 [["status", :pending]]Possible fixes
https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/workers/stuck_ci_jobs_worker.rb