Skip to content

Log scheduled pipeline creation failure

Rémy Coutable requested to merge log-scheduled-pipeline-creation-failures into master

What does this MR do and why?

This adds logging for the cases when a scheduled pipeline fails to be created due to a Ci::CreatePipelineService::CreateError error.

That would ease the debugging, for instance https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/963#note_680524106.

Example of such logging (look at the extra.run_pipeline_schedule_worker.pipeline_creation_error key):

2021-09-20_14:07:17.18678 rails-background-jobs : {"severity":"INFO","time":"2021-09-20T14:07:17.186Z","class":"RunPipelineScheduleWorker","args":["2","1"],"retry":3,"queue":"pipeline_creation:run_pipeline_schedule","backtrace":true,"version":0,
"queue_namespace":"pipeline_creation","jid":"b6447e7d79c885edd751be1f","created_at":"2021-09-20T14:07:05.489Z","meta.user":"root","meta.project":"gitlab-org/security/gitlab","meta.root_namespace":"gitlab-org","meta.caller_id":"Projects::PipelineSchedulesController#play",
"meta.remote_ip":"127.0.0.1","meta.feature_category":"continuous_integration","meta.client_id":"user/1","correlation_id":"01FG1RFVYQPB3GQX2Z2QT52EJM","idempotency_key":"resque:gitlab:duplicate:pipeline_creation:run_pipeline_schedule:df123ce4b4c8a1304bab530fa93b8be45d345878200cc723b1e9e8d49fea2408",
"enqueued_at":"2021-09-20T14:07:05.509Z","job_size_bytes":5,"pid":30967,"message":"RunPipelineScheduleWorker JID-b6447e7d79c885edd751be1f: done: 11.675608 sec","job_status":"done","scheduling_latency_s":0.001458,"gitaly_calls":3,"gitaly_duration_s":0.065525,
"rugged_calls":29,"rugged_duration_s":0.021343,"redis_calls":1623,"redis_duration_s":0.464513,"redis_read_bytes":192357,"redis_write_bytes":88773,"redis_cache_calls":1622,"redis_cache_duration_s":0.464391,"redis_cache_read_bytes":192347,
"redis_cache_write_bytes":88556,"redis_queues_calls":1,"redis_queues_duration_s":0.000122,"redis_queues_read_bytes":10,"redis_queues_write_bytes":217,"db_count":64,"db_write_count":2,"db_cached_count":10,
"external_http_count":1,"external_http_duration_s":0.3057050000061281,"cpu_s":10.375158,
"extra.run_pipeline_schedule_worker.pipeline_creation_error":"'DAST-fullscan-ruleset1' job needs 'review-deploy' job, but 'review-deploy' is not in any previous stage\n'DAST-fullscan-ruleset2' job needs 'review-deploy' job, but 'review-deploy' is not in any previous stage\n'DAST-fullscan-ruleset3' job needs 'review-deploy' job, but 'review-deploy' is not in any previous stage\n'DAST-fullscan-ruleset4' job needs 'review-deploy' job, but 'review-deploy' is not in any previous stage\n'DAST-fullscan-ruleset5' job needs 'review-deploy' job, but 'review-deploy' is not in any previous stage\n'DAST-fullscan-ruleset6' job needs 'review-deploy' job, but 'review-deploy' is not in any previous stage\n'DAST-fullscan-rule-90020' job needs 'review-deploy' job, but 'review-deploy' is not in any previous stage\n'DAST-fullscan-rule-40018' job needs 'review-deploy' job, but 'review-deploy' is not in any previous stage\n'DAST-fullscan-rule-40014' job needs 'review-deploy' job, but 'review-deploy' is not in any previous stage\n'DAST-fullscan-rule-6' job needs 'review-deploy' job, but 'review-deploy' is not in any previous stage\n'DAST-fullscan-rule-40012' job needs 'review-deploy' job, but 'review-deploy' is not in any previous stage",
"duration_s":11.675608,"completed_at":"2021-09-20T14:07:17.186Z","db_duration_s":0.06649}

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Rémy Coutable

Merge request reports