Tracing by correlation_id can be misleading
Sometimes correlation_id
gets assigned incorrectly.
I think the problem here is wrongly assigned
correlation_id
. TheUpdateAllMirrorsWorker
job can be created by user request by callingProject.import_state.force_import_job!
so some particularcorrelation_id
will be assigned in this case, at the same time,UpdateAllMirrorsWorker
itself picks up other projects waiting for mirroring but they are not related to thiscorrelation_id
in any way.
The last thing I could not understand how
correlation_id
is passed toUpdateAllMirrorsWorker
again and again. Here is how https://gitlab.com/gitlab-org/gitlab-ee/blob/65770d3e7a9cd503b2d65eb2869085386f078716/ee/app/workers/update_all_mirrors_worker.rb#L30 we reschedule it again and again and this is how this mistake is spread.