Ci::ArchiveTracesCronWorker keeps failing false positively
Summary
Ci::ArchiveTracesCronWorker is a cron worker to archive stale live traces.
Today, I found that job_trace_archive_failed_total metric on Prometheus keeps rising on dev.gitlab.org.
Here is the corresponding log on Kibana
https://log.gitlab.net/app/kibana#/discover?_g=h@91bd11d&_a=h@fae6573
The problem is that the worker tries to archive job traces which are just waiting for BuildFinishedWorker.
If those traces are rigth on the track, the cron worker should not do anything.
/cc @ayufan