Make the ExpirePipelineCacheWorker idempotent

Part of #178 (closed)

The ExpirePipelineCacheWorker processed 500k "duplicate" jobs the past 7 days, and spent 24 hours on that. Duplicate jobs are jobs that get scheduled when there is already a job in the queue for the same worker with the same arguments.

If the job was (marked as) idempotent, we would be able to deduplicate those jobs when they get scheduled.

This worker expires the etag cache for a pipeline, and updates the status in redis based on the current state. It is already idempotent, we just need to add the spec and mark it as such.

Edited by Bob Van Landuyt