Make the WebhookWorker idempotent
Part of gitlab-com/gl-infra/scalability#178
The WebhookWorker processed 1.5 million "duplicate" jobs the past 7 days, and spent 70 hours on that. Duplicate jobs are jobs that get scheduled when there is already a job in the queue for the same worker with the same arguments.
If the job was (marked as) idempotent, we would be able to deduplicate those jobs when they get scheduled.
This worker makes a request based on the webhook passed, the data in the arguments is the payload of the request. So an identical job would deliver the exact same payload. Most of the "duplicate" jobs in the that queue seem to originate from /api/:version/projects/:id and are update events for projects. The payload is the same, so it does not include the attribute that was updated. Should those payloads be delivered multiple times?