Skip delay job when not needed

Related to: infrastructure#1222 (closed)

E.g: https://gitlab.com/cki-project/kernel-ark/-/pipelines/2358056558/

Beware: gitlab-org/gitlab#365604

Jira: CKI-7438