Support timing out aged queued jobs
Before raising this MR, consider whether the following are required, and complete if so:
-
Unit tests -
Metrics -
Documentation update(s)
If not required, please explain in brief why not.
Description
Support timing out aged queued jobs by cancelling them.
The implementation is similar to the job-pruner, by periodically querying aged jobs above threshold. Jobs are aborted directly by a batch operation on DB rows by setting the status code as UNAVAILABLE
.
This MR also refactors the execution-timeout
function, to set the status code as DEADLINE_EXCEEDED
.
See more details at: https://grpc.github.io/grpc/core/md_doc_statuscodes.html
Edited by Zehao Chen