[bst-1] Properly handle grpc exceptions

Background

Some errors from cas cache server should be better handled. ABORTED should raise a temporary error (so that the scheduler will retry the job). On UNAVAILABLE we should just retry the request.

See documentation:

https://developers.google.com/maps-booking/reference/grpc-api/status_codes

This would help mitigate issue #810 (closed).

Examples of failures

Acceptance Criteria

  • All gRPC calls error UNAVAILABLE and ABORTED should be handled
  • ABORTED error should restart the job
  • UNAVAILABLE errors should retry the request
Edited by Tristan Van Berkom