Implement startup probe for MG
🧩 Problem to solve
Per this issue, there is no startup probe for the codegen model.
💡 Proposal
This probe should verify if the codegen model is ready. The required grpc endpoint is available through the InferenceServerClient
.
⚔ ️ Plan of attack
This implementation adds a startup probe to the model-gateway yaml.
Notes:
Might want to edit the timing on startup and liveness probes. As per this implementation, it appears that the timing is different in this implementation in SR (here)
@dbernardi to investigate this further, but can be fixed in a merge request later if need be.
Edited by Dylan Bernardi