Status of Triton in Model Gateway / AI Gateway
This question was raised in the context of the GA milestone, and the possibility of managing the Model Gateway/AI Gateway on top of the new Runway Internal Developer Platform currently being developed within the Infra Platform group.
If Runway were to be used, we wouldn't be able to run the Triton Inference service.
Questions:
- Are user requests currently being sent to Triton?
- Are there any (other) reasons why we wouldn't be able to deliver the Gateway at GA on Runway?
cc @marin @mray2020 @lmcandrew @igorwwwwwwwwwwwwwwwwwwww @ggillies @tle_gitlab