Skip to content

ML Infra : Dynamic Batching

https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md#dynamic-batcher

To have dynamic batching in prod, we need to pad sequencies during preprocssing

Edited by Mon Ray