Skip to content

Make context window limit configured by model

Problem to solve

In Duo Workflow Service, context is limited to a hardcoded value of MAX_CONTEXT_TOKENS currently set to 1M.

This value is global and does not depend on the model running. It will create issues if we use another model with a different context window limit.

Proposal

Make this value configurable based on the model.

Further details

Links / references