Max Token/Message Control

[aimachinedream]

A big reason why the pickup on the Assistants API has been so slow, is the token use can get out of control. With the new V2 support you added, this can be addressed.

"Context window management
The Assistants API automatically manages the truncation to ensure it stays within the model's maximum context length. You can customize this behavior by specifying the maximum tokens you'd like a run to utilize and/or the maximum number of recent messages you'd like to include in a run."

So some UI to let us set the max messages, and tokens a run will use. It will also fix another issue where in very long conversations the bots tend to go off message. With a maximum number of recent messages, we can limit this.

[marcolivierbouch]

Hello @aimachinedream,

The feature to set maximum tokens is now delivered!