Token Limiting #434
SethTurin
started this conversation in
Feature Requests & Suggestions
Replies: 2 comments
-
If you feel that it’s easy to toggle or adjust as needed, please do so! Though the current implementation won’t give you the most accurate token usage. I am working on that through a PR to a dependency library |
Beta Was this translation helpful? Give feedback.
0 replies
-
tokens per minute, request per minute, and the number of |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
For my use case, I needed to set limits to how many tokens can be use in a timeframe, so I wrote some code for it. It ended up being a bigger edit that I anticipated, but I think the integration is clean and reusable.
If others feel this would be a useful feature, I can put it into a pull request.
Beta Was this translation helpful? Give feedback.
All reactions