[Bug]: If I use assistants, it will allow me to go over token budget

[virtuman]

What happened?

When I use OpenAI directly as an endpoint, it will block my requests as i come to the limit of tokens. However, when i use Assistants, it ignores the fact that I went over limit, and continues to process all requests.

is it expected behavior?

Steps to Reproduce

login to account with minimal or negative credit
try to run request on openai assistant - it should block if runs over allowed limits
then select any assistant with openai backend, and run same request
-> it will continue to process requests as normal, without taking into consideration that the allotted token limits is overused.

What browsers are you seeing the problem on?

Chrome

Relevant log output

No response

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Thanks I’ve triaged the issue and will push a fix soon.

[badcom]

I got this error when using the gpt-4-vision-preview model. Related to this bug?

[danny-avila]

I got this error when using the gpt-4-vision-preview model. Related to this bug?

No, this is specific to OpenRouter and using their "free" account which limits context

[badcom]

I'm not using OpenRouter, though. It isn't one of my endpoints. My OPENROUTER_API_KEY does have a key set to it, but I'm not using that endpoint when I get that error message. That's the odd thing.

[danny-avila]

OPENROUTER_API_KEY

Do not use this variable, see #2097