You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I don't expect we'll be able to add this to the library itself anytime soon, but I'll keep it in the back of my mind. In the meantime, will be happy to send folks towards your library.
Confirm this is a feature request for the Python library and not the underlying OpenAI API.
Describe the feature or improvement you're requesting
Introduce the ability to use a prioritized set of backends and load-balance across different OpenAI instances.
Additional context
I wrote a Python OpenAI Load Balancer as an example of what may be useful.
The text was updated successfully, but these errors were encountered: