LLM-provider agnostic client

[timatim]

Is your feature request related to a problem? Please describe.
In my development work, I've found it tedious when working with multiple providers having to initiate a client for each provider and patch them separately, while keep track of one object for each provider throughout my app.

This felt very redundant to me because those clients are all stateless apart from an api key. This is of course not an issue with the instructor lib itself, but imo it is a good place for instructor to come in play as it is already somewhat working towards (whether intentionally or not) having a provider-agnostic interface.

Describe the solution you'd like
A generic LLM client patched with instructor that invokes different client depending on the model name supplied during a call. This also allows extensions into other things like token usage tracking. I have a working solution for this that I'd be happy to contribute but wanted to check if this is of interest/considered by the team, or deemed unnecessary/out of scope.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[jxnl]

i recommend https://docs.litellm.ai/docs/

[timatim]

Thank you, was not aware of this!