[Refactor] Refactory of OpenAICompatible LLM setting #759
Labels
enhancement
New feature or request
good first issue
Good for newcomers
help wanted
Extra attention is needed
Right now, the OpenAICompatible setting is broken (see also #735 #723 #713)!!
Describe the solution you'd like
Refactor this class with 4 fields:
And use
CustomOpenAI
as_pyclass
defined in thefactory/custom_llm.py
.This latest should be written like so:
This provides a basic connection for most runners that have OpenAI API compatibility (this is the reason why the model_kwargs are an empty dict, to provide only a simple connection!)
Additional context
If the user needs more customization from one provider than, he can build the specific client!
The community provide and publish some of this custom clients on the registry!
The text was updated successfully, but these errors were encountered: