Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Refactor] Refactory of OpenAICompatible LLM setting #759

Closed
valentimarco opened this issue Mar 22, 2024 · 0 comments
Closed

[Refactor] Refactory of OpenAICompatible LLM setting #759

valentimarco opened this issue Mar 22, 2024 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@valentimarco
Copy link
Collaborator

Right now, the OpenAICompatible setting is broken (see also #735 #723 #713)!!

Describe the solution you'd like

Refactor this class with 4 fields:

  1. url
  2. temperature
  3. model_name
  4. api_key

And use CustomOpenAI as _pyclass defined in the factory/custom_llm.py.
This latest should be written like so:

from langchain_openai.chat_models import ChatOpenAI


class CustomOpenAI(ChatOpenAI):

    def __init__(self, **kwargs):
        super().__init__(
            openai_api_key=kwargs['api_key'],
            model_kwargs={},
            **kwargs
        )

        self.openai_api_base = kwargs['base_url']

This provides a basic connection for most runners that have OpenAI API compatibility (this is the reason why the model_kwargs are an empty dict, to provide only a simple connection!)

Additional context
If the user needs more customization from one provider than, he can build the specific client!
The community provide and publish some of this custom clients on the registry!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
Status: Done
Development

No branches or pull requests

2 participants