Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Mistral API analogous to OpenAI API #384

Open
pchalasani opened this issue Jan 25, 2024 · 4 comments
Open

Add Mistral API analogous to OpenAI API #384

pchalasani opened this issue Jan 25, 2024 · 4 comments
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@pchalasani
Copy link
Contributor

https://github.com/mistralai/client-python

@pchalasani pchalasani added enhancement New feature or request good first issue Good for newcomers labels Jan 25, 2024
@rachittshah
Copy link

Hey @pchalasani I'd love to take this up.

I'll refer the TGI endpoint implementation by huggingface to build this TGI endpoint details

@pchalasani
Copy link
Contributor Author

@rachittshah If I understood the TGI post correctly, we would simply set api_base to be access models on TGI. We already have this in langroid, i.e. you can set OpenAIGPTConfig.chat_model = "local/localhost:5000/v1") and under the hood the api_base and api_key are set to the appropriate values.

What I had in mind with this issue is directly using the Mistral python API client to access models on their servers. Provided their API is OpenAI-compatible (which I'm not sure yet, as they don't seem to mention OpenAI anywhere in their API docs), this would be very simple, i.e. adding a couple of model name enums in langroid/language_models/openai_gpt.py and switching to their chat/completion endpoint based on the chat_model setting (just like we currently switch between litellm and openai completion calls).

@rachittshah
Copy link

@pchalasani understood, and it makes sense to have a switch between litellm and openai call.

I'm new to Langdrioid, but I'd love to pick this up!

@pchalasani
Copy link
Contributor Author

@rachittshah sure go ahead , thanks. I can elaborate a bit more what I had in mind later when I have time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants