You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@rachittshah If I understood the TGI post correctly, we would simply set api_base to be access models on TGI. We already have this in langroid, i.e. you can set OpenAIGPTConfig.chat_model = "local/localhost:5000/v1") and under the hood the api_base and api_key are set to the appropriate values.
What I had in mind with this issue is directly using the Mistral python API client to access models on their servers. Provided their API is OpenAI-compatible (which I'm not sure yet, as they don't seem to mention OpenAI anywhere in their API docs), this would be very simple, i.e. adding a couple of model name enums in langroid/language_models/openai_gpt.py and switching to their chat/completion endpoint based on the chat_model setting (just like we currently switch between litellm and openai completion calls).
https://github.com/mistralai/client-python
The text was updated successfully, but these errors were encountered: