adding support for claude, azure, cohere, and replicate

[krrishdholakia]

Hi @eli64s,

Noticed you're calling OpenAI via REST Endpoints. I'm working on litellm (simple library to standardize LLM API Calls - https://github.com/BerriAI/litellm) and was wondering if we could be helpful.

I added support for Claude, Azure, Cohere and Llama2 (via Replicate) with an else-if statement. The code is pretty similar to the OpenAI class - as litellm follows the same pattern as the openai-python sdk.

Would love to know if this helps.

Happy to add additional tests / update documentation, if the initial PR looks good to you.

[krrishdholakia]

also curious - why call it via REST endpoints vs. the sdk or using langchain?

[IIBenII]

Hi @krrishdholakia ,
I tried your PR with Azure endpoint but it's not working,

• litellm not in pyproject.toml
• the code implement a function to test the token https://github.com/eli64s/readme-ai/blob/main/readmeai/conf.py#L33 and the function failed
• your if "gpt" in self.engine will not work as working with azure endpoint engine containt gpt inside https://docs.litellm.ai/docs/completion/supported#azure-openai-chat-completion-models

I can help to implement a better implementation if needed

[eli64s]

Hey @krrishdholakia, this is a cool project!

Using REST endpoints was more of wanting control and flexibility at first, but I've been exploring new implementations recently with Langchain and other frameworks such as LlamaIndex.

Litellm looks like a nice way to standardize and consolidate things. Could you add a few items when you get a moment:

• Update all requirement files with litellm
• Update the code if theres been any changes to litellm

Thanks!
Eli

[krrishdholakia]

Hey @eli64s - here's a screenshot of it working with claude-instant-1.

i added the option to save a LLM_API_KEY in os.environ. It felt more natural for me to save my non-openai llm key there vs. in OPENAI_API_KEY.

Let me know if there's anything else missing for this PR to be merged.