[Local LLMs] Support ChatGLM3, whose has function call abilities

[yhyu13]

Hi,

THe latestest release of chatglm3 enables openai gpt like function call abilities https://github.com/THUDM/ChatGLM3/blob/main/tool_using/README.md

It seems we need to re-format the function calling conventions to adapt to chatglm3 so that we can have a local llms able to follow tool calling

[dreamworks2050]

yes please this is a great idea

[cpacker]

If ChatGLM3 supports native OpenAI ChatCompletion function calling, all you have to do is swap out OPENAI_API_BASE and it should "work", see the top of this README: https://github.com/cpacker/MemGPT/tree/main/memgpt/local_llm

From a brief look at a google translated README, it seems like you might have to write your own wrapper code to support their API calls, because they seem custom to ChatGLM3 (eg "role" / "content" + "tool").

If there's enough demand for this we might try to add this ourselves but in the meantime you can try to implement it following these instructions and open a PR: https://github.com/cpacker/MemGPT/tree/main/memgpt/local_llm#-adding-support-for-new-llms--improving-performance

[yhyu13]

Another problem with all these open source LLMs is that they are not 100% rate of actually replying with function call if you specifiy "function_call" token in openai api call. (OpenAI api has demand a firm function call reply if you specifiy "function_call" token, otherwise, it is up to the LLM to choose if reply with function call)

So we have to retry and retry until success. I guess OpenAI GPT4 might do it the same way behind the scene?