How does llama cpp backend work?

[chsasank]

I am curious to see the latest update where IPEX is actually working as backend for llama.cpp. This is a very unusual architecture. Interested to know how this work! Can you point me to the right source files?

[jason-dai]

@chsasank To clarify, IPEX is not working as backend for llama.cpp. Many performance critical operations in IPEX-LLM is implemented in C++ (or sycl); by exposing a C++ interface, we can use these operations in IPEX-LLM as a backend for llama.cpp.

[chsasank]

Can I see the code for this? Where is it implemented in the repo?

[chsasank]

It looks like bigdl-core-cpp are the cpp bindings. Where is the source for this package?