Add OpenELM

[3Samourai]

Apple released several open source LLMs that are designed to run on-device.
Huggingface Link

[thinkverse]

Not supported in llama.cpp yet, there's an issue for it, ggerganov/llama.cpp#6868, labeled as a good first issue, if someone with C++ and Python experience wants to tackle it. 👍

[mertbozkir]

Interesting, I haven't seen this issue, and was trying to upload this model. 🫣

[userforsource]

What is the requirement for llama.cpp.
As far as i see ollama does not depend on llama.cpp

[userforsource]

What is the requirement for llama.cpp.
As far as i see ollama does not depend on llama.cpp

[thinkverse]

As far as i see ollama does not depend on llama.cpp

Ollama makes heavy use of llama.ccp, it's the backend Ollama uses. When you start Ollama, it starts a llama.cpp server. When you chat with an LLM using Ollama is forwards it to the llama.cpp server.

ollama/llm/server.go

Line 73 in 2bed629

func NewLlamaServer(gpus gpu.GpuInfoList, model string, ggml *GGML, adapters, projectors []string, opts api.Options) (LlamaServer, error) {

ollama/server/routes.go

Lines 1315 to 1320 in 2bed629

	if err := runner.llama.Completion(c.Request.Context(), llm.CompletionRequest{
	Prompt: prompt,
	Format: req.Format,
	Images: images,
	Options: opts,
	}, fn); err != nil {

You can see the llama.cpp submodude under https://github.com/ollama/ollama/tree/main/llm.

[userforsource]

ggerganov/llama.cpp#6960

[thinkverse]

Update on the OpenELM support, a draft PR has been opened ggerganov/llama.cpp#6986, and @joshcarp is looking for anyone to help out. I'm certain it would be appreciated if anyone who has experience with C++, Python, or something related can help. 👍