New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
crash loading llama-3-chinese-8b-instruct model #4080
Comments
Here is part of the server log that is directly related to this issue: |
Can you try running the server with |
To be frank, I'm yet to find out how to use those two env variables! |
Here is the serve log from the cpu-only machine: |
Now I guess I know how to re-start the server. Here is what I got from the server console after I set the first env. var., re-starte the server and re-run the model: |
And here is what I got from the server console after I used the 2nd env. var., the one for the CPU: |
Thanks for the logs. It looks like the model is triggering a crash inside llama.cpp during load, regardless of GPU type. |
Just FYI: After I have updated to 0.1.33, the message now is: And on 0.1.35, the message is: |
Never mind. It turns to be incomplete GGUF files downloaded by vanilla GIT command line. Sorry for the faulse alarm! |
What is the issue?
When trying run a model created from a GGUF model, the captioned error happens. The model can be downloade from: https://modelscope.cn/models/ChineseAlpacaGroup/llama-3-chinese-8b-instruct/summary
OS
Windows
GPU
Nvidia
CPU
Intel
Ollama version
0.132
The text was updated successfully, but these errors were encountered: