You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
On Ubuntu 22.04.04 using the upstream kernel driver (Linux dell 6.5.0-1020-oem #21-Ubuntu SMP PREEMPT_DYNAMIC Wed Apr 3 14:54:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux) with an Intel Arc A770 inference usually causes a segfault after some number of tokens.
### Issue submission checklist
- [X] I'm reporting an issue. It's not a question.
- [X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
- [X] There is reproducer code and related data files such as images, videos, models, etc.
The text was updated successfully, but these errors were encountered:
OpenVINO Version
2024.1.0
Operating System
Other (Please specify in description)
Device used for inference
GPU
Framework
PyTorch
Model used
https://huggingface.co/fakezeta/openchat-3.5-0106-openvino-int8
Issue description
On Ubuntu 22.04.04 using the upstream kernel driver (Linux dell 6.5.0-1020-oem #21-Ubuntu SMP PREEMPT_DYNAMIC Wed Apr 3 14:54:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux) with an Intel Arc A770 inference usually causes a segfault after some number of tokens.
Originally reported to LocalAI: mudler/LocalAI#2219
I had issues installing the out-of-tree driver from Intel so have not tried that. There are no problems using the iGPU on the same machine or the CPU.
c.c. @fakezeta
Step-by-step reproduction
It can be reproduced either in LocalAI or by running https://github.com/fakezeta/openchat.openvino/tree/main
All that is required is to ask (any?) model a longer question, such as to describe the output of
lscpu
. It will usually segfault.Relevant log output
Already disassembled here: mudler/LocalAI#2219 (comment)
The text was updated successfully, but these errors were encountered: