You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When I enable llamafile GPU support with -ngl 9999, it exits with the error
ggml_cuda_compute_forward: RMS_NORM failed
CUDA error: invalid device function
current device: 0, in function ggml_cuda_compute_forward at ggml-cuda.cu:11444
err
GGML_ASSERT: ggml-cuda.cu:9198: !"CUDA error"
I have also tried re-running after installing the AMD HIP SDK but this made no difference.
Contrary to the runtime messages, amdclang++.exe was in my Windows PATH.
Hey I believe integrated GPU are not supported, probably better to run on CPU at this time, by passing -ngl 0 instead of 9999
Also I have seen a few open issues with the same error/warning at the start when using AMD, so am not sure if I should open a new issues.
The line that says : get_rocm_bin_path: note: amdclang++.exe not found on $PATH
The actually file located there is named clang++.exe , in Windows, however on Linux is called amdclang++.exe
Perhaps there could be an Operating System check before looking for amdclang++ or clang++.
or maybe it is something else
Hey I believe integrated GPU are not supported, probably better to run on CPU at this time, by passing -ngl 0 instead of 9999
OK. I was curious to know whether additional acceleration could be obtained by combining the iGPU with the CPU.
In any case, I reckon the docs should explicitly state that AMD iGPUs are not supported (if they're not). Ideally, llamafile should also report this at runtime.
Also I have seen a few open issues with the same error/warning at the start when using AMD, so am not sure if I should open a new issues. The line that says : get_rocm_bin_path: note: amdclang++.exe not found on $PATH The actually file located there is named clang++.exe , in Windows, however on Linux is called amdclang++.exe
Perhaps there could be an Operating System check before looking for amdclang++ or clang++. or maybe it is something else
Yes. I copied clang++.exe to amdclang++.exe to overcome this. Both executables were in the $PATH, but the get_rocm_bin_path: note: amdclang++.exe not found on $PATH message still appeared.
I'm running llamafile 0.8.1 on a Windows 10 mini PC with a AMD Ryzen 9 5900HX CPU
CPU Architecture: AMD Cezanne (Zen 3, Ryzen 5000)
GPU: AMD Radeon RX Vega 8
The mini PC has 64GB RAM installed.
When I enable llamafile GPU support with -ngl 9999, it exits with the error
My command line is:
llamafile-0.8.1.exe -ngl 9999 -m dolphin-2.9-llama3-8b-Q5_K_M.gguf
I have also tried re-running after installing the AMD HIP SDK but this made no difference.
Contrary to the runtime messages, amdclang++.exe was in my Windows PATH.
The text was updated successfully, but these errors were encountered: