Trying to compile for hipblas/ROCm with cmake causes ggml.h not found error

[lyssieth]

I have the latest commit at time of writing (8f253ef), and am trying to use cmake to compile it instead of make, because I'm trying to debug an issue with building Rust bindings for it (which uses cmake).

I've been able to reproduce the issue with cmake only, so I believe it's not related to what I'm trying to do, but rather an issue in how this project builds itself.

System info:

• Ryzen 5900X CPU
• 7900XTX GPU (gfx1100)
• ROCm 6.0.2 from Arch

The reproduction process is as follows:

1. Fresh clone of the repo
2. mkdir build && cd build
3. cmake -GNinja -DWHISPER_HIPBLAS=on -DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang -DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++ ..
4. ninja

And the output that produces on my machine:

[4/20] Building CXX object CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o
FAILED: CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o
/opt/rocm/llvm/bin/clang++ -DGGML_USE_CUDA -DGGML_USE_HIPBLAS -DUSE_PROF_API=1 -D_GNU_SOURCE -D_XOPEN_SOURCE=600 -D__HIP_PLATFORM_AMD__=1 -isystem /opt/rocm/include -Wall                                       -Wextra                                     -Wpedantic                                  -Wcast-qual                              -O3 -DNDEBUG -std=gnu++11 -fPIC -mllvm -amdgpu-early-inline-all=true -mllvm -amdgpu-function-calls=false -x hip -MD -MT CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o -MF CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o.d -o CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o -c /tmp/whisper.cpp/ggml-cuda.cu
In file included from /tmp/whisper.cpp/ggml-cuda.cu:5:
/tmp/whisper.cpp/ggml-cuda/common.cuh:3:10: fatal error: 'ggml.h' file not found
#include "ggml.h"
         ^~~~~~~~
1 error generated when compiling for gfx906.
[12/20] Building CXX object CMakeFiles/whisper.dir/whisper.cpp.o
/tmp/whisper.cpp/whisper.cpp:206:29: warning: unused function 'ggml_mul_mat_pad' [-Wunused-function]
static struct ggml_tensor * ggml_mul_mat_pad(struct ggml_context * ctx, struct ggml_tensor * x, struct ggml_tensor * y, int pad = 32) {
                            ^
1 warning generated.
[13/20] Building CXX object examples/server/CMakeFiles/server.dir/server.cpp.o
ninja: build stopped: subcommand failed.

Setting DAMDGPU_TARGETS=gfx1100 produces the same error, so I doubt it's related.

Edit: I'll also add that building with make WHISPER_HIPBLAS=1 doesn't cause any errors.

[lyssieth]

Manually editing the generated build.ninja file's ggml.cuda.cu.o chunk to the following:

build CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o: CXX_COMPILER__ggml-rocm_unscanned_Release /home/lys/Downloads/whisper.cpp/ggml-cuda.cu || cmake_object_order_depends_target_ggml-rocm
  DEFINES = -DGGML_USE_CUDA -DGGML_USE_HIPBLAS -DUSE_PROF_API=1 -D_GNU_SOURCE -D_XOPEN_SOURCE=600 -D__HIP_PLATFORM_AMD__=1
  DEP_FILE = CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o.d
  FLAGS = -Wall                                       -Wextra                                     -Wpedantic                                  -Wcast-qual                              -O3 -DNDEBUG -std=gnu++11 -fPIC -mllvm -amdgpu-early-inline-all=true -mllvm -amdgpu-function-calls=false -x hip
  INCLUDES = -isystem /opt/rocm/include -I/home/lys/Downloads/whisper.cpp/. # <--- Edit is here, adding the include for `whisper.cpp/.`
  OBJECT_DIR = CMakeFiles/ggml-rocm.dir
  OBJECT_FILE_DIR = CMakeFiles/ggml-rocm.dir

Makes it build again, but then I get:

./bin/main: symbol lookup error: /home/lys/Downloads/whisper.cpp/build/libwhisper.so.1: undefined symbol: _Z19ggml_cuda_op_im2colR25ggml_backend_cuda_contextP11ggml_tensor

[lyssieth]

The first bug (where ggml.h is not found) can be easily replicated in the rocm/rocm-terminal docker image.

After installing the dependencies (sudo apt update && sudo apt install -y hipblas rocblas)

1. mkdir build && cd build
2. cmake -DWHISPER_HIPBLAS=ON -DWHISPER_BUILD_TESTS=OFF -DWHISPER_BUILD_EXAMPLES=OFF -DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang -DCMAKE_CXX_COMPILER='/opt/rocm/llvm/bin/clang++' ..
3. cmake --build .

And it fails with the ggml.h not found error as expected.

[lyssieth]

Okay, it builds now. There's an issue, however, with running the main example.

(FILE_PATH_REMOVED == path to where I'm storing my whisper ggml models and wav files for testing)

❯ ./bin/main -m FILE_PATH_REMOVED/ggml-medium.en.bin FILE_PATH_REMOVED/untitled.wav
whisper_init_from_file_with_params_no_state: loading model from 'FILE_PATH_REMOVED/ggml-medium.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51864
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 1024
whisper_model_load: n_audio_head  = 16
whisper_model_load: n_audio_layer = 24
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 1024
whisper_model_load: n_text_head   = 16
whisper_model_load: n_text_layer  = 24
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 4 (medium)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs       = 99
whisper_backend_init: using CUDA backend
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:   no
ggml_cuda_init: CUDA_USE_TENSOR_CORES: yes
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon RX 7900 XTX, compute capability 11.0, VMM: no
whisper_model_load:    ROCm0 total size =  1533.14 MB
whisper_model_load: model size    = 1533.14 MB
whisper_backend_init: using CUDA backend
whisper_init_state: kv self size  =  132.12 MB
whisper_init_state: kv cross size =  147.46 MB
whisper_init_state: compute buffer (conv)   =   28.68 MB
whisper_init_state: compute buffer (encode) =  594.22 MB
whisper_init_state: compute buffer (cross)  =    7.85 MB
whisper_init_state: compute buffer (decode) =  138.87 MB

system_info: n_threads = 4 / 24 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | CUDA = 1 | COREML = 0 | OPENVINO = 0

main: processing 'FILE_PATH_REMOVED/untitled.wav' (184971 samples, 11.6 sec), 4 threads, 1 processors, 5 beams + best of 5, lang = en, task = transcribe, timestamps = 1 ...

ggml_cuda_compute_forward: IM2COL failed
CUDA error: shared object initialization failed
  current device: 0, in function ggml_cuda_compute_forward at /tmp/whisper.cpp/ggml-cuda.cu:2304
  err
GGML_ASSERT: /tmp/whisper.cpp/ggml-cuda.cu:60: !"CUDA error"
ptrace: Operation not permitted.
No stack.
The program is not being run.
fish: Job 1, './bin/main -m FILE_PATH_REMOVED…' terminated by signal SIGABRT (Abort)

[lyssieth]

It seems that using the CC/CXX=hipcc described in #2102 makes it work fine :)