Integrate Metal #34

jwijffels · 2024-01-27T21:33:37Z

No description provided.

jwijffels · 2024-01-31T08:19:41Z

jwijffels · 2024-02-01T19:51:30Z

Hello @jmgirard
About that Metal enabling. I saw I gave in the wrong path for these in Makevars, - fixed that in 4c0b272

Could you retry with installing. Thanks!

Sys.setenv(WHISPER_ACCELERATE = "1")
Sys.setenv(WHISPER_METAL = "1")
remotes::install_github("bnosac/audio.whisper", force = TRUE)

jmgirard · 2024-02-01T22:25:44Z

Package installed this time without errors (just the same warnings as before about cblas_sgemm being deprecated). Running the speed test now.

jmgirard · 2024-02-02T03:56:06Z

Accelerate only = 10.24 min
Accelerate + METAL = 12.95 min
METAL only = 14.92 min
Neither = 35.22 min

jwijffels · 2024-02-02T08:55:30Z

Thanks for the timings. From the README of whisper.cpp

On Apple Silicon, the inference runs fully on the GPU via Metal:

Do you have an Apple Silicon with GPU?

jwijffels · 2024-02-02T11:26:29Z

Accelerate only = 10.24 min Accelerate + METAL = 12.95 min METAL only = 14.92 min Neither = 35.22 min

This was whisper("medium") right?

jmgirard · 2024-02-02T13:30:12Z

Yes I have an M2 chip, which is apple silicon. It was the medium model while plugged in (vs battery).

jmgirard · 2024-02-02T16:57:01Z

I was surprised that Accelerate + METAL was slower than Accelerate only. Is that expected?

jwijffels · 2024-02-02T17:17:25Z

I don´t know. Maybe ggerganov/whisper.cpp#89 can provide some hints on that.
I presume you can see your GPU card being active?
I can only show the comparison with the use of OpenBLAS.
If you use trace = TRUE as argument in the predict call, you will see the printing out of the optimisations you will also find in whisper.cpp output.

jwijffels · 2024-02-03T07:15:43Z

Can you use ´model <- whisper('medium', use_gpu = TRUE)´ that is needed when you want to use your GPU.
I´ve added this in 721cb5c to start on the cuda enabling, but metal might also need it.
I still need to document it.

jmgirard · 2024-02-03T14:58:05Z

> options(timeout=60*5)
> model <- whisper("medium", use_gpu = TRUE)
trying URL 'https://huggingface.co/ggerganov/whisper.cpp/resolve/d15393806e24a74f60827e23e986f0c10750b358/ggml-medium.bin'
Content type 'application/octet-stream' length 1533763059 bytes (1462.7 MB)
==================================================
downloaded 1462.7 MB

Downloading finished, model stored at '/Users/jmgirard/ggml-medium.bin'
whisper_init_from_file_with_params_no_state: loading model from '/Users/jmgirard/ggml-medium.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51865
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 1024
whisper_model_load: n_audio_head  = 16
whisper_model_load: n_audio_layer = 24
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 1024
whisper_model_load: n_text_head   = 16
whisper_model_load: n_text_layer  = 24
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 4 (medium)
whisper_model_load: adding 1608 extra tokens
whisper_model_load: n_langs       = 99
whisper_backend_init: using Metal backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: default.metallib not found, loading from source
ggml_metal_init: GGML_METAL_PATH_RESOURCES = nil
ggml_metal_init: error: could not use bundle path to find ggml-metal.metal, falling back to trying cwd
ggml_metal_init: loading 'ggml-metal.metal'
Error in whisper_load_model(out$file, ...) : 
  c++ exception (unknown reason)

jwijffels · 2024-02-03T15:08:53Z

Ah an error showing it tries to use metal. Does that give the same error with the tiny model?

jmgirard · 2024-02-03T15:12:52Z

Yes, same error.

jwijffels · 2024-02-03T15:14:57Z

Can you show the compilation trace of the installation so that I can see how it linked to Metal on your system.

jmgirard · 2024-02-03T15:17:24Z

Sure, I assume you mean this:

> Sys.setenv(WHISPER_ACCELERATE = "1")
> Sys.setenv(WHISPER_METAL = "1")
> remotes::install_github("bnosac/audio.whisper", force = TRUE)
Downloading GitHub repo bnosac/audio.whisper@HEAD
── R CMD build ──────────────────────────────────────────────────
✔  checking for file ‘/private/var/folders/kr/tx86v16n5bx_djz_z2cpvfkc0000gq/T/Rtmpy2Igb1/remotesae3524adccf0/bnosac-audio.whisper-8a2b1f4/DESCRIPTION’ ...
─  preparing ‘audio.whisper’:
✔  checking DESCRIPTION meta-information
─  cleaning src
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘audio.whisper_0.3.1.tar.gz’
   
Installing package into ‘/opt/homebrew/lib/R/4.3/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘audio.whisper’ ...
** using staged installation
** libs
using C++ compiler: ‘Apple clang version 15.0.0 (clang-1500.1.0.2.5)’
using C++11
using SDK: ‘MacOSX14.2.sdk’
I whisper.cpp build info: 
I UNAME_S:  Darwin
I UNAME_P:  arm
I UNAME_M:  arm64
I PKG_CFLAGS:   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_CPPFLAGS: -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_LIBS:   -framework Accelerate -framework Foundation -framework Metal -framework MetalKit

clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-quants.c -o whisper_cpp/ggml-quants.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-backend.c -o whisper_cpp/ggml-backend.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-alloc.c -o whisper_cpp/ggml-alloc.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml.c -o whisper_cpp/ggml.o
whisper_cpp/ggml.c:9828:17: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
                cblas_sgemm(CblasRowMajor, CblasNoTrans, CblasTrans,
                ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
whisper_cpp/ggml.c:10238:9: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
        cblas_sgemm(CblasRowMajor, transposeA, CblasNoTrans, m, n, k, 1.0, a, lda, b, n, 0.0, c, n);
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
2 warnings generated.
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/whisper.cpp -o whisper_cpp/whisper.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/common-ggml.cpp -o whisper_cpp/common-ggml.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/common.cpp -o whisper_cpp/common.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c rcpp_whisper.cpp -o rcpp_whisper.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c RcppExports.cpp -o RcppExports.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2 -fobjc-exceptions  -c whisper_cpp/ggml-metal.m -o whisper_cpp/ggml-metal.o
clang++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/opt/homebrew/Cellar/r/4.3.2/lib/R/lib -L/opt/homebrew/opt/gettext/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/xz/lib -L/opt/homebrew/lib -o audio.whisper.so whisper_cpp/ggml-quants.o whisper_cpp/ggml-backend.o whisper_cpp/ggml-alloc.o whisper_cpp/ggml.o whisper_cpp/whisper.o whisper_cpp/common-ggml.o whisper_cpp/common.o rcpp_whisper.o RcppExports.o whisper_cpp/ggml-metal.o -framework Accelerate -framework Foundation -framework Metal -framework MetalKit -L/opt/homebrew/Cellar/r/4.3.2/lib/R/lib -lR -lintl -Wl,-framework -Wl,CoreFoundation
installing to /opt/homebrew/lib/R/4.3/site-library/00LOCK-audio.whisper/00new/audio.whisper/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (audio.whisper)

jwijffels · 2024-02-03T15:21:15Z

Thanks, will look into this later. Good to know already it is now trying to use the GPU.

jwijffels · 2024-02-04T01:14:21Z

Probably just need to put ggml-metal.metal in src folder instead of src/whisper_cpp folder or define the location of src/whisper_cpp in environment variable GGML_METAL_PATH_RESOURCES

…iable GGML_METAL_PATH_RESOURCES is set

jwijffels · 2024-02-04T18:25:23Z

Hello Jeffrey, this should be fixed now I believe with commit 908454f.
I've added that ggml-metal.metal file in the package and made sure it is referenced with environment variable GGML_METAL_PATH_RESOURCES when you call whisper("medium", use_gpu = TRUE)

Could you reïnstall and try again?

jmgirard · 2024-02-05T00:21:23Z

> model <- whisper("medium", use_gpu = TRUE)
whisper_init_from_file_with_params_no_state: loading model from '/Users/jmgirard/ggml-medium.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51865
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 1024
whisper_model_load: n_audio_head  = 16
whisper_model_load: n_audio_layer = 24
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 1024
whisper_model_load: n_text_head   = 16
whisper_model_load: n_text_layer  = 24
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 4 (medium)
whisper_model_load: adding 1608 extra tokens
whisper_model_load: n_langs       = 99
whisper_backend_init: using Metal backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: default.metallib not found, loading from source
ggml_metal_init: GGML_METAL_PATH_RESOURCES = /opt/homebrew/lib/R/4.3/site-library/audio.whisper/metal
ggml_metal_init: loading '/opt/homebrew/lib/R/4.3/site-library/audio.whisper/metal/ggml-metal.metal'
Error in whisper_load_model(out$file, ...) : 
  c++ exception (unknown reason)

Trace

> Sys.setenv(WHISPER_ACCELERATE = "1")
> Sys.setenv(WHISPER_METAL = "1")
> remotes::install_github("bnosac/audio.whisper", force = TRUE)
Downloading GitHub repo bnosac/audio.whisper@HEAD
── R CMD build ──────────────────────────────────────────────────
✔  checking for file ‘/private/var/folders/kr/tx86v16n5bx_djz_z2cpvfkc0000gq/T/RtmpKtk3Zm/remotese57dd31cd2e/bnosac-audio.whisper-3933d05/DESCRIPTION’ ...
─  preparing ‘audio.whisper’:
✔  checking DESCRIPTION meta-information
─  cleaning src
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘audio.whisper_0.3.1.tar.gz’
   
Installing package into ‘/opt/homebrew/lib/R/4.3/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘audio.whisper’ ...
** using staged installation
** libs
using C++ compiler: ‘Apple clang version 15.0.0 (clang-1500.1.0.2.5)’
using C++11
using SDK: ‘MacOSX14.2.sdk’
I whisper.cpp build info: 
I UNAME_S:  Darwin
I UNAME_P:  arm
I UNAME_M:  arm64
I PKG_CFLAGS:   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_CPPFLAGS: -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_LIBS:   -framework Accelerate -framework Foundation -framework Metal -framework MetalKit

clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-quants.c -o whisper_cpp/ggml-quants.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-backend.c -o whisper_cpp/ggml-backend.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-alloc.c -o whisper_cpp/ggml-alloc.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml.c -o whisper_cpp/ggml.o
whisper_cpp/ggml.c:9828:17: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
                cblas_sgemm(CblasRowMajor, CblasNoTrans, CblasTrans,
                ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
whisper_cpp/ggml.c:10238:9: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
        cblas_sgemm(CblasRowMajor, transposeA, CblasNoTrans, m, n, k, 1.0, a, lda, b, n, 0.0, c, n);
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
2 warnings generated.
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/whisper.cpp -o whisper_cpp/whisper.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/common-ggml.cpp -o whisper_cpp/common-ggml.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/common.cpp -o whisper_cpp/common.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c rcpp_whisper.cpp -o rcpp_whisper.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c RcppExports.cpp -o RcppExports.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2 -fobjc-exceptions  -c whisper_cpp/ggml-metal.m -o whisper_cpp/ggml-metal.o
clang++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/opt/homebrew/Cellar/r/4.3.2/lib/R/lib -L/opt/homebrew/opt/gettext/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/xz/lib -L/opt/homebrew/lib -o audio.whisper.so whisper_cpp/ggml-quants.o whisper_cpp/ggml-backend.o whisper_cpp/ggml-alloc.o whisper_cpp/ggml.o whisper_cpp/whisper.o whisper_cpp/common-ggml.o whisper_cpp/common.o rcpp_whisper.o RcppExports.o whisper_cpp/ggml-metal.o -framework Accelerate -framework Foundation -framework Metal -framework MetalKit -L/opt/homebrew/Cellar/r/4.3.2/lib/R/lib -lR -lintl -Wl,-framework -Wl,CoreFoundation
installing to /opt/homebrew/lib/R/4.3/site-library/00LOCK-audio.whisper/00new/audio.whisper/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (audio.whisper)

jwijffels · 2024-02-05T09:12:34Z

Thanks for your time. Looks like it crashed at https://github.com/bnosac/audio.whisper/blob/master/src/whisper_cpp/ggml-metal.m#L285-L303 when loading and next compiling the metal file.
Good thing is I managed to reproduce this using continuous integration. https://github.com/bnosac/audio.whisper/actions/runs/7781881608/job/21217187251. I'll look into it.

jwijffels · 2024-02-05T12:37:24Z

I did some debugging using the continuous integration. The root cause of this seems to be a conflict between R.h and the printing of the debug log.

whisper_model_load: adding 1608 extra tokens
  whisper_model_load: n_langs       = 99
  whisper_backend_init: using Metal backend
  ggml_metal_init: allocating
  ggml_metal_init: found device: Apple Paravirtual device
  ggml_metal_init: picking default device: Apple Paravirtual device
  ggml_metal_init: default.metallib not found, loading from source
  ggml_metal_init: GGML_METAL_PATH_RESOURCES = /Users/runner/work/audio.whisper/audio.whisper/check/audio.whisper.Rcheck/audio.whisper/metal
  ggml_metal_init: loading '/Users/runner/work/audio.whisper/audio.whisper/check/audio.whisper.Rcheck/audio.whisper/metal/ggml-metal.metal'
  2024-02-05 08:59:59.428 R[5328:23186] +[NSString stringWithContentsOfFile:encoding:Rf_error:]: unrecognized selector sent to class 0x7ff844c39820
  Error in whisper_load_model(out$file, use_gpu = TRUE, ...) : 
    c++ exception (unknown reason)
  Calls: whisper -> whisper_load_model

I've created a branch 'metal' which disables all these R printing and when looking at the continuous integration run I can see it worked - although the continuous integration runs apparently on Mac 7 which is not sufficient for Metal (hope you have some newer Mac).

Trace

── R CMD build ─────────────────────────────────────────────────────────────────
* checking for file ‘/private/var/folders/kx/gw6dssyn19d9qjs9mvh4hkz80000gn/T/RtmpPhoO2V/file23bc789[30](https://github.com/bnosac/audio.whisper/actions/runs/7781881608/job/21217187251#step:8:31)0c1/audio.whisper/DESCRIPTION’ ... OK
* preparing ‘audio.whisper’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building ‘audio.whisper_0.3.1.tar.gz’

Installing package into ‘/Users/runner/work/_temp/Library’
(as ‘lib’ is unspecified)
* installing *source* package ‘audio.whisper’ ...
** using staged installation
** libs
using C++ compiler: ‘Apple clang version 14.0.0 (clang-1400.0.29.202)’
using C++11
using SDK: ‘’
I whisper.cpp build info: 
I UNAME_S:  Darwin
I UNAME_P:  i386
I UNAME_M:  x86_64
I PKG_CFLAGS:   -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_CPPFLAGS: -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_LIBS:   -framework Accelerate -framework Foundation -framework Metal -framework MetalKit

clang -arch x86_64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/Users/runner/work/_temp/Library/Rcpp/include' -I/opt/R/x86_64/include   -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -falign-functions=64 -Wall -g -O2  -c whisper_cpp/ggml-quants.c -o whisper_cpp/ggml-quants.o
whisper_cpp/ggml-quants.c:54:23: warning: unused function 'mul_sum_i8_pairs' [-Wunused-function]
static inline __m128i mul_sum_i8_pairs(const __m128i x, const __m128i y) {
                      ^
whisper_cpp/ggml-quants.c:85:19: warning: unused function 'hsum_i[32](https://github.com/bnosac/audio.whisper/actions/runs/7781881608/job/21217187251#step:8:33)_4' [-Wunused-function]
static inline int hsum_i32_4(const __m128i a) {
                  ^
whisper_cpp/ggml-quants.c:150:23: warning: unused function 'packNibbles' [-Wunused-function]
static inline __m128i packNibbles( __m256i bytes )
                      ^
whisper_cpp/ggml-quants.c:1[33](https://github.com/bnosac/audio.whisper/actions/runs/7781881608/job/21217187251#step:8:34)7:14: warning: unused function 'make_qkx1_quants' [-Wunused-function]
static float make_qkx1_quants(int n, int nmax, const float * restrict x, uint8_t * restrict L, float * restrict the_min,
             ^
4 warnings generated.
clang -arch x86_64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/Users/runner/work/_temp/Library/Rcpp/include' -I/opt/R/x86_64/include   -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -falign-functions=64 -Wall -g -O2  -c whisper_cpp/ggml-backend.c -o whisper_cpp/ggml-backend.o
whisper_cpp/ggml-backend.c:840:13: warning: unused function 'sched_print_assignments' [-Wunused-function]
static void sched_print_assignments(ggml_backend_sched_t sched, struct ggml_cgraph * graph) {
            ^
1 warning generated.
clang -arch x86_64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/Users/runner/work/_temp/Library/Rcpp/include' -I/opt/R/x86_64/include   -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -falign-functions=64 -Wall -g -O2  -c whisper_cpp/ggml-alloc.c -o whisper_cpp/ggml-alloc.o
clang -arch x86_64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/Users/runner/work/_temp/Library/Rcpp/include' -I/opt/R/x86_64/include   -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -falign-functions=64 -Wall -g -O2  -c whisper_cpp/ggml.c -o whisper_cpp/ggml.o
whisper_cpp/ggml.c:2192:19: warning: unused function 'ggml_up32' [-Wunused-function]
static inline int ggml_up32(int n) {
                  ^
whisper_cpp/ggml.c:17591:13: warning: unused function 'ggml_opt_get_grad' [-Wunused-function]
static void ggml_opt_get_grad(int np, struct ggml_tensor * const ps[], float * g) {
            ^
2 warnings generated.
clang++ -arch x86_64 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/Users/runner/work/_temp/Library/Rcpp/include' -I/opt/R/x86_64/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c whisper_cpp/whisper.cpp -o whisper_cpp/whisper.o
clang++ -arch x86_64 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/Users/runner/work/_temp/Library/Rcpp/include' -I/opt/R/x86_64/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c whisper_cpp/common-ggml.cpp -o whisper_cpp/common-ggml.o
clang++ -arch x86_64 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/Users/runner/work/_temp/Library/Rcpp/include' -I/opt/R/x86_64/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c whisper_cpp/common.cpp -o whisper_cpp/common.o
clang++ -arch x86_64 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/Users/runner/work/_temp/Library/Rcpp/include' -I/opt/R/x86_64/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c rcpp_whisper.cpp -o rcpp_whisper.o
clang++ -arch x86_64 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/Users/runner/work/_temp/Library/Rcpp/include' -I/opt/R/x86_64/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c RcppExports.cpp -o RcppExports.o
clang -arch x86_64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/Users/runner/work/_temp/Library/Rcpp/include' -I/opt/R/x86_64/include    -fPIC  -falign-functions=64 -Wall -g -O2 -fobjc-exceptions  -c whisper_cpp/ggml-metal.m -o whisper_cpp/ggml-metal.o
clang++ -arch x86_64 -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/x86_64/lib -o audio.whisper.so whisper_cpp/ggml-quants.o whisper_cpp/ggml-backend.o whisper_cpp/ggml-alloc.o whisper_cpp/ggml.o whisper_cpp/whisper.o whisper_cpp/common-ggml.o whisper_cpp/common.o rcpp_whisper.o RcppExports.o whisper_cpp/ggml-metal.o -framework Accelerate -framework Foundation -framework Metal -framework MetalKit -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: -undefined dynamic_lookup may not work with chained fixups
installing to /Users/runner/work/_temp/Library/00LOCK-audio.whisper/00new/audio.whisper/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (audio.whisper)
trying URL 'https://huggingface.co/ggerganov/whisper.cpp/resolve/d15393806e24a74f60827e23e986f0c10750b[35](https://github.com/bnosac/audio.whisper/actions/runs/7781881608/job/21217187251#step:8:36)8/ggml-tiny.bin'
Content type 'application/octet-stream' length 77691713 bytes (74.1 MB)
==================================================
downloaded 74.1 MB

Downloading finished, model stored at '/Users/runner/work/audio.whisper/audio.whisper/ggml-tiny.bin'
whisper_init_from_file_with_params_no_state: loading model from '/Users/runner/work/audio.whisper/audio.whisper/ggml-tiny.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51865
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = [38](https://github.com/bnosac/audio.whisper/actions/runs/7781881608/job/21217187251#step:8:39)4
whisper_model_load: n_audio_head  = 6
whisper_model_load: n_audio_layer = 4
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 384
whisper_model_load: n_text_head   = 6
whisper_model_load: n_text_layer  = 4
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 1 (tiny)
whisper_model_load: adding 1608 extra tokens
whisper_model_load: n_langs       = 99
whisper_backend_init: using Metal backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple Paravirtual device
ggml_metal_init: picking default device: Apple Paravirtual device
ggml_metal_init: default.metallib not found, loading from source
ggml_metal_init: GGML_METAL_PATH_RESOURCES = /Users/runner/work/_temp/Library/audio.whisper/metal
ggml_metal_init: loading '/Users/runner/work/_temp/Library/audio.whisper/metal/ggml-metal.metal'
ggml_metal_init: GPU name:   Apple Paravirtual device
ggml_metal_init: hasUnifiedMemory              = false
ggml_metal_init: recommendedMaxWorkingSetSize  =  1073.74 MB
ggml_metal_init: maxTransferRate               = built-in GPU
whisper_backend_init: Metal GPU does not support family 7 - falling back to CPU
ggml_metal_free: deallocating
whisper_model_load:      CPU buffer size =    77.18 MB
whisper_model_load: model size    =   77.11 MB
whisper_backend_init: using Metal backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple Paravirtual device
ggml_metal_init: picking default device: Apple Paravirtual device
ggml_metal_init: default.metallib not found, loading from source
ggml_metal_init: GGML_METAL_PATH_RESOURCES = /Users/runner/work/_temp/Library/audio.whisper/metal
ggml_metal_init: loading '/Users/runner/work/_temp/Library/audio.whisper/metal/ggml-metal.metal'
ggml_metal_init: GPU name:   Apple Paravirtual device
ggml_metal_init: hasUnifiedMemory              = false
ggml_metal_init: recommendedMaxWorkingSetSize  =  1073.74 MB
ggml_metal_init: maxTransferRate               = built-in GPU
whisper_backend_init: Metal GPU does not support family 7 - falling back to CPU
ggml_metal_free: deallocating
whisper_init_state: kv self size  =    8.26 MB
whisper_init_state: kv cross size =    9.22 MB
whisper_init_state: compute buffer (conv)   =   12.17 MB
whisper_init_state: compute buffer (encode) =   64.92 MB
whisper_init_state: compute buffer (cross)  =    4.01 MB
whisper_init_state: compute buffer (decode) =   96.02 MB

system_info: n_threads = 1 / 4 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | METAL = 1 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0 | 

whisper_print_timings:     load time =   157.06 ms
whisper_print_timings:     fallbacks =   0 p /   0 h
whisper_print_timings:      mel time =     0.00 ms
whisper_print_timings:   sample time =     0.00 ms /     1 runs (    0.00 ms per run)
whisper_print_timings:   encode time =  1210.80 ms /     1 runs ( 1210.80 ms per run)
whisper_print_timings:   decode time =     0.00 ms /     1 runs (    0.00 ms per run)
whisper_print_timings:   batchd time =     0.00 ms /     1 runs (    0.00 ms per run)
whisper_print_timings:   prompt time =     0.00 ms /     1 runs (    0.00 ms per run)
whisper_print_timings:    total time =  1[40](https://github.com/bnosac/audio.whisper/actions/runs/7781881608/job/21217187251#step:8:41)8.[47](https://github.com/bnosac/audio.whisper/actions/runs/7781881608/job/21217187251#step:8:48) ms

You can install as follows to test it. I'll see how can work around this printing issue later.

Sys.setenv(WHISPER_ACCELERATE = "1")
Sys.setenv(WHISPER_METAL = "1")
remotes::install_github("bnosac/audio.whisper@metal", force = TRUE)

library(audio.whisper)
model <- whisper("medium", use_gpu = TRUE)

jmgirard · 2024-02-05T14:31:15Z

Wow! Big improvement there.

Sys.setenv(WHISPER_ACCELERATE = "1")
Sys.setenv(WHISPER_METAL = "1")
remotes::install_github("bnosac/audio.whisper@metal", force = TRUE)
library(audio.whisper)
model <- whisper("medium", use_gpu = TRUE)
trans <- predict(model, newdata = "output.wav", language = "en", n_threads = 1)
trans$timing
#> 1.977917 mins

jwijffels · 2024-02-05T15:09:29Z

Note to myself: the definition of error in R_ext/Error.h which is loaded in ggml.h and is needed to cleanly exit R in case of an unexpected error conflicts with NSError * error which is used in ggml-metal.m. Don't know yet how to solve this.

jwijffels · 2024-02-05T15:10:39Z

Wow! Big improvement there.

Sys.setenv(WHISPER_ACCELERATE = "1")
Sys.setenv(WHISPER_METAL = "1")
remotes::install_github("bnosac/audio.whisper@metal", force = TRUE)
library(audio.whisper)
model <- whisper("medium", use_gpu = TRUE)
trans <- predict(model, newdata = "output.wav", language = "en", n_threads = 1)
trans$timing
#> 1.977917 mins

Nice indeed. And does that speed up if you specify n_threads = 2 or n_threads = 4?

jmgirard · 2024-02-05T15:12:02Z

With 4 threads, it goes down to 1.359346 mins

jwijffels · 2024-02-05T20:36:45Z

I've incorporated the this Metal integration in the latest release, including fixes for the issue mentionned here: #34 (comment)
I'll remove that branch, you can just install version 0.3.1 from now on.

jwijffels added a commit that referenced this issue Jan 29, 2024

#34 Enable Metal in Makevars

c1a3a88

jwijffels added a commit that referenced this issue Jan 30, 2024

#34 enable metal

32758b8

jwijffels mentioned this issue Jan 31, 2024

upgrade to 1.5.0 #24

Closed

jwijffels added a commit that referenced this issue Feb 4, 2024

#34 put ggml-metal.metal in inst folder and make sure environment var…

908454f

…iable GGML_METAL_PATH_RESOURCES is set

jwijffels added a commit that referenced this issue Feb 5, 2024

#34 Enable Metal and avoid clashing of definition of error with R.h

2fdc52f

jwijffels added a commit that referenced this issue Feb 5, 2024

#34 README

1443570

jwijffels closed this as completed Feb 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate Metal #34

Integrate Metal #34

jwijffels commented Jan 27, 2024

jwijffels commented Jan 31, 2024

jwijffels commented Feb 1, 2024

jmgirard commented Feb 1, 2024 •

edited

jmgirard commented Feb 2, 2024

jwijffels commented Feb 2, 2024

jwijffels commented Feb 2, 2024

jmgirard commented Feb 2, 2024

jmgirard commented Feb 2, 2024

jwijffels commented Feb 2, 2024 •

edited

jwijffels commented Feb 3, 2024 •

edited

jmgirard commented Feb 3, 2024

jwijffels commented Feb 3, 2024

jmgirard commented Feb 3, 2024

jwijffels commented Feb 3, 2024

jmgirard commented Feb 3, 2024

jwijffels commented Feb 3, 2024

jwijffels commented Feb 4, 2024 •

edited

jwijffels commented Feb 4, 2024 •

edited

jmgirard commented Feb 5, 2024 •

edited

jwijffels commented Feb 5, 2024

jwijffels commented Feb 5, 2024 •

edited

jmgirard commented Feb 5, 2024

jwijffels commented Feb 5, 2024 •

edited

jwijffels commented Feb 5, 2024

jmgirard commented Feb 5, 2024

jwijffels commented Feb 5, 2024

Integrate Metal #34

Integrate Metal #34

Comments

jwijffels commented Jan 27, 2024

jwijffels commented Jan 31, 2024

jwijffels commented Feb 1, 2024

jmgirard commented Feb 1, 2024 • edited

jmgirard commented Feb 2, 2024

jwijffels commented Feb 2, 2024

jwijffels commented Feb 2, 2024

jmgirard commented Feb 2, 2024

jmgirard commented Feb 2, 2024

jwijffels commented Feb 2, 2024 • edited

jwijffels commented Feb 3, 2024 • edited

jmgirard commented Feb 3, 2024

jwijffels commented Feb 3, 2024

jmgirard commented Feb 3, 2024

jwijffels commented Feb 3, 2024

jmgirard commented Feb 3, 2024

jwijffels commented Feb 3, 2024

jwijffels commented Feb 4, 2024 • edited

jwijffels commented Feb 4, 2024 • edited

jmgirard commented Feb 5, 2024 • edited

jwijffels commented Feb 5, 2024

jwijffels commented Feb 5, 2024 • edited

jmgirard commented Feb 5, 2024

jwijffels commented Feb 5, 2024 • edited

jwijffels commented Feb 5, 2024

jmgirard commented Feb 5, 2024

jwijffels commented Feb 5, 2024

jmgirard commented Feb 1, 2024 •

edited

jwijffels commented Feb 2, 2024 •

edited

jwijffels commented Feb 3, 2024 •

edited

jwijffels commented Feb 4, 2024 •

edited

jwijffels commented Feb 4, 2024 •

edited

jmgirard commented Feb 5, 2024 •

edited

jwijffels commented Feb 5, 2024 •

edited

jwijffels commented Feb 5, 2024 •

edited