Forget rocm? Vulkan

[mlsterpr0]

I just wanted to give a big thank you for vulkan. I had no idea... Just gave it a try and was blown away. I don't have graphics card, but using vulkan with 10 gpu layers, and its just flying. Ryzen 7735hs 4gb Dedicated memory. It's significantly faster. I had no luck with rocm, but now don't need it. 7b gguf model speed is 7 T/s. Promt processing also much faster, everything is faster:)

[LostRuins]

Don't thank me, thank @0cc4m

[wh33t]

Can vulkan make use of tensor_split to have multiple AMD cards in-use like cublas + nvidia?

[LostRuins]

Yes. This needs to be done from the command line, however. See the usage of --usevulkan