Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Webgpu backend #585

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Open

Conversation

audiovention
Copy link

Hi all,

I have whipped up a very quick and dirty webgpu backend for ggml. This is by no means ready for merging, it's been written in a day, supports only silu operation, and has a lot of extra files and questions around how to do things properly.
The implementation is copy/pasted from the Metal backend and adjusted.

Note - working on native with Mozilla's wgpu lib, not even tested on emscripten target, and might have issues on Chrome's Dawn.

@ggerganov I wanted to drop it here, if anyone wants to take over, as I'll be quite busy and plan to get back to it in a few months.

The branch doesn't have anything you need to make it run, here's an example repo that runs a working example:
https://github.com/audiovention/webgpu-ggml

The repo contains precompiled wgpu libs for Win/Linux/macos downloaded from https://eliemichel.github.io/LearnWebGPU/getting-started/hello-webgpu.html

The impl is mostly based on the example here:
https://github.com/gfx-rs/wgpu-native/blob/trunk/examples/compute/main.c#L28
Also I've copied a couple of source files from that repo, which currently take care of loading the shader, they can be easily removed.

Things that need to be addressed:

  • GGML_WGPU_ADD_KERNEL - doesn't pass the kernel name, my c-macro skills are lacking
  • ggml_wgpu_free is not implemented
  • ggml_wgpu_get_tensor is abysmal, but I really wanted to see numbers on my screen
  • the operation tensor buffers (src0,src1,dst) are not mapped to the kernel, instead I map the entire storage buffer and pass the offsets to the kernel - I tried to map the tensor buffers from where they start, but webgpu has restrictions on buffer offset alignment. I believe ggml can already handle that, but as I said I really wanted numbers on my screen
  • the webgpu headers/libs are not currently incorporated in the repo, I have no idea how to do this nicely, one option is to add the Chrome Dawn as a submodule and build it together with ggml from source

I'm open to answer any questions with my limited webgpu experience, and as I said I'll be happy if someone takes over.

@ggerganov
Copy link
Owner

Great - very useful! This is exactly what we need to get us started.
This will stay a bit in the background for some time as there are more pressing issues to resolve. So unless there is community effort in implementing this further, I'll integrate this at some point in the future, most likely after the backends interface work is completed.

@KerfuffleV2
Copy link

Not sure if it's useful, but I just found this today: https://github.com/cryscan/web-rwkv

So it's in Rust, but it's based on WebGPU. Also there are a bunch of WGSL files for stuff like matmul, softmax, SiLU etc in there: https://github.com/cryscan/web-rwkv/tree/main/src/shaders

Has a compatible license as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants