Whisper Unity3d bindings

[Macoron]

First of all, thanks to all developers of whisper.cpp. You are doing great job by making this open source and fast!

I've just finished bindings for Unity3d and place them in this repository called whisper.unity. It wraps native C++ methods of whisper.cpp into simplified C# API that can be used for creating 3d interactive software in Unity (games, 3d chatbots, VR apps, etc). Not all original whisper methods and parameters are properly supported yet, but I hope it will be fixed relatively soon.

Right now it supports Windows, MacOS, Linux, iOS and Android. Maybe WebGL in future releases.

[Macoron]

Few updates with a new features for Unity bindings.

subtitles.mp4

whisper.tiny in subtitles demo, color shows confidence level for each token

multilang.mp4

"whisper-small.bin" model tested in English, German and Russian from microphone

Recently I added custom Github Actions to build whisper.cpp from sources and update binary libraries of Unity bindings. I think it might be useful for other bindings developers, so I will leave pipeline here in case someone want to reuse it.

https://github.com/Macoron/whisper.unity/blob/master/.github/workflows/build_cpp.yml

[akareactor]

Thanks for your efforts! Is there any demo project for Whisper?

[Macoron]

Not compiled, but you can just download repo and run it using Unity Editor. There are samples scenes for AudioClips transcription, microphone input, subtitles mode, etc.

https://github.com/Macoron/whisper.unity

[akareactor]

Thank you very much!
Link https://github.com/Macoron/whisper.unity.git?path=/Packages/com.whisper.unity has no samples in package, downloaded it from page