WebGL support

[Macoron]

Whisper.cpp supports webasm and even has some online demos running into browser.

I didn't research that, but it seems that Unity support native C++ plugins for WebGL builds. So in theory, you can use cmake to compile whisper.cpp into .bc and embed it into web unity build.

Also Unity doesn't support microphone input for WebGL out of the box and you need third-party plugin to record anything.

[Macoron]

Did some tests today. Here is rough list of tasks that need to be done:

• Unity doesn't support C# multithreading. Whisper blocks main unity thread and app hangs until inference complete. The good news: Unity 2022.2+ supports C++ multithreading. If I add parallel job creation in whisper.cpp code - this should work.
• Bump Unity project to 2022 LTS
• Inference time is very slow comparing with official whisper.cpp demo. Unity transcribe "jfk.wav" for 15 seconds, while official demo works for 3 seconds. Researching this.
• No microphone support. Need to look for some MIT license compatible microphone library for Unity.

If you want to play around with it, check webgl branch.

[Bodekaer]

Hi @Macoron, any updates on webgl support. Did you get it running?

[Macoron]

Not much, biggest blocker is that Unity doesn't support C# multithreading. But you can checkout webgl branch. It's a bit outdated, but still working.

[D4N005H]

Not much, biggest blocker is that Unity doesn't support C# multithreading. But you can checkout webgl branch. It's a bit outdated, but still working.

Based on this source, Async/Await is possible for WebGL with this addon: /~https://github.com/Cysharp/UniTask

[janmajayamohanty-inteligen]

Got it working on Unity 2022.3.39f1. Thanks

[janmajayamohanty-inteligen]

The Webgl library does not implement whisper_init_from_file_with_params . It would be of immense help to have the Latest library for webgl.

[jiaxinnnnnc]

can i just check if this has been resolved and it can now work with webgl? if so how can i get it to work, sorry i am new to all this

[Thaina]

No microphone support. Need to look for some MIT license compatible microphone library for Unity.

Is it possible to add /~https://github.com/bnco-dev/unity-webgl-microphone in the meantime

[Nileneb]

There is a framework, which supports async operations in unity: /~https://github.com/Cysharp/UniTask.git

[DanielaAuroch]

Web GL builds are currently failing due to Runtime/Utils.MicrophoneRecord.cs making use of the Microphone Unity class which is not available in Web GL. These usages should be wrapped around #if !UNITY_WEBGL to allow for Web GL builds to succeed.