iOS example app

[ggerganov]

Implement a very basic iOS application using whisper.cpp

The ggwave-objc project can be used as a good starting point. It already provides the audio capture functionality. We just need to pass the captured data to whisper.cpp.

[ggerganov]

Just finished the initial implementation of the sample app and it is amazing!

whisper-iphone-13-mini.mp4

Edit: and I had forgotten to enable the Accelerate framework.. after enabling it - it's x2.5 times faster compared to what you see in the video! 🚀

---

Edit: And here is the latest version using the Accelerate framework + base.en model:

whisper-iphone-13-mini-2.mp4

[cmp-nct]

Impressive

[yujinqiu]

great! will it support stream mode ? @ggerganov

[ggerganov]

It's easy to add, but I will leave it as an exercise for someone with more experience in Objective-C and iOS apps

[ggerganov]

There is now basic streaming mode support added to the whisper.objc application:

whisper-iphone-13-mini-3.mp4

[siilats]

How do you enable the large project? it fails to allocate even if I manually enter the sizes

[ggerganov]

I haven't tried, but I believe you won't be able to load the large model because iOS apps have a memory limit of 2GB

[siilats]

It looks like there is an entitlement to get up to 16gb on iPad Pro https://9to5mac.com/2021/06/25/apps-can-request-access-to-more-ram-with-ios-15-entitlement-exceeding-normal-system-memory-limits/amp/ It looks like the parameters in iOS app are hardcoded not read from the bin file, right?

On Thu, Dec 8, 2022 at 12:19 AM Georgi Gerganov ***@***.***> wrote: I haven't tried, but I believe you won't be able to load the large model because iOS apps have a memory limit of 2GB — Reply to this email directly, view it on GitHub <#97 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAI2ZY44CORSMBPSP3JQHF3WMGDW7ANCNFSM6AAAAAARQRYMOE> . You are receiving this because you commented.Message ID: ***@***.***>

-- Keith Siilats ***@***.*** www.bytelogics.com Fax: 646-352-4705 Phone: +1 (646) 881-4378

[ggerganov]

The model size is not hardcoded. The only thing that is hardcoded in the current example is the name of the model file:

whisper.cpp/examples/whisper.objc/whisper.objc/ViewController.m

Line 53 in 832b4f3

NSString *modelPath = [[NSBundle mainBundle] pathForResource:@"ggml-base.en" ofType:@"bin"];

You just have to change it to ggml-large.bin and make sure that the model is uploaded to the iPhone / iPad.

[siilats]

Yeah I’m getting Malloc error and the model type is not 5 even though I changed it to large. It looks like it doesn’t get the 20 layers from the file.

On Thu, Dec 8, 2022 at 10:59 AM Georgi Gerganov ***@***.***> wrote: The model size is not hardcoded. The only thing that is hardcoded in the current example is the name of the model file: https://github.com/ggerganov/whisper.cpp/blob/832b4f34c9a155352c9f4c121faf92dcf759f86d/examples/whisper.objc/whisper.objc/ViewController.m#L53 You just have to change it to ggml-large.bin and make sure that the model is uploaded to the iPhone. — Reply to this email directly, view it on GitHub <#97 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAI2ZYZMHBYGRE34HGBND6DWMIOWXANCNFSM6AAAAAARQRYMOE> . You are receiving this because you commented.Message ID: ***@***.***>

-- Keith Siilats ***@***.*** www.bytelogics.com Fax: 646-352-4705 Phone: +1 (646) 881-4378

[ggerganov]

I just tested it with iPhone 14 Simulator and it works.
I only changed the model filename to ggml-large.bin and added the model to the project:

[siilats]

Hi, I also ran it successfully on the Iphone 14 simulator but when I try on the actual phone I get this error, I added the large memory entitlement.
Details

The app “whisper.objc” on iPhone quit unexpectedly.
Domain: IDEDebugSessionErrorDomain
Code: 11
Failure Reason: Message from debugger: Terminated due to memory issue
User Info: {
DVTErrorCreationDateKey = "2022-12-20 01:43:16 +0000";
IDERunOperationFailingWorker = DBGLLDBLauncher;
}

Analytics Event: com.apple.dt.IDERunOperationWorkerFinished : {
"device_model" = "iPhone14,3";
"device_osBuild" = "16.1.1 (20B101)";
"device_platform" = "com.apple.platform.iphoneos";
"launchSession_schemeCommand" = Run;
"launchSession_state" = 2;
"launchSession_targetArch" = arm64;
"operation_duration_ms" = 5083;
"operation_errorCode" = 11;
"operation_errorDomain" = IDEDebugSessionErrorDomain;
"operation_errorWorker" = DBGLLDBLauncher;
"operation_name" = IDEiPhoneRunOperationWorkerGroup;
"param_consoleMode" = 0;
"param_debugger_attachToExtensions" = 0;
"param_debugger_attachToXPC" = 1;
"param_debugger_type" = 5;
"param_destination_isProxy" = 0;
"param_destination_platform" = "com.apple.platform.iphoneos";
"param_diag_MainThreadChecker_stopOnIssue" = 0;
"param_diag_MallocStackLogging_enableDuringAttach" = 0;
"param_diag_MallocStackLogging_enableForXPC" = 1;
"param_diag_allowLocationSimulation" = 1;
"param_diag_checker_tpc_enable" = 1;
"param_diag_gpu_frameCapture_enable" = 0;
"param_diag_gpu_shaderValidation_enable" = 0;
"param_diag_gpu_validation_enable" = 0;
"param_diag_memoryGraphOnResourceException" = 0;
"param_diag_queueDebugging_enable" = 1;
"param_diag_runtimeProfile_generate" = 0;
"param_diag_sanitizer_asan_enable" = 0;
"param_diag_sanitizer_tsan_enable" = 0;
"param_diag_sanitizer_tsan_stopOnIssue" = 0;
"param_diag_sanitizer_ubsan_stopOnIssue" = 0;
"param_diag_showNonLocalizedStrings" = 0;
"param_diag_viewDebugging_enabled" = 1;
"param_diag_viewDebugging_insertDylibOnLaunch" = 1;
"param_install_style" = 0;
"param_launcher_UID" = 2;
"param_launcher_allowDeviceSensorReplayData" = 0;
"param_launcher_kind" = 0;
"param_launcher_style" = 0;
"param_launcher_substyle" = 0;
"param_runnable_appExtensionHostRunMode" = 0;
"param_runnable_productType" = "com.apple.product-type.application";
"param_runnable_type" = 2;
"param_testing_launchedForTesting" = 0;
"param_testing_suppressSimulatorApp" = 0;
"param_testing_usingCLI" = 0;
"sdk_canonicalName" = "iphoneos16.2";
"sdk_osVersion" = "16.2";
"sdk_variant" = iphoneos;
}

System Information

macOS Version 13.0.1 (Build 22A400)
Xcode 14.2 (21534) (Build 14C18)
Timestamp: 2022-12-19T20:43:16-05:00

[siilats]

When I did analyse I got this:

[brandonmcgraw]

Is it possible to process a transcription from a saved WAV file (vs. getting from the microphone) in your iOS implementation of Whisper?

[ggerganov]

The whisper.swiftui example already supports loading WAV files - give it a try.

[ridemove]

Does real-time transcription have a limit? I'm only reaching 400 seconds max then it stops.

Model = base

[ggerganov]

The whisper.objc is actually limited to 30 seconds:

whisper.cpp/examples/whisper.objc/whisper.objc/ViewController.h

Line 14 in 4a3f0d3

#define MAX_AUDIO_SEC 30

It's just a basic example app. It can be improved to support longer transcriptions, but it will make it more complicated.

[ridemove]

Thanks!

Also as new results are generated, the previous ones are captured. As the number of captured results increases, the processing time also increases significantly. Is there a way to reduce the number of previous results captured to improve performance?

[Akeel14]

Thanks!

Also as new results are generated, the previous ones are captured. As the number of captured results increases, the processing time also increases significantly. Is there a way to reduce the number of previous results captured to improve performance?

I'm following this cuz also looking for a solution but I think it's for accuracy.