Possible reason for increased encoding time? (v1.4.0 --> v.1.5.2)

[LVCSRer]

Hello.

I was using v1.4.0 for developing iOS app, and recently found there was major update and applied to my app.

I use iPhone 12 pro and base.en model with Core ML.
I tested with my own recorded 15 seconds long audio, and below figure is average of 10 times iteration.

For Core ML without Metal setting, overall time was reduced due to decode time reduction, but encode time is increased.

Does anyone know possible reason for this behavior?

[ggerganov]

If you change this:

whisper.cpp/coreml/whisper-encoder.mm

Lines 25 to 29 in ab0a859

	// select which device to run the Core ML model on
	MLModelConfiguration *config = [[MLModelConfiguration alloc] init];
	config.computeUnits = MLComputeUnitsCPUAndGPU;
	//config.computeUnits = MLComputeUnitsCPUAndNeuralEngine;
	//config.computeUnits = MLComputeUnitsAll;

To this:

    // select which device to run the Core ML model on
    MLModelConfiguration *config = [[MLModelConfiguration alloc] init];
    //config.computeUnits = MLComputeUnitsCPUAndGPU;
    //config.computeUnits = MLComputeUnitsCPUAndNeuralEngine;
    config.computeUnits = MLComputeUnitsAll;

Does it help?

[LVCSRer]

Yes! Setting to both MLComputeUnitsCPUAndNeuralEngine and MLComputeUnitsAll reduces encode time by half for iPhone 12.
Both setting require more time for first model initialization than MLComputeUnitsCPUAndGPU. (Maybe because of using ANE?)

Thanks for the advice!