Whisper-live taking same time on CPU and GPU to transcribe an audio

[prem1303]

I am using whisper-live==0.2.1 , faster-whisper==0.10.0 and Ctranslate2==4.0.0

Transcribing a 30-second audio file currently requires the same amount of time whether processed on a CPU or GPU, approximately 2 minutes. Any guidance on enhancing GPU performance to expedite this task would be greatly valued.

[trungkienbkhn]

@prem1303 , hello. FW 0.10.0 is broken tag, could you update FW and Ctranslate2 to latest version (FW 1.0.1 + Ctranslate2 4.2.0 + CUDA 12) ?

[prem1303]

Thanks @trungkienbkhn ,
I tried but still getting same time on GPU, any other suggestions????

[trungkienbkhn]

@prem1303 Could you show your code and attach example audio ? I will try testing them to evaluate throughput. I think you could try using smaller models or distil models to reduce time execution.

[prem1303]

I am using (https://github.com/collabora/WhisperLive) I have set up a client-side application locally and deployed a server.py (server side) on a GPU remote host. connecting the client and server using WebSockets and passing audio files using a Flask API.

[prem1303]

I solved the issues.
Thank you very much for your support and time.

[ruifengma]

@prem1303 do you use docker to deploy the model?