transcription results are inconsistent and timestamps are None type. Issue appears in the latest version of the transformers==4.38.1.

[kranipa]

transcription results are inconsistent and timestamps are None type. Issue appears in the latest version of the transformers==4.38.1.

example code.

processor = WhisperProcessor.from_pretrained("distil-whisper/distil-large-v2")
model = AutoModelForSpeechSeq2Seq.from_pretrained("distil-whisper/distil-large-v2", 
                                                                  torch_dtype=torch.float32, 
                                                                  low_cpu_mem_usage=True, 
                                                                  use_safetensors=True)    

pipe = pipeline(
                        "automatic-speech-recognition",
                        model=model,
                        tokenizer=processor.tokenizer,
                        feature_extractor=processor.feature_extractor,
                        max_new_tokens=128,
                        batch_size=16,
                        torch_dtype=torch.float32,
                        device='cpu',
                        return_timestamps=True,
                    )
    
pipe(audio_file)

SAMPLE FILE https://github.com/pyannote/pyannote-audio/blob/develop/tutorials/assets/sample.wav

transformers==4.37.2

{'text': " Hello? Hello? Hello? I didn't know you were there. Neither did I. Okay, I thought, you know, I heard a beep. This is Diane in New Jersey. And I'm Sheila and Texas, originally from Chicago. Oh, I'm originally from Chicago also. I'm in New Jersey now, though. Well, there isn't that much difference. At least, you know, they all call me a Yankee down here, so what kind of thinks. Oh, I don't hear that, th", 'chunks': [{'timestamp': (0.0, 11.92), 'text': " Hello? Hello? Hello? I didn't know you were there. Neither did I. Okay, I thought, you know,"}, {'timestamp': (11.92, 17.92), 'text': " I heard a beep. This is Diane in New Jersey. And I'm Sheila and Texas, originally from Chicago."}, {'timestamp': (17.92, 21.6), 'text': " Oh, I'm originally from Chicago also. I'm in New Jersey now, though."}, {'timestamp': (21.6, 28.0), 'text': " Well, there isn't that much difference. At least, you know, they all call me a Yankee down here, so what kind of thinks."}, {'timestamp': (28.0, None), 'text': " Oh, I don't hear that, th"}]}

timestamps are None for transformers==4.38.1

WARNING MESSAGE ==>

Whisper did not predict an ending timestamp, which can happen if audio is cut off in the middle of a word. Also make sure WhisperTimeStampLogitsProcessor was used during generation.

RESULTS ==>

{'text': " Hello? Hello? Oh, hello, I didn't know you were there. Neither did I. Okay, I thought, you know, I heard a beep. This is Diane in New Jersey. And I'm Sheila and Texas, originally from Chicago. Oh, I'm originally from Chicago also. I'm in New Jersey now, though. Well, there isn't that much difference. At least, you know, they all call me a Yankee down here, so what kind of that. Oh, I don't hear that in New Jersey now.", 'chunks': [{'timestamp': (None, None), 'text': " Hello? Hello? Oh, hello, I didn't know you were there. Neither did I. Okay, I thought, you know, I heard a beep. This is Diane in New Jersey. And I'm Sheila and Texas, originally from Chicago. Oh, I'm originally from Chicago also. I'm in New Jersey now, though. Well, there isn't that much difference. At least, you know, they all call me a Yankee down here, so what kind of that. Oh, I don't hear that in New Jersey now."}]}