Challenges with .train() function in faster-whisper for Fine-tuning WhisperModel

[SDeyGitHubber]

This discussion aims to explore the challenges encountered while using the faster-whisper library for fine-tuning a Whisper model, specifically focusing on the apparent absence of a .train() function within the WhisperModel class.

1. Context:

• I'm working on fine-tuning a Whisper model for Bengali speech recognition using a dataset of 100 audio files.
• The faster-whisper library appears to be a promising choice for this task.
• However, attempts to call model.train() on a WhisperModel instance resulted in an AttributeError.

2. Challenges:

2.a.) Missing .train() function:

• The WhisperModel class seemingly lacks a built-in .train() function, which is typically used to switch the model to training mode.

• This makes it difficult to fine-tune the model on a custom dataset.
  2.b) Limited Documentation:

• The faster-whisper documentation might not explicitly address the process of fine-tuning or the training mode within WhisperModel.

[gongouveia]

I believe. that is not how it works.
You have to train a model from whisper with huggingface transformers. then convert the model to be used back to whisper format.
Convert this model again to be used in faster whisper

[SDeyGitHubber]

Let me give it a try and check it out