[WIP] S2ST recipe for SpeechMatrix

[juice500ml]

What?

Features

• Enable utilizing pretrained kmeans model
• Enable utilizing HiFiGAN pretrained vocoder
• Add shallow decoder for source unit estimation
• Add huggingface ASR models for ASR BLEU calculation
• Refactor: Add option for choosing between maintaining vocabulary vs. filtering OOV cases
• Refactor: Connect run.sh's src_lang and tgt_lang parameter with local/data.sh
• Refactor: We have to manually skip Stage 4. Add --skip_stages or other flags for s2st.sh
• Refactor: fairseq version that is currently installed by ESPnet does not support HiFiGAN
• Refactor: Handle different vocoder types more elegantly

Bug fixes

• Fix bugs related to the case where we only have speech data for training (i.e., use_src_lang=false, use_tgt_lang=false)
• Fixed fix_data_dir filtering bug (due to wav.scp.${src_lang} and wav.scp.${tgt_lang} being custom files)

Data preparation

• FLEURS data preparation
• EPST data preparation
• SpeechMatrix data preparation
• Remove fairseq dependency on data preparation by copying & modifying
• Add necessary exception handling for additional python packages
• SpeechMatrix valid/test splits
• Refactor data_prep.py (too big of a file with too many repetitions)
• Refactor run.sh to handle different test_sets per language pairs
• Refactor this commit: 3af3f41

Modeling

• Conducted hyperparameter tuning to find the optimal architecture
• Conducted hyperparameter tuning to find the optimal learning rate

Why?

See also