Skip to content

Automatically generate, translate, and overlay subtitles for any video.

License

Notifications You must be signed in to change notification settings

YJ-20/auto-subtitle-translate

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Automatic subtitles & translates in your videos

This repository uses ffmpeg and OpenAI's Whisper to automatically generate and overlay subtitles on any video. Then, it uses pretrained Llama2 to translate the subtitles to 50 languages.

Installation

To get started, you'll need Python 3.7 or newer. Install the binary by running the following command:

pip install git+https://github.com/YJ-20/auto-subtitle-llama

You'll also need to install ffmpeg, which is available from most package managers:

# on Ubuntu or Debian
sudo apt update && sudo apt install ffmpeg

# on MacOS using Homebrew (https://brew.sh/)
brew install ffmpeg

# on Windows using Chocolatey (https://chocolatey.org/)
choco install ffmpeg

Usage

The following command will generate a subtitled/video.mp4 file contained the input video with overlayed subtitles.

auto_subtitle_llama /path/to/video.mp4 -o subtitled/

The default setting (which selects the large model) works well for transcribing English. You can optionally use a bigger model for better results (especially with other languages). The available models are tiny, tiny.en, base, base.en, small, small.en, medium, medium.en, large.

auto_subtitle_llama /path/to/video.mp4 --model medium

Adding --translate_to language_code will translate the subtitles into one of the 50 languages:

auto_subtitle_llama /path/to/video.mp4 --translate_to language_code

Language Code

Language Arabic Czech German English Spanish Estonian Finnish French Gujarati Hindi Italian Japanese Kazakh Korean Lithuanian Latvian Burmese Nepali Dutch Romanian Russian Sinhala Turkish Vietnamese Chinese Afrikaans Azerbaijani Bengali Persian Hebrew Croatian Indonesian Georgian Khmer Macedonian Malayalam Mongolian Marathi Polish Pashto Portuguese Swedish Swahili Tamil Telugu Thai Tagalog Ukrainian Urdu Xhosa Galician Slovene
Code ar_AR cs_CZ de_DE en_XX es_XX et_EE fi_FI fr_XX gu_IN hi_IN it_IT ja_XX kk_KZ ko_KR lt_LT lv_LV my_MM ne_NP nl_XX ro_RO ru_RU si_LK tr_TR vi_VN zh_CN af_ZA az_AZ bn_IN fa_IR he_IL hr_HR id_ID ka_GE km_KH mk_MK ml_IN mn_MN mr_IN pl_PL ps_AF pt_XX sv_SE sw_KE ta_IN te_IN th_TH tl_XX uk_UA ur_PK xh_ZA gl_ES sl_SI

Run the following to view all available options:

auto_subtitle_llama --help

License

This script is open-source and licensed under the MIT License. For more details, check the LICENSE file.