(SOLVED) How to Use Post Processing Tool: Translation?

[quantuan125]

If I want to translate the transcribed subtitles from another language to English. How would I do that?

Thank you in advance

[abdeladim-s]

Hi @quantuan125,

• If the target language is just English, then you can use whisper directly, from the model configs set the language to the language of the media file, and set task to translate and then transcribe.
• Otherwise, if you want to use the post-processing tool, transcribe your file first, then just select the model you want to use and the source and target languages and then translate.

Let me know if you find any other issues.

[quantuan125]

By the way,

I'm running into this CUDA issue when using post-processing translation tool. Do you know what might causes this isuee?

The video was first transcribed by the WhisperX models_large v2(which is super impressive) and I was using CUDA 11.8 to perform this process. However the same can not be done when I select CUDA:0 on the "device" of the post-processing tool tab. Any suggestions? Should I run it with the CPU instead?

EDIT: I can't run with CPU either. Any modification of configuration on post-processing tool results in the same error.

FULL ERROR:
RuntimeError: CUDA error: an illegal instruction was encountered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
Traceback:
File "C:\Users\quant\AppData\Local\Programs\Python\Python311\Lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 552, in _run_script
exec(code, module.dict)
File "C:\Users\quant\AppData\Local\Programs\Python\Python311\Lib\site-packages\subsai\webui.py", line 523, in
run()
File "C:\Users\quant\AppData\Local\Programs\Python\Python311\Lib\site-packages\subsai\webui.py", line 516, in run
webui()
File "C:\Users\quant\AppData\Local\Programs\Python\Python311\Lib\site-packages\subsai\webui.py", line 389, in webui
translated_subs = tools.translate(subs=subs,
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\quant\AppData\Local\Programs\Python\Python311\Lib\site-packages\subsai\main.py", line 190, in translate
translated_sub.text = translation_model.translate(text=sub.text,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\quant\AppData\Local\Programs\Python\Python311\Lib\site-packages\dl_translate_translation_model.py", line 194, in translate
encoded.to(self.device)
File "C:\Users\quant\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\tokenization_utils_base.py", line 772, in to
self.data = {k: v.to(device=device) for k, v in self.data.items()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\quant\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\tokenization_utils_base.py", line 772, in
self.data = {k: v.to(device=device) for k, v in self.data.items()}
^^^^^^^^^^^^^^^^^^^

[abdeladim-s]

Hi @quantuan125,

• use WhisperX just if you want word level timestamps, otherwise use FasterWhisper (which is being used in whisperX under the hood) or if you have a good GPU you can use the original whisper implementation. That way you will have the task to translate to english. As a said if the target language is english just use the translate task, it will give better results than the post processing tool (I guess).

• Regarding the translation models, to be brief, they are different in their architecture and the dataset they trained on, and also how many languages they support and so on .. your best bet (depending on the source language) is to try all the models on a small subtitles file and see which one gave you good results. Otherwise you can just google the model name and you can see the characteristics of each model so you can compare between them.

[abdeladim-s]

• Regarding the error, I think you have a problem with your cuda installation. just leave the device as auto and see if it works.

[quantuan125]

@abdeladim-s

Thank you for the suggestion. I'm currently experimenting with all models and configurations so it will take sometimes for me to learn which is best for a certain situation. At the moment, I need both the transcription in Danish and English for the purpose of my experimentation and that's why I was curious on the post-processing translation tool. Regardless, I will look into all of your great suggestions and insights.

Regarding the error, I'm currently searching for the solution which apparently quite common when you have pytorch 2.0.0 and cuda11.8 installed (as suggested for the installation in another discussion post). This has not only been affecting the post-translation tool but also my ability to perform [linto-ai/whisper-timestamped] model as well as the "task: translation" using cuda processing.

Just out of curiosity: which cuda and pytorch version are you using and recommend?

[abdeladim-s]

@quantuan125,

• Yes just experiment with the different configurations until you find the perfect set for your situation .. and let me know if you need any other help.
• For me, I have a relatively old GPU, I am using cuda 11.7 and torch version 2.0.1+cu117. But for you, it depends on your GPU model, if it is a new one, just go with cuda 12, otherwise downgrade to cuda 11.7 if 11.8 didn't work well. I know, it is quite frustrating to find the one that works, but just be patient and experiment with different versions until you find it.

Hope this helps.