ArrayFire Exception (Device out of memory:101) while decoding using Beam search decoder

[abramovi]

Bug Description

I am trying to run inference on a 30 min audio file using Beam search but it fail immediately with the following error:
terminate called after throwing an instance of 'af::exception' what(): ArrayFire Exception (Device out of memory:101): In function af_err af_matmul(void**, af_array, af_array, af_mat_prop, af_mat_prop) In file src/api/c/blas.cpp:237

Reproduction Steps

[Steps to reproduce. Please include, if needed for your issue:
This is the command I run:
!../../build/bin/asr/fl_asr_tutorial_inference_ctc \ --am_path=am_transformer_ctc_stride3_letters_300Mparams.bin \ --tokens_path=tokens.txt \ --lexicon_path=lexicon.txt \ --lm_path=lm_common_crawl_small_4gram_prun0-6-15_200kvocab.bin \ --logtostderr=true \ --sample_rate=16000 \ --beam_size=50 \ --beam_size_token=30 \ --beam_threshold=100 \ --lm_weight=1.5 \ --word_score=0 \ --audio_list=audio.lst

this is the output:
`I0406 17:32:38.839179 11262 InferenceCTC.cpp:66] Gflags after parsing
--flagfile=;--fromenv=;--tryfromenv=;--undefok=;--tab_completion_columns=80;--tab_completion_word=;--help=false;--helpfull=false;--helpmatch=;--helpon=;--helppackage=false;--helpshort=false;--helpxml=false;--version=false;--am_path=am_transformer_ctc_stride3_letters_300Mparams.bin;--audio_list=audio.lst;--beam_size=50;--beam_size_token=30;--beam_threshold=100;--lexicon_path=lexicon.txt;--lm_path=lm_common_crawl_small_4gram_prun0-6-15_200kvocab.bin;--lm_weight=1.5;--sample_rate=16000;--tokens_path=tokens.txt;--word_score=0;--alsologtoemail=;--alsologtostderr=false;--colorlogtostderr=false;--drop_log_memory=true;--log_backtrace_at=;--log_dir=;--log_link=;--log_prefix=true;--logbuflevel=0;--logbufsecs=30;--logemaillevel=999;--logfile_mode=436;--logmailer=/bin/mail;--logtostderr=true;--max_log_size=1800;--minloglevel=0;--stderrthreshold=2;--stop_logging_if_full_disk=false;--symbolize_stacktrace=true;--v=0;--vmodule=;

I0406 17:32:38.839272 11262 InferenceCTC.cpp:89] [Inference tutorial for CTC] Reading acoustic model from am_transformer_ctc_stride3_letters_300Mparams.bin
I0406 17:32:40.061904 11262 InferenceCTC.cpp:140] [Inference tutorial for CTC] Network is loaded.
I0406 17:32:40.433898 11262 InferenceCTC.cpp:152] [Inference tutorial for CTC] Number of classes/tokens in the network: 29
I0406 17:32:40.433930 11262 InferenceCTC.cpp:155] [Inference tutorial for CTC] Number of words in the lexicon: 200001
I0406 17:32:40.667213 11262 InferenceCTC.cpp:166] [Inference tutorial for CTC] Language model is constructed.
I0406 17:32:41.800864 11262 InferenceCTC.cpp:177] [Inference tutorial for CTC] Trie is planted.
I0406 17:32:41.801892 11262 InferenceCTC.cpp:196] [Inference tutorial for CTC] Beam search decoder is created

terminate called after throwing an instance of 'af::exception'
what(): ArrayFire Exception (Device out of memory:101):
In function af_err af_matmul(void**, af_array, af_array, af_mat_prop, af_mat_prop)
In file src/api/c/blas.cpp:237`

Platform and Hardware

[Please list your operating system, [GPU] hardware, compiler, and other details if relevant]
I am running on CPU machine with 4 core and 8 gb ram

[Manni-Arora]

Was getting the same error. Try reducing the size of your audio.