[WIP] Add integration tests for ESPnetEZ

[Masao-Someki]

What?

Adds integration tests for ESPnet EZ

• ASR1
  • basic
  • streaming
  • RNN-Transducer
  • ASR Transducer (standalone)
• ASR2
• TTS1
• GAN-TTS1
• ENH1
  • basic
  • data augment
  • with preprocess
  • dynamic mixing
• ENH-TSE1
  • basic
  • unk_nspk
  • random enrollment
  • variable nspk random enrollment
• ENH-ASR1
• SSL1 (No fine-tune)
• ST1
  • basic
  • streaming
• SPK1
• S2T1
• S2ST1
• LM1
• SLU

Why?

To ensure that ESPnetEZ supports the same tasks as espnet2.

See also

#5584

[codecov]

Codecov Report

Attention: Patch coverage is 10.71429% with 25 lines in your changes are missing coverage. Please review.

Project coverage is 35.62%. Comparing base (3858d84) to head (b5cb974).

Files	Patch %	Lines
espnetez/dataset.py	9.09%	10 Missing ⚠️
espnetez/trainer.py	9.09%	10 Missing ⚠️
espnet2/tasks/st.py	0.00%	3 Missing ⚠️
espnetez/config.py	33.33%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #5725       +/-   ##
===========================================
- Coverage   71.58%   35.62%   -35.96%     
===========================================
  Files         760      758        -2     
  Lines       69841    69825       -16     
===========================================
- Hits        49996    24878    -25118     
- Misses      19845    44947    +25102     

Flag	Coverage Δ
test_integration_espnet1	62.92% <ø> (ø)
test_integration_espnetez	?
test_python_espnet1	18.20% <0.00%> (-0.01%)	⬇️
test_python_espnet2	?
test_python_espnetez	13.81% <10.71%> (-0.14%)	⬇️
test_utils	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Masao-Someki]

Hi, when I try to run the integration test for the gan-tts task, I encounter the following error. I'm using the egs2/mini_an4/tts/conf/train_vits_debug.yaml to test the training.
Does anyone know how to avoid the following error?

RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed).
Saved intermediate values of the graph are freed when you call .backward() or autograd.grad().
Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

[sw005320]

@kan-bayashi, can you help the issue?

[Masao-Someki]

I also encountered the following error when I tried to run the ENF-TSE task.

Traceback (most recent call last):
  File "../../../test/espnetez/test_integration_espnetez.py", line 361, in <module>
    trainer.train()
  File "/hdd/doc/espnet/espnetez/trainer.py", line 170, in train
    self.task_class.main(self.train_config)
  File "/hdd/doc/espnet/espnet2/tasks/abs_task.py", line 1154, in main
    cls.main_worker(args)
  File "/hdd/doc/espnet/espnet2/tasks/abs_task.py", line 1471, in main_worker
    cls.trainer.run(
  File "/hdd/doc/espnet/espnet2/train/trainer.py", line 317, in run
    all_steps_are_invalid = cls.train_one_epoch(
  File "/hdd/doc/espnet/espnet2/train/trainer.py", line 557, in train_one_epoch
    for iiter, (utt_id, batch) in enumerate(
  File "/hdd/doc/espnet/espnet2/train/reporter.py", line 268, in measure_iter_time
    retval = next(iterator)
  File "/hdd/doc/espnet/espnet2/iterators/chunk_iter_factory.py", line 156, in build_iter
    raise RuntimeError(
RuntimeError: All sequences must has same length: 16000 != 5000

I noticed that changing the training configuration works for this test, but I think this is not the correct way to fix this because there would be no issue if we ran the integration test from run.sh.
@sw005320 would you tell me who could be the proper person to ask about this problem?

# before
chunk_excluded_key_prefixes:
  - "enroll_ref"

# after
chunk_excluded_key_prefixes:
  - "enroll_ref"
  - "speech_ref" # add this
  - "speech_mix" # add this