[BUG] darts should provide way to input multiprocessing_context parameter for dataloader when fit

[joshua-xia]

Describe the bug
I use Parallel(n_jobs=2, backend="loky") to boot two training job, and the dataloader workers number set to be 3

when training the folliowing error happened:

pred_values_dict_list = Parallel(n_jobs=2, backend="loky")(

File "/devtools/anaconda3/envs/quant-env/lib/python3.9/site-packages/joblib/parallel.py", line 1952, in call
return output if self.return_generator else list(output)
File "/devtools/anaconda3/envs/quant-env/lib/python3.9/site-packages/joblib/parallel.py", line 1595, in _get_outputs
yield from self._retrieve()
File "/devtools/anaconda3/envs/quant-env/lib/python3.9/site-packages/joblib/parallel.py", line 1699, in _retrieve
self._raise_error_fast()
File "/devtools/anaconda3/envs/quant-env/lib/python3.9/site-packages/joblib/parallel.py", line 1734, in _raise_error_fast
error_job.get_result(self.timeout)
File "/devtools/anaconda3/envs/quant-env/lib/python3.9/site-packages/joblib/parallel.py", line 736, in get_result
return self._return_or_raise()
File "/devtools/anaconda3/envs/quant-env/lib/python3.9/site-packages/joblib/parallel.py", line 754, in _return_or_raise
raise self._result
AttributeError: 'Process' object has no attribute 'env'

the reason is when create dataloader, we should input multiprocessing_context=get_context("loky"), but there is no place to setting the multiprocessing_context from neither model parameter or fit parameter.

so I think we should be able to set multiprocessing_context in fit function when we set num_loader_workers>0

[madtoinou]

Hi @joshua-xia,

I think that this problem will be solved by #2295, if you cannot wait for it to be merged and released, you inspire yourself from the changes to apply it to your local darts install.

[joshua-xia]

I did not see the multiprocessing_context from that issue 's patch, I strongly suggest the committer add this parameter.