Multi GPU training speed

[kunazure]

Hello ， I am using neural_sp to train aishell-1 and another task, but I find that when i use 4 GPU to train ASR model, it needs 8 hours per epoch, which is slow than I use single GPU，the time even grow when I add more GPU. Do you have this problem or there is something wrong with my envirment?
I use your tools to install the envirment, python=3.7.9 and pytorch=10.0 ,with cuda 10.1。

[hirofumi0810]

Hi, I think it's time to use DDP in pytorch. I'm still working on it internally. I'll update code soon.