EmoDubber: Towards High Quality and Emotion Controllable Movie Dubbing

Environment

1. Python >= 3.10
2. Clone this repository:

git clone https://github.com/GalaxyCong/DubFlow.git
cd DubFlow

3. Install python requirements:

pip install -r requirements.txt

Dataset

• Chem dataset
• GRID dataset
• V2C-Animation dataset

Training (Flow Matching)

First training:

python train_flow.py --config_path ./Configs/config.yml

Inference (Flow-based User Emotion Controlling)

python Inference_Wav_Setting1.py --config_path ./Configs/config.yml

python Inference_Wav_Setting2.py --config_path ./Configs/config.yml

python Inference_Wav_Setting3.py --config_path ./Configs/config.yml

Our Checkpoints

Baidu Drive, Google Drive

License

Code: MIT License

Citing

If you find our work useful, please consider citing:

@article{cong2024emodubber,
  title={EmoDubber: Towards High Quality and Emotion Controllable Movie Dubbing},
  author={Cong, Gaoxiang and Pan, Jiadong and Li, Liang and Qi, Yuankai and Peng, Yuxin and Hengel, Anton van den and Yang, Jian and Huang, Qingming},
  journal={arXiv preprint arXiv:2412.08988},
  year={2024}
}