Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

This seems amazing, but only english.... #5

Open
juangea opened this issue Sep 27, 2024 · 2 comments
Open

This seems amazing, but only english.... #5

juangea opened this issue Sep 27, 2024 · 2 comments

Comments

@juangea
Copy link

juangea commented Sep 27, 2024

Hi there.

StyleTTS and this seems amazing, but for some of us I think it's very hard or nearly impossible to train a new language, could be great to have some more languages or maybe I'm wrong and it's not that hard to train a new language.

I have one 4090, I never did a training on a new language, I could be happy to help and share a model, but I cannot have the 4090 working 7 days a week during several weeks because I use it for work.

Are there plans to add / train more languages?

@dkounadis
Copy link
Member

Hallo,

The SHIFT TTS is based on a phenomenon that if one feeds 4x speed style (speaker/reference) to StyleTTS2 the latter becomes very emotional.

The figures below show how much the emotion probability increases by using 4x speed than 1x speed, i.e. natural speech styles.

fig_english_WIN=40_HOP=10_HFdisc

fig_foreign_WIN=40_HOP=10_HFdisc

In this repo we have pre-generated 134 interesting styles for StyleTTS2.

You can use those styles also in a non-English StyleTTS2 - e.g. this issue

  • I don't know if we will train a non-english checkpoint

@juangea
Copy link
Author

juangea commented Sep 30, 2024

Thanks @dkounadis .

I really hope you can train a multi-language model, it will be the most useful TTS out there, so far the best solution to get something emotional is Bark, and it’s not very much controlable and it’s a bit random.

No other open source can be compared regarding multilingual, and this could be awesome. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants