AI SONG TRANSLATIONS?

[smoky turtle]

I saw people on youtube using some kind of AI to redub songs into their desired language with the same voice and vocals as the orignal singer

Does anyone know where I can find this AI??

AI SONG TRANSLATIONS?

[polar slate]

https://github.com/R3gm/SoniTranslate could help you

you could use rvc voice models of the original singers

haven't tested it tho

[timber canyon]

They all use eleven labs dubbing

I used it too it's free up to 5 minutes

Iirc

You can also use it with YouTube links

[smoky turtle]

I thought elevenslabs sounded awful when translating songs and vocals

[smoky turtle]

it also said that they gave the ai the lyrics in the language they wanted

[smoky turtle]

I'm gonna try this, thank you!

[polar slate]

you're welcome, to make it use the same voice as the singer, you'd need an rvc model of that singer tho

[smoky turtle]

@polar slate I tried it and it doesnt mimic the vocals the ai just reads the text

[smoky turtle]

I wonder what AI they used

[polar slate]

Ah shit😭

Yea technically

Its speech recognition that then goes to TTS

I remember there was an rvc tab where you could use models to make them look similar

But tts is prolly not good for singing

[smoky turtle]

Definitely not for singing

[gentle needle]

That's right, it's not designed for that purpose

Diffsinger might be what you're looking for

[cerulean shale]

Example?

[smoky turtle]

Here's an example

This song is orginally Italian

https://youtu.be/Wth_i4fbl_Y?si=p4jrjDEelSRVcZg5

And someone managed to make a perfect french version

[smoky turtle]

https://youtu.be/W1kvCb7lzI4?si=O9T_EGzhpnAOQZQw

[cerulean shale]

Looking at the description the creator credits someone that does fandubs and covers of that show. A possiblity is that they got that person to first dub the song to french using their voice and then convert it using an rvc model trained on the original song to match the original song's voice.

[smoky turtle]

That's what I thought at first too, but those people only gave her the lyrics and the AI did the rest

[smoky turtle]

What's diffsinger

[rain cobalt]

its my dream same version

https://www.youtube.com/watch?v=Q_Za9d7y-ck

https://www.youtube.com/watch?v=vzJVevUrVks

https://www.youtube.com/watch?v=b-ADe-Wa4Eg&t=136s

https://www.youtube.com/watch?v=GIFmILIVwg0

https://www.youtube.com/watch?v=TYlHd6VDPZk

https://www.youtube.com/watch?v=0Jif1vp95r0

https://www.youtube.com/watch?v=g6sCOoBdFUM

https://www.youtube.com/watch?v=m-LqU9DkyB8&t=137s