ElevenLabs releases Dubbing v2, a sync-aware model that translates audio while preserving speaker emotion, accents, and timing
Demos show the system translating MrBeast videos into Spanish.
#1245Alex Volkov@ALTRYNE🤯 Language barriers? Where we're going we won't have any language barriers
Dubbing v2 by @ElevenLabs blew my mind so much I had to stop editing @thursdai_pod and show you (100% not paid lol)
it's insane, it understands tonality, emotion, does voice transfer & even accent!?
ElevenLabs@ElevenLabsIntroducing Dubbing v2, our revolutionary new dubbing model. For the first time, the emotion and performance of the original content is carried over into every language.
I didn't have this in the short but both me and @nisten voices were dubbed, here's me and Nisten in Hebrew, and while for natural listeners this may sound not perfectly 100%, it's still damn impressive!
If he'd speak Hebrew, he's sound 100% like this!
Alex Volkov@altryne🤯 Language barriers? Where we're going we won't have any language barriers Dubbing v2 by @ElevenLabs blew my mind so much I had to stop editing @thursdai_pod and show you (100% not paid lol) it's insane, it understands tonality, emotion, does voice transfer & even accent!?
I also am a native Russian speaker, and my wife said this doesn't sound like me, but honestly, could you even tell?
@nisten sounds native af
Alex Volkov@altryneI didn't have this in the short but both me and @nisten voices were dubbed, here's me and Nisten in Hebrew, and while for natural listeners this may sound not perfectly 100%, it's still damn impressive! If he'd speak Hebrew, he's sound 100% like this!
Notice how they translated the swear words, the natural pauses in my speaking cadence, the emotion and the intensity of when I speak softly and then loudly, this is beyond... anything I've experienced before.
Truly amazing. The lips are not synced, but that's... solved with other models, like in this example, I used the @synclabs via @fal and while expensive, it really does look like we're speaking the language!
Alex Volkov@altryneI also am a native Russian speaker, and my wife said this doesn't sound like me, but honestly, could you even tell? @nisten sounds native af