Hot take: AI lip-sync might be what finally ends the sub vs dub war

Reddit r/ArtificialInteligence Tools

Summary

AI lip-sync tools like sync.so can redraw mouth movements to match dubbed audio, potentially neutralizing the long-standing argument that dubs break immersion due to mismatched mouth movements.

The sub vs dub debate always had one mechanical problem nobody disputed. dubs don't match mouths. even peak funimation only got so close because the animators drew the mouth for japanese phonemes. nothing the voice actor does fixes that. AI lip-sync tools redraw the mouth frame by frame to match the dub audio. [sync.so](http://sync.so) has demos of this on real footage and it looks shockingly natural. tried it on a clip myself last week, its costly af, applied to an anime scene, the mouths actually fit the english. This doesn't fix bad voice direction or sloppy localization scripts. those are still a dub problem. but it kills the strongest argument the sub side ever had. the whole 'dubs ruin immersion because mouths don't match' line stops working.
Original Article

Similar Articles

ElevenLabs Dubbing v2

Reddit r/singularity

ElevenLabs launched Dubbing v2, an AI dubbing model that preserves the original speaker's emotion, tone, and performance across 90+ languages by conditioning on the original audio directly, offering broadcast-quality dubbing at a fraction of the cost.

How Descript engineers multilingual video dubbing at scale

OpenAI Blog

Descript redesigned its translation pipeline using OpenAI reasoning models to optimize multilingual video dubbing at scale, achieving 15% increase in translated video exports and 13-43% improvement in duration adherence across languages by addressing the challenge of matching speech duration to video timing constraints.

Voiser AI

Product Hunt

Voiser AI offers human-like AI voiceovers in over 140 languages.

At what point do we stop calling ai generated video slop

Reddit r/artificial

The article argues that AI-generated video has surpassed the 'slop' threshold, with major studios and streaming services adopting it for cost savings and commercial viability, forcing a redefinition of quality in content creation.

Vaani

Product Hunt

Vaani is a lip-synced AI dubbing tool for creators, brands, and studios.