Lipsmatch AI lip sync
Updated
What is AI lip sync, and do you need it?
What the technology changes in a video, what it keeps, and how to tell if your dub needs it.
The short answer
AI lip sync changes a speaker's mouth to match new audio, most often a dub in another language. It shapes the lips, teeth and jaw to each sound, frame by frame. The eyes, the expression and the edit stay as filmed. You need it when viewers see a face speak. Lipsmatch is AI lip sync for films, ads and talks. The speaker's own voice, recreated by our technology, speaks the new language, and the mouth matches every word.
One scene, in English and in Hindi
This is a Lipsmatch technology demonstration on a film we do not own. The scene is from Good Will Hunting. We made it only to show the technology. It has no link to the film's makers or cast. Tap Original to hear it as released in English. Tap Lipsmatch to see the same moment in Hindi. The lips form every Hindi word. The eyes, the hands and the camera move exactly as before. Keep the sound on.
AI lip sync, in plain words
AI lip sync is technology that changes a filmed mouth so it speaks new words.
A plain dub replaces the voice and leaves the picture as it was. So the lips keep moving to the old language. Viewers hear one language and see another. AI lip sync changes the picture to fit the new voice.
The software finds each face in every frame and follows the mouth through the shot. For each sound in the new audio, it builds the mouth shape that sound needs. On m, b and p, for example, the lips close. Then it blends the new mouth into the original frame.
People also say lip sync dubbing, visual dubbing or AI visual dubbing. All 3 mean a dub with the mouth changed to match. If the work is good, viewers forget that the video was ever in another language.
On social apps, lip sync also means miming to a song. This guide is about the dubbing meaning.
What AI lip sync changes, and what it keeps
The change stays in a small area around the mouth. Every other part of the frame comes straight from your original file.
| Part of the video | With AI lip sync |
|---|---|
| Lips, teeth and jaw | Shaped again for each sound of the new audio |
| Chin and lower cheeks | Move with the new jaw where a sound needs it |
| Eyes, brows and expression | Kept as filmed |
| Head moves, hands and gestures | Kept as filmed |
| Light, colour and grain | Kept, and copied onto the new mouth |
| People who only listen | Kept as filmed |
| Cuts, shot length and pace | Kept, so the video runs the same length |
| The voice | Comes from the new dub |
Swipe the table sideways to see every column.
Do you need lip sync? Look at the faces
The bigger the speaking face on screen, the more lip sync matters. Here is a first guide by kind of video. If no face speaks, our comparison of subtitles, voice-over and dubbing shows simpler methods.
| Kind of video | Lip sync | Why |
|---|---|---|
| Ad film with close-ups | Yes | Viewers judge the face, so the ad must look made in their language |
| Film or series scene | Yes | The acting shows in the faces, often in close-up |
| Creator or teacher talking to camera | Yes | The face fills the frame for most of the video |
| Recorded talk with close shots of the speaker | Yes | Viewers watch the speaker's face for long stretches |
| Interview or video podcast | Often | It helps most on the close shots of each speaker |
| Lecture with slides and a small face | Rarely | Viewers watch the slides more than the mouth |
| Event filmed only in wide shots | Rarely | Faces are small, so a good dub or subtitles can be enough |
| Screen recording with a voice-over | No | No mouth is on screen, so a voice track or subtitles are enough |
Swipe the table sideways to see every column.
The shots that make AI lip sync hard
Some shots give the software less to work with. Weak lip sync shows first in these frames, so they get the closest checks.
- Side angles. As the face turns, less of the mouth shows. The new mouth has to follow the line of the jaw and the cheek. A face turned fully away needs no change at all.
- Objects in front of the mouth. A hand, a microphone, a cup or a phone can pass across the lips. The new mouth sits behind the object, and the object's edge stays clean in every frame.
- Several faces. Each speaker gets a separate track through the shot, and the listeners keep their own mouths. When 2 people talk over each other, each mouth follows its own voice.
- Low light. A dark scene shows less detail around the lips and more noise. The new mouth needs the same shadow and grain to blend in.
- Fast camera moves. A quick pan or a sudden turn of the head blurs the mouth. The new mouth needs the same blur. The tracking must not break during the move.
- Beards, teeth and big close-ups. When a face fills the screen, stubble, a moustache and the edges of the teeth show every small error. These frames get the most care.
Why timing still matters with lip sync
AI lip sync changes the shape of the mouth. Every shot keeps its length. So each new line must fit in the seconds that the old line used.
Dubbing research makes the same point. A 2022 paper on automatic dubbing says a good dub keeps the speech and pause pattern of the original. Each new stretch of speech must take about as long as the old one. A line that runs long goes past a nod, a breath or a cut.
This is hard, because languages say things at different speeds. In one study, people read the same texts aloud in 8 languages. English gave the most information per second. The same message took as long or longer in each of the other 7. Indian languages run long too. Researchers at IIT Madras, IIIT Hyderabad and IIT Bombay dubbed English lectures into Indian languages. The new audio took 1.2 to 1.5 times the English running time.
With AI lip sync, the mouth follows the new words. So the translator no longer has to find words that fit the actor's old lip shapes. Professional dubbers already rank that task low. Across 54 dubbed titles, researchers found that natural delivery and a faithful meaning usually came first.
The time limit stays. A good translation keeps the full meaning in natural words. Each line fits its time, so the speaker keeps a calm, natural pace.
How a dub becomes a lip-matched video
These are our steps from a dub to a finished video, after the new audio is ready. Our technology page shows each stage in more detail.
Step by step
Time the dub to the edit
We time each new line to the cut. It starts and ends with the breath and the gesture of the speaker.
Track every speaking face
The software follows each person who speaks, shot by shot. Faces that only listen stay untouched.
Shape the new mouth
Frame by frame, the lips, teeth and jaw take the shape of each sound in the new audio.
Finish the fine detail
Our team works shot by shot on teeth edges, beard lines, skin texture and motion blur.
Check every close-up
Our team checks each close-up at full size before you see it. Nothing goes out until you approve it.
How to judge AI lip sync quality
Test any lip sync on your own footage, ours included. Play it full screen with the sound on, and look for these points.
- Closed lips. On m, b and p, the lips must meet. A mouth that stays open on these sounds is the easiest error to spot.
- Steady teeth. Teeth must stay sharp and in place. Soft or flickering teeth are a sign of a weak result.
- Clean edges. Look at the chin, the corners of the mouth and the beard line. Look for a soft patch or a border around the changed area. A good result has none.
- The hard moments. Pause where a hand passes, the head turns or the camera moves. The mouth must stay sharp and in place through each one.
- The same person. The speaker must still look like themselves, with their own smile and their own way of speaking.
- Voice and mouth together. A matched mouth with a flat voice still feels wrong. Judge the sound and the picture as one.
Where Lipsmatch comes in
Lipsmatch is AI lip sync as a finished service, for films, ads and talks. Our team works through every shot and checks each version before it reaches you.
If you have a finished dub in any language, from any studio or tool, send it. We change the speaker's mouth to match each word. If you need the dub as well, we translate your own script and check every line. Our technology then recreates the speaker's own voice in the new language, with the same delivery as the original take. Last, we match the mouth to it.
We spend the most time on side angles, hands near the mouth and big close-ups. Our lip sync work aired in 12 languages, across 75+ broadcast masters. On client work, we change a mouth only with the approvals you already have for the face and voice on screen.
If you publish often, a monthly plan sets the hours and the languages at the start. The work then comes in regular batches. Agencies and post houses can offer the work under their own name through our partner terms.
AI lip sync questions
What is Lipsmatch?
Lipsmatch is AI lip sync for films, ads and talks. The speaker's own voice, recreated by our technology, speaks the new language, and the mouth matches every word. You can bring your own dub, or we make it. Our team checks every version before you get it.
Is AI lip sync the same as a deepfake?
The aim is different. A deepfake puts words in a person's mouth. A lip sync dub carries the speaker's own message into another language. On client work, we do it only with the approvals you already have. The law can still count a lifelike dub as a deepfake. In the EU, from 2 August 2026, you must disclose it when it is shown there, and a film needs only a light disclosure. Rules for AI dubbing explains what to label.
Is AI lip sync the same as voice cloning?
No. They do 2 separate jobs. Recreating a voice makes a person sound like themselves in a new language. AI lip sync changes the picture, so their mouth forms the new words. A full video translation often uses both. With Lipsmatch, you can bring your own dub, and we change the mouth. Or we recreate the voice and then match the mouth.
Can you add lip sync to our existing dub?
Yes. Send the original video and the dubbed audio for each language, made by any studio or any tool. We change the speaker's mouth to fit your dub, frame by frame. You get one new video file for each language, with the same cuts as your original.
Does YouTube auto dubbing do lip sync?
YouTube is testing lip sync on its own automatic dubs. It is an experimental feature for some channels and languages only. It works only with YouTube's synthetic voice, and you cannot edit that dub. Lipsmatch matches the mouth to the speaker's own recreated voice or to a dub you bring. Our team checks every version. Each Lipsmatch language is a finished video. You can post it as Shorts, in a playlist or on a channel for that language.
Does AI lip sync work on old or low quality video?
Often, yes. The software needs to see the mouth clearly. Low resolution, heavy compression and soft focus give it less detail. Then the new mouth can look softer than the face around it. Send a short clip first. We tell you what to expect before you send the rest.
Which languages work with AI lip sync?
Lip sync follows the sounds of speech, so the method works across languages. We work in 90+ languages and accents. Our lip sync aired in 12 of them: Hindi, English, Tamil, Telugu, Kannada, Malayalam, Bengali, Marathi, Punjabi, Gujarati, Assamese and Odia. Each of the 12 has its own language page.
Do viewers notice when the lips do not match?
Yes, when they watch the face. A small 2025 study of dubbed lectures had 18 students. Students and teachers in it called lips that did not match the sound annoying. One student said that the lip-synced version felt natural. The quality ratings in the study showed no clear overall gain from lip sync. So lip sync helps most in close-ups, where viewers look at the mouth.
Explore related
Your speaker, in a new language
Send one scene where a face speaks. We send it back in your language, with the mouth matched, at no cost and with no commitment.