AI Lip Sync for Filmmakers: How-To and Best Tools (2026)

AI lip sync makes a character’s mouth match a voice track, either by repainting the mouth on footage you already have or by generating a new talking shot from a still image and audio. As of October 17, 2026, the tool you should use depends on what you start with: an existing take, a single still, or a shot you haven’t made yet. Dedicated lip-sync models are the right choice when the audio already exists, such as a recorded actor, a dub or a song. Video models that generate dialogue in the same pass are often easier when the scene doesn’t exist yet.

Quick answer: the best AI lip sync tools right now

• Best for re-syncing existing footage: sync. labs’ sync-3 (up to 4K, handles profiles and hands in front of the mouth, about $0.11 to $0.13 per second).
• Best for making a photo talk: HeyGen Avatar IV or Hedra Character 3 (one image plus your audio).
• Best for carrying an actor’s real performance: Runway Act-Two (you film the line, it moves the expression onto your character).
• Best free, local option: LatentSync 1.6 for mouth-only dubbing, or InfiniteTalk when the head and body should move with the speech too.
• Best when the shot doesn’t exist yet: a video model that takes your voice file as a reference, such as MiniMax H3 (where its license applies).

Prices and specs below are dated and come from each developer’s own pages. Disclosure: we make CinemaDrop, a hosted AI filmmaking studio that includes a lip-sync step in its storyboards. We describe it in one section near the end and don’t rank it against the tools here.

How AI lip sync works: four kinds of tools

Most “AI lip sync” tools do one of four different jobs. Picking the wrong kind is the most common reason a result looks off.

1. Mouth replacement (video dubbing)

You give the model a video and a new audio file. It finds the face, reads the speech (LatentSync uses Whisper audio embeddings, for example) and repaints only the mouth region frame by frame. Everything else in the take stays: the camera move, the blocking, the eyes. This is the right tool for fixing a line, dubbing a scene into another language or syncing a performer to a song.

2. Audio-driven talking video (photo to video)

You give the model one image and audio, and it generates a new video where the face, head and sometimes hands move with the speech. HeyGen Avatar IV, Hedra Character 3, InfiniteTalk and Kandinsky S2V work this way. It’s the answer to “make a photo talk,” and it’s how most AI characters get their dialogue scenes.

3. Native dialogue in a video model

Many 2026 video models generate speech, lip movement and sound in one pass from a prompt. Some also accept a reference audio file. In Fuser’s September 2026 test, MiniMax H3 used the supplied recording as the dialogue track, while Seedance 2.5 kept the words and rhythm but delivered a new performance of its own. If you need your exact recording, check which behavior a model has before you build a scene around it.

4. Performance transfer

Instead of audio, you record yourself or an actor delivering the line, and the tool moves that facial performance onto a character. Runway Act-Two takes a 3- to 30-second driving video. You get a human delivery rather than a model’s guess, which matters for comedy timing and emotional beats.

Best AI lip sync tools compared (October 2026)

Each row lists what the tool takes in, its limits and its published price, checked October 17, 2026. Prices change often, so confirm on the developer’s page before you budget a project.

Sources: sync. labs model docs and pricing page; HeyGen’s credit guide and pricing page; Hedra’s Character 3 model page; Runway’s Act-Two help article; the GitHub READMEs for each open model, all checked October 17, 2026.

A quick way to choose: if the take already exists and only the words change, use mouth replacement. If all you have is a still, use a photo-to-video model. If you can film the line yourself, performance transfer gives you the most control over acting.

Free and open-source lip sync models

If you have a decent GPU, open models remove the per-second bill and keep your footage on your own machine. They need a local setup, usually Python or ComfyUI, and each license is different.

• LatentSync 1.6 (ByteDance, released June 11, 2025). An audio-conditioned latent diffusion model trained at 512×512 to fix the blurry mouths of earlier versions. Inference needs at least 18 GB of VRAM; the older 1.5 runs in 8 GB. Apache 2.0. See the LatentSync repository.
• MuseTalk 1.5 (Tencent Music’s Lyra Lab, March 2025). Built for speed: 30+ fps on an NVIDIA V100, editing a 256×256 face region. The code is MIT and the README says the weights can be used for any purpose, including commercially. Its sample test data is research-only.
• InfiniteTalk (MeiGen-AI, August 2025). It regenerates the whole frame, so the head, shoulders and expression follow the new audio instead of only the lips, and it has no length cap. It runs on top of Wan 2.1’s 14B image-to-video model at 480p or 720p, with a low-VRAM mode. Apache 2.0. See the InfiniteTalk repository. Its team released a follow-up framework, LongCat-Video-Avatar 1.5, on May 21, 2026.
• Kandinsky S2V 1.0 (Kandinsky Lab, paper dated October 15, 2026). The newest entry: a 2B Lite and a 21B Pro model that turn a reference image, a speech track and a text prompt into long talking video at 25 fps, plus distilled versions that run 5.4 times faster. The project page says the code and checkpoints are released under MIT. We couldn’t confirm the full weights on Hugging Face on October 17, so check the model page before you plan around it.

Two open video models also generate dialogue with lip sync as part of a clip: Kandinsky 6.0 Video (MIT, 5-second clips with 44 kHz audio, released October 6) and MiniMax H3 (license excludes the US, EU, UK and South Korea). Our guide to open-source AI video generators covers their GPU needs and licenses in detail.

How to lip sync an AI character, step by step

This workflow works with any of the tools above. The order matters more than the tool.

1. Lock the line first. Record the actor, or generate the voice with a text-to-speech model, before you make the shot. Lip sync follows audio, so changing the words later means redoing the sync.
2. Clean the audio. Give the lip-sync model dialogue only, with no music, room tone or effects. Background sound muddies the mouth shapes. Add music and effects after the sync is done.
3. Pad the end. Add half a second or more of silence after the last word. In Fuser’s test, LatentSync’s first output came back shorter than the input and cut the last word; padding fixed it. Most tools trim to whichever of the audio or video is shorter.
4. Make (or pick) a shot built for sync. A medium close-up with the face toward the camera, the mouth visible and steady light. See the next section for framing.
5. Choose the tool by your starting point. Existing take: mouth replacement. Still image: photo to talking video. Actor on camera: performance transfer.
6. Run a short test first. Sync two or three seconds that include a pause and a hard consonant (b, m or p). If the lips close on those and rest during the pause, run the full line.
7. Review at 100% and in motion. Check teeth, the inside of the mouth and the jawline on a big screen. Small errors that vanish on a phone show up in a festival screening.
8. Mix in your editor. Put the synced clip on the timeline, add room tone, music and effects, and match levels across shots. For effects you don’t have, an AI sound effects generator can create them from a text prompt.

Shots that lip sync well

Lip-sync models are only as good as the face they’re given. When you generate the image or clip that will speak, plan it like a dialogue setup:

• Framing: a medium close-up or close-up. In a wide shot the mouth is a few pixels tall and any model will struggle. Our guide to camera shot types explains each size.
• Angle: frontal to three-quarter. sync-3 is built to handle profiles, but most other models do best when both corners of the mouth are visible.
• Nothing over the mouth: no hands, mics, scarves or hair across the lips unless your tool says it handles obstructions.
• A neutral, slightly open mouth in the source frame gives the model room to move. A wide grin or tight lips in the still tends to stay baked in.
• The same character every time. If the face drifts between shots, the sync will look right while the character looks wrong. See our guide to consistent AI characters.

Here’s a template for the shot that will speak. Fill in the brackets and keep the mouth area clear:

Medium close-up of [character], facing the camera at a slight three-quarter angle,
mouth relaxed and slightly open, nothing covering the lips.
Soft key light from [side], steady exposure on the face.
[Location] softly out of focus behind them.
Camera locked off, no fast movement. [Style or lens notes].

AI lip sync for dubbing and translation

AI dubbing translates the dialogue into another language, voices it, then lip syncs the speaker to the new audio. Done well, the finished lip sync video doesn’t look like a badly dubbed film. For filmmakers this means one cut can travel to more festivals and audiences.

• All-in-one: HeyGen’s video translation with lip sync costs 6 credits a minute in Speed mode or 10 in Precision mode. sync. labs offers dubbing too, billed separately from the lip-sync step.
• Do it yourself: translate and record or generate the new line, then run it through a mouth-replacement model. This gives you control over casting and delivery.
• Watch the timing: translated lines are often longer. Trim the translation or allow small edits in the cut, because lip sync can’t stretch a 3-second shot to fit 5 seconds of speech.

Common AI lip sync problems and how to fix them

Consent, likeness and disclosure

Lip sync can put words in a real person’s mouth, so treat it as a rights question, not just a technical one. Only sync real people who have agreed to it in writing, including the voice you use. Read your tool’s terms: several block real-person likenesses (Seedance 2.5 refused a photoreal face in Fuser’s test). When a dubbed or AI-voiced performance goes out publicly, label it, and keep your source files and permissions with the project.

Lip sync inside a storyboard

When your film has many dialogue shots, keeping each line, voice and synced clip attached to its shot saves a lot of file juggling. That’s how CinemaDrop handles it. Each storyboard shot has a Speech panel for ElevenLabs text-to-speech (50+ voices with bracket emotion tags, available on the free Lite plan) and a Lip Sync tab that turns the shot’s image or video plus audio into a talking clip. The Lip Sync tab lists five models: VEED Fabric 1.0, Wan 2.6, Infinite Talk, OmniHuman 1.5 and Kling Avatar v2. Lip sync and video start on the Basic plan ($8 a month), and the free Lite plan’s 50 one-time credits cover scripts, images, storyboards and voices. When the scene is ready, export the storyboard’s assets as a ZIP and finish the cut and mix in your editor. CinemaDrop doesn’t offer custom voice cloning, so for an actor’s own voice, record it and bring the file in. If you’re making a performance piece, our AI music video guide walks through syncing a song from start to finish.

AI lip sync FAQ

Is there a free AI lip sync tool?

Yes. LatentSync, MuseTalk and InfiniteTalk are free and open source if you can run them on your own GPU. Among hosted tools, HeyGen’s free plan includes three videos a month up to one minute, and sync. labs gives free accounts one sync-3 generation a month up to 15 seconds (both as of October 17, 2026).

What is the most realistic AI lip sync?

For re-syncing existing footage, sync-3 is the current high end: it reads the whole shot, handles profiles and obstructions, and outputs up to 4K. For talking video from a still, HeyGen Avatar IV and Hedra Character 3 are the common choices. Realism still depends most on clean audio and a clear, well-lit face.

Can AI lip sync work with multiple characters?

Some tools can. sync-3 detects who is speaking when several people are in frame. For other models, the reliable way is to sync one speaker per shot, or crop to each speaker and composite, then cut between them as you would in a normal dialogue scene.

Can I lip sync a cartoon or animated character?

Yes. Hedra’s own Character 3 examples include talking animals and claymation characters, and InfiniteTalk’s team says its LongCat-Video-Avatar 1.5 follow-up handles anime and animals. Mouth-replacement models work best on faces with human-like mouths.

Do I still need a lip-sync tool if my video model makes dialogue?

Not always. Many current video models generate speech with matching lips in one pass. You still need a dedicated tool when the audio already exists, such as a recorded actor, a translation or a song, or when you want to change a line without regenerating the whole shot.

Is AI lip sync legal to use?

Using it on your own characters, or on people who have given consent, is generally fine. Syncing a real person without permission can break platform rules and likeness or publicity laws, depending on where you are. Get written consent and label the result.

The bottom line

Choose your AI lip sync tool by what you already have. Use mouth replacement (sync-3, lipsync-2-pro, LatentSync) for footage that exists, photo-to-video (HeyGen, Hedra, InfiniteTalk) for a still, and performance transfer (Act-Two) when the acting matters most. Lock and clean the audio first, frame for the mouth, test the pauses and plosives, and mix last. For the bigger workflow around dialogue scenes, see our hybrid AI filmmaking workflow guide. We’ll update this comparison as new models ship.