Soongsil University, Chung-Ang University team develops technology to disrupt lip-sync deepfakes
Summary
A joint research team from Soongsil University and Chung-Ang University developed MUTE, a technology that uses audio protection to disrupt lip synchronization in deepfake videos. The team said it is the first technology to do so using audio protection alone. MUTE adds subtle signals to audio before it is uploaded, making generated videos show noticeable mismatches between speech and lip movements. The research was accepted for presentation at NeurIPS 2026.
Key Facts
- The team reported that MUTE reduced lip-sync accuracy by an average of 47.5 percent across five AI video-generation models.
- The protective effect persisted after noise removal and other audio enhancement processes.
- The research was supported by a research and development project of the Personal Information Protection Commission and the Korea Internet & Security Agency.
- The paper is titled MUTE: Multi-Level Alignment Uncoupling Against Talking-Head Exploitation for Voice Protection.
- The team said its next step is to develop technology that protects both audio and images.
Paper set for presentation at NeurIPS 2026 in Sydney in December
From left in the top row are professor Choi Dae-sun of Soongsil University’s AI Security Center, doctoral student Kim Dong-hyun and master’s student Kim Seung-min. From left in the bottom row are undergraduate student Kim Da-in, professor Kwon Jun-seok of Chung-Ang University and doctoral student Hong Jin. Courtesy of Soongsil University and Chung-Ang University
A joint research team from Soongsil University and Chung-Ang University has developed a technology to disrupt lip synchronization in deepfake videos that use artificial intelligence (AI) to make a person's lip movements match an audio track.
Soongsil University said Tuesday that the team, led by professor Choi Dae-sun of its AI Security Center and professor Kwon Jun-seok of Chung-Ang University, developed the technology, dubbed MUTE.
According to the team, MUTE is the first technology to disrupt lip synchronization in deepfake videos using audio protection alone.
The paper, titled “MUTE: Multi-Level Alignment Uncoupling Against Talking-Head Exploitation for Voice Protection,” was accepted for presentation at the Conference on Neural Information Processing Systems (NeurIPS) 2026, one of the world’s leading AI conferences.
The study was conducted as part of a research and development project supported by the Personal Information Protection Commission and the Korea Internet & Security Agency (KISA).
The research team included doctoral student Kim Dong-hyun, master’s student Kim Seung-min and undergraduate student Kim Da-in from Soongsil University, as well as doctoral student Hong Jin from Chung-Ang University.
Recent advances in AI technology have made it possible to create sophisticated fake videos that make people appear to speak naturally based on just a single photograph and a short audio clip.
Because efforts to detect and remove deepfakes often begin only after harm has occurred, technologies that protect personal information before it is misused are becoming increasingly important.
A schematic diagram shows how a technology developed by a joint research team from Soongsil University and Chung-Ang University disrupts lip synchronization in deepfake videos. Courtesy of Soongsil University and Chung-Ang University
MUTE adds subtle protective signals to audio before it is uploaded online. These signals are virtually imperceptible to human listeners, preserving the original voice’s natural sound while disrupting AI systems’ ability to generate lip-synced deepfake videos.
Deepfake videos generated using MUTE-protected audio exhibit noticeable mismatches between speech and lip movements, making it easier for viewers to recognize that a video may have been manipulated.
The research team found that MUTE reduced lip-sync accuracy by an average of 47.5 percent across five AI video-generation models. The protective effect persisted even after noise removal and other audio enhancement processes.
“As AI technology advances, deepfake misuse is evolving beyond the manipulation of photos and voices to the creation of sophisticated fake videos that make it appear as though real people are speaking,” professor Choi said.
He added, “To counter this threat, we need technologies that protect personal information, such as facial images and voice recordings, before they are made public.”
“This study confirmed that audio protection alone can significantly disrupt lip synchronization in deepfake videos. Adding image protection could disrupt synchronization further, making such videos more difficult to misuse,” said doctoral student Kim Dong-hyun.
“As the next step, we will develop technology to protect both audio and images, helping counter deepfake threats targeting individuals’ faces and voices,” he added.
Explore More
- Q.
- Q.
- Q.