K-pop idols targeted by AI deepfake apps

A recent wave of advertisements using artificial intelligence (AI)-generated deepfake videos of K-pop idols has stirred backlash on social media, raising fresh concerns over the growing misuse of AI to create sexually suggestive content targeting celebrities.

These advertisements for AI-powered apps recently triggered widespread concern in online communities, with users criticizing the spread of inappropriate content on short-form video platforms like TikTok and Instagram.

The ads in question promote AI tools that generate videos of two people kissing, prompting users to upload their own photo alongside an image of their favorite K-pop idol to create a fabricated clip.

Screenshots of these ads circulated online, with K-pop fans calling them disturbing and invasive. "Even after reporting it, the ad kept coming back ... I've seen even worse ones," one online community user wrote.

Under Korean law, the non-consensual creation, manipulation, distribution or threat to distribute sexually explicit content using digital tools, such as AI, is a criminal offense.

Despite the legal framework, deepfake sex crimes have surged in recent years, emerging as one of the most pressing threats facing K-pop idols. The crimes are often driven by obsessive fan behavior or malicious intent to harm public figures.

Just last month, police arrested dozens of men in their teens, 20s and 30s for creating or sharing sexually manipulated videos featuring female celebrities and internet personalities.

According to authorities on April 11, 23 operators of a Telegram chat room were arrested on charges including violations of the Juvenile Protection Act and the Act on Special Cases Concerning the Punishment, etc. of Sexual Crimes.

Thirteen of the suspects were formally indicted, while an additional 60 users suspected of participating in the chat room were also arrested.

Entertainment agencies have begun to take more proactive steps in response to the growing threat.

On the same day the police announced the arrests, the multinational entertainment company HYBE confirmed that some of the suspects had targeted its artists. The company said the breakthrough was the result of its collaboration with law enforcement, including a direct hotline with the police.

According to HYBE, the agency signed a memorandum of understanding (MOU) with the Northern Gyeonggi Police on Feb. 25 to address the deepfake crimes. Since then, the company has actively reported illegal content to authorities and shared information to assist investigations.

"We will continue to respond to crimes that infringe on the portrait rights and dignity of our artists with zero tolerance and no settlements," said HYBE CEO Lee Jae-sang in a statement.

Several days later, Cube Entertainment, home to girl group i-dle, also announced legal action in response to manipulated videos and online abuse directed at its members.

On April 18, the label gave a statement addressing the sexually offensive content and online attacks on the group.

"We are currently securing evidence and preparing all possible civil and criminal legal actions through our legal representatives," the company said.

It added that anyone who distributes manipulated videos or posts defamatory content would face full legal consequences, with no leniency or settlements.

Korean law stipulates that creating or distributing sexually manipulated deepfake content is punishable by up to seven years in prison or a fine of up to 50 million won ($35,700). If carried out for profit, the penalty increases to a minimum of three years in prison.

Possessing, purchasing, storing or viewing such content carries a penalty of up to three years in prison or a fine of up to 30 million won.