Can Korea’s AI law stop deepfake crimes as Musk’s Grok comes under fire?

Elon Musk's AI company xAI is at the center of a controversy over the spread of deepfake images. AFP-Yonhap

As Elon Musk's artificial intelligence company xAI faces backlash over deepfake images, Korea's Framework Act on the Development of Artificial Intelligence is set to take effect next week. All eyes are on whether this legislation — the world’s first comprehensive AI regulation — can effectively fulfill its role.

However, skepticism is growing about the law’s effectiveness, with many predicting it will face practical hurdles in blocking xAI’s deepfake services or restricting access.

Industry officials said Tuesday that xAI’s AI model, Grok, continues to offer features on Musk’s social media platform X that turn everyday photos into sexually explicit deepfake images.

While Malaysia and Indonesia have restricted access to the platform and other nations have launched legal investigations — prompting xAI to limit the feature to paid subscribers — concerns remain.

The act, which takes effect Jan. 22, was enacted to strengthen AI operators’ responsibilities and create a foundation aimed at fostering trust in an AI-driven society. Preventing deepfake crimes has been cited as a key goal.

The law applies to overseas operators and requires them to designate domestic agents to fulfill legal obligations.

However, the transparency obligation to label deepfakes is not specifically included in the domestic agent’s role. This ambiguity could delay prevention or response when communicating with foreign operators.

Instead, the act itself mandates that AI-generated content must be labeled. It requires visible watermarks on deepfake content that are hard to distinguish from reality. Violations trigger a correction order, and failure to comply can result in a fine of up to 30 million won ($20,300).

Under these rules, Grok’s deepfake images must carry watermarks. But since the act includes a minimum one-year grace period, immediate enforcement is impossible.

Analysts say that even after the grace period ends, directly blocking or restricting foreign AI services will remain difficult due to potential trade friction.

“Under current laws, it is hard to do more than impose fines if overseas companies like xAI do not cooperate voluntarily,” said Jung Chang-woo, a lawyer at Lee & Ko.

Experts advise that until the new AI act takes full effect, deepfake cases should be handled under existing laws such as the Information and Communications Network Act or the Personal Information Protection Act.

“The binding force of the act alone is weak,” said Yeo Hyun-dong, a lawyer at Yoon & Yang LLC. “Regulations must be supplemented by sanctioning specific violations in conjunction with existing laws.”

The government maintains its stance on applying minimal regulation while monitoring the situation for a certain period. A Ministry of Science and ICT official said, “As AI technology is still developing through trial and error, we will watch the situation for now to allow for self-correction.”

This article from the Hankook Ilbo, the sister publication of The Korea Times, is translated by a generative AI system and edited by The Korea Times.