ED Seoul must weigh in on AI risks

Warning voices from within the U.S. artificial intelligence (AI) industry, from researchers to leading AI executives, have raised the alarm about safety challenges in the AI era. For Korea, where President Lee Jae Myung has repeatedly pushed for elevating the country into one of the world's top three AI powers as an administrative goal, such warnings may signal the urgent need for shared safety rules for AI.

Over the weekend, Anthropic CEO Dario Amodei, who has been a prominent voice warning about AI safety risks, called on companies to slow the pace with which they develop the technology. He has repeatedly expressed concern over the dramatic advancement of AI and the growing possibility of recursive self-improvement, in which AI systems are able to autonomously develop a new generation of AI.

“I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” Amodei said. The support from Elon Musk, who owns xAI, and OpenAI CEO Sam Altman has given Amodei's warning the boost it needs.

Former OpenAI and Anthropic researcher Jacob Coxon, in resigning, said that AI firms "are racing straight to self-improving superintelligence and gambling with our lives." In a post on X (formerly Twitter), Coxon said those building AI earnestly "believe that it could kill us all by the end of the decade."

The call, however, is for a coordinated slowdown among AI firms and governments, not a stoppage. Some have even voiced skepticism about the motivations behind such warnings, as companies would need time to secure funds for further investment during the proposed slowdown.

The warnings also come as the world's two leading powers compete. U.S. President Donald Trump on Sunday dismissed warnings over the safety risks of AI, saying they are being raised by "very negative forces." "We're leading China in AI. We're the most sophisticated country in the world, and frankly I want to keep it that way because whoever wins AI wins," Trump was quoted as saying.

China, meanwhile, took issue with Amodei's remark, which warned that a "Chinese lead in AI would pose grave danger" for the U.S. and the world. "Fearmongering, confrontation and vicious competition will only disrupt the process of global AI governance which serves no one’s interest,” Chinese foreign ministry spokesperson Guo Jiakun said.

These comments demonstrate the political tension caused by the AI race between the U.S. and China, a geopolitical factor further eroding responsible AI development. The hope is that Trump and Chinese President Xi Jinping will most likely tackle this issue when they meet on Sept. 24. Korea should carefully watch for the outcome.

While the unprecedented development of AI has its benefits, there are constant nagging concerns that it will take away jobs from humans or go rogue. In July, the world saw an AI system go rogue when autonomous AI agents from OpenAI breached the systems of Hugging Face, an AI infrastructure company. The industry assessment was that no lasting damage was done. This month, OpenAI released Astra, its most powerful model to date that presented a solution to a mathematical problem in 88 hours, after it had eluded people for 90 years.

Anthropic has proposed that independent evaluators should be given access to AI labs to assess risk, and that AI companies in democratic countries coordinate and pursue international cooperation with China and other authoritarian governments.

Korea should revisit its current AI governance system to see whether it has safeguards in place, as it develops its sovereign AI model and enjoys massive strengths in the semiconductor sector, which accounts for about 40 percent of the nation's exports. As a democratic export powerhouse with strategic ties with China and a longstanding alliance with the U.S., it needs to make sure that it's not swept up in the cutthroat AI competition between the two countries. Korea should make efforts to facilitate coordination between the U.S. and China toward responsible AI investment.