ED Seoul must weigh in on AI risks

Warning voices from within the U.S. artificial intelligence (AI) industry, from researchers to executives, have raised the alarm about safety challenges related to the technology. For Korea, where President Lee Jae Myung has repeatedly pushed for the country to become one of the world's top three AI powers as a goal for his administrative, these warnings signal an urgent need for shared safety regulations.

Over the weekend, Anthropic CEO Dario Amodei, who has been warning about AI safety risks, called on companies to slow the pace at which companies are working to develop the technology. He has repeatedly expressed concern over the dramatic advancement of AI and the growing possibility of recursive self-improvement, in which AI systems are able to autonomously develop a new generation of AI.

“I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” Amodei said. Support from rivals like Elon Musk, who owns SpaceXAI, and OpenAI CEO Sam Altman has given Amodei's warning even more weight.

Former OpenAI and Anthropic researcher Jacob Coxon said when he resigned over concerns about AI safety that firms "are racing straight to self-improving superintelligence and gambling with our lives." In a post on X (formerly Twitter), Coxon said those building AI earnestly "believe that it could kill us all by the end of the decade."

The call is for a coordinated slowdown among AI firms and governments, not stopping research altogether. Some have even voiced skepticism about the motivations behind Amodei's warnings, suggesting AI firms intend to use the slowdown to shore up their existing leads in the sector.

The warnings also come as the world's two leading powers compete. U.S. President Donald Trump on Sunday dismissed warnings over the safety risks of AI, saying they are being raised by "very negative forces."

"We're leading China in AI. We're the most sophisticated country in the world, and frankly I want to keep it that way because whoever wins AI wins." Trump was quoted as saying.

China also took issue with Amodei's remarks, which warned that a "Chinese lead in AI would pose grave danger" for the U.S. and the world.

"Fearmongering, confrontation and vicious competition will only disrupt the process of global AI governance which serves no one’s interest,” Chinese foreign ministry spokesperson Guo Jiakun said.

These comments demonstrate the political tension caused by the AI race between the U.S. and China, a geopolitical factor further eroding responsible AI development. The hope is that Trump and Chinese President Xi Jinping will most likely tackle this issue when they meet on Sept. 24. Korea should carefully watch for the outcome.

While the development of AI has demonstrated many benefits, there are constant concerns that it will take jobs away from humans or go rogue. In July, the world saw an AI system go rogue when autonomous AI agents from OpenAI breached the systems of Hugging Face, an AI infrastructure company. The industry assessment was that no lasting damage was done.

Anthropic has proposed that independent evaluators should be given access to AI labs to assess risk, firms set common safety standards and democratic countries pursue international coordination with China and other authoritarian governments.

Korea should revisit its current AI governance system to see whether it has safeguards in place, as it develops its sovereign AI model and enjoys massive strengths in the semiconductor sector, which accounts for about 40 percent of the nation's exports. As a democratic export powerhouse with strategic ties with China and a longstanding alliance with the U.S., it needs to make sure that it's not swept up in the cutthroat AI competition between the two countries. Korea should make efforts to facilitate coordination between the U.S. and China toward responsible AI investment.