ED Korea's devastating AI blind spot

Like some experts observed, technological progress has outpaced humanity’s ability to understand it and our capacity to address the problems it creates. The artificial intelligence (AI)-powered cyberattacks on Korea’s financial institutions are a vivid illustration of how ill-prepared the nation is for the AI age.

The race to exploit system flaws and vulnerabilities is in full swing among cybercriminals, and without effective countermeasures, further damage will be inevitable — and could be far greater than that caused by cyberattacks in the pre-AI era.

The series of cyber incidents involving seven Korean financial institutions last week sends a warning signal: In the AI age, similar intrusions can occur at any time and across all sectors, potentially causing significant harm on an unprecedented scale.

There has been heated debate over how Korea can become a global leader in AI. But little attention is paid to the warning that unrestrained technology can endanger humans, rather than simply making life more affordable and convenient.

Korea is now paying the price for ignoring the destructive side of AI technology.

Last week, Shinhan Bank detected a data breach affecting about 25,000 customers and immediately reported it to financial authorities. The bank found that an intruder had identified vulnerabilities in its website and stolen customers' personal data, including their names, mobile phone numbers and annual incomes. The Financial Security Institute (FSI) found that the hacker had used AI to launch the cyberattack and had disrupted the systems at six other financial institutions as well.

AI agent-driven cyberattacks are unprecedented, raising concerns among financial authorities and institutions about the potential for devastating attacks. Unlike traditional AI systems or chatbots, which primarily respond to users’ queries, AI agents can perceive, reason, plan, act and learn. These capabilities can enable threat actors to increase the speed, scale and sophistication of their cyberattacks.

Accordingly, cyberattacks carried out with the help of AI agents could have irreversible and devastating consequences.

Aware of the seriousness of the incident, the government took immediate action. The Ministry of Science and ICT, in collaboration with the Korea Internet & Security Agency, launched an emergency response system to prevent the spread of AI-powered data breaches. Meanwhile, three financial supervisory organizations — the Financial Services Commission, the Financial Supervisory Service and the FSI — joined forces to investigate the unprecedented AI-powered breach and monitor whether other private-sector institutions had also been affected.

Artex AI, a Chinese AI system, is believed to have been used in the intrusion. The AI agent was initially developed as a “white-hat” security tool for conducting simulated cyberattacks and autonomously identifying system vulnerabilities. Malicious actors, however, have repurposed the technology to identify weaknesses and orchestrate real attacks.

The dangers posed by AI agents were foretold.

In May, Google warned of AI-powered cyberattacks and their potentially devastating consequences, saying they could reach an industrial scale. In a report released that month, the tech giant’s Threat Intelligence Group found that cybercriminals and state-sponsored actors from China, Russia and North Korea appeared to be widely using AI models such as Gemini and ChatGPT to bolster their attacks. Hackers were using AI to detect vulnerabilities in target software and hardware, develop malware and undermine defensive responses. “The era of AI-driven vulnerabilities and exploitation is already here,” John Hultquist, the group's chief analyst, was quoted as saying.

Yet such warnings went largely unheeded.

Korea turned a deaf ear to the warnings about the destructive side of technology, focusing instead on overtaking global AI leaders as if AI is capable of solving every problem facing society.

The AI-powered cyberattacks targeting Korean banks are a vivid reminder of what can happen when cybercriminals exploit technology. They also offer a glimpse into the troubling reality of what can happen when powerful technology falls into the wrong hands.

To prevent further AI-powered cyberattacks, Korea must introduce effective guardrails. But an immediate response alone will not be enough.

The attacks should instead be viewed as an opportunity to overcome the nation’s complacency about the AI age and put the brakes on its blind pursuit of technological advancement. Policymakers and industry leaders should listen to growing concerns about the direction of technological development.

A global debate is already underway over whether frontier AI development should be slowed. At its core is the question of how to balance technological progress with the risks it creates. Advocates of greater restraint argue that AI models and their capabilities are advancing faster than humanity’s ability to understand, govern and control them.

The day when humans become subordinate to their own creations may not be a distant prospect. It may already be unfolding.