Human translators rout AI in much-hyped translation event

By Yoon Sung-won

Humans proved they have the competitive edge in comprehension of original texts compared to translation programs, despite the rapid improvement of artificial intelligence (AI) technologies, Tuesday.

On a scale of 30 points, four human translators received around 25 points whereas the best score by one of the three translation programs was 15. The two other AIs had less than 10 points, the organizers said.

"Human translators received high scores," said Kwak Joong-chol, chairman of the Korean Association of Translator & Interpreters (KATI), who led the three-member evaluation panel.

Kwak said the reason for the humans' landslide victory was the difficulty of the original texts.

"The original texts were hard to translate for anyone. They were never been translated before," he said. "The four translators all have more than five years of experience. The AIs turned out to be no match as of yet."

Kwak pointed out the AI programs revealed more problems in translating literature. He also said AIs do not revise their work, which is their biggest weakness.

The KATI chairman, who also is a professor at Hankuk University of Foreign Studies' Graduate School of Translation and Interpretation, said accuracy, linguistic expression and logical structure were the key evaluation points.

Kwak also said AI-based translation programs, which can self-improve by learning from databases thanks to a mechanism called "neural machine translation," will continue to perform better.

"It is highly likely AI programs will surpass humans in the future as it did in the game of go," he said. "Human translators and interpreters will be eliminated from the market unless they follow the trend to improve themselves."

The event was jointly hosted by the International Interpretation and Translation Association (IITA) and Sejong University. It was the first translation competition between AI-based programs and human experts in Korea.

"No matter how fast the translation programs are, many will doubt they can perfectly translate subtle expressions of emotion in literature," IIAT Chairman Kim Dong-ik said. "We hope the event shows the relative strengths and weaknesses of AI translation programs and human experts."

In Tuesday's event, four human professional translators, who asked for anonymity for privacy, and three translation programs by Google, Naver and Systran were given two sets of English and Korean original texts in literary and non-literary sectors.

The English original texts had 300 words each and the Korean texts had 700 words each including spaces. The four human translators chose one original text each to translate within 50 minutes. Searching the internet was also allowed for the human translators, according to the organizers. The three programs processed all four original texts.

Excerpts from the Korean novel titled "Mother and Daughter" by Kang Kyeong-ae and English novel "Thank You for Being Late" by Thomas Friedman were taken for the literary sector. For the non-literary sector, excerpts from an op-ed by Korean news daily Hankook Ilbo and a Fox News business article titled "How a Movie Propelled Lego Back to the World's Most Powerful Brand" by Ben Brown were picked.

"Translation programs still have limits," said Sam Kim, vice president of Systran International's Global Strategy Planning Office. "Reports by Google and Naver also show such programs still make mechanical errors in processing subtle nuances and emotional expressions. Humans and machines can collaborate in such part."

Kim also said the rules of Tuesday's competition were unfavorable to machines.

"The artificial neural network consumes much computing power. This increased server capacity consumption and slowed down processing pace," he said. "Compared to human experts who can access library database, machines can be seen as disadvantageous. In specified translations, however, machines will perform better with fewer mistakes."

Saltlux Vice President Shin Seok-hwan said, "Though research shows machine translators can perform about 85 to 90 percent of human experts, they still make absurd mistakes. Today's event is more about how much humans can benefit from using these machines and boost work efficiency of professional translators."

Before the showdown, human experts were expected to outperform the AIs. The organizers said human professionals would produce more accurate translations whereas AI-based programs will surely overwhelm their human counterparts in speed.

"Future discussions about the capability of AI-based programs should expand to the areas of poetry translation and religious sermon interpretation," said Hur Myung-soo, president ofthe Korea Association of Translation Studies.

Site Map E-paper Subscribe Register LogIn
Site Map E-paper Subscribe Register LogIn