Surprise: The Humans Won

The results are in. Human translators still beat AI. Not that they don’t do so everyday. But the showdown in Korea saw four professional translators post much higher accuracy scores than those achieved on average by the machine translation programs of Google, South Korean web company Naver, and commercial automated translation provider Systran. Machine translation these days is often branded as artificial intelligence (AI).

As reported by Yonhap, both sides were given 50 minutes to finish their work. The human translators squared off against Google Translate, Naver Papago, and Systran’s machine translation system, crunching through random articles, translating the English text into Korean, Korean into English. Their work was then evaluated by two other professional translators.

Organizer IAPTI (International Association of Professional Translators and Interpreters) said the human translators scored a median 25 out of 30 in Korean-English translation, while the AI side averaged 10–15. Of course, trying to judge translation quality with mathematical accuracy is fraught with difficulty.

Advertisement

According to the organizers, the unsurprising finding was that the NMT engines were unable to understand context. They added that 90% of NMT-translated texts were “grammatically awkward.”

In short, no breakthrough findings out of Korea. But the industry should welcome initiatives like this, which bring a lighthearted element of competition to one of the industry’s hot button issues.