Intento Updated Its MT Evaluation Report (23 Systems, 48 Language Pairs)

Berkeley, California, Jan. 18, 2018 — Intento (https://inten.to) has published a report on the evaluation of 23 different stock (pre-trained) Machine Translation models across 48 popular language pairs: https://bit.ly/stock_mt_jan2019

There are more than 20 different Machine Translation systems available on the market. Intento, which provides tools to use Machine Translation in a vendor-agnostic fashion, evaluates stock MT models since May 2017. As they show in this report, the Machine Translation landscape is complex and fast-changing, as the best MT system changed for 21 language pairs since July 2018.

Key highlights for the last report:

Advertisement

1. Two NMT engines added for the quantitative evaluation: ModernMT and SDL BeGlobal, while SMT systems from Microsoft and IBM were officially deprecated and removed.
2. We have discovered that PROMT models improved a lot in Russian, Tencent – in Russian and German, Yandex – in Japanese, Portuguese, German, and Chinese, Baidu – in English and Russian, Systran PNMT got a boost in almost every pair they support.
3. Yandex cut their MT price 50% (to ~$7).
4. Amazon, DeepL, IBM, Microsoft, and Youdao increased their language coverage.
5. “The best engine” changed for 21 language pairs: there are still 9 required to get the best quality for all pairs, but those 9 are very different from July 2018.

With the goal of full transparency and reproducibility, Intento runs the evaluation on publicly available datasets and describes every step of the process. The report is available for free to all interested parties at https://bit.ly/stock_mt_jan2019.

For further information, please contact Konstantin Savenkov, Co-Founder & CEO of Intento (ks@inten.to)