Which Engines Work for Me? Machine Translation as a Managed Service at One Hour Translation

Yaron Kaufman, One Hour Translation at SlatorCon Amsterdam 2019

Yaron Kaufman, Co-founder and Chief Marketing Officer of One Hour Translation, invited the audience at SlatorCon Amsterdam 2019 to imagine a world in which machine translation (MT) has already been perfected. It is free, automated, and equal in quality to human translation.

Kaufman painted the scene: Websites flawlessly localized into hundreds of languages; immigration without language barriers; more accessible educational materials; software that translates itself. For language service providers (LSPs), Kaufman envisions localization managers taking on a new and more important role.

“We’re talking about people who know how to apply tools for specific jobs, specific industries. We’re talking about people who focus on the local experience,” Kaufman said.

Yes, But

While it has been well established that neural machine translation (NMT) has boosted machine translation quality over the past three years, the technology’s continuing flaws have also become apparent. Kaufman cited the omission of words in MT output, changed meanings, inconsistent translations, and the need for significant post-editing as issues that still need to be addressed.

This means clients may come to One Hour Translation unsure of whether to pursue MT for their translation needs. They have heard that it is promising, but not perfect.

“There are two main reasons why customers are approaching us and asking about taking advantage of machine translation,” Kaufman said. “One is price, and the second one, which I like more, is, ‘I have tons of content to translate.’”

As an online LSP offering translation in over 100 languages, One Hour Translation often works with clients as they weigh the pros and cons of investing in MT. Kaufman shared four questions that help guide these conversations.

NMT Score

Clients first want to clarify the mixed messages they may have received: Is MT “good enough” to merit further consideration? To answer this question, One Hour Translation runs its own experiments, feeding 100 general phrases into several of the most popular MT engines, including Amazon Translate, Baidu, DeepL, Google Translate, or Yandex. One Hour Translation then analyzes the results and publishes a quarterly report, the NMT Score, which is made available to the public.

So far, One Hour Translation has seen MT engines improve quickly — even from one quarter to the next — and, unsurprisingly, has noted that some engines are stronger in certain language pairs than others.

“We know that some of your competitors are already in this game, and you need to be ready for tomorrow” — Yaron Kaufman, Co-founder and CMO, One Hour Translation

The caveat, of course, is that this research is done using only general texts. “But when we focus on certain domains, certain industries, such as the gaming industry, we see a different picture,” Kaufman explained. “We see that those generic engines do not produce such good results as they do for general texts.”

This is one reason that the kind of content a client needs to translate will play a role in determining whether MT is an appropriate option. If it is, the content type will also factor into the client’s potential savings.

One Hour Translation provides actual figures by running client-specific content through several different MT engines and then having human evaluators rate the results.

Machine Translation as a Managed Service

Based on those ratings, the client’s content can be grouped into requiring “MT without further work,” “MT plus post-editing,” or “human translation from scratch,” with “MT-only” translation offering the greatest possible savings.

Clients want to know how they can benefit from MT today, and those figures help answer the question. In general, One Hour Translation sees clients save an average of 30% of their translation costs when using MT or a combination of MT, post-editing, and pure human translation.

Skeptics, however, might question investing in MT now, when the technology is changing so rapidly that MT might soon be perfect and free of charge. In addition to the immediate savings, Kaufman reminds clients to consider the bigger picture for their business.

“My recommendation is to be in this game. Even if you’re going to use generic engines, and not dedicated, trainable machine translation, this is my recommendation,” Kaufman said. “We know that some of your competitors are already in this game, and you need to be ready for tomorrow.”