logo image
  • News
    • People Moves
    • Deal Wins
    • Demand Drivers
    • M&A and Funding
    • Financial Results
    • Technology
    • Academia
    • Industry News
    • Features
    • Machine Translation
    • — Divider —
    • Slator Pro
    • — Divider —
    • Press Releases
    • Sponsored Content
  • Data & Research
    • Research Reports & Pro Guides
    • Language Industry Investor Map
    • Real-Time Charts of Listed LSPs
    • Language Service Provider Index
  • Podcasts & Videos
  • Events
    • Design Thinking – February 2021
    • — Divider —
    • SlatorCon Coverage
    • Other Events
  • Directory
  • RFP Center
  • Jobs
MENU
  • News
    • People Moves
    • Deal Wins
    • Demand Drivers
    • M&A and Funding
    • Financial Results
    • Technology
    • Academia
    • Industry News
    • Features
    • Machine Translation
    • — Divider —
    • Slator Pro
    • — Divider —
    • Press Releases
    • Sponsored Content
  • Data & Research
    • Research Reports & Pro Guides
    • Language Industry Investor Map
    • Real-Time Charts of Listed LSPs
    • Language Service Provider Index
  • Podcasts & Videos
  • Events
    • Design Thinking – February 2021
    • — Divider —
    • SlatorCon Coverage
    • Other Events
  • Directory
  • RFP Center
  • Jobs

Advertise on Slator! Download the 2021 Online Media Kit Now

  • Slator Market Intelligence
  • Slator Advertising Services
  • Slator Advisory
  • Login
Search
Generic filters
Exact matches only
Advertisement
Bridging the Gap Between NMT’s Theoretical Promise and Practical Limitations

1 year ago

October 23, 2019

Bridging the Gap Between NMT’s Theoretical Promise and Practical Limitations

Sponsored Content ·

by Ofer Shoshan

On October 23, 2019

1 year ago
Sponsored Content ·

by Ofer Shoshan

On October 23, 2019

Bridging the Gap Between NMT’s Theoretical Promise and Practical Limitations

I was recently on a call with Sara, the localization manager of one of our enterprise customers, a big ecommerce company. She was frustrated. Ideally, she would like to translate millions of words a month. Human translation takes way too long and is too expensive. Neural Machine Translation (NMT) looks more promising than human translation and previous translation technologies, but putting it to practical use did not really work.

On the one hand, it seems like NMT can potentially solve a lot of her current localization issues, as she needs to process high volumes quickly with high quality. On the other hand, getting NMT to work in the real world is difficult. Different NMT engines handle different content and different languages at varying quality levels, the quality of the NMT engines is changing all the time, and the workflow required to produce quality translations based on NMT is often too complex.

Many of our enterprise customers face the same issues. There is a big gap between the theoretical capabilities of NMT engines and the practical capabilities in a real-world production environment.

Advertisement

Clearly, NMT has great potential to totally transform and disrupt the translation industry by providing instant, human-quality translation at a very low cost. In just three to four years, NMT surpassed all previous translation technologies developed over the 50 years since the Cold War. Having said that, the biggest issue is NMT quality variance between different engines and between different sentences translated with the same engine.

There is a big gap between the theoretical capabilities of NMT engines and the practical capabilities in a real-world production environment

During the next few years, until NMT reaches human parity, many organizations will have already started using imperfect NMT engines to reduce translation cost and turnaround time.

The key question is, what can executives in companies that sell globally do today to ensure they stay competitive and leverage NMT correctly?

First, it should be clear that the goal is getting your organization to a point where your  translations are accurate, high quality, containing your terms, jargon, style, and glossary, and delivered instantly at a low cost. This means that whenever you change something in your source language, you immediately have it in all other languages; i.e., all your customers are on the same page all the time, with very little overhead.

The biggest issue is NMT quality variance between different engines and between different sentences translated with the same engine

The other goal is to keep up with the competition — if your competitors use NMT and you do not then, effectively, you will lag behind and be less competitive.

In practice, there are three points to consider.

  • Check if NMT is right for you – Depending on your type of material and language pairs.

NMT quality is not unified across engines, languages, and types of material. For example, Amazon may be the best engine for your content when translated into French, while Google may be better with the same material into Russian; and there might not be a good engine right now to translate your content into Japanese.

  • What NMT engines to use – More than one is key!

Since quality varies a lot between NMT engines, as well as within the same engine, and it changes with time, it is important to identify specifically which NMT engine to use for each material and language pair at any given time. In addition, you should consider training a dedicated NMT engine just for you. 

  • When to start – Once above a minimal quality threshold.

To benefit from NMT, it should be used when the basic quality is above a certain level. If dedicated NMT training is required, it takes more time and should start even before such a level is reached.

Is NMT for you? The answer is probably yes; the question is when?

Before investing any resources in NMT, it is worth checking if the current quality of NMT engines fits your company’s needs. Since NMT quality changes all the time and since automatic measures are not good enough, we launched ONEs – OHT NMT Evaluation score. (Read more about it here and here.)

ONEs is an objective assessment of the quality of NMT engines, based on a unique human evaluation. Once a quarter, we run tens of thousands of string evaluations, using hundreds of linguists, while using different NMT engines to translate into several target languages.

ONEs can provide a general answer regarding the current state of NMT for your language pairs. 

Assuming NMT looks promising, you need to answer a more specific question: Which engines perform best with your material per target language?

Since the quality of different NMT engines — Amazon, Google, Microsoft, etc. — varies a lot across languages and types of material, and since the quality changes on a monthly basis, it makes sense to check the translation quality of a sample of your specific material.

From our experience, using just one NMT engine is typically not good enough

From our experience, using just one NMT engine is typically not good enough, and does not provide an optimal outcome. You need to check your material with different NMT engines, and with your target language pairs, to see which engine performs best in each language. It will take a while before the generic engines can handle all materials and all target languages at the same quality level.

Another important aspect is: Not to look at the average quality of a document, a page, or even a set of strings!

From our experience, the average NMT quality across several sentences, even in the same paragraph, will almost always look bad because there is high quality variance. That is, when running NMT on a set of sentences, the first sentence may have a perfect machine translation, the second will require some human post-editing, and the third will need to be retranslated by a human translator from scratch.

We check for customers the quality produced by several NMT engines, such as Amazon, Google, etc., by running a sample of the customer’s material. The result of this examination is a table that tells you what NMT engine to use per material and language pair, and what level of quality you should expect; i.e., what level of manual work will be required in each case.

Finally, should you start or should you wait?

In the NMT report we produce for customers, we calculate the estimated potential savings per target language and type of material using the optimal combination of NMT engines. In general, if potential savings are at 30% or more, you should start using NMT. The reason is simple — at this level, the savings are substantial enough for both you and your competitors to use NMT.

In relevant cases, we also recommend training a dedicated NMT engine. Still, we recommend to first use existing NMTs in an optimal way, and then start training a dedicated engine. One thing to consider is that training a dedicated engine takes time and depends on the volume of existing high-quality translations that can serve as input during training. Dedicated training, which we call MyNMT, will be discussed separately.

There are other considerations, of course; mainly regarding the use of the right software platform that can send each sentence to the right NMT engine dynamically, and that can select the right workflow (pre-processing, quality control, etc.)

I will discuss these points and others in my presentation at SlatorCon Amsterdam on November 28, 2019. More details and registration please visit the event website.

TAGS

Amazon TranslateGoogle TranslateMicrosoft Translatorneural machine translationOfer ShoshanOne Hour Translation
SHARE
Ofer Shoshan

By Ofer Shoshan

Ofer Shoshan is the CEO of One Hour Translation, providing enterprises with HALO - Hybrid AI Localization platform, Neural Machine Translation, and Multilingual human services. OHT allows great companies communicate with great customers quickly and efficiently across any language. Learn more are www.onehourtranslation.com

Advertisement

SUBSCRIBE TO THE SLATOR WEEKLY

Language Industry Intelligence
In Your Inbox. Every Friday

SUBSCRIBE

SlatorSweepSlatorPro
ResearchRFP CENTER

PUBLISH

PRESS RELEASEDIRECTORY LISTING
JOB ADEVENT LISTING

Bespoke advisory including speaking, briefings and M&A

SLATOR ADVISORY
Advertisement

Featured Reports

See all
Slator 2020 Language Industry M&A and Funding Report

Slator 2020 Language Industry M&A and Funding Report

by Slator

Slator 2021 Data-for-AI Market Report

Slator 2021 Data-for-AI Market Report

by Slator

Slator 2020 Medtech Translation and Localization Report

Slator 2020 Medtech Translation and Localization Report

by Slator

Pro Guide: Sales and Marketing for Language Service Providers

Pro Guide: Sales and Marketing for Language Service Providers

by Slator

Press Releases

See all
Donna Thomas Joins Visual Data Media Services as Senior Vice President of Sales, Americas

Donna Thomas Joins Visual Data Media Services as Senior Vice President of Sales, Americas

by Visual Data Media Services

iDISC Awarded ISO 27001 Information Security Management Certification

iDISC Awarded ISO 27001 Information Security Management Certification

by iDISC

XTRF Launches a Bi-Monthly Free Networking Event for Localization Professionals

XTRF Launches a Bi-Monthly Free Networking Event for Localization Professionals

by XTRF

Upcoming Events

See All
  1. Memsource MT Post-Editing Pricing Models Webinar

    Pricing Models for MT Post-Editing Workshop

    by Memsource

    · February 3

    Hear a panel of innovative localization professionals share different approaches for MT post-editing pricing.

    More info FREE

Featured Companies

See all
Text United

Text United

Memsource

Memsource

Wordbank

Wordbank

Protranslating

Protranslating

Seprotec

Seprotec

Versacom

Versacom

SDL

SDL

Smartling

Smartling

Lingotek

Lingotek

XTM International

XTM International

Smartcat

Smartcat

Translators without Borders

Translators without Borders

STAR Group

STAR Group

memoQ Translation Technologies

memoQ Translation Technologies

Advertisement

Popular articles

Why Netflix Shut Down Its Translation Portal Hermes

Why Netflix Shut Down Its Translation Portal Hermes

by Esther Bond

The Slator 2020 Language Service Provider Index

The Slator 2020 Language Service Provider Index

by Slator

Top Language Industry Quotes of 2020

Top Language Industry Quotes of 2020

by Monica Jamieson

SlatorPod: The Weekly Language Industry Podcast

connect with us

footer logo

Slator makes business sense of the language services and technology market.

Our Company

  • Support
  • About us
  • Terms & Conditions
  • Privacy Policy

Subscribe to the Slator Weekly

Language Industry Intelligence
In Your Inbox. Every Friday

© 2021 Slator. All rights reserved.

Sign up to the Slator Weekly

Join over 13,000 subscribers and get the latest language industry intelligence every Friday

Your information will never be shared with third parties. No Spam.