Technology • By Gino Diño On

Here’s How Google Improves Translation Quality for Youtube’s UI

Research at Google published a blog post detailing how they used a newly developed Language Quality Survey to improve the translated, multilingual user interface (UI) of Youtube based on user feedback. Now, they want everyone else to use it free of charge.

“The perceived language quality of translated user interfaces (UIs) can have a significant impact on the overall quality and usability of a product,” the blog post reads. It explains that Google had experts perform in-depth reviews of its translated UI texts — so-called “language find-its” — to improve Youtube. It highlights that the developers who design the UI do not necessarily know about the quality of translated texts if they do not speak the languages the text was translated into. The linguists who work on UI translations may also not know the full context of the strings they need to work on or separate translators could work on a single project. All of these factors snowball into increased risk of low quality UI translations.

The “language find-its” were mostly useful, they were also expensive and time-consuming, so Google decided to develop a channel for users to provide feedback themselves. The following ten questions of the Language Quality Survey was the result:

  1. How friendly or unfriendly is the text used in the
    interface? By “friendly” we mean that the language used shows that
    respects and likes their users.
  2. How casual or formal is the text used in the
    interface? By “casual” we mean that the language used is relaxed, like friends speaking to each other. By “formal” we mean that the language is academic, similar to the text of an essay or a legal document.
  3. How professional is the text used in the
    interface? By “professional” we mean that the language is well-written and shows that
    cares about quality.
  4. How natural or unnatural is the text used in the
    interface? Natural here means that the language used represents the way people normally speak to each other.
  5. How easy or difficult to understand is the text used in the
    interface?
  6. How appropriate or inappropriate do you consider the text in the
    interface?
  7. How often do you encounter grammatical errors in the text used in the
    interface?
  8. How often do you encounter typos / spelling errors in the text used in the
    interface?
  9. How often do you encounter untranslated words that are not in English in the text used in the
    interface?
  10. How satisfied or dissatisfied are you with the quality of language in the
    interface when using English?

Users can provide answers based on a five point range corresponding to Always, Often, Sometimes, Rarely, and Never, phrased differently depending on the question. According to Google’s research, the first five questions focus on readability: how smooth and natural the translated text reads. The next four questions focuses on linguistic correctness: the frequency of inconsistencies in the text. Every time a user chose the two most negative answers (i.e. never or rarely user-friendly or casual or professional), the survey brought up a text form to obtain qualitative feedback. Users were asked to explain why they chose these answers.

All in all, Google managed to identify problem areas in Youtube’s translated UI texts, analyze quantitative and qualitative feedback from their users, and even use the Language Quality Survey in conjunction with their “language find-its” to optimize their UI translations. Google saved money and time by cutting back on their costly and time intensive “language find-its” thanks to this survey.

The modern web browsing experience is deeply affected by UI and the quality of engagement users get from it. Add to that the fact that trends like e-commerce is driving demand for more website localization and Google itself is warning websites against using machine translated content without quality assurance, this newly developed Language Quality Survey may just prove useful.

Google is offering their Language Quality Survey to anyone interested in using it for their quality assessments. If it worked for Google and Youtube, it will probably work for other developers struggling with UI translations.