We Tested Google Bard on Machine Translation

Google Bard on Machine Translation

If you plan to tinker with Bard’s translation capabilities, be prepared for some self-aware caveats, as Google’s large language model (LLM) is “still working to learn more languages.”

Google reminds users repeatedly that, as an experiment, Bard’s responses may be inaccurate or inappropriate. “When in doubt, use the ‘Google it’ button to check Bard’s responses,” Google recommends.

Initially rolled out in the US and the UK, Google VP and General Manager Sissie Hsiao shared in a May 10, 2023 blog post that Bard is being opened up to over 180 countries and territories, and “on track to support 40 languages soon,” though no timeline was provided. 

Bard is currently available in US English, Japanese, and Korean — but Bard does not seem to be aware of that fact.

“I can translate between 133 languages,” Bard replied when asked which languages it can translate. “I can also translate between many different dialects of the same language, such as American English, British English, Australian English, Indian English, and Jamaican English.”

In practice, when asked to work in a language other than the officially supported three, Bard begged off: “I am an LLM trained to respond in a subset of languages at this time, so I can’t assist you with that.”

Bard successfully translated text from Korean into English, Korean into Japanese, and Japanese into Korean.

Translating from English into Korean, Bard offered three drafts, though they appeared to be identical. 

Working from English into Japanese, Bard produced two translations, describing the second as “a more literal translation of the text.”

Bard declined to translate from Japanese into English, responding, in Japanese, “As a large scale language model I am still learning and cannot answer that question.”

Perhaps Bard’s most “human” quality is the way it oscillates between professed modesty and overconfidence. 

Bard confirmed that it can subtitle a movie, explaining in detail the steps it would take to transcribe the dialogue, translate it, and time the subtitles, and even promising to send the user a subtitle file in a desired language if needed.

Which languages? According to Bard, “I can generate subtitles in a variety of languages, including English, French, Spanish, German, Italian, Portuguese, Russian, Chinese, Japanese, and Korean.”

Presented with a link to a video on YouTube and asked to transcribe the audio, Bard hallucinated a speech about the importance of a good work ethic by a man wearing a suit and tie. But not to worry — Bard is working hard, and still learning.

Important Read: Why Large Language Models Hallucinate When Machine Translating ‘in the Wild’