Venture funding continues for startups working on automated text-generation tools and services. While these investments are not going into the core translation and localization space that Slator typically covers, natural language processing (NLP) is going through a major commercialization phase, which will likely leave a mark in translation as well.
After OthersideAI and Verbit, comes AI21 Labs, developer of (semi-automated) writing productivity tool, Wordtune. The Israeli startup has just raised USD 25m, according to a LinkedIn post by investor Pitango First, which led the round. Fellow Tel Aviv VC, TPY Capital, among others, also participated.
Dubbed as “Israel’s largest venture capital fund” in a November 19, 2020 press release announcing the round, Pitango first invested in AI21 in January 2019, when it led a USD 9.5m round.
The latest investment brings AI21’s total funds raised to USD 34.5m, according to Ori Goshen (Co-CEO), who co-founded AI21 in 2017 with Yoav Shoham (Co-CEO), and Amnon Shashua (Chairman).
Goshen told Slator, “We raised more funding to double-down on our AI technology and continue to build additional category-defining products.” He declined to go into the current ownership structure of AI21 Labs.
The company is based in Tel Aviv, as are most of its 40 full-time employees. Asked if they had any plans to relocate (as fellow Israeli-founded Verbit did) in the foreseeable future, Goshen would only say, “AI21 headquarters and most of our development will remain in Tel Aviv. Of course, we have business operations around the globe.”
Goshen recalled that when he and Co-CEO, Shoham, first met, they talked about “the state of AI, and the discrepancy of the hoopla around AI of the 80s and that of today. Then the discussion morphed into one about semantics and symbolic methods. (Today, it’s about statistics and neural nets.) They realized there was an opportunity to combine the two. Slightly later, Amnon Shashua, an expert in deep learning with a similar passion for turning machines into true thought partners, joined the quest.”
Déjà Vu
AI21 Lab’s showcase software, Wordtune, suggests alternative ways to write what one has typed. Users of translation productivity (a.k.a. CAT) tools will find the interface familiar: Place the cursor over a word and a bunch of synonyms will appear in drop down. Highlight a sentence and you can choose among several paraphrased alternatives in two modes: casual (represented by a sneaker) or formal (a briefcase).
Wordtune is currently available as a Chrome extension via five-day free trial. Once you install it on Chrome, the purple Wordtune dot (menu) will appear on other platforms as well; Gmail, Google Docs, Slack, Whatsapp, LinkedIn, Twitter, Facebook, etc. For now, it is available in English only, but does support other languages (e.g., French, Russian, Portuguese, Spanish, German, Hindi, Chinese, Arabic) through the plug-in’s “Word Finder feature.”
Asked about Wordtune’s underlying tech and algorithm, and whether they used GPT-3 (currently dubbed the world’s largest language model, as email productivity startup OthersideAI did), Goshen declined to comment.
He told Slator, “We launched Wordtune on October 27, 2020. Much of what the team has done in the past three years contributed to Wordtune in one way or another.”
What the team has done is outlined in the company website. For example, they created a model called HAIM using the same Transformer-based architecture as GPT-3 precursor GPT-2 and Allen’s Grover. The developers describe HAIM as “a language model that inserts synthetic text between human-written beginning and ending,” the goal being to improve fluency.
Slator 2020 Medtech Translation and Localization Report
As the developers put it: “Impressive as it is, the text generated by GPT-2 and Grover is far from perfect. In particular, the models’ output tends to diverge from the human-written input as the generation progresses. Sooner or later, the generators go off-topic, lose coherence, or contradict the preceding text. Such behaviors are a major problem for a user trying to convey a message or express an idea.”
The AI21 team also created HAIMKE, which turns bullet points to text, similar to what OthersideAI does, as covered by Slator.
SenseBERT, another model they worked on, is explained in a paper on pre-print server arXiV. SenseBERT “achieves significantly improved lexical disambiguation” based on Google’s NLP model BERT. In short, “driving some sense into BERT,” the team said.
According to Goshen, “Most of our technology is not public. SenseBERT and a few other technologies were made public. HAIM and HAIMKE are neither technology nor product, but demos.”
So what inspired them to come up with a writing productivity tool as a use case. Goshen said, “The way we write hasn’t drastically changed since Microsoft Word in 1980; and the way we read hasn’t really changed since the printing press in 1440. We believe both can be made AI-first experiences, with the machine becoming a true writing and reading partner. Wordtune is a first step in that journey.”
Your information will not be shared with third parties. No Spam.