Beyond enterprise AI platforms, the world of horizontal AI (e.g., computer vision, natural language processing or NLP, and voice) continues to be incredibly vibrant. So said Matt Turck, Managing Director at New York-based venture capital firm, FirstMark.

In an extensive report on Major Trends in the 2019 Data & AI Landscape, Turk pointed out that key trends in AI include major improvements in NLP, particularly through the application of transfer learning.

Venture capitalists Nathan Benaich and Ian Hogarth agree. Benaich is the founder of VC firm Air Street Capital. Hogarth is a UK-based angel investor. In their 130-slide State of Artificial Intelligence (AI) report published on June 28, 2019, Benaich and Hogarth highlighted recent research breakthroughs in NLP, such as Google AI’s BERT, Allen Institute’s ELMo, OpenAI’s GPT-2 Transformer, and ULMFiT.

According to the report, these language models are typically trained on very large amounts of publicly available, unlabeled text from the Internet; and they learn via transfer learning. That is, knowledge gained from one problem is stored and applied to a different or related problem, “thereby reducing the need for additional training data and compute,” wrote the duo. Such pretrained language models can substantially improve NLP performance.

Slator 2019 Language Industry Market Report

Data and Research

33 pages. Total market size, key verticals, services & tech landscape, market share by segment, M&A, and outlook.

$385 BUY NOW

Among Benaich and Hogarth’s six predictions for AI over the next year, one concerns NLP. The pair see a wave of new startups that will apply recent breakthroughs from NLP research and collectively raise over USD 100m in the next 12 months.

Another VC firm, MMC Ventures, concurs. In a comprehensive presentation entitled The State of AI: Divergence, MMC said, “transfer learning-powered models are improving the state of the art in language processing in areas of universal utility — text classification, summation, text generation, question answering and sentiment analysis.” MMC, too, flags BERT, ELMo, OpenAI, and ULMFiT as the most important recent NLP breakthroughs.

When Google decided to open-source BERT in November 2018, it claimed “anyone in the world can train their own state-of-the-art question answering system (or a variety of other models) in about 30 minutes on a single Cloud TPU, or in a few hours using a single GPU.”

OpenAI, which counts Elon Musk among its founders, released a new language model called GPT-2 in February 2019. The non-profit AI research company claimed GPT-2 could “predict” what the next words of a user would be after letting them input a sentence or phrase. Apparently, the system was so convincingly good that it had to be pulled out of the public domain for fear of it being misused.

Benaich and Hogarth also see potential dangers of recent NLP research breakthroughs. They point out that NLP could be weaponized because as “machines get better at reading and writing there is increasing scope for fraud (scalable ‘spearfishing’ attacks over email for example) and computational propaganda.”

MMC Ventures, meanwhile, highlights two far less sinister use cases for NLP. According to the VC, in customer service, “chatbots using natural language processing can offer 24/7 product information and answers to policyholders’ enquiries in a scalable, inexpensive and personalised channel.”

In the legal space (case law, discovery, due diligence), NLP “can identify, classify and utilise content from databases and unstructured documents at scale and speed, saving legal firms time and cost for document review,” MMC said.

Conspicuous by its absence in any of the three reports is one of NLP’s main uses cases: machine translation. Only Benaich and Hogarth pluck out a single Facebook NMT paper on leveraging monolingual data to make machine translation more widely applicable.

At SlatorCon San Francisco 2019, natural language processing will take center stage.

• Nvidia Senior Deep Learning Engineer, Chip Huyen, will discuss the latest advances in NLP and MT.
• Appen CEO Mark Brayan will talk about how to make language-based AI solutions work in the real world.
• Venture investors Tomasz Tunguz (Partner at Redpoint Ventures, an investor in Lilt) and Sri Chandrasekar (Partner at Point72 Ventures, lead investor in PolyAI’s Series A) will join the Institutional Investors panel discussion.