NLG in Journalism

When we think of AI our minds lean towards robotic assistants, self driving cars, and whatever Boston Dynamics are currently working on. However, look closely, towards the back of the room, just behind Alexa’s discarded human form and you’ll notice a quiet, but fascinating area of AI, ironically overlooked in the news.

Natural Language Generation in journalism has been making waves for the past few years. In 2018, a Natural Language Generating AI named Tobi was responsible for reporting on the results of Swiss referendums for 2,222 municipalities. Similarly, in 2017, Press Association won a £621,000 grant to start producing automated regional news. They partnered up with Urbs Media and are working towards producing up to 30,000 local news stories every month. This is currently the major strength of Automated Journalism – the ability to produce huge quantities of news stories out of structured data.

The current limitations, however, are fairly obvious. There is no conscious writer behind these articles, but rather a series of templates and decision trees. It is in fact difficult in many cases to justify the descriptor “AI” when it comes to these kinds of techniques and software, or perhaps we just give too much credit because of the grandiose associations of the word AI.

This is changing as more advanced machine learning and NLG technologies begin to surface. What it takes to make a machine write, or talk, like a human is becoming clearer, and it may not be so long until AI journalists are producing more than just templates and statistics. Moving forward, however, will require a fundamentally different approach.

Research scientist, and NLG specialist, Boris Katz explains in an interview with MIT Tech Review that “If you look at machine-learning advances, all the ideas came 20 to 25 years ago.” And in regards to our apparent cutting edge technologies such as Virtual Assistants, they’re doing little more than “counting words and numbers.”

In many instances the progression of other technologies creates an illusion that we are on the cusp of smart conversational NLG. Massive strides in Natural Language Processing, and voice recognition, for example, have resulted in software which is incredible at listening and interpreting, such as Alexa, but when it comes to speaking, little more than canned responses, and google search results can be produced.

Despite the difficulties in AI and automated journalism, interest in the field remains very high. The Computation + Journalism conference in Miami is testament to this, drawing many industry professionals and academics from around the world. Talks this year covered subjects from the ethics of data scraping, to micro audiences, to AI fact checking. While these subjects are being explored academically, there remains no focal point in the industry in terms of real technological solutions. Machine learning techniques have not been capable of producing sophisticated automated journalism, and traditional techniques lack the complexity and reasoning required. There is a gap wide open for an emerging technology or solution, and it is only a matter of time until it is filled.