Six questions to ask yourself when reading about AI

• Updated on: 2019-09-13
• Read original article here

Hardly a week goes by without some breathless bit of AI news touting a “major” new discovery or warning us we are about to lose our jobs to the newest breed of smart machines.

Rest easy. As two scientists who have spent our careers studying AI, we can tell you that a large fraction of what’s reported is overhyped.

Consider this pair of headlines from last year describing an alleged breakthrough in machine reading: “Robots Can Now Read Better than Humans, Putting Millions of Jobs at Risk” and “Computers Are Getting Better than Humans at Reading.” The first, from Newsweek, is a more egregious exaggeration than the second, from CNN, but both wildly oversell minor progress.

To begin with, there were no actual robots involved, and no actual jobs were remotely at risk. All that really happened was that Microsoft made a tiny bit of progress and put out a press release saying that “AI…can read a document and answer questions about it as well as a person.”

That sounded much more revolutionary than it really was. Dig deeper, and you would discover that the AI in question was given one of the easiest reading tests you could imagine—one in which all of the answers were directly spelled out in the text. The test was about highlighting relevant words, not comprehending text.

Suppose, for example, that I hand you a piece of paper with this short passage:

The Microsoft system was built to answer questions like “Who went for a walk?” in which the answer (“Chloe and Alexander”) is directly spelled out in the text. But if you were to ask it a simple question like “Did Chloe see the cat?” (which she must have, because she went to pet it) or “Was Chloe frightened by the cat?” (which she must not have been, because she went to pet it), it would not have been able to find the answers, as they weren’t spelled out in the text. Inferring what isn’t said is at the heart of reading, and it simply wasn’t tested.

Microsoft didn’t make that clear, and neither did Newsweek or CNN.

Practically every time one of the tech titans puts out a press release, we get a reprise of this same phenomenon: minor progress portrayed as revolution. In another example, two years ago Facebook created a proof-of-concept AI program that could read a five-line summary of The Lord of the Rings and answer questions about where people and things ended up. (“Where was the Ring? At Mount Doom.”) The result was a slew of ridiculously over-enthusiastic articles, explaining how reading fantasy literature was the key to getting AI’s to read, with headlines like Slate’s “Facebook Thinks It Has Found the Secret to Making Bots Less Dumb.” (They didn’t.)

The consequence of this kind of over-reporting in the media? The public has come to believe that AI is much closer to being solved than it really is. So whenever you hear about a supposed success in AI, here’s a list of six questions you should ask:

AI really is coming, eventually, but it is further away than most people think. To get a realistic picture, take everything you read about dramatic progress in AI with a healthy dose  of skepticism—and rejoice in your (for now) uniquely human ability to do so.

This essay is adapted fromRebooting AI,published this month by Pantheon.