We may hope that machines will eventually compete with men in all purely intellectual fields. –Alan Turing, 1950
In 1950, Alan Turing theorized a test of computer intelligence. The experiment he imagined, soon after coined the “Turing test,” asks a subject to blindly converse with a machine and another human. If the machine can fool the questioner into thinking it is a machine, than it has achieved true intelligence. Because we judge the intelligence of other humans based on external indicators, it was only fair that we hold machines to the same standard. (For a more complex analysis of the test, its philosophical implications, and the prolific career of Alan Turing see this page).
On Tuesday, in round 2 in the Jeopardy! IBM Challenge, past champions Ken Jennings and Brad Rutter got crushed by room-size IBM supercomputer, Watson. Here is the second half of that show, including the Jeopardy round and lots of IBM material advertising for the greatness of the artificial intelligence software that is Watson:
http://www.youtube.com/watch?v=HR2_M8kL_3o
At times, the human contestants sound more like the computer than the other way around, as they rattle off humorless knee-jerk responses to trivial questions (at least when they could get a word in edgewise against Watson). Yet, there have been plenty of situations in which the “computerness” of Watson shines through. It (he?) has generally been struggling with art-related clues, and sometimes repeats incorrect answers (IBM needs to work on his group listening skills!). In Final Jeopardy on Tuesday, it made a common-sense mistake handling this question: Its largest is named for a World War II hero; its second largest for a World War II battle? It gave “What is Toronto?????,” which is obviously wrong because it is not in the U.S. At least it seemed to recognize the answer was a long shot, wagering only a small portion of its winnings on the answer (the closest thing I’ve seen to a computer acknowledging that it is pushing its luck). The two humans correctly identified Chicago.
Last night, Watson took home the $1 million prize for IBM by racking up a final score of $77,147, vs. Jennings’ $24,000 and Rutter’s $21,600. I just discoverd an interactive game on nyt.com that allows you to play against Watson and see a bit how the processing works. Check it out! Sometime today, IBM is expected to announce a collaboration with Columbia University and the University of Maryland to develop a “physician’s assistant service.”
Of course a computer beating human competitors at Jeopardy does not mean it has passed a Turing Test. So, let’s turn to another case where machine is mimicking human. Last fall, the New York Times covered the story of Statsheet.com in an article titled, “When the Software is the Sportswriter.” What is unique about Statsheet.com is that it allows college basketball and football fans to call up not just stats on their favorite team, but also a write-up describing past performance and predictions for upcoming games. If I want to get the latest on Michigan State’s basketball team, I can either go to this page with stats, schedules, etc., or I can go read The Spartan Ball and see the latest headlines. The catch is that those headlines, and the short blurbs that run beneath them, are entirely computer generated. In most cases, the algorithm that the site uses is fairly transparent: “Michigan State Drops One To #2 Ranked Ohio State, 71-61. / Following the Penn State win with a loss, Michigan State is struggling. In Columbus on February 15th the Spartans were beaten by the Buckeyes, 71-61, in Big Ten play.” If you click a level deeper, you tend to get bullet lists of stat-heavy commentary; e.g., “Prior to this game, MSU played 62 Top 25 opponents in the last 5 seasons with a record of 30-32.” One clever twist on the site is that descriptions of results are phrased in more positive terms when you are on the home page for that team. For instance, when I switch over to the (Ohio State) Buckeye Beat, I see this headline for yesterday’s game: “Ohio State Gets the Win Over Michigan State, 71-61.” By my judgment, Statsheet.com passes the Turing test only if compared to the worst kind of sports writer, one who communicates solely with statistics strung together between cliches.
That’s Turing test failure number two. Where does that leave us? I propose that the next frontier is a robo-stand-up-comedian. I read statsheet.com for over 20 minutes and didn’t laugh a single time. Only a human statistician could come up with this one: “A historian, an engineer and a statistician are duck hunting. a duck rises from the lake. The historian fires first, and shoots 10′ over the duck. Then the engineer shoulders the shotgun and shoots 10′ under the duck. The statistician exclaims ‘got him!’”