5.3 Man vs. Machine: How IBM Built a Jeopardy Champion

Course Info

Instructor

Prof. Dimitris Bertsimas

Departments

Sloan School of Management

As Taught In

Spring 2017

Level

Graduate

Topics

Learning Resource Types

Lecture Videos

Lecture Notes

Problem Sets with Solutions

Download Course

5 Text Analytics

Video 1: IBM Watson

View video page

The slides from all videos in this Lecture Sequence can be downloaded here: How IBM Built a Jeopardy Champion (PDF - 1.1MB).

Quick Question

What were the goals of IBM when they set out to build Watson? Select all that apply.

To build a computer that could compete with the best human players at Jeopardy!. check

To build a computer for Jeopardy! that could be used as a contestant on every show. close

To build a computer that could generate questions on Jeopardy!. close

To build a computer that could answer questions that are commonly believed to require human intelligence. check

Explanation The main goals of IBM were to build a computer that could answer questions that are commonly believed to require human intelligence, and to therefore compete with the best human players at *Jeopardy!*.

Continue: Video 2: The Game of Jeopardy

Quick Question

For which of the following reasons is Jeopardy! challenging? Select all that apply.

A wide variety of categories. check

Expert knowledge is required in a few specific categories. close

Speed is required - you have to buzz in faster than your competitors. check

The categories and clues are often cryptic. check

Explanation *Jeopardy!* is challenging because there are a wide variety of categories, speed is required, and the categories and clues are cryptic. Expert knowledge is not generally required.

Quick Question

Which of the following two questions do you think would be easiest for a computer to answer?

Was Abraham Lincoln generally considered a happy man? close

What year was Abraham Lincoln born? check

Explanation The second question would be the easiest, because the answer is a fact. The first question is much more subjective.

Quick Question

Select the LAT of the following Jeopardy question: NICHOLAS II WAS THE LAST RULING CZAR OF THIS ROYAL FAMILY (Hint: The answer is “The Romanovs”)

NICHOLAS II close

THE LAST RULING CZAR close

THIS ROYAL FAMILY check

Explanation The LAT in the first question is "THIS ROYAL FAMILY" and the LAT in the second question is "THIS DEVICE". Remember that if you replace the LAT with the correct answer, the sentence should make sense.

Select the LAT of the following Jeopardy question: REGARDING THIS DEVICE, ARCHIMEDES SAID, “GIVE ME A PLACE TO STAND ON, AND I WILL MOVE THE EARTH” (Hint: The answer is “A lever”)

THIS DEVICE check

ARCHIMEDES close

A PLACE close

THE EARTH close

Quick Question

To predict which candidate answer is correct, we said that Watson uses logistic regression. Which of the other techniques that we have learned could be used instead? Select all that apply.

Linear Regression close

CART check

Random Forests check

Explanation CART and Random Forests are both techniques that are also used for classification, and could provide confidence probabilities.

Video 2: The Game of Jeopardy

View video page

Errata: Only the first two rounds of Jeopardy have six categories, each with five questions. Final Jeopardy consists of only one question, with the players wagering a dollar amount up to their current winnings.

Video 3: Watson’s Database and Tools

View video page

Video 4: How Watson Works - Steps 1 and 2

View video page

Video 5: How Watson Works - Steps 3 and 4

View video page

Video 6: The Results

View video page

The figure in this video of Watson’s progress comes from the academic paper “Building Watson: An Overview of the DeepQA Project”, by D. Ferrucci, E. Brown, J. Chu-Carroll, J. Fan, D. Gondek, A. A. Kalyanpur, A. Lally, J. W. Murdock, E. Nyberg, J. Prager, N. Schlaefer, and C. Welty, Association for the Advancement of Artificial Intelligence, 2010.

Back: Quick Question