WEBVTT

00:00:00.500 --> 00:00:03.020
In order to build
circuits to manipulate,

00:00:03.020 --> 00:00:06.030
transmit or store information,
we are going to need some

00:00:06.030 --> 00:00:08.700
engineering tools to see
if we're choosing a good

00:00:08.700 --> 00:00:10.730
representation for
the information --

00:00:10.730 --> 00:00:13.660
that's the subject
of this chapter.

00:00:13.660 --> 00:00:16.520
We'll study different ways of
encoding information as bits

00:00:16.520 --> 00:00:18.780
and learn the mathematics
that help us determine

00:00:18.780 --> 00:00:20.970
if our encoding is a good one.

00:00:20.970 --> 00:00:24.800
We'll also look into what we can
do if our representation gets

00:00:24.800 --> 00:00:26.380
corrupted by errors.

00:00:26.380 --> 00:00:28.890
It would be nice to detect
that something bad has happened

00:00:28.890 --> 00:00:32.659
and possibly even
correct the problem.

00:00:32.659 --> 00:00:35.860
Let's start by asking
"what is information?"

00:00:35.860 --> 00:00:37.370
From our engineering
perspective,

00:00:37.370 --> 00:00:40.590
we'll define information as
data communicated or received

00:00:40.590 --> 00:00:43.230
that resolves uncertainty
about a particular fact

00:00:43.230 --> 00:00:44.660
or circumstance.

00:00:44.660 --> 00:00:47.060
In other words, after
receiving the data

00:00:47.060 --> 00:00:49.190
we'll know more about
that particular fact

00:00:49.190 --> 00:00:50.900
or circumstance.

00:00:50.900 --> 00:00:53.380
The greater the uncertainty
resolved by the data,

00:00:53.380 --> 00:00:55.534
the more information
the data has conveyed.

00:00:57.150 --> 00:00:59.040
LET'S LOOK AT AN
EXAMPLE: a card has

00:00:59.040 --> 00:01:01.340
been chosen at random
from a normal deck of 52

00:01:01.340 --> 00:01:02.940
playing cards.

00:01:02.940 --> 00:01:05.050
Without any data
about the chosen card,

00:01:05.050 --> 00:01:08.450
there are 52 possibilities
for the type of the card.

00:01:08.450 --> 00:01:12.220
Now suppose you receive one of
the following pieces of data

00:01:12.220 --> 00:01:13.720
about the choice.

00:01:13.720 --> 00:01:17.470
A. You learn the suit
of the card is "heart".

00:01:17.470 --> 00:01:21.440
This narrows the choice
to down one of 13 cards.

00:01:21.440 --> 00:01:25.610
B. You learn instead the card
is NOT the Ace of Spades.

00:01:25.610 --> 00:01:29.080
This still leaves 51
cards that it might be.

00:01:29.080 --> 00:01:32.740
C. You learn instead that
the card is a face card, that

00:01:32.740 --> 00:01:35.140
is, a Jack, Queen or King.

00:01:35.140 --> 00:01:38.050
So the choice is
one of 12 cards.

00:01:38.050 --> 00:01:42.190
D. You learn instead that the
card is the "suicide king."

00:01:42.190 --> 00:01:44.370
Our little blue friend is
showing us that this is

00:01:44.370 --> 00:01:47.420
actually a particular card – the
King of Hearts where the king

00:01:47.420 --> 00:01:49.910
is sticking the sword
through his head!

00:01:49.910 --> 00:01:54.690
No uncertainty here, we know
exactly what the choice was.

00:01:54.690 --> 00:01:56.650
Which of the possible
pieces of data

00:01:56.650 --> 00:01:58.900
conveys the most information?

00:01:58.900 --> 00:02:02.070
In other words, which data
resolves the most uncertainty

00:02:02.070 --> 00:02:04.110
about the chosen card?

00:02:04.110 --> 00:02:06.930
Similarly, which data
conveys the least

00:02:06.930 --> 00:02:08.600
amount of information?

00:02:08.600 --> 00:02:10.280
Before we talk about
the mathematics

00:02:10.280 --> 00:02:12.690
behind the correct answers
to these questions,

00:02:12.690 --> 00:02:15.910
vote for your answers
in the following poll.