WEBVTT
00:00:01.290 --> 00:00:06.490
Given a probability, p, of an event occurring,
we know that the amount of information that
00:00:06.490 --> 00:00:13.760
you get from being told that a particular
event occurred is log2(1/p).
00:00:13.760 --> 00:00:20.670
Using that information, let's try to answer
the following questions.
00:00:20.670 --> 00:00:26.480
Someone picks a name out of a hat known to
contain the names of 5 women and 3 men, and
00:00:26.480 --> 00:00:29.110
tells you a man has been selected.
00:00:29.110 --> 00:00:32.800
How much information have they given you about
the selection?
00:00:32.800 --> 00:00:36.890
The probability of a man being selected in
this problem is 3/8.
00:00:36.890 --> 00:00:45.430
So the amount of information that you have
been given
00:00:45.430 --> 00:00:48.370
is log2(1/(3/8)) = log2(8/3).
00:00:48.370 --> 00:00:53.150
Another way to think about this is that if
you start off with N choices, and the additional
00:00:53.150 --> 00:00:58.480
information that you are given reduces your
choices to M, then the amount of information
00:00:58.480 --> 00:01:03.640
that you have been given is log2(N/M).
00:01:03.640 --> 00:01:09.150
In this example N = 8 because that is the
original number of choices.
00:01:09.150 --> 00:01:14.570
The reduced number of choices once you are
told that a man has been selected is 3, so
00:01:14.570 --> 00:01:16.400
M = 3.
00:01:16.400 --> 00:01:27.820
This means that the amount of information
you received is log2(8/3).
00:01:27.820 --> 00:01:33.160
For the second question, you are asked to
guess a random 4-bit 2's complement number.
00:01:33.160 --> 00:01:37.000
I then tell you that the number is greater
than 0.
00:01:37.000 --> 00:01:39.930
How much information have you been given?
00:01:39.930 --> 00:01:45.320
For this problem the original number of choices
is the number of permutations of 4 binary
00:01:45.320 --> 00:01:51.100
bits which is 16, so N = 16.
00:01:51.100 --> 00:01:56.729
Once you are told that the number is greater
than 0, this reduces your choice to 7, so
00:01:56.729 --> 00:01:58.870
M = 7.
00:01:58.870 --> 00:02:05.180
That means that the amount of information
that you have been given is log2(16/7).