WEBVTT 00:00:00.000 --> 00:00:01.455 align:middle line:90% [SQUEAKING] 00:00:01.455 --> 00:00:03.395 align:middle line:90% [RUSTLING] 00:00:03.395 --> 00:00:07.760 align:middle line:90% [CLICKING] 00:00:07.760 --> 00:00:13.242 align:middle line:90% 00:00:13.242 --> 00:00:14.700 align:middle line:84% ANKUR MOITRA: All right, everybody. 00:00:14.700 --> 00:00:16.480 align:middle line:90% Welcome back to class. 00:00:16.480 --> 00:00:18.860 align:middle line:84% So today we're going to pick up where we left off. 00:00:18.860 --> 00:00:21.680 align:middle line:84% So remember, last time we talked about the pigeonhole principle. 00:00:21.680 --> 00:00:24.520 align:middle line:84% I showed you a whole bunch of applications of it. 00:00:24.520 --> 00:00:26.665 align:middle line:84% And then we started this longer topic 00:00:26.665 --> 00:00:28.540 align:middle line:84% that's going to take us a couple of lectures, 00:00:28.540 --> 00:00:30.920 align:middle line:90% which is probability. 00:00:30.920 --> 00:00:35.040 align:middle line:84% So just a quick recap of where we left off last time. 00:00:35.040 --> 00:00:37.960 align:middle line:84% So last time I started to introduce the basic terminology 00:00:37.960 --> 00:00:39.960 align:middle line:90% that we need for probability. 00:00:39.960 --> 00:00:41.800 align:middle line:84% We had these key definitions, like we 00:00:41.800 --> 00:00:45.000 align:middle line:84% had the notion of a sample space, which is really 00:00:45.000 --> 00:00:48.280 align:middle line:84% just telling you what are all the possible outcomes 00:00:48.280 --> 00:00:49.800 align:middle line:90% of your experiment. 00:00:49.800 --> 00:00:52.680 align:middle line:84% You could think about something very small and finite, 00:00:52.680 --> 00:00:56.720 align:middle line:84% like I roll a die and I get a number between 1 and 6. 00:00:56.720 --> 00:00:58.720 align:middle line:84% Or you could even think about a sample space 00:00:58.720 --> 00:01:01.210 align:middle line:90% as being an infinite space. 00:01:01.210 --> 00:01:03.650 align:middle line:84% For purposes of our class, we're only 00:01:03.650 --> 00:01:06.650 align:middle line:84% going to consider discrete sample spaces, 00:01:06.650 --> 00:01:08.690 align:middle line:84% but you can even consider sample spaces 00:01:08.690 --> 00:01:12.610 align:middle line:84% like what happens if I take a coin and I flip it as many times 00:01:12.610 --> 00:01:14.810 align:middle line:90% until I get my first heads? 00:01:14.810 --> 00:01:17.850 align:middle line:84% That's something which has an infinite sample space. 00:01:17.850 --> 00:01:20.330 align:middle line:84% And what's important is that to the sample space 00:01:20.330 --> 00:01:24.170 align:middle line:84% we associate what's called a probability measure P. 00:01:24.170 --> 00:01:27.890 align:middle line:84% So all this does is it takes in some particular outcome 00:01:27.890 --> 00:01:30.530 align:middle line:84% from the range of all possible outcomes. 00:01:30.530 --> 00:01:34.050 align:middle line:84% It better assign a non-negative probability to that. 00:01:34.050 --> 00:01:36.010 align:middle line:84% And additionally, your probability measure 00:01:36.010 --> 00:01:38.930 align:middle line:84% had better be normalized so that the sum of p 00:01:38.930 --> 00:01:41.890 align:middle line:84% of x over your entire sample space 00:01:41.890 --> 00:01:44.690 align:middle line:84% had better be normalized so that it's equal to 1, 00:01:44.690 --> 00:01:47.142 align:middle line:84% so that these probabilities make sense. 00:01:47.142 --> 00:01:48.850 align:middle line:84% In fact, there are a lot of distributions 00:01:48.850 --> 00:01:51.370 align:middle line:84% that we're going to see over and over again. 00:01:51.370 --> 00:01:54.250 align:middle line:84% One of the simplest and most ubiquitous distributions 00:01:54.250 --> 00:01:56.810 align:middle line:84% is called the uniform distribution. 00:01:56.810 --> 00:02:00.660 align:middle line:84% So here, we have some finite sample space S. 00:02:00.660 --> 00:02:03.140 align:middle line:84% And then we just look at the probability measure 00:02:03.140 --> 00:02:05.780 align:middle line:84% that assigns the same fixed probability 00:02:05.780 --> 00:02:08.060 align:middle line:90% to every possible outcome. 00:02:08.060 --> 00:02:10.759 align:middle line:84% So in particular, in order for it to be normalized, 00:02:10.759 --> 00:02:15.900 align:middle line:84% each of those probabilities had better be 1 over the size of S. 00:02:15.900 --> 00:02:17.760 align:middle line:84% So this is where we left off last time. 00:02:17.760 --> 00:02:21.500 align:middle line:84% We also defined this notion of what an event was. 00:02:21.500 --> 00:02:23.780 align:middle line:84% Sometimes we're not interested in one 00:02:23.780 --> 00:02:27.420 align:middle line:84% particular outcome, but really a collection of outcomes. 00:02:27.420 --> 00:02:29.260 align:middle line:84% Something like when I take a die and I 00:02:29.260 --> 00:02:33.220 align:middle line:84% roll it and I get an even number, that's not just any one 00:02:33.220 --> 00:02:37.700 align:middle line:84% atomic outcome, but it's really three outcomes for 2, 4, and 6. 00:02:37.700 --> 00:02:40.300 align:middle line:84% And we talk about the probability of an event 00:02:40.300 --> 00:02:42.780 align:middle line:84% as really just being the sum of the probability 00:02:42.780 --> 00:02:48.420 align:middle line:84% measure over all of the outcomes that land in the subset T. 00:02:48.420 --> 00:02:51.420 align:middle line:84% So this is where we left off last time. 00:02:51.420 --> 00:02:54.540 align:middle line:84% And today, what we're really working towards 00:02:54.540 --> 00:02:56.190 align:middle line:84% is we're going to talk about something 00:02:56.190 --> 00:02:59.350 align:middle line:84% that's very important, but a little bit counterintuitive. 00:02:59.350 --> 00:03:02.670 align:middle line:84% We're going to talk about independence and conditioning. 00:03:02.670 --> 00:03:06.510 align:middle line:84% So what's important to build up to this is really the idea that 00:03:06.510 --> 00:03:09.350 align:middle line:84% we cannot just consider the probabilities of individual 00:03:09.350 --> 00:03:14.950 align:middle line:84% events, but we can combine them in interesting ways. 00:03:14.950 --> 00:03:20.310 align:middle line:84% So for example, if I give you two events, T1 and T2, 00:03:20.310 --> 00:03:23.410 align:middle line:84% I can think about the union of T1 and T2. 00:03:23.410 --> 00:03:25.310 align:middle line:84% What this means is I'm really just treating 00:03:25.310 --> 00:03:28.510 align:middle line:84% this as the set of outcomes I'm interested in. 00:03:28.510 --> 00:03:32.150 align:middle line:84% This is another set of possible outcomes I'm interested in. 00:03:32.150 --> 00:03:34.470 align:middle line:84% And I'm forming a new event that takes 00:03:34.470 --> 00:03:38.230 align:middle line:84% everything that happens in either one of these or maybe 00:03:38.230 --> 00:03:39.390 align:middle line:90% both. 00:03:39.390 --> 00:03:42.510 align:middle line:84% So in particular, in words, what does this new event 00:03:42.510 --> 00:03:45.070 align:middle line:90% that I've constructed mean? 00:03:45.070 --> 00:03:55.366 align:middle line:84% It's the event that at least one of these events, T1 or T2, 00:03:55.366 --> 00:03:55.866 align:middle line:90% happened. 00:03:55.866 --> 00:03:58.890 align:middle line:90% 00:03:58.890 --> 00:04:01.450 align:middle line:84% So that's one way to combine events. 00:04:01.450 --> 00:04:04.270 align:middle line:84% I could also think about combining events by, 00:04:04.270 --> 00:04:08.697 align:middle line:84% for example, taking T1 intersect T2. 00:04:08.697 --> 00:04:10.030 align:middle line:90% And then what am I going to get? 00:04:10.030 --> 00:04:13.650 align:middle line:84% I'm going to get a new event that's asking me whether both T1 00:04:13.650 --> 00:04:15.490 align:middle line:90% and T2 happened. 00:04:15.490 --> 00:04:20.190 align:middle line:84% And so this now leads to a very interesting concept, 00:04:20.190 --> 00:04:24.090 align:middle line:84% which is the notion of conditional probability. 00:04:24.090 --> 00:04:29.650 align:middle line:84% And I promise that this will sound very simple. 00:04:29.650 --> 00:04:32.170 align:middle line:84% But in the examples we'll see, some of them 00:04:32.170 --> 00:04:34.810 align:middle line:84% will become pretty counterintuitive. 00:04:34.810 --> 00:04:37.530 align:middle line:84% In fact, when you look, conditional probability 00:04:37.530 --> 00:04:39.970 align:middle line:84% is really in the world all around you 00:04:39.970 --> 00:04:41.470 align:middle line:90% in very important ways. 00:04:41.470 --> 00:04:44.290 align:middle line:84% We'll talk about some real world examples. 00:04:44.290 --> 00:04:46.150 align:middle line:84% So first, let me tell you what this means. 00:04:46.150 --> 00:04:48.570 align:middle line:84% Let me just give you the definition. 00:04:48.570 --> 00:04:51.130 align:middle line:84% We can also combine two events, A and B, 00:04:51.130 --> 00:04:56.660 align:middle line:84% in this interesting way, where we want to assert for a fact 00:04:56.660 --> 00:04:58.900 align:middle line:90% that event B really did happen. 00:04:58.900 --> 00:05:02.500 align:middle line:84% And we want to look at if I tell you that event B happened, 00:05:02.500 --> 00:05:06.820 align:middle line:84% what's the new probability that this other event A happened. 00:05:06.820 --> 00:05:10.500 align:middle line:84% So you can think about this in a few different ways. 00:05:10.500 --> 00:05:14.420 align:middle line:84% So in words, it's the probability 00:05:14.420 --> 00:05:24.608 align:middle line:84% of event A given or conditioned on that event B happens. 00:05:24.608 --> 00:05:28.160 align:middle line:84% See, one way to think about this is really in terms of pictures. 00:05:28.160 --> 00:05:31.060 align:middle line:90% 00:05:31.060 --> 00:05:34.460 align:middle line:84% So we could start off with our full sample space 00:05:34.460 --> 00:05:37.660 align:middle line:84% S. That's all of the different possibilities. 00:05:37.660 --> 00:05:41.940 align:middle line:84% We could take one event A right here. 00:05:41.940 --> 00:05:45.020 align:middle line:84% And what's going on is that our probability measure really 00:05:45.020 --> 00:05:48.820 align:middle line:84% describes a way of throwing a dart at this board. 00:05:48.820 --> 00:05:52.630 align:middle line:84% Now, this dart always lands inside this box S, 00:05:52.630 --> 00:05:56.150 align:middle line:84% because that's the range of all possible outcomes. 00:05:56.150 --> 00:05:57.870 align:middle line:84% And then the probability of the event A 00:05:57.870 --> 00:06:01.150 align:middle line:84% is just the probability that that dart lands inside this 00:06:01.150 --> 00:06:03.590 align:middle line:90% oval that I've drawn right here. 00:06:03.590 --> 00:06:05.270 align:middle line:84% So now when I have a pair of events 00:06:05.270 --> 00:06:09.350 align:middle line:84% and I have A and B right here, well, what I can do 00:06:09.350 --> 00:06:11.190 align:middle line:84% is imagine I do this experiment where 00:06:11.190 --> 00:06:13.190 align:middle line:90% I throw my dart at the board. 00:06:13.190 --> 00:06:16.070 align:middle line:84% And we already know it has to land inside the set 00:06:16.070 --> 00:06:21.310 align:middle line:84% S. But then what if I tell you that it's landed in this set B? 00:06:21.310 --> 00:06:24.190 align:middle line:84% And I promise you that event B happens. 00:06:24.190 --> 00:06:26.530 align:middle line:84% What I want to know is, given that promise, 00:06:26.530 --> 00:06:29.150 align:middle line:84% what's the new probability that it actually 00:06:29.150 --> 00:06:31.590 align:middle line:90% lands inside the set A? 00:06:31.590 --> 00:06:34.110 align:middle line:84% So the only place where it could then land 00:06:34.110 --> 00:06:37.630 align:middle line:84% and event A could happen is right here in the intersection. 00:06:37.630 --> 00:06:41.350 align:middle line:84% And that's exactly how we define the conditional probability. 00:06:41.350 --> 00:06:43.230 align:middle line:84% It's literally just the probability 00:06:43.230 --> 00:06:46.550 align:middle line:84% of the intersection of the two events divided 00:06:46.550 --> 00:06:49.840 align:middle line:84% by the probability that I'm conditioning on. 00:06:49.840 --> 00:06:51.320 align:middle line:90% And this part is intuitive. 00:06:51.320 --> 00:06:53.720 align:middle line:84% Imagine that I have the same chance 00:06:53.720 --> 00:06:58.280 align:middle line:84% of hitting every single point inside this rectangle. 00:06:58.280 --> 00:07:00.920 align:middle line:84% Then really, what's going on is that when I tell you 00:07:00.920 --> 00:07:04.520 align:middle line:84% that event B has happened, the probability of B 00:07:04.520 --> 00:07:09.040 align:middle line:84% is how I'm renormalizing my space of possible outcomes, 00:07:09.040 --> 00:07:13.640 align:middle line:84% because I know that now my dart is uniform over the set B. 00:07:13.640 --> 00:07:16.240 align:middle line:84% And what's important is not the amount of space 00:07:16.240 --> 00:07:19.160 align:middle line:84% that a takes up inside S, but the amount 00:07:19.160 --> 00:07:23.800 align:middle line:84% of space that A intersect B takes up inside B. 00:07:23.800 --> 00:07:24.820 align:middle line:90% So does this make sense? 00:07:24.820 --> 00:07:26.800 align:middle line:84% Are there any questions about the definition 00:07:26.800 --> 00:07:29.840 align:middle line:90% of conditional probability? 00:07:29.840 --> 00:07:31.080 align:middle line:90% Good? 00:07:31.080 --> 00:07:32.120 align:middle line:90% All right. 00:07:32.120 --> 00:07:35.040 align:middle line:84% So let's develop a few different ways to think 00:07:35.040 --> 00:07:36.660 align:middle line:90% about conditional probability. 00:07:36.660 --> 00:07:39.900 align:middle line:84% Let me actually give one other simple example, 00:07:39.900 --> 00:07:45.280 align:middle line:84% which is related to it, which will have its own quirks. 00:07:45.280 --> 00:07:50.170 align:middle line:84% The other crucial notion that I want to introduce today 00:07:50.170 --> 00:07:52.770 align:middle line:90% is the notion of independence. 00:07:52.770 --> 00:07:56.610 align:middle line:84% So we talked about two events, A and B. 00:07:56.610 --> 00:07:59.350 align:middle line:84% And there's this very important concept, 00:07:59.350 --> 00:08:02.090 align:middle line:84% which we're going to use all the time, which 00:08:02.090 --> 00:08:03.270 align:middle line:90% is called independence. 00:08:03.270 --> 00:08:09.130 align:middle line:84% So we say that two events A and B are 00:08:09.130 --> 00:08:21.290 align:middle line:84% independent if the probability of A intersect B 00:08:21.290 --> 00:08:27.330 align:middle line:84% is equal to the probability of A times the probability of B. 00:08:27.330 --> 00:08:29.770 align:middle line:84% So this is, in some sense, the simplest way 00:08:29.770 --> 00:08:32.370 align:middle line:84% that the probability of intersecting two events 00:08:32.370 --> 00:08:35.590 align:middle line:84% could work is that whether A happens, 00:08:35.590 --> 00:08:39.330 align:middle line:84% you think of it as it contributes this P of A. 00:08:39.330 --> 00:08:43.409 align:middle line:84% And whether B happens contributes this P of B. 00:08:43.409 --> 00:08:45.890 align:middle line:84% But in fact, just as an exercise, 00:08:45.890 --> 00:08:51.060 align:middle line:84% using our definition for conditional probability, 00:08:51.060 --> 00:08:55.220 align:middle line:84% we can see that independence is really equivalent to a much 00:08:55.220 --> 00:08:59.620 align:middle line:84% simpler thing, which is that probability of A given B 00:08:59.620 --> 00:09:04.300 align:middle line:84% is equal to the probability of A. 00:09:04.300 --> 00:09:07.860 align:middle line:84% So this comes just from dividing through by the probability of B 00:09:07.860 --> 00:09:10.980 align:middle line:84% And using my definition for conditional independence. 00:09:10.980 --> 00:09:13.840 align:middle line:84% So these are two completely equivalent conditions. 00:09:13.840 --> 00:09:15.460 align:middle line:90% You can check either one. 00:09:15.460 --> 00:09:19.180 align:middle line:84% And that's what it means for two things to be independent. 00:09:19.180 --> 00:09:23.540 align:middle line:84% So what's going on is that in my dartboard example in pictures, 00:09:23.540 --> 00:09:28.460 align:middle line:84% I might have my entire rectangle represents S. I might have 00:09:28.460 --> 00:09:32.420 align:middle line:84% this left half represents A. And maybe this bottom half 00:09:32.420 --> 00:09:35.500 align:middle line:84% represents B. And so what's going on 00:09:35.500 --> 00:09:38.880 align:middle line:84% is if I look at the probability of A, 00:09:38.880 --> 00:09:41.900 align:middle line:84% that's how much area this rectangle for A 00:09:41.900 --> 00:09:44.420 align:middle line:84% takes up out of the entire rectangle, 00:09:44.420 --> 00:09:46.120 align:middle line:84% but then when I condition on B, it 00:09:46.120 --> 00:09:48.640 align:middle line:84% doesn't change what that fraction is. 00:09:48.640 --> 00:09:52.120 align:middle line:84% So this is one way to think about what independence is. 00:09:52.120 --> 00:09:54.080 align:middle line:84% And I want to caution you that even though it's 00:09:54.080 --> 00:09:56.120 align:middle line:84% a very simple definition, it's actually very 00:09:56.120 --> 00:09:58.480 align:middle line:90% counterintuitive in some ways. 00:09:58.480 --> 00:10:00.880 align:middle line:84% There are easy pairs of events that 00:10:00.880 --> 00:10:03.902 align:middle line:84% are natural to think about why they're independent. 00:10:03.902 --> 00:10:05.360 align:middle line:84% And then there are some events that 00:10:05.360 --> 00:10:08.200 align:middle line:84% are independent that are very counterintuitive. 00:10:08.200 --> 00:10:10.600 align:middle line:84% So let's do the simple one first. 00:10:10.600 --> 00:10:13.320 align:middle line:84% Let's do some simple examples to get some intuition 00:10:13.320 --> 00:10:15.640 align:middle line:90% for independence. 00:10:15.640 --> 00:10:24.320 align:middle line:84% So imagine that we throw a dice twice. 00:10:24.320 --> 00:10:27.720 align:middle line:84% And this is a fair die, so it's the uniform distribution 00:10:27.720 --> 00:10:30.680 align:middle line:90% over all six possibilities. 00:10:30.680 --> 00:10:33.960 align:middle line:84% And when I look at the pair of throws, 00:10:33.960 --> 00:10:38.680 align:middle line:84% my sample space now has 36 possibilities. 00:10:38.680 --> 00:10:42.440 align:middle line:84% For every possibility for the first throw x1, 00:10:42.440 --> 00:10:45.650 align:middle line:84% every possibility for the second throw x2. 00:10:45.650 --> 00:10:49.050 align:middle line:84% So I look at all possibilities of x1 x2 00:10:49.050 --> 00:10:55.290 align:middle line:84% that are in this set 1, 2, up to 6. 00:10:55.290 --> 00:10:57.690 align:middle line:84% So this is my sample space, and everything is 00:10:57.690 --> 00:11:00.450 align:middle line:90% uniform over the sample space. 00:11:00.450 --> 00:11:03.010 align:middle line:84% All of the 36 possibilities for the choice 00:11:03.010 --> 00:11:06.290 align:middle line:84% of some number between 1 and 6, and some number between 1 and 6, 00:11:06.290 --> 00:11:08.890 align:middle line:90% are all equally likely. 00:11:08.890 --> 00:11:12.610 align:middle line:84% And now I can define some events. 00:11:12.610 --> 00:11:20.650 align:middle line:84% Let's look at events A, which is that x1 is odd. 00:11:20.650 --> 00:11:29.050 align:middle line:84% And let's look at the event that B, that x2 is at most 3. 00:11:29.050 --> 00:11:31.710 align:middle line:84% So probably you all have very good intuition for this. 00:11:31.710 --> 00:11:36.810 align:middle line:84% So who thinks these two events A and B are independent? 00:11:36.810 --> 00:11:37.730 align:middle line:90% OK. 00:11:37.730 --> 00:11:40.900 align:middle line:90% Who thinks they're not? 00:11:40.900 --> 00:11:44.140 align:middle line:90% Who is still tired? 00:11:44.140 --> 00:11:44.940 align:middle line:90% OK. 00:11:44.940 --> 00:11:45.640 align:middle line:90% All right. 00:11:45.640 --> 00:11:46.980 align:middle line:90% Good honesty. 00:11:46.980 --> 00:11:47.800 align:middle line:90% All right. 00:11:47.800 --> 00:11:50.120 align:middle line:90% So these two are independent. 00:11:50.120 --> 00:11:52.460 align:middle line:84% Now, one way to check independence 00:11:52.460 --> 00:11:55.260 align:middle line:84% is literally just to follow the definition. 00:11:55.260 --> 00:11:58.040 align:middle line:84% So when in doubt, just compute the thing. 00:11:58.040 --> 00:12:01.620 align:middle line:84% But we're going to see part of the intuition for why 00:12:01.620 --> 00:12:05.360 align:middle line:84% they're independents we'll see in the computation itself. 00:12:05.360 --> 00:12:08.260 align:middle line:84% So what does it mean to be independent? 00:12:08.260 --> 00:12:09.160 align:middle line:90% Let's check. 00:12:09.160 --> 00:12:14.060 align:middle line:84% Let's look at the probability of A and B. So in our sample space, 00:12:14.060 --> 00:12:16.380 align:middle line:90% there are 36 possibilities. 00:12:16.380 --> 00:12:18.540 align:middle line:84% How many possibilities meet the condition 00:12:18.540 --> 00:12:21.330 align:middle line:84% that both event A and event B hold? 00:12:21.330 --> 00:12:28.100 align:middle line:90% 00:12:28.100 --> 00:12:29.660 align:middle line:90% Not a hard question. 00:12:29.660 --> 00:12:30.940 align:middle line:90% Yeah? 00:12:30.940 --> 00:12:31.720 align:middle line:90% A quarter of them. 00:12:31.720 --> 00:12:32.220 align:middle line:90% Yeah. 00:12:32.220 --> 00:12:35.700 align:middle line:90% So let me just say 9 out of 36. 00:12:35.700 --> 00:12:37.660 align:middle line:90% Why is it 9? 00:12:37.660 --> 00:12:42.230 align:middle line:84% Because I have three possibilities for x1 is odd. 00:12:42.230 --> 00:12:43.830 align:middle line:90% 1, 3, 5. 00:12:43.830 --> 00:12:48.190 align:middle line:84% I have three possibilities for x2 is at most 3, 1, 2, 3. 00:12:48.190 --> 00:12:52.150 align:middle line:84% And every choice of one of the possibilities for A 00:12:52.150 --> 00:12:55.510 align:middle line:84% and every choice for one of the possibilities of B 00:12:55.510 --> 00:13:00.030 align:middle line:84% defines a possible outcome that satisfies both A and B. 00:13:00.030 --> 00:13:02.630 align:middle line:84% So they're independent because they have this kind of product 00:13:02.630 --> 00:13:06.167 align:middle line:84% structure that whatever happens for A, 00:13:06.167 --> 00:13:07.750 align:middle line:84% I can choose something there, and then 00:13:07.750 --> 00:13:10.350 align:middle line:84% I can define something that happens for B. 00:13:10.350 --> 00:13:12.130 align:middle line:84% So literally, if you think about it, 00:13:12.130 --> 00:13:14.590 align:middle line:84% it really represents this picture, if you 00:13:14.590 --> 00:13:16.710 align:middle line:90% think about it the right way. 00:13:16.710 --> 00:13:20.310 align:middle line:84% So in any case, we can check that this really 00:13:20.310 --> 00:13:25.870 align:middle line:84% does equal the probability of A times the probability of B. 00:13:25.870 --> 00:13:27.730 align:middle line:84% Because what's the probability of A? 00:13:27.730 --> 00:13:29.140 align:middle line:90% It's just 1/2. 00:13:29.140 --> 00:13:30.830 align:middle line:90% 1/2 of the roles are odd. 00:13:30.830 --> 00:13:32.830 align:middle line:90% What's the probability of B? 00:13:32.830 --> 00:13:36.970 align:middle line:84% It's 1/2 because 1/2 of them or 3 or less. 00:13:36.970 --> 00:13:37.470 align:middle line:90% So 00:13:37.470 --> 00:13:40.640 align:middle line:84% The way to think about it is that independence 00:13:40.640 --> 00:13:42.840 align:middle line:90% is sometimes intuitive. 00:13:42.840 --> 00:13:46.320 align:middle line:84% Because what's going on here is I could literally think 00:13:46.320 --> 00:13:49.120 align:middle line:90% about rolling two separate die. 00:13:49.120 --> 00:13:53.000 align:middle line:84% I roll one die here, and I care about what the roll is. 00:13:53.000 --> 00:13:55.200 align:middle line:84% And then you go to another room in building two 00:13:55.200 --> 00:13:56.920 align:middle line:90% and you roll another die. 00:13:56.920 --> 00:13:58.840 align:middle line:84% And they're independent, because we literally 00:13:58.840 --> 00:14:00.460 align:middle line:90% are not talking to each other. 00:14:00.460 --> 00:14:04.040 align:middle line:84% These events are just happening in different spaces. 00:14:04.040 --> 00:14:07.200 align:middle line:84% But independence is a bit more counterintuitive than that, 00:14:07.200 --> 00:14:09.400 align:middle line:84% because there are other ways that events 00:14:09.400 --> 00:14:12.000 align:middle line:84% can be independent that satisfy our definition, that 00:14:12.000 --> 00:14:14.200 align:middle line:90% aren't so easy to check. 00:14:14.200 --> 00:14:16.960 align:middle line:84% Let me also mention that a lot of times 00:14:16.960 --> 00:14:20.300 align:middle line:84% we use the shorthand, which just means that A 00:14:20.300 --> 00:14:24.440 align:middle line:90% and B are independent as events. 00:14:24.440 --> 00:14:24.940 align:middle line:90% All right. 00:14:24.940 --> 00:14:27.140 align:middle line:84% So let's go a little further with this example 00:14:27.140 --> 00:14:30.360 align:middle line:84% so that I can convince you that independence is subtle. 00:14:30.360 --> 00:14:33.960 align:middle line:84% So let's define a new event C, which 00:14:33.960 --> 00:14:40.810 align:middle line:84% is going to be the event x1 minus x2 is equal to 3. 00:14:40.810 --> 00:14:43.090 align:middle line:84% Now, this is a bit stranger of an event, 00:14:43.090 --> 00:14:47.850 align:middle line:84% because it involves both x1 and x2. 00:14:47.850 --> 00:14:51.370 align:middle line:84% Now, what are the possible outcomes in the sample space 00:14:51.370 --> 00:14:53.390 align:middle line:90% where event C happens? 00:14:53.390 --> 00:14:56.650 align:middle line:90% Can anyone list them for me? 00:14:56.650 --> 00:14:59.013 align:middle line:90% STUDENT: [INAUDIBLE] 00:14:59.013 --> 00:14:59.930 align:middle line:90% ANKUR MOITRA: Perfect. 00:14:59.930 --> 00:15:02.250 align:middle line:84% And those are the only possibilities. 00:15:02.250 --> 00:15:04.650 align:middle line:84% So another way to define this event 00:15:04.650 --> 00:15:06.930 align:middle line:84% is really as the subset of the sample 00:15:06.930 --> 00:15:10.490 align:middle line:84% space, which we're interested in, which is the possibility 4, 00:15:10.490 --> 00:15:11.490 align:middle line:90% 1. 00:15:11.490 --> 00:15:13.890 align:middle line:90% That's the possibility 5, 2. 00:15:13.890 --> 00:15:17.450 align:middle line:84% And lastly, we have the possibility 6, 3. 00:15:17.450 --> 00:15:19.712 align:middle line:84% That's the only subset of the sample space 00:15:19.712 --> 00:15:20.670 align:middle line:90% that I'm interested in. 00:15:20.670 --> 00:15:22.770 align:middle line:84% Those are the only things that can happen where 00:15:22.770 --> 00:15:26.650 align:middle line:90% this event C is on, is true. 00:15:26.650 --> 00:15:28.790 align:middle line:84% And we can look at the same type of thing. 00:15:28.790 --> 00:15:32.330 align:middle line:84% Let's look at our definition for independence. 00:15:32.330 --> 00:15:36.310 align:middle line:84% Let's look at the probability that A and C see both happen. 00:15:36.310 --> 00:15:39.470 align:middle line:90% So what was A? x1 is odd. 00:15:39.470 --> 00:15:44.430 align:middle line:84% And there's only one possibility on my list where x1 is odd 00:15:44.430 --> 00:15:46.190 align:middle line:90% and C happens. 00:15:46.190 --> 00:15:51.470 align:middle line:84% So this means the probability of my intersection is 1 over 36. 00:15:51.470 --> 00:15:56.190 align:middle line:84% Unfortunately, this is not equal to the probability of A times 00:15:56.190 --> 00:16:00.310 align:middle line:84% the probability of C. We know the probability of A is 1/2, 00:16:00.310 --> 00:16:04.370 align:middle line:84% and we know the probability of C is 3 over 36, 00:16:04.370 --> 00:16:07.030 align:middle line:84% just based on our description of the event. 00:16:07.030 --> 00:16:11.710 align:middle line:84% So these two events A and C are not independent. 00:16:11.710 --> 00:16:14.790 align:middle line:90% A is not independent from C. 00:16:14.790 --> 00:16:18.230 align:middle line:84% And maybe this is intuitive too, because these two events 00:16:18.230 --> 00:16:20.230 align:middle line:90% depend on the same things. 00:16:20.230 --> 00:16:24.630 align:middle line:84% They both depend on what happens for roll one. 00:16:24.630 --> 00:16:28.910 align:middle line:84% But what I claim is that there are slight changes to event C 00:16:28.910 --> 00:16:31.950 align:middle line:84% that actually make these things be independent. 00:16:31.950 --> 00:16:35.200 align:middle line:84% Let's define a fourth and final event. 00:16:35.200 --> 00:16:37.040 align:middle line:84% Let's look at this event D. That's 00:16:37.040 --> 00:16:42.720 align:middle line:84% x1 minus x2 is equal to 4 instead of 3. 00:16:42.720 --> 00:16:46.200 align:middle line:84% Now we have only two possibilities. 00:16:46.200 --> 00:16:50.280 align:middle line:84% We have the possibility 5, 1, and we have the possibility 6, 00:16:50.280 --> 00:16:50.960 align:middle line:90% 2. 00:16:50.960 --> 00:16:56.240 align:middle line:84% And we can just check our definition for independence. 00:16:56.240 --> 00:17:00.460 align:middle line:84% So the probability of A and D. Well, 00:17:00.460 --> 00:17:02.780 align:middle line:84% there's only one thing that survives. 00:17:02.780 --> 00:17:07.720 align:middle line:84% So my probability of the intersection is still 1 over 36. 00:17:07.720 --> 00:17:09.960 align:middle line:84% And you can check that this really is equal 00:17:09.960 --> 00:17:12.720 align:middle line:84% to the probability of A, which is 1/2, 00:17:12.720 --> 00:17:18.240 align:middle line:84% times the probability of D, which is 2 over 36. 00:17:18.240 --> 00:17:21.839 align:middle line:84% So this is an example which would be a little bit hard 00:17:21.839 --> 00:17:25.920 align:middle line:84% to guess unless you literally did the calculation, because it 00:17:25.920 --> 00:17:30.320 align:middle line:84% doesn't look like they're independent, but they still are. 00:17:30.320 --> 00:17:33.210 align:middle line:84% So are there any questions about independence? 00:17:33.210 --> 00:17:35.530 align:middle line:90% Does this make sense? 00:17:35.530 --> 00:17:36.610 align:middle line:90% OK. 00:17:36.610 --> 00:17:37.650 align:middle line:90% All right. 00:17:37.650 --> 00:17:41.170 align:middle line:90% So let's go a little further. 00:17:41.170 --> 00:17:44.410 align:middle line:84% I want to tell you about some more powerful tools 00:17:44.410 --> 00:17:48.170 align:middle line:84% in probability that play very nicely with conditioning 00:17:48.170 --> 00:17:50.770 align:middle line:90% and independence. 00:17:50.770 --> 00:17:52.890 align:middle line:84% Let me tell you about this basic fact 00:17:52.890 --> 00:17:55.390 align:middle line:84% that we're going to use all over the place. 00:17:55.390 --> 00:17:58.130 align:middle line:84% And then I'll give you a way to visualize it. 00:17:58.130 --> 00:18:02.470 align:middle line:84% This fact is called the law of total probability. 00:18:02.470 --> 00:18:09.770 align:middle line:90% 00:18:09.770 --> 00:18:12.850 align:middle line:84% Even though we already defined whatever we need, just 00:18:12.850 --> 00:18:16.233 align:middle line:84% in terms of the sample space and the probability measure, 00:18:16.233 --> 00:18:17.650 align:middle line:84% there are a lot of tricks that are 00:18:17.650 --> 00:18:20.650 align:middle line:84% going to happen where there are clever ways 00:18:20.650 --> 00:18:23.490 align:middle line:84% to compute probabilities that don't involve listing 00:18:23.490 --> 00:18:26.690 align:middle line:84% the entire sample space and just adding things up 00:18:26.690 --> 00:18:28.170 align:middle line:90% in a boring way. 00:18:28.170 --> 00:18:29.610 align:middle line:84% So a lot of these tools are really 00:18:29.610 --> 00:18:32.060 align:middle line:84% a way to organize and think about the sample 00:18:32.060 --> 00:18:35.860 align:middle line:84% space to try and make probability more intuitive. 00:18:35.860 --> 00:18:40.580 align:middle line:84% And the basic version of the law of total probability 00:18:40.580 --> 00:18:45.360 align:middle line:84% is that if I have an event A and I care about its probability, 00:18:45.360 --> 00:18:46.940 align:middle line:84% we already have a definition for how 00:18:46.940 --> 00:18:49.940 align:middle line:84% to compute that probability just by going over the entire sample 00:18:49.940 --> 00:18:51.140 align:middle line:90% space. 00:18:51.140 --> 00:18:53.740 align:middle line:84% But what I claim is that we can break it up 00:18:53.740 --> 00:18:59.540 align:middle line:84% into two different terms, one where A and B both happen. 00:18:59.540 --> 00:19:01.700 align:middle line:84% And then I can add in the probability 00:19:01.700 --> 00:19:05.980 align:middle line:84% that A happens and B does not happen. 00:19:05.980 --> 00:19:09.660 align:middle line:84% So this is just if I'm interested in the probability 00:19:09.660 --> 00:19:15.060 align:middle line:84% that A happens, well either A happens and B happens 00:19:15.060 --> 00:19:17.740 align:middle line:84% or A happens and B doesn't happen. 00:19:17.740 --> 00:19:20.660 align:middle line:84% These together cover the space of all things that 00:19:20.660 --> 00:19:23.620 align:middle line:84% could happen so that A still occurs, 00:19:23.620 --> 00:19:25.860 align:middle line:90% and they don't overlap at all. 00:19:25.860 --> 00:19:27.860 align:middle line:84% So in particular, the proof of this 00:19:27.860 --> 00:19:30.050 align:middle line:90% is there really is no proof. 00:19:30.050 --> 00:19:33.910 align:middle line:84% It's literally just we take the definition of what 00:19:33.910 --> 00:19:35.770 align:middle line:90% the probability of A is. 00:19:35.770 --> 00:19:40.830 align:middle line:84% It's the sum over all x in my set A of p of x. 00:19:40.830 --> 00:19:43.750 align:middle line:84% And all I'm doing is I'm breaking the sum up into two 00:19:43.750 --> 00:19:50.310 align:middle line:84% pieces, it's the sum over all x in A and B of p of x. 00:19:50.310 --> 00:19:55.523 align:middle line:84% That's literally this right here plus the sum over all x in A 00:19:55.523 --> 00:19:59.510 align:middle line:90% and not B of probability of x. 00:19:59.510 --> 00:20:05.510 align:middle line:84% And so pictorially, we can go back to our favorite diagram. 00:20:05.510 --> 00:20:08.470 align:middle line:84% We have this event A that we care about. 00:20:08.470 --> 00:20:11.950 align:middle line:84% We have this other event B. And then 00:20:11.950 --> 00:20:15.890 align:middle line:84% we can break up this event A into two pieces, 00:20:15.890 --> 00:20:18.510 align:middle line:84% one where A happens and B does not, 00:20:18.510 --> 00:20:23.230 align:middle line:84% and one where A happens and B does. 00:20:23.230 --> 00:20:27.280 align:middle line:84% And together, they cover the space. 00:20:27.280 --> 00:20:29.320 align:middle line:84% So, in fact, there's an even fancier version 00:20:29.320 --> 00:20:31.560 align:middle line:84% of the law of total probability, because no one 00:20:31.560 --> 00:20:34.440 align:middle line:84% says you have to do this with one other event. 00:20:34.440 --> 00:20:38.160 align:middle line:84% In fact, you can do this with a ton of other events. 00:20:38.160 --> 00:20:43.560 align:middle line:84% So let's talk about the most general version of the law 00:20:43.560 --> 00:20:45.640 align:middle line:90% of total probability. 00:20:45.640 --> 00:20:48.800 align:middle line:84% We can be interested in the probability of some event A, 00:20:48.800 --> 00:20:51.560 align:middle line:84% and we can write this as the sum over a collection 00:20:51.560 --> 00:20:56.800 align:middle line:84% of other events of the probability of A and BI. 00:20:56.800 --> 00:21:00.480 align:middle line:84% But now we need some conditions on the BIs. 00:21:00.480 --> 00:21:05.840 align:middle line:84% We need that all of the BIs are disjoint. 00:21:05.840 --> 00:21:11.160 align:middle line:84% So BI intersect BJ is the empty set for I 00:21:11.160 --> 00:21:14.680 align:middle line:84% not equal to J. They're pairwise disjoint. 00:21:14.680 --> 00:21:19.600 align:middle line:84% And we also need that together, they cover the entire space. 00:21:19.600 --> 00:21:23.760 align:middle line:84% The union of all of these BIs is my sample space. 00:21:23.760 --> 00:21:26.660 align:middle line:84% So in particular for this most general definition 00:21:26.660 --> 00:21:29.320 align:middle line:84% for the law of total probability, really, 00:21:29.320 --> 00:21:31.660 align:middle line:84% the way that I recover that special case we already 00:21:31.660 --> 00:21:36.860 align:middle line:84% talked about is I consider two events, B1 and B2, and B2 00:21:36.860 --> 00:21:39.540 align:middle line:90% is the complement of B1. 00:21:39.540 --> 00:21:42.580 align:middle line:84% And that way, I just get two terms and the sum. 00:21:42.580 --> 00:21:44.140 align:middle line:90% They're definitely disjoint. 00:21:44.140 --> 00:21:47.060 align:middle line:84% B1 and its complement don't intersect. 00:21:47.060 --> 00:21:49.340 align:middle line:84% And together, they cover the space. 00:21:49.340 --> 00:21:52.140 align:middle line:84% But instead of breaking up my entire sample space 00:21:52.140 --> 00:21:54.780 align:middle line:84% into only two possibilities, I can 00:21:54.780 --> 00:21:57.600 align:middle line:84% think about breaking it up into three, four, five, 00:21:57.600 --> 00:22:00.220 align:middle line:90% how many ever is convenient. 00:22:00.220 --> 00:22:04.620 align:middle line:84% So any questions about the law of total probability? 00:22:04.620 --> 00:22:06.100 align:middle line:90% Good? 00:22:06.100 --> 00:22:08.260 align:middle line:90% All right. 00:22:08.260 --> 00:22:11.960 align:middle line:84% So let's go back to visualizing these things. 00:22:11.960 --> 00:22:17.980 align:middle line:84% I want to try to make these things as intuitive as possible. 00:22:17.980 --> 00:22:21.820 align:middle line:84% So in fact, there's a clever way to think 00:22:21.820 --> 00:22:26.220 align:middle line:84% about them using a diagram, which I find very helpful. 00:22:26.220 --> 00:22:28.750 align:middle line:90% 00:22:28.750 --> 00:22:34.150 align:middle line:84% So we can think about our random process 00:22:34.150 --> 00:22:37.710 align:middle line:84% as really taking some sort of path in a tree. 00:22:37.710 --> 00:22:41.650 align:middle line:84% Maybe we start off our random process, 00:22:41.650 --> 00:22:46.030 align:middle line:84% and then we have a whole bunch of possibilities, like B1, 00:22:46.030 --> 00:22:49.270 align:middle line:90% B2, B3. 00:22:49.270 --> 00:22:52.830 align:middle line:84% So what I'm deciding here is really 00:22:52.830 --> 00:22:59.440 align:middle line:84% the answer to the question, which of the BIs happens? 00:22:59.440 --> 00:23:03.030 align:middle line:90% 00:23:03.030 --> 00:23:07.230 align:middle line:84% Because in the setup of the law of total probability, 00:23:07.230 --> 00:23:09.670 align:middle line:84% all these BIs are pairwise disjoint, 00:23:09.670 --> 00:23:11.670 align:middle line:90% and they cover the space. 00:23:11.670 --> 00:23:15.310 align:middle line:84% So when I start this process, before I actually tell you 00:23:15.310 --> 00:23:18.070 align:middle line:84% what the outcome is, where the dart lands, 00:23:18.070 --> 00:23:21.510 align:middle line:84% I'm first going to tell you which region the dart lands in. 00:23:21.510 --> 00:23:22.980 align:middle line:90% Does it land in B1? 00:23:22.980 --> 00:23:24.420 align:middle line:90% Does it land in B2? 00:23:24.420 --> 00:23:26.360 align:middle line:90% Does it land in B3? 00:23:26.360 --> 00:23:28.840 align:middle line:84% The assumption that these things are pairwise disjoint 00:23:28.840 --> 00:23:33.080 align:middle line:84% just means that one and only one of these things happens. 00:23:33.080 --> 00:23:34.600 align:middle line:84% But I have a different probability 00:23:34.600 --> 00:23:36.600 align:middle line:84% of traversing these edges depending 00:23:36.600 --> 00:23:39.280 align:middle line:84% on what the probability of those events are. 00:23:39.280 --> 00:23:44.320 align:middle line:84% And then finally, I can look at conditioned 00:23:44.320 --> 00:23:49.260 align:middle line:84% on B1 happening whether the event A actually happens. 00:23:49.260 --> 00:23:51.380 align:middle line:90% So I can continue this process. 00:23:51.380 --> 00:23:54.960 align:middle line:90% 00:23:54.960 --> 00:24:01.000 align:middle line:84% And then what I can ask is, how does, knowing that BI happens, 00:24:01.000 --> 00:24:09.660 align:middle line:84% how does that change the probability of A? 00:24:09.660 --> 00:24:12.880 align:middle line:90% 00:24:12.880 --> 00:24:15.680 align:middle line:84% So this is a nice way to visualize it. 00:24:15.680 --> 00:24:18.600 align:middle line:84% Because what's going on in here is 00:24:18.600 --> 00:24:22.630 align:middle line:84% you can write it out instead using conditional probability. 00:24:22.630 --> 00:24:24.690 align:middle line:84% Another equivalent way to do this is it's 00:24:24.690 --> 00:24:27.370 align:middle line:84% the sum over I of the probability 00:24:27.370 --> 00:24:33.490 align:middle line:84% of A conditioned on BI times the probability of BI. 00:24:33.490 --> 00:24:35.270 align:middle line:84% Literally nothing is happening here. 00:24:35.270 --> 00:24:39.330 align:middle line:84% I'm just using the definition of what conditional probability is. 00:24:39.330 --> 00:24:43.650 align:middle line:84% And then you can see that algebraically, what's happening 00:24:43.650 --> 00:24:46.650 align:middle line:84% is first you're telling me which BI happened. 00:24:46.650 --> 00:24:49.570 align:middle line:84% That contributes this term right here. 00:24:49.570 --> 00:24:52.370 align:middle line:84% And then once you tell me that BI happens, 00:24:52.370 --> 00:24:55.130 align:middle line:84% I have to also take into account the remaining conditional 00:24:55.130 --> 00:24:56.210 align:middle line:90% probability. 00:24:56.210 --> 00:24:58.550 align:middle line:84% And when I add up over all the possibilities, 00:24:58.550 --> 00:25:01.650 align:middle line:84% that gives me the total probability of the event. 00:25:01.650 --> 00:25:04.310 align:middle line:84% So these are very basic algebraic manipulations, 00:25:04.310 --> 00:25:05.220 align:middle line:90% but they're powerful. 00:25:05.220 --> 00:25:07.850 align:middle line:90% 00:25:07.850 --> 00:25:10.730 align:middle line:90% So any questions? 00:25:10.730 --> 00:25:12.770 align:middle line:90% All makes perfect sense? 00:25:12.770 --> 00:25:15.917 align:middle line:84% STUDENT: So what are the nodes, the bottom? 00:25:15.917 --> 00:25:16.750 align:middle line:90% ANKUR MOITRA: Right. 00:25:16.750 --> 00:25:20.780 align:middle line:84% So this would be these are all of the possible outcomes 00:25:20.780 --> 00:25:22.420 align:middle line:90% in the sample space. 00:25:22.420 --> 00:25:24.900 align:middle line:84% So what I'm doing is instead of just telling you first 00:25:24.900 --> 00:25:28.020 align:middle line:84% where the dart lands, like which x happens, 00:25:28.020 --> 00:25:31.380 align:middle line:84% first I'm going to tell you whether B1 happens, B2 happens, 00:25:31.380 --> 00:25:33.100 align:middle line:90% or B3 happens. 00:25:33.100 --> 00:25:36.300 align:middle line:84% Once I tell you that B1 happens, I know my dart lands 00:25:36.300 --> 00:25:37.580 align:middle line:90% in that space. 00:25:37.580 --> 00:25:40.780 align:middle line:84% I look at all the x's that belong to B1, 00:25:40.780 --> 00:25:43.020 align:middle line:84% and maybe I just shade in the x's 00:25:43.020 --> 00:25:47.220 align:middle line:84% that belong to that subset that are also in my set A. 00:25:47.220 --> 00:25:50.340 align:middle line:84% So for example, the fraction of nodes here that are colored 00:25:50.340 --> 00:25:54.500 align:middle line:84% is exactly the probability of A conditioned on B1. 00:25:54.500 --> 00:25:57.560 align:middle line:84% So you can see that I have this probability of taking B1, 00:25:57.560 --> 00:25:59.620 align:middle line:84% and then I have a conditional probability. 00:25:59.620 --> 00:26:02.340 align:middle line:84% Or I have the probability of taking this branch and then 00:26:02.340 --> 00:26:04.260 align:middle line:90% another conditional probability. 00:26:04.260 --> 00:26:07.640 align:middle line:84% This is pictorially what the law of probability is telling you. 00:26:07.640 --> 00:26:11.220 align:middle line:84% You can think about it many different ways. 00:26:11.220 --> 00:26:17.410 align:middle line:84% This is tricky, so don't be embarrassed to ask questions. 00:26:17.410 --> 00:26:18.750 align:middle line:90% That's why I'm here. 00:26:18.750 --> 00:26:21.750 align:middle line:90% Any other questions? 00:26:21.750 --> 00:26:24.830 align:middle line:84% Everyone thinks this is way too easy? 00:26:24.830 --> 00:26:27.510 align:middle line:90% Way too hard? 00:26:27.510 --> 00:26:28.990 align:middle line:90% Just right? 00:26:28.990 --> 00:26:31.350 align:middle line:90% OK, I'll take it. 00:26:31.350 --> 00:26:31.860 align:middle line:90% All right. 00:26:31.860 --> 00:26:34.630 align:middle line:90% 00:26:34.630 --> 00:26:35.410 align:middle line:90% Let's see. 00:26:35.410 --> 00:26:40.110 align:middle line:90% 00:26:40.110 --> 00:26:42.145 align:middle line:84% All right, let's check our understanding, 00:26:42.145 --> 00:26:44.770 align:middle line:84% because some of these things are a little bit counterintuitive. 00:26:44.770 --> 00:26:48.550 align:middle line:90% 00:26:48.550 --> 00:26:53.350 align:middle line:84% How many people have heard of the Monty Hall problem? 00:26:53.350 --> 00:26:58.270 align:middle line:84% OK, how many people feel totally comfortable with Monty Hall 00:26:58.270 --> 00:26:59.810 align:middle line:84% problem and think I should skip it 00:26:59.810 --> 00:27:02.190 align:middle line:84% or how many people want to hear it? 00:27:02.190 --> 00:27:05.430 align:middle line:90% Who wants me to skip it? 00:27:05.430 --> 00:27:08.350 align:middle line:84% OK, that's a very small number, so going to do it. 00:27:08.350 --> 00:27:12.310 align:middle line:84% So you've all seen the Monty Hall problem before. 00:27:12.310 --> 00:27:16.400 align:middle line:84% What I claim is that a lot of these ways of thinking 00:27:16.400 --> 00:27:18.700 align:middle line:84% about things in terms of the law of total probability, 00:27:18.700 --> 00:27:21.040 align:middle line:84% in terms of this visual I gave you here, 00:27:21.040 --> 00:27:22.680 align:middle line:84% they give you a nice intuitive way 00:27:22.680 --> 00:27:25.040 align:middle line:90% to think about what's going on. 00:27:25.040 --> 00:27:28.360 align:middle line:84% So I'll give you maybe an explanation 00:27:28.360 --> 00:27:30.520 align:middle line:90% you haven't seen before. 00:27:30.520 --> 00:27:34.200 align:middle line:84% So remember in the Monty Hall problem, there's a game show 00:27:34.200 --> 00:27:38.120 align:middle line:90% and there's three doors. 00:27:38.120 --> 00:27:42.160 align:middle line:84% And behind one of the doors is a nice prize. 00:27:42.160 --> 00:27:44.680 align:middle line:90% Maybe there's a car. 00:27:44.680 --> 00:27:47.760 align:middle line:84% But you don't know which door has the prize. 00:27:47.760 --> 00:27:52.860 align:middle line:84% So the way that the game works is that first, you pick a door. 00:27:52.860 --> 00:27:55.960 align:middle line:90% 00:27:55.960 --> 00:27:58.980 align:middle line:84% Now, whatever door you pick, if I pick this door, 00:27:58.980 --> 00:28:02.200 align:middle line:84% there's definitely another door that doesn't have the prize. 00:28:02.200 --> 00:28:04.220 align:middle line:84% If I pick this door that has the prize, 00:28:04.220 --> 00:28:07.320 align:middle line:84% there's still another door that doesn't have the prize. 00:28:07.320 --> 00:28:12.080 align:middle line:84% So either way, the host can always, no matter what 00:28:12.080 --> 00:28:17.260 align:middle line:84% you pick for your first door, reveal to you another door 00:28:17.260 --> 00:28:22.260 align:middle line:84% that you didn't select which does not have the price, 00:28:22.260 --> 00:28:25.620 align:middle line:84% and show you that you shouldn't pick that door. 00:28:25.620 --> 00:28:30.220 align:middle line:84% And the basic question is, do you switch? 00:28:30.220 --> 00:28:32.280 align:middle line:84% So this is a very famous example. 00:28:32.280 --> 00:28:33.760 align:middle line:84% I think the first time you see it, 00:28:33.760 --> 00:28:36.700 align:middle line:84% it is a bit hard to wrap your head around. 00:28:36.700 --> 00:28:39.300 align:middle line:84% But it's been done in virtually every class that 00:28:39.300 --> 00:28:43.620 align:middle line:84% covers probability so often that maybe it's become 00:28:43.620 --> 00:28:46.140 align:middle line:90% a little bit more intuitive. 00:28:46.140 --> 00:28:49.060 align:middle line:84% As you all know, the answer is that you switch, 00:28:49.060 --> 00:28:52.660 align:middle line:84% that then you'll win this game with 2/3 probability. 00:28:52.660 --> 00:28:55.620 align:middle line:84% Now, you can figure that out by computing 00:28:55.620 --> 00:28:56.840 align:middle line:90% conditional probabilities. 00:28:56.840 --> 00:28:58.940 align:middle line:84% You have to be very careful about the definitions 00:28:58.940 --> 00:29:00.580 align:middle line:90% of your events. 00:29:00.580 --> 00:29:02.620 align:middle line:84% But I think a very intuitive way to think 00:29:02.620 --> 00:29:06.660 align:middle line:84% about it is in terms of this tree 00:29:06.660 --> 00:29:10.380 align:middle line:84% that I drew before, because we can think about all the things 00:29:10.380 --> 00:29:12.670 align:middle line:90% that can happen in the game. 00:29:12.670 --> 00:29:14.690 align:middle line:84% So in the beginning, when you choose a door, 00:29:14.690 --> 00:29:18.510 align:middle line:84% remember that the prize is put behind a door at random. 00:29:18.510 --> 00:29:23.750 align:middle line:84% So whatever door you choose, you have a 2/3 probability 00:29:23.750 --> 00:29:28.190 align:middle line:84% of selecting a door that has no prize, 00:29:28.190 --> 00:29:32.670 align:middle line:84% and you have a 1/3 chance of selecting the door that has 00:29:32.670 --> 00:29:34.110 align:middle line:90% the prize. 00:29:34.110 --> 00:29:36.510 align:middle line:84% Of course, the game isn't over then, 00:29:36.510 --> 00:29:39.190 align:middle line:84% and we can just think about what happens depending 00:29:39.190 --> 00:29:41.470 align:middle line:90% on your choice of strategy. 00:29:41.470 --> 00:29:44.510 align:middle line:84% You could, at that juncture, decide to switch. 00:29:44.510 --> 00:29:47.670 align:middle line:90% You could decide to not switch. 00:29:47.670 --> 00:29:49.790 align:middle line:90% And the same thing happens here. 00:29:49.790 --> 00:29:55.350 align:middle line:84% You could decide to switch or not switch. 00:29:55.350 --> 00:29:57.950 align:middle line:84% And the way to think about it is that once I 00:29:57.950 --> 00:30:01.470 align:middle line:84% select a door that doesn't have the prize, 00:30:01.470 --> 00:30:04.830 align:middle line:84% then the host has no choice but to reveal to me the only 00:30:04.830 --> 00:30:07.150 align:middle line:84% other door that doesn't have the prize. 00:30:07.150 --> 00:30:11.600 align:middle line:84% And then when I switch, I definitely win. 00:30:11.600 --> 00:30:14.480 align:middle line:84% So I win as soon as I select a door 00:30:14.480 --> 00:30:17.760 align:middle line:84% in the beginning with no prize if I'm committed to the strategy 00:30:17.760 --> 00:30:18.840 align:middle line:90% switch. 00:30:18.840 --> 00:30:23.000 align:middle line:84% And if I don't switch, then of course, I lose. 00:30:23.000 --> 00:30:26.720 align:middle line:84% And the same thing happens here, but in reverse. 00:30:26.720 --> 00:30:30.720 align:middle line:84% If I got unlucky and I selected the door with the prize 00:30:30.720 --> 00:30:33.160 align:middle line:84% and I committed to the strategy switch, 00:30:33.160 --> 00:30:35.560 align:middle line:84% then the host would show me one of these two doors 00:30:35.560 --> 00:30:37.160 align:middle line:84% without the prize, and I would switch, 00:30:37.160 --> 00:30:39.760 align:middle line:90% and I would lose my prize. 00:30:39.760 --> 00:30:42.340 align:middle line:84% But this only happens with probability 1/3, 00:30:42.340 --> 00:30:45.320 align:middle line:84% because that's the probability that this tree associates with 00:30:45.320 --> 00:30:47.920 align:middle line:90% it, and otherwise I'd win. 00:30:47.920 --> 00:30:51.400 align:middle line:84% So you can see that once I've committed to the fact 00:30:51.400 --> 00:30:54.400 align:middle line:84% that I'm going to switch, all I need to do 00:30:54.400 --> 00:30:56.860 align:middle line:84% is I need to get not super lucky, 00:30:56.860 --> 00:30:59.900 align:middle line:84% just I need to select the door which doesn't have the prize, 00:30:59.900 --> 00:31:04.960 align:middle line:84% and then automatically you can see that I will win. 00:31:04.960 --> 00:31:07.560 align:middle line:84% So we'll do more examples of these kinds of things 00:31:07.560 --> 00:31:12.050 align:middle line:84% later, especially in recitation, like drawing out these diagrams 00:31:12.050 --> 00:31:14.650 align:middle line:90% to organize the state space. 00:31:14.650 --> 00:31:17.810 align:middle line:84% Let me tell you one more really cool tool, 00:31:17.810 --> 00:31:21.930 align:middle line:84% and then we'll do some real world applications of it. 00:31:21.930 --> 00:31:24.510 align:middle line:84% This involves, again, conditional probability. 00:31:24.510 --> 00:31:27.770 align:middle line:90% That's called Bayes' rule. 00:31:27.770 --> 00:31:29.420 align:middle line:90% And it's a very basic formula. 00:31:29.420 --> 00:31:34.210 align:middle line:90% 00:31:34.210 --> 00:31:37.130 align:middle line:84% So imagine that we compute something 00:31:37.130 --> 00:31:41.210 align:middle line:84% the probability of B given A. It turns out 00:31:41.210 --> 00:31:44.770 align:middle line:84% there's a way to express this in terms 00:31:44.770 --> 00:31:49.890 align:middle line:84% of the conditional probability the other way around. 00:31:49.890 --> 00:31:54.490 align:middle line:84% You just have to correct by the right factors of P of B and P 00:31:54.490 --> 00:31:58.450 align:middle line:84% of A. So this is a very powerful rule. 00:31:58.450 --> 00:32:02.030 align:middle line:84% The proof is quite straightforward. 00:32:02.030 --> 00:32:04.050 align:middle line:90% It's just algebraic. 00:32:04.050 --> 00:32:07.860 align:middle line:84% Because all we have to do is just invoke the definitions. 00:32:07.860 --> 00:32:11.540 align:middle line:84% So by definition, how did we define 00:32:11.540 --> 00:32:13.980 align:middle line:90% the probability of A given B? 00:32:13.980 --> 00:32:18.380 align:middle line:84% It was just the probability of A and B 00:32:18.380 --> 00:32:22.060 align:middle line:84% divided by the probability of B. That's literally 00:32:22.060 --> 00:32:24.140 align:middle line:90% what our definition was. 00:32:24.140 --> 00:32:27.020 align:middle line:84% I can take this definition, and I can just 00:32:27.020 --> 00:32:31.180 align:middle line:84% multiply by this correction term, which is probability 00:32:31.180 --> 00:32:34.980 align:middle line:90% of B over probability of A. 00:32:34.980 --> 00:32:38.120 align:middle line:84% And if I do this on both sides of my equation, 00:32:38.120 --> 00:32:40.640 align:middle line:84% well, one side I'm going to get this right hand side. 00:32:40.640 --> 00:32:44.580 align:middle line:84% So I'll get probability of A given B times 00:32:44.580 --> 00:32:50.260 align:middle line:84% probability of B over the probability of A. 00:32:50.260 --> 00:32:52.420 align:middle line:84% And on my other side, the probability 00:32:52.420 --> 00:32:54.140 align:middle line:90% of B's are going to cancel. 00:32:54.140 --> 00:33:00.300 align:middle line:84% I'm going to get probability of A and B over probability of A. 00:33:00.300 --> 00:33:03.260 align:middle line:84% And I claim that this is the same thing as the probability 00:33:03.260 --> 00:33:08.510 align:middle line:84% of B given A. And that's because it doesn't matter what order I 00:33:08.510 --> 00:33:09.970 align:middle line:90% intersect these two things. 00:33:09.970 --> 00:33:12.270 align:middle line:84% Probability of A and B is the same thing 00:33:12.270 --> 00:33:14.950 align:middle line:84% as the probability of B and A. And then 00:33:14.950 --> 00:33:17.510 align:middle line:84% this is, again, literally the definition 00:33:17.510 --> 00:33:19.790 align:middle line:90% of conditional probability. 00:33:19.790 --> 00:33:22.910 align:middle line:84% So the proof of this Bayes rule is very simple. 00:33:22.910 --> 00:33:25.830 align:middle line:84% In fact, anyone know anything historically 00:33:25.830 --> 00:33:28.090 align:middle line:84% about why people were interested in this. 00:33:28.090 --> 00:33:31.110 align:middle line:84% It's kind of an interesting historical fact. 00:33:31.110 --> 00:33:32.590 align:middle line:90% Yeah? 00:33:32.590 --> 00:33:34.130 align:middle line:84% STUDENT: They wanted to prove God. 00:33:34.130 --> 00:33:37.070 align:middle line:84% ANKUR MOITRA: Yeah, they wanted to prove God, the existence 00:33:37.070 --> 00:33:38.758 align:middle line:90% of God using Bayes' rule. 00:33:38.758 --> 00:33:40.550 align:middle line:84% I don't think they got there, and we're not 00:33:40.550 --> 00:33:42.630 align:middle line:84% going to get there today either, but we'll still 00:33:42.630 --> 00:33:44.208 align:middle line:90% do some fun stuff with it. 00:33:44.208 --> 00:33:45.250 align:middle line:90% It's kind of interesting. 00:33:45.250 --> 00:33:48.390 align:middle line:84% A lot of the foundations of probability 00:33:48.390 --> 00:33:50.830 align:middle line:84% have some theological component to them, 00:33:50.830 --> 00:33:54.750 align:middle line:84% but I'll leave that as a discussion for another time. 00:33:54.750 --> 00:33:56.310 align:middle line:90% So we've got Bayes' rule. 00:33:56.310 --> 00:34:00.140 align:middle line:84% Let's do a powerful real world example. 00:34:00.140 --> 00:34:04.850 align:middle line:90% 00:34:04.850 --> 00:34:07.960 align:middle line:84% The numbers in this example are all completely real. 00:34:07.960 --> 00:34:10.770 align:middle line:90% 00:34:10.770 --> 00:34:14.090 align:middle line:90% So let me tell you some facts. 00:34:14.090 --> 00:34:26.480 align:middle line:84% So approximately 1% of women aged 40 to 50 get breast cancer. 00:34:26.480 --> 00:34:29.250 align:middle line:90% 00:34:29.250 --> 00:34:31.550 align:middle line:84% That is a shockingly high percentage, 00:34:31.550 --> 00:34:34.730 align:middle line:90% but it's a true fact. 00:34:34.730 --> 00:34:37.810 align:middle line:84% And one of the ways that you can try and screen for breast cancer 00:34:37.810 --> 00:34:39.230 align:middle line:90% is a mammogram. 00:34:39.230 --> 00:34:41.929 align:middle line:90% 00:34:41.929 --> 00:34:45.010 align:middle line:84% So a mammogram is not a perfect test. 00:34:45.010 --> 00:34:49.060 align:middle line:84% So it has a 10% false positive rate. 00:34:49.060 --> 00:34:51.730 align:middle line:90% 00:34:51.730 --> 00:34:55.170 align:middle line:84% I'll tell you what I mean by that in a second. 00:34:55.170 --> 00:34:58.500 align:middle line:84% And it has similarly for its false negative rate. 00:34:58.500 --> 00:35:02.170 align:middle line:90% 00:35:02.170 --> 00:35:05.080 align:middle line:84% So let's look at two events which we really care about, 00:35:05.080 --> 00:35:08.260 align:middle line:90% because our test is imperfect. 00:35:08.260 --> 00:35:14.820 align:middle line:84% We can look at A is the event that we test positive. 00:35:14.820 --> 00:35:17.380 align:middle line:84% And we can look at the event B that we really 00:35:17.380 --> 00:35:20.200 align:middle line:84% have the condition that's being tested for. 00:35:20.200 --> 00:35:24.560 align:middle line:90% In this case, breast cancer. 00:35:24.560 --> 00:35:27.460 align:middle line:90% These are the two events. 00:35:27.460 --> 00:35:28.700 align:middle line:90% Yeah? 00:35:28.700 --> 00:35:29.708 align:middle line:90% STUDENT: [INAUDIBLE] 00:35:29.708 --> 00:35:30.500 align:middle line:90% ANKUR MOITRA: Yeah. 00:35:30.500 --> 00:35:30.920 align:middle line:90% It is. 00:35:30.920 --> 00:35:31.420 align:middle line:90% Yeah. 00:35:31.420 --> 00:35:33.240 align:middle line:90% So developed in that time frame. 00:35:33.240 --> 00:35:35.940 align:middle line:90% 00:35:35.940 --> 00:35:38.260 align:middle line:84% So here, just to be clear, so what 00:35:38.260 --> 00:35:41.580 align:middle line:84% do I mean by the false positive rate? 00:35:41.580 --> 00:35:47.020 align:middle line:84% So this would be the probability of A given not B. 00:35:47.020 --> 00:35:50.100 align:middle line:84% So this is the chance the test says positive 00:35:50.100 --> 00:35:52.300 align:middle line:90% even though you don't have it. 00:35:52.300 --> 00:35:55.660 align:middle line:84% And we can also look at the false negative rate, which 00:35:55.660 --> 00:35:59.860 align:middle line:84% is the probability of not A given B. 00:35:59.860 --> 00:36:01.810 align:middle line:84% So you actually do have the condition, 00:36:01.810 --> 00:36:04.350 align:middle line:90% but the test doesn't pick it up. 00:36:04.350 --> 00:36:07.030 align:middle line:84% So both of these things, a lot of these tests, 00:36:07.030 --> 00:36:09.790 align:middle line:84% we have much higher accuracy tests. 00:36:09.790 --> 00:36:13.750 align:middle line:84% But the idea is that you usually deploy some lower accuracy 00:36:13.750 --> 00:36:18.830 align:middle line:84% test as a screening procedure to try and get a rough idea. 00:36:18.830 --> 00:36:21.670 align:middle line:84% But then the question is, how much stock should you actually 00:36:21.670 --> 00:36:25.710 align:middle line:90% put in the test results? 00:36:25.710 --> 00:36:27.910 align:middle line:84% So one of the conditional probabilities 00:36:27.910 --> 00:36:34.510 align:middle line:84% that's very important is what's the probability of B given A? 00:36:34.510 --> 00:36:35.770 align:middle line:90% So what does this mean? 00:36:35.770 --> 00:36:38.730 align:middle line:84% You test positive, and then you're very worried, 00:36:38.730 --> 00:36:41.350 align:middle line:84% but you want to know whether you actually have the condition 00:36:41.350 --> 00:36:44.270 align:middle line:90% or not. 00:36:44.270 --> 00:36:48.870 align:middle line:84% So let me take a poll from the audience. 00:36:48.870 --> 00:36:50.470 align:middle line:84% Let's just say approximately, what 00:36:50.470 --> 00:36:53.430 align:middle line:84% do you think the probability of B given A is? 00:36:53.430 --> 00:36:57.950 align:middle line:84% After all, the mammogram has 10% false positive and false 00:36:57.950 --> 00:36:58.930 align:middle line:90% negative rate. 00:36:58.930 --> 00:37:02.000 align:middle line:84% So maybe you think you've got a 90% chance 00:37:02.000 --> 00:37:04.120 align:middle line:90% of having the condition. 00:37:04.120 --> 00:37:08.560 align:middle line:90% Maybe it's 75%. 00:37:08.560 --> 00:37:10.236 align:middle line:90% Maybe it's 25%. 00:37:10.236 --> 00:37:13.960 align:middle line:90% 00:37:13.960 --> 00:37:18.520 align:middle line:90% Maybe it's around 10%. 00:37:18.520 --> 00:37:21.800 align:middle line:90% Maybe it's 1%. 00:37:21.800 --> 00:37:25.280 align:middle line:84% So you all said you understand conditional probability. 00:37:25.280 --> 00:37:28.880 align:middle line:84% You all said you're ready for a test on it. 00:37:28.880 --> 00:37:30.560 align:middle line:90% So here's my test. 00:37:30.560 --> 00:37:32.180 align:middle line:84% So what do you think the answer is? 00:37:32.180 --> 00:37:35.280 align:middle line:84% And this is a very medically important question, 00:37:35.280 --> 00:37:38.400 align:middle line:84% because when you test positive for something, 00:37:38.400 --> 00:37:40.260 align:middle line:90% you have many treatment options. 00:37:40.260 --> 00:37:43.160 align:middle line:84% Some of them are more or less invasive than others. 00:37:43.160 --> 00:37:45.940 align:middle line:84% So how sure are you that you have that condition? 00:37:45.940 --> 00:37:51.000 align:middle line:84% Is it worth it to do some very invasive and risky procedure? 00:37:51.000 --> 00:37:53.010 align:middle line:84% So who thinks the answer is one, 90%? 00:37:53.010 --> 00:37:56.000 align:middle line:90% 00:37:56.000 --> 00:38:00.930 align:middle line:84% Who thinks the answer is two, 75%? 00:38:00.930 --> 00:38:02.330 align:middle line:90% Three, 25%. 00:38:02.330 --> 00:38:04.330 align:middle line:90% You guys have to vote. 00:38:04.330 --> 00:38:05.510 align:middle line:90% You can't abstain. 00:38:05.510 --> 00:38:06.310 align:middle line:90% You're a doctor. 00:38:06.310 --> 00:38:07.830 align:middle line:90% Someone's relying on you. 00:38:07.830 --> 00:38:09.250 align:middle line:90% Let's try this again. 00:38:09.250 --> 00:38:11.200 align:middle line:84% Who thinks the answer is one, 90%? 00:38:11.200 --> 00:38:13.970 align:middle line:90% 00:38:13.970 --> 00:38:14.790 align:middle line:90% Just take a vote. 00:38:14.790 --> 00:38:16.090 align:middle line:90% Take a chance. 00:38:16.090 --> 00:38:19.810 align:middle line:84% I'm not going to grade you on this. 00:38:19.810 --> 00:38:21.740 align:middle line:84% Who thinks the answer is two, 75%? 00:38:21.740 --> 00:38:24.650 align:middle line:90% 00:38:24.650 --> 00:38:27.570 align:middle line:84% Who thinks the answer is three, 25%? 00:38:27.570 --> 00:38:30.490 align:middle line:84% Who thinks the answer is four, 10%? 00:38:30.490 --> 00:38:33.130 align:middle line:84% Who thinks the answer is five, 1%? 00:38:33.130 --> 00:38:42.090 align:middle line:84% OK, so if you answered two, then you're in good company, 00:38:42.090 --> 00:38:44.950 align:middle line:84% because that's the average physician's answer. 00:38:44.950 --> 00:38:48.290 align:middle line:84% There was actually a survey of hundreds of physicians. 00:38:48.290 --> 00:38:51.770 align:middle line:84% And when you average their scores, you get 75%. 00:38:51.770 --> 00:38:53.570 align:middle line:84% Unfortunately, you're in a group that 00:38:53.570 --> 00:38:56.980 align:middle line:84% completely got the wrong answer, because the answer is not 00:38:56.980 --> 00:38:59.180 align:middle line:90% at all 75%. 00:38:59.180 --> 00:39:01.320 align:middle line:90% It's actually about 8%. 00:39:01.320 --> 00:39:03.260 align:middle line:90% So 10% is very close. 00:39:03.260 --> 00:39:05.180 align:middle line:84% For anyone who answered four, could you 00:39:05.180 --> 00:39:08.620 align:middle line:90% give me some kind of intuition? 00:39:08.620 --> 00:39:09.500 align:middle line:90% Yeah. 00:39:09.500 --> 00:39:12.040 align:middle line:84% STUDENT: I guess 1% is really low to begin with. 00:39:12.040 --> 00:39:13.100 align:middle line:90% So I don't know. 00:39:13.100 --> 00:39:16.100 align:middle line:90% [INAUDIBLE] 00:39:16.100 --> 00:39:17.480 align:middle line:84% ANKUR MOITRA: Yeah, that's right. 00:39:17.480 --> 00:39:21.600 align:middle line:84% That's great Intuition Because if you think about it, already, 00:39:21.600 --> 00:39:23.200 align:middle line:84% I mean, this is a high percentage, 00:39:23.200 --> 00:39:26.300 align:middle line:84% but it's still in an absolute sense low. 00:39:26.300 --> 00:39:30.140 align:middle line:84% So you can think that the vast majority, when 00:39:30.140 --> 00:39:35.060 align:middle line:84% you think about a typical person who doesn't have it, 99% chance, 00:39:35.060 --> 00:39:37.340 align:middle line:84% then there's still a reasonable chance 00:39:37.340 --> 00:39:40.580 align:middle line:84% that they would still test positive if they were tested. 00:39:40.580 --> 00:39:43.120 align:middle line:84% And so among the people who are tested positive, 00:39:43.120 --> 00:39:45.660 align:middle line:84% that's the region in the dartboard I care about. 00:39:45.660 --> 00:39:47.940 align:middle line:84% Still, the disproportionate share of them 00:39:47.940 --> 00:39:50.700 align:middle line:84% are people who aren't going to have it. 00:39:50.700 --> 00:39:53.300 align:middle line:84% So this is actually from a very famous book 00:39:53.300 --> 00:39:58.160 align:middle line:84% that I encourage you to read if you want to be depressed. 00:39:58.160 --> 00:40:01.320 align:middle line:84% It's called Thinking Fast and Slow by Kahneman. 00:40:01.320 --> 00:40:03.760 align:middle line:84% And it goes through all of these examples about how 00:40:03.760 --> 00:40:07.520 align:middle line:84% peoples, and especially doctors, it kind of picks on doctors, 00:40:07.520 --> 00:40:10.080 align:middle line:84% make very counterintuitive decisions. 00:40:10.080 --> 00:40:14.520 align:middle line:84% So when I used to teach 042 more regularly, 00:40:14.520 --> 00:40:15.820 align:middle line:90% I would still do this example. 00:40:15.820 --> 00:40:17.720 align:middle line:84% I think it's a beautiful example. 00:40:17.720 --> 00:40:22.760 align:middle line:84% And then in 2016, we had our first kid, a baby girl. 00:40:22.760 --> 00:40:26.120 align:middle line:84% And I remember that when we were doing all these tests, not 00:40:26.120 --> 00:40:29.000 align:middle line:84% breast cancer, but other things like testing 00:40:29.000 --> 00:40:32.295 align:middle line:84% for the risk of Down syndrome, one of the crazy things 00:40:32.295 --> 00:40:33.920 align:middle line:84% is that at the time, what you do is you 00:40:33.920 --> 00:40:35.980 align:middle line:90% do an amniocentesis to do that. 00:40:35.980 --> 00:40:39.280 align:middle line:84% That's where you put a needle into the belly, 00:40:39.280 --> 00:40:42.160 align:middle line:84% and then you extract fluid to try and figure out 00:40:42.160 --> 00:40:45.332 align:middle line:84% whether various genetic conditions are happening. 00:40:45.332 --> 00:40:48.040 align:middle line:84% The crazy thing is that a lot of the things they were testing for 00:40:48.040 --> 00:40:50.340 align:middle line:84% had very low percentages of occurring. 00:40:50.340 --> 00:40:53.290 align:middle line:84% They're terrible things when they happen, like Down syndrome. 00:40:53.290 --> 00:40:56.730 align:middle line:84% But the risk of causing a miscarriage 00:40:56.730 --> 00:40:59.970 align:middle line:84% with an amniocentesis, when I looked it up, 00:40:59.970 --> 00:41:02.650 align:middle line:90% was something like 1 inch 700. 00:41:02.650 --> 00:41:06.330 align:middle line:84% So I have to say that real world examples from having kids 00:41:06.330 --> 00:41:08.790 align:middle line:84% makes these things feel very differently. 00:41:08.790 --> 00:41:10.290 align:middle line:84% So if you're curious, I can tell you 00:41:10.290 --> 00:41:12.050 align:middle line:84% all kinds of conditional probabilities 00:41:12.050 --> 00:41:14.910 align:middle line:84% that doctors get wrong that relate to child rearing, 00:41:14.910 --> 00:41:18.090 align:middle line:84% but that's another topic for another time. 00:41:18.090 --> 00:41:20.390 align:middle line:84% So we have a little bit of time left. 00:41:20.390 --> 00:41:22.950 align:middle line:84% What I'm going to do is I'm going to start the next topic. 00:41:22.950 --> 00:41:27.210 align:middle line:84% We're going to take these ideas even further. 00:41:27.210 --> 00:41:32.050 align:middle line:84% So far, what we've talked about is mostly events. 00:41:32.050 --> 00:41:34.070 align:middle line:84% We talked about ways to reason about events. 00:41:34.070 --> 00:41:36.290 align:middle line:84% We talked about what events were, how to combine them 00:41:36.290 --> 00:41:37.730 align:middle line:90% in different ways. 00:41:37.730 --> 00:41:39.410 align:middle line:84% And we talked about these fancy ways 00:41:39.410 --> 00:41:41.530 align:middle line:84% about reasoning about their probability, 00:41:41.530 --> 00:41:43.650 align:middle line:84% either through the law of total probability 00:41:43.650 --> 00:41:46.810 align:middle line:84% or using conditional probabilities. 00:41:46.810 --> 00:41:50.130 align:middle line:84% We talked about how to reason about it through this tree 00:41:50.130 --> 00:41:51.780 align:middle line:90% diagram I gave you. 00:41:51.780 --> 00:41:54.900 align:middle line:84% That's a way to organize the sample space. 00:41:54.900 --> 00:41:57.800 align:middle line:84% We talked about the Monty Hall problem. 00:41:57.800 --> 00:42:00.700 align:middle line:84% We talked about some real world examples. 00:42:00.700 --> 00:42:03.620 align:middle line:84% Well, it turns out that there's an even more powerful 00:42:03.620 --> 00:42:06.900 align:middle line:84% abstraction that we can do beyond events 00:42:06.900 --> 00:42:09.460 align:middle line:84% where things get even more interesting. 00:42:09.460 --> 00:42:14.340 align:middle line:84% Some of the theory goes over, but it changes in cool ways. 00:42:14.340 --> 00:42:17.020 align:middle line:84% So let me explain this in terms of how we're going 00:42:17.020 --> 00:42:19.280 align:middle line:90% to generalize what we've seen. 00:42:19.280 --> 00:42:22.660 align:middle line:84% So last topic for today I want to tell you about 00:42:22.660 --> 00:42:25.040 align:middle line:84% is what are called random variables. 00:42:25.040 --> 00:42:28.540 align:middle line:90% 00:42:28.540 --> 00:42:31.220 align:middle line:90% So where are we right now? 00:42:31.220 --> 00:42:34.220 align:middle line:84% We talked about events, which are 00:42:34.220 --> 00:42:38.250 align:middle line:90% a subset of the sample space. 00:42:38.250 --> 00:42:42.900 align:middle line:90% 00:42:42.900 --> 00:42:46.140 align:middle line:84% We talked about a lot of properties of events. 00:42:46.140 --> 00:42:50.230 align:middle line:84% We talked about the property of independence 00:42:50.230 --> 00:42:52.730 align:middle line:84% and how, in some cases, it can be intuitive, 00:42:52.730 --> 00:42:55.270 align:middle line:84% like when the experiments are literally done separated 00:42:55.270 --> 00:42:56.390 align:middle line:90% from each other. 00:42:56.390 --> 00:42:58.150 align:middle line:84% Other times, it can be counterintuitive 00:42:58.150 --> 00:43:00.630 align:middle line:84% and you just have to check the definition. 00:43:00.630 --> 00:43:04.870 align:middle line:90% We talked about conditioning. 00:43:04.870 --> 00:43:07.250 align:middle line:84% We talked about unions and intersections. 00:43:07.250 --> 00:43:11.910 align:middle line:84% And then random variables, instead 00:43:11.910 --> 00:43:14.670 align:middle line:84% of being a subset of the sample space, what they are 00:43:14.670 --> 00:43:20.830 align:middle line:84% is they're functions defined on the sample space. 00:43:20.830 --> 00:43:23.030 align:middle line:84% So what they do is they take an outcome 00:43:23.030 --> 00:43:27.430 align:middle line:84% and they ascribe to that outcome some real value. 00:43:27.430 --> 00:43:29.850 align:middle line:90% So this is the intuitive thing. 00:43:29.850 --> 00:43:31.890 align:middle line:90% Let's do the definition. 00:43:31.890 --> 00:43:36.030 align:middle line:90% 00:43:36.030 --> 00:43:48.040 align:middle line:84% So a random variable is a function f 00:43:48.040 --> 00:43:51.860 align:middle line:84% that goes from my sample space and ascribes to each outcome 00:43:51.860 --> 00:43:54.640 align:middle line:90% some real value. 00:43:54.640 --> 00:43:57.360 align:middle line:84% And in fact, we'll be interested in a lot of properties 00:43:57.360 --> 00:43:59.040 align:middle line:90% of random variables. 00:43:59.040 --> 00:44:02.760 align:middle line:84% So one of the most useful and ubiquitous properties 00:44:02.760 --> 00:44:04.700 align:middle line:90% is called its expectation. 00:44:04.700 --> 00:44:07.000 align:middle line:84% So let's just define that now too. 00:44:07.000 --> 00:44:14.680 align:middle line:84% So the associated expectation of the random variable 00:44:14.680 --> 00:44:18.160 align:middle line:84% as just literally the average over your probability 00:44:18.160 --> 00:44:20.880 align:middle line:84% space of the values the function takes. 00:44:20.880 --> 00:44:22.140 align:middle line:90% So I write it here. 00:44:22.140 --> 00:44:25.800 align:middle line:84% This is a little block E. And it's 00:44:25.800 --> 00:44:29.600 align:middle line:84% defined to be equal to the sum over all outcomes 00:44:29.600 --> 00:44:33.880 align:middle line:84% in my space of the probability of that outcome 00:44:33.880 --> 00:44:40.560 align:middle line:84% times the real value of the corresponding random variable. 00:44:40.560 --> 00:44:43.000 align:middle line:84% So this is just a weighted average 00:44:43.000 --> 00:44:45.640 align:middle line:84% of the value of the random variable, weighted 00:44:45.640 --> 00:44:48.510 align:middle line:84% by the probability of each one of these outcomes. 00:44:48.510 --> 00:44:52.810 align:middle line:84% So random variables come up all the time. 00:44:52.810 --> 00:44:56.090 align:middle line:90% We don't just care about events. 00:44:56.090 --> 00:45:00.050 align:middle line:84% One of the simplest ways it might come up 00:45:00.050 --> 00:45:03.658 align:middle line:84% is even in understanding something like betting, 00:45:03.658 --> 00:45:05.450 align:middle line:84% where there are a lot of different outcomes 00:45:05.450 --> 00:45:06.870 align:middle line:90% that you could be betting on. 00:45:06.870 --> 00:45:08.850 align:middle line:84% And then something like the random variable 00:45:08.850 --> 00:45:12.290 align:middle line:84% would describe the payoffs to different parties. 00:45:12.290 --> 00:45:15.810 align:middle line:84% So let's do some simple examples to get some intuition for what 00:45:15.810 --> 00:45:17.230 align:middle line:90% the random variable is. 00:45:17.230 --> 00:45:27.690 align:middle line:90% 00:45:27.690 --> 00:45:31.300 align:middle line:84% Let's do one where the sample space is infinite, just for fun. 00:45:31.300 --> 00:45:38.050 align:middle line:90% 00:45:38.050 --> 00:45:48.580 align:middle line:84% So let's toss a coin until you get a heads. 00:45:48.580 --> 00:45:50.120 align:middle line:84% We'll start off with a fair coin, 00:45:50.120 --> 00:45:52.440 align:middle line:84% and then we'll vary this experiment. 00:45:52.440 --> 00:45:56.170 align:middle line:84% So first, what is the sample space? 00:45:56.170 --> 00:46:04.020 align:middle line:90% 00:46:04.020 --> 00:46:07.920 align:middle line:84% Well, we could have the first coin as heads already, 00:46:07.920 --> 00:46:09.460 align:middle line:90% and we stop there. 00:46:09.460 --> 00:46:12.100 align:middle line:90% We could have tails, heads. 00:46:12.100 --> 00:46:15.500 align:middle line:84% Could have tails, tails, heads, and so on. 00:46:15.500 --> 00:46:19.100 align:middle line:84% So any sequence of tails, any length from 0 00:46:19.100 --> 00:46:22.460 align:middle line:84% to infinity with a heads appended at the end, that's 00:46:22.460 --> 00:46:24.340 align:middle line:90% our sample space. 00:46:24.340 --> 00:46:27.100 align:middle line:84% And we also care about what's the associated probability 00:46:27.100 --> 00:46:28.340 align:middle line:90% measure. 00:46:28.340 --> 00:46:39.980 align:middle line:84% So furthermore, the probability of having I T's and then one H 00:46:39.980 --> 00:46:44.840 align:middle line:90% is equal to 1/2 to the I plus 1. 00:46:44.840 --> 00:46:47.700 align:middle line:84% Because I better get my first thing is a tails. 00:46:47.700 --> 00:46:49.140 align:middle line:90% That's half probability. 00:46:49.140 --> 00:46:51.240 align:middle line:84% My second thing is tails and so on. 00:46:51.240 --> 00:46:53.480 align:middle line:84% And there are I plus 1 things here. 00:46:53.480 --> 00:46:57.200 align:middle line:84% So the chance of getting exactly the string is just 1/2 raised 00:46:57.200 --> 00:46:59.600 align:middle line:90% to the I plus 1. 00:46:59.600 --> 00:47:01.040 align:middle line:84% And now we have enough information 00:47:01.040 --> 00:47:03.880 align:middle line:84% that we can compute the expected value. 00:47:03.880 --> 00:47:08.360 align:middle line:84% So let's look at the random variable we'll be interested in 00:47:08.360 --> 00:47:15.480 align:middle line:84% will be just the number of coin tosses. 00:47:15.480 --> 00:47:19.660 align:middle line:84% So of course, this game in principle could go on forever, 00:47:19.660 --> 00:47:22.200 align:middle line:84% but the expected value gives us some baseline 00:47:22.200 --> 00:47:24.800 align:middle line:90% of what we expect it to be. 00:47:24.800 --> 00:47:28.040 align:middle line:84% We're going to talk about a lot fancier tools later 00:47:28.040 --> 00:47:29.840 align:middle line:90% in a few lectures' time. 00:47:29.840 --> 00:47:32.500 align:middle line:84% Because when we get back to random variables, 00:47:32.500 --> 00:47:34.880 align:middle line:84% we're going to talk about tools for arguing 00:47:34.880 --> 00:47:37.440 align:middle line:84% why random variables in some cases 00:47:37.440 --> 00:47:40.240 align:middle line:84% are very close to their expectation. 00:47:40.240 --> 00:47:42.810 align:middle line:84% That's very important in things like understanding 00:47:42.810 --> 00:47:44.450 align:middle line:90% the error of polling. 00:47:44.450 --> 00:47:46.330 align:middle line:84% But for right now, we just want to work 00:47:46.330 --> 00:47:49.130 align:middle line:84% with what a random variable is and get 00:47:49.130 --> 00:47:54.810 align:middle line:84% some crude idea of what we expect it to be approximately. 00:47:54.810 --> 00:47:59.370 align:middle line:84% So the natural way to do this, to compute the expectation of f, 00:47:59.370 --> 00:48:03.950 align:middle line:84% is literally just to use the expression we gave you. 00:48:03.950 --> 00:48:10.330 align:middle line:84% So there's a 1/2 probability that the game is over instantly 00:48:10.330 --> 00:48:14.370 align:middle line:84% and that we get heads to begin with. 00:48:14.370 --> 00:48:20.610 align:middle line:84% There's 1/4 probability that this game goes on for two total 00:48:20.610 --> 00:48:21.970 align:middle line:90% flips. 00:48:21.970 --> 00:48:27.630 align:middle line:84% There is 1/8 probability that it goes on for three total flips 00:48:27.630 --> 00:48:28.770 align:middle line:90% and so on. 00:48:28.770 --> 00:48:30.990 align:middle line:84% And so we can write this as a series. 00:48:30.990 --> 00:48:38.730 align:middle line:84% It's the sum from k equals 1 to infinity of k times 1 over 2 00:48:38.730 --> 00:48:41.100 align:middle line:90% to the k. 00:48:41.100 --> 00:48:43.020 align:middle line:84% Already on the homework, we've shown 00:48:43.020 --> 00:48:47.055 align:middle line:84% you some examples of how to evaluate sums like this. 00:48:47.055 --> 00:48:49.180 align:middle line:84% We're going to talk about this in a lot more detail 00:48:49.180 --> 00:48:51.020 align:middle line:84% when we get to the counting unit, which 00:48:51.020 --> 00:48:53.140 align:middle line:84% is right after the probability unit. 00:48:53.140 --> 00:48:55.920 align:middle line:84% So we'll give you tools not only for evaluating the sums, 00:48:55.920 --> 00:48:59.380 align:middle line:84% but for thinking about what kinds of things they count. 00:48:59.380 --> 00:49:02.540 align:middle line:84% But in any case, we can evaluate this very quickly 00:49:02.540 --> 00:49:06.460 align:middle line:84% just by arranging things in the right way. 00:49:06.460 --> 00:49:08.340 align:middle line:84% So from this expression, it's not 00:49:08.340 --> 00:49:11.020 align:middle line:84% that easy to guess what the answer is. 00:49:11.020 --> 00:49:15.980 align:middle line:84% But if we break up this quarter, this 2 times 1/4 into the sum 00:49:15.980 --> 00:49:20.340 align:middle line:90% of 2/4, it becomes a lot easier. 00:49:20.340 --> 00:49:22.540 align:middle line:84% So let's write it out pictorially. 00:49:22.540 --> 00:49:26.820 align:middle line:84% We have 1/2 from the first term, and then we have 2/4. 00:49:26.820 --> 00:49:28.236 align:middle line:90% And then we have 3/8. 00:49:28.236 --> 00:49:33.460 align:middle line:90% 00:49:33.460 --> 00:49:35.660 align:middle line:90% And this thing goes on. 00:49:35.660 --> 00:49:37.910 align:middle line:84% And what we can do is we can evaluate 00:49:37.910 --> 00:49:42.110 align:middle line:84% the sum in a different way, line by line, row by row first. 00:49:42.110 --> 00:49:46.830 align:middle line:84% So if we have 1/2 plus 1/4 plus 1/8 plus 1/16 and so on. 00:49:46.830 --> 00:49:48.470 align:middle line:84% Well, this, you probably all know, 00:49:48.470 --> 00:49:52.470 align:middle line:84% the answer is 1 for what that part of the sum contributes. 00:49:52.470 --> 00:49:56.870 align:middle line:84% And if instead we have 1/4 plus 1/8 plus 1/16, well, 00:49:56.870 --> 00:49:58.990 align:middle line:90% that's going to give us 1/2. 00:49:58.990 --> 00:50:01.510 align:middle line:84% And the next thing will give us 1/4. 00:50:01.510 --> 00:50:04.750 align:middle line:84% And now when we sum up all of these guys along the column, 00:50:04.750 --> 00:50:06.390 align:middle line:90% we're going to get 2. 00:50:06.390 --> 00:50:08.830 align:middle line:90% So this is very natural. 00:50:08.830 --> 00:50:11.070 align:middle line:84% The expected number of flips for a coin 00:50:11.070 --> 00:50:14.550 align:middle line:84% until you get your first heads is 2. 00:50:14.550 --> 00:50:15.140 align:middle line:90% Makes sense. 00:50:15.140 --> 00:50:21.430 align:middle line:90% 00:50:21.430 --> 00:50:25.470 align:middle line:84% And in fact, we can do this in a more clever way too. 00:50:25.470 --> 00:50:27.610 align:middle line:84% We'll use this in a minute, actually. 00:50:27.610 --> 00:50:33.030 align:middle line:84% So let me explain the even more general setting. 00:50:33.030 --> 00:50:37.640 align:middle line:84% Which leads me to something called mean time to failure. 00:50:37.640 --> 00:50:40.200 align:middle line:84% This is something that comes up in a lot of computer science 00:50:40.200 --> 00:50:41.960 align:middle line:90% applications. 00:50:41.960 --> 00:50:44.340 align:middle line:84% Let me give you the story behind it, 00:50:44.340 --> 00:50:48.320 align:middle line:84% but it'll really be the same kind of exercise we just did. 00:50:48.320 --> 00:50:53.800 align:middle line:84% Imagine we've got some computer as a part of some giant 00:50:53.800 --> 00:50:55.720 align:middle line:90% computing system. 00:50:55.720 --> 00:51:01.960 align:middle line:84% And it's not a perfect computer, and it fails independently 00:51:01.960 --> 00:51:14.080 align:middle line:84% on each day with some probability P. 00:51:14.080 --> 00:51:16.460 align:middle line:84% Well, one of the things you might be interested in, 00:51:16.460 --> 00:51:23.560 align:middle line:84% what's called the mean time to failure, 00:51:23.560 --> 00:51:26.440 align:middle line:84% is how many days you expect the computer 00:51:26.440 --> 00:51:29.520 align:middle line:90% to last before you replace it. 00:51:29.520 --> 00:51:31.700 align:middle line:84% This is a very basic probabilistic statement. 00:51:31.700 --> 00:51:34.280 align:middle line:84% It shows up all the time, especially 00:51:34.280 --> 00:51:36.450 align:middle line:84% in things like actuarial sciences 00:51:36.450 --> 00:51:39.050 align:middle line:90% or literally in computing. 00:51:39.050 --> 00:51:43.050 align:middle line:84% Because when you're building giant computing systems 00:51:43.050 --> 00:51:47.330 align:middle line:84% like ChatGPT, you have to create this huge infrastructure 00:51:47.330 --> 00:51:49.610 align:middle line:84% of computers, and they often fail. 00:51:49.610 --> 00:51:52.370 align:middle line:84% And you have to have some idea for how often that's 00:51:52.370 --> 00:51:56.330 align:middle line:84% going to happen and how much the cost is to replace these parts, 00:51:56.330 --> 00:52:00.050 align:middle line:84% to figure out economically what you're sinking into the system. 00:52:00.050 --> 00:52:03.270 align:middle line:84% But we can do the same exercise we did the same way. 00:52:03.270 --> 00:52:05.610 align:middle line:84% I'll do it in a slightly more clever way. 00:52:05.610 --> 00:52:09.730 align:middle line:84% So we have this random variable T. That is the time 00:52:09.730 --> 00:52:12.410 align:middle line:90% that this computer fails. 00:52:12.410 --> 00:52:15.090 align:middle line:84% So we have some probability P that it 00:52:15.090 --> 00:52:17.130 align:middle line:90% happens on the first day. 00:52:17.130 --> 00:52:21.570 align:middle line:84% And then we have some probability 1 minus P times P 00:52:21.570 --> 00:52:23.770 align:middle line:84% that it happens on the second day. 00:52:23.770 --> 00:52:28.010 align:middle line:84% Let's just do out one more, 1 minus P squared times P, 00:52:28.010 --> 00:52:30.810 align:middle line:84% then it happens on the third day. 00:52:30.810 --> 00:52:32.950 align:middle line:84% But there's a clever trick for doing 00:52:32.950 --> 00:52:38.510 align:middle line:84% this, which is that this is equal to P from this first term 00:52:38.510 --> 00:52:46.270 align:middle line:84% times 1 minus P times 1 plus the expectation of T. 00:52:46.270 --> 00:52:48.950 align:middle line:90% So this looks a bit magical. 00:52:48.950 --> 00:52:51.870 align:middle line:84% Can anyone give me some intuition for what exactly 00:52:51.870 --> 00:52:53.690 align:middle line:90% I just did in this expression? 00:52:53.690 --> 00:52:58.390 align:middle line:90% 00:52:58.390 --> 00:52:59.570 align:middle line:90% Why is this true? 00:52:59.570 --> 00:53:00.070 align:middle line:90% Yeah. 00:53:00.070 --> 00:53:02.750 align:middle line:90% 00:53:02.750 --> 00:53:04.810 align:middle line:90% That's right, that's right. 00:53:04.810 --> 00:53:07.210 align:middle line:90% So first I got my P from here. 00:53:07.210 --> 00:53:10.470 align:middle line:84% I pulled out this 1 minus P, because everything's got a 1 00:53:10.470 --> 00:53:12.030 align:middle line:90% minus P in front of it. 00:53:12.030 --> 00:53:15.270 align:middle line:84% But then the important thing is that when 00:53:15.270 --> 00:53:19.950 align:middle line:84% I don't fail on the first day, the entire process starts over 00:53:19.950 --> 00:53:21.070 align:middle line:90% again. 00:53:21.070 --> 00:53:23.970 align:middle line:84% It has no memory of what happened in the past. 00:53:23.970 --> 00:53:26.190 align:middle line:90% I've gone one extra day. 00:53:26.190 --> 00:53:29.550 align:middle line:84% But then whatever happens to that same identical random 00:53:29.550 --> 00:53:31.680 align:middle line:84% variable that I care about is going 00:53:31.680 --> 00:53:34.500 align:middle line:84% to be what happens to the process thereon out. 00:53:34.500 --> 00:53:37.060 align:middle line:84% So in fact, this makes it much easier. 00:53:37.060 --> 00:53:39.380 align:middle line:84% Instead of messing around with these infinite sums, 00:53:39.380 --> 00:53:42.400 align:middle line:84% I can just rearrange things algebraically. 00:53:42.400 --> 00:53:46.680 align:middle line:84% And what I'll get is that the expectation of T 00:53:46.680 --> 00:53:55.760 align:middle line:84% is equal to 1 over P. Any questions? 00:53:55.760 --> 00:53:57.200 align:middle line:90% Make sense? 00:53:57.200 --> 00:53:58.240 align:middle line:90% All right. 00:53:58.240 --> 00:54:00.080 align:middle line:84% So the mean time to failure just depends 00:54:00.080 --> 00:54:02.660 align:middle line:84% on this failure probability in this very simple way. 00:54:02.660 --> 00:54:08.160 align:middle line:90% 00:54:08.160 --> 00:54:11.300 align:middle line:84% So let's get back to expectation. 00:54:11.300 --> 00:54:14.360 align:middle line:84% And then I'm going to tell you something unlike the Monty Hall 00:54:14.360 --> 00:54:18.200 align:middle line:84% problem that you're definitely going to find counterintuitive. 00:54:18.200 --> 00:54:20.520 align:middle line:84% So we talked about random variables. 00:54:20.520 --> 00:54:21.910 align:middle line:90% We talked about events. 00:54:21.910 --> 00:54:24.880 align:middle line:90% 00:54:24.880 --> 00:54:28.500 align:middle line:84% Actually, there's a way to go back and forth between the two. 00:54:28.500 --> 00:54:36.490 align:middle line:90% 00:54:36.490 --> 00:54:41.490 align:middle line:84% So it turns out that we can actually treat events 00:54:41.490 --> 00:54:43.630 align:middle line:90% as random variables. 00:54:43.630 --> 00:54:47.312 align:middle line:90% 00:54:47.312 --> 00:54:48.770 align:middle line:84% This will be a very important trick 00:54:48.770 --> 00:54:51.930 align:middle line:84% that we're going to use all the time. 00:54:51.930 --> 00:54:55.930 align:middle line:84% So this is called the indicator function. 00:54:55.930 --> 00:55:11.090 align:middle line:84% So in particular, if you give me an event A, well, 00:55:11.090 --> 00:55:14.750 align:middle line:84% I can define this function that has this funny notation. 00:55:14.750 --> 00:55:19.450 align:middle line:84% Usually you write it as a bold faced 1 with an A subscript. 00:55:19.450 --> 00:55:21.470 align:middle line:84% But this is just a random variable. 00:55:21.470 --> 00:55:25.430 align:middle line:84% It takes in some outcome from the sample space, 00:55:25.430 --> 00:55:28.260 align:middle line:90% and it spits out a real value. 00:55:28.260 --> 00:55:34.980 align:middle line:84% It spits out the value 1 if your outcome 00:55:34.980 --> 00:55:37.960 align:middle line:90% belongs to the event, fine. 00:55:37.960 --> 00:55:41.500 align:middle line:90% And it outputs a 0 otherwise. 00:55:41.500 --> 00:55:46.860 align:middle line:84% So in particular, if you give me an event, 00:55:46.860 --> 00:55:49.320 align:middle line:84% this describes an associated random variable. 00:55:49.320 --> 00:55:51.060 align:middle line:90% This is just a recipe for it. 00:55:51.060 --> 00:55:53.400 align:middle line:84% My random variable takes on real values. 00:55:53.400 --> 00:55:56.060 align:middle line:84% Actually, it takes on values just 1 or 0. 00:55:56.060 --> 00:56:00.700 align:middle line:84% It's 1 if the event is on, it's 0 otherwise. 00:56:00.700 --> 00:56:02.860 align:middle line:84% And now, the good thing is that a lot 00:56:02.860 --> 00:56:06.900 align:middle line:84% of what we do with random variables, 00:56:06.900 --> 00:56:09.540 align:middle line:84% we can use to reason about events. 00:56:09.540 --> 00:56:11.980 align:middle line:84% So let's connect this back to events. 00:56:11.980 --> 00:56:17.140 align:middle line:84% So I can look at, for example, the expectation of my indicator 00:56:17.140 --> 00:56:21.198 align:middle line:90% function over a random outcome. 00:56:21.198 --> 00:56:22.740 align:middle line:84% So let's just be clear, because we've 00:56:22.740 --> 00:56:26.990 align:middle line:84% introduced a lot of notation already today for probability. 00:56:26.990 --> 00:56:29.250 align:middle line:84% Remember that when I write this bold E, 00:56:29.250 --> 00:56:32.310 align:middle line:84% I'm looking at that experiment where I throw my dart 00:56:32.310 --> 00:56:35.270 align:middle line:84% and I get some outcome in my space. 00:56:35.270 --> 00:56:38.270 align:middle line:90% That tells me what x I have. 00:56:38.270 --> 00:56:41.510 align:middle line:84% And remember that an event is just whether or not 00:56:41.510 --> 00:56:44.510 align:middle line:84% x belongs to A. But what's going on here 00:56:44.510 --> 00:56:48.290 align:middle line:84% is that I have this function that just outputs 1 00:56:48.290 --> 00:56:52.270 align:middle line:84% if x belongs to A and outputs 0 otherwise. 00:56:52.270 --> 00:56:53.910 align:middle line:84% So in this case, we can write out 00:56:53.910 --> 00:56:56.830 align:middle line:84% what the definition of the expectation is. 00:56:56.830 --> 00:57:00.910 align:middle line:84% It's the sum over all outcomes in my set 00:57:00.910 --> 00:57:04.350 align:middle line:84% of the probability of the outcome times the value 00:57:04.350 --> 00:57:05.460 align:middle line:90% of the random variable. 00:57:05.460 --> 00:57:09.150 align:middle line:90% 00:57:09.150 --> 00:57:11.950 align:middle line:84% And this is the same thing as the sum over all 00:57:11.950 --> 00:57:15.910 align:middle line:90% x in A of the probability of x. 00:57:15.910 --> 00:57:19.630 align:middle line:84% Because the only way that this term survives 00:57:19.630 --> 00:57:24.550 align:middle line:84% is if x belongs to A. Otherwise, the term is 0, and it goes away. 00:57:24.550 --> 00:57:26.280 align:middle line:84% So this is really just bookkeeping 00:57:26.280 --> 00:57:30.280 align:middle line:84% for keeping track of which P of x is contribute to my sum. 00:57:30.280 --> 00:57:33.040 align:middle line:84% And this way, I change the subscript of my sum 00:57:33.040 --> 00:57:37.760 align:middle line:84% from being over all outcomes to only the outcomes in my set. 00:57:37.760 --> 00:57:41.200 align:middle line:84% So in particular, the nice thing is that we get back 00:57:41.200 --> 00:57:43.480 align:middle line:90% the probability of the event. 00:57:43.480 --> 00:57:46.280 align:middle line:84% So you can see that we can use random variables 00:57:46.280 --> 00:57:49.760 align:middle line:84% as a placeholder for computing things like the probabilities 00:57:49.760 --> 00:57:53.720 align:middle line:84% of events just by looking at their expectation. 00:57:53.720 --> 00:57:55.460 align:middle line:84% So this is just a bunch of bookkeeping. 00:57:55.460 --> 00:57:57.120 align:middle line:90% Does this make sense? 00:57:57.120 --> 00:57:58.440 align:middle line:90% Any questions? 00:57:58.440 --> 00:58:01.440 align:middle line:90% All right. 00:58:01.440 --> 00:58:04.480 align:middle line:84% So now one of the really powerful things 00:58:04.480 --> 00:58:09.960 align:middle line:84% about expectation is that it behaves really nicely 00:58:09.960 --> 00:58:10.980 align:middle line:90% in some settings. 00:58:10.980 --> 00:58:14.080 align:middle line:84% It behaves as nicely as you could want. 00:58:14.080 --> 00:58:16.220 align:middle line:84% So let me tell you a powerful fact. 00:58:16.220 --> 00:58:18.520 align:middle line:84% Some of you have probably already seen it before. 00:58:18.520 --> 00:58:21.800 align:middle line:90% But linearity of expectation. 00:58:21.800 --> 00:58:33.240 align:middle line:84% So it turns out that when you add random variables, 00:58:33.240 --> 00:58:34.990 align:middle line:84% computing their expectation is easy. 00:58:34.990 --> 00:58:43.580 align:middle line:90% 00:58:43.580 --> 00:58:48.380 align:middle line:84% So in particular, I can look at a random variable f of x. 00:58:48.380 --> 00:58:51.820 align:middle line:84% I can look at a random variable g of x. 00:58:51.820 --> 00:58:55.420 align:middle line:84% I mean, it seems intuitive, but it's not true. 00:58:55.420 --> 00:58:56.700 align:middle line:90% Yeah? 00:58:56.700 --> 00:58:59.940 align:middle line:84% STUDENT: If A and B both happen, then it 00:58:59.940 --> 00:59:02.720 align:middle line:84% gets double counted on the left side but not on the right side. 00:59:02.720 --> 00:59:04.540 align:middle line:84% ANKUR MOITRA: That's right, that's right. 00:59:04.540 --> 00:59:09.220 align:middle line:84% So what's going on here is we can go back to our diagram. 00:59:09.220 --> 00:59:14.820 align:middle line:84% We have these two events A and B. 00:59:14.820 --> 00:59:19.980 align:middle line:84% And when I look at the expectation of 1A of x, well, 00:59:19.980 --> 00:59:22.890 align:middle line:84% I'm just looking at this dart and throwing it in here. 00:59:22.890 --> 00:59:25.670 align:middle line:84% And every time it lands inside A, I'm getting $1. 00:59:25.670 --> 00:59:28.790 align:middle line:84% Every time it lands outside, I'm getting no dollars. 00:59:28.790 --> 00:59:32.390 align:middle line:84% And I'm looking at the expected number of dollars I win. 00:59:32.390 --> 00:59:34.590 align:middle line:84% And that's the same thing as the probability 00:59:34.590 --> 00:59:36.790 align:middle line:84% that the dart actually lands in that set. 00:59:36.790 --> 00:59:40.170 align:middle line:84% And now same thing is true for B. 00:59:40.170 --> 00:59:43.110 align:middle line:84% But when I add these two things up, what's going on 00:59:43.110 --> 00:59:46.430 align:middle line:84% is if I actually landed in the intersection of these two, 00:59:46.430 --> 00:59:48.050 align:middle line:90% I'd be getting $2. 00:59:48.050 --> 00:59:52.950 align:middle line:84% So really what's going on is that this is not equal, 00:59:52.950 --> 00:59:56.630 align:middle line:84% because this is no longer an indicator random variable. 00:59:56.630 --> 00:59:59.950 align:middle line:84% It doesn't just take the value 0 and 1. 00:59:59.950 --> 01:00:02.630 align:middle line:84% It also takes the value 2 sometimes. 01:00:02.630 --> 01:00:05.470 align:middle line:84% The only way this would be true is if A and B were actually 01:00:05.470 --> 01:00:07.670 align:middle line:90% disjoint. 01:00:07.670 --> 01:00:08.170 align:middle line:90% All right. 01:00:08.170 --> 01:00:09.950 align:middle line:90% Does that make sense? 01:00:09.950 --> 01:00:11.590 align:middle line:90% All right. 01:00:11.590 --> 01:00:12.220 align:middle line:90% Sounds good. 01:00:12.220 --> 01:00:15.310 align:middle line:90% 01:00:15.310 --> 01:00:21.880 align:middle line:84% So I'll tell you one really interesting problem, 01:00:21.880 --> 01:00:26.240 align:middle line:84% and then we'll end there, and we'll go on to other topics 01:00:26.240 --> 01:00:28.240 align:middle line:90% next time. 01:00:28.240 --> 01:00:32.520 align:middle line:84% So you all have seen the Monty Hall problem before. 01:00:32.520 --> 01:00:33.490 align:middle line:90% That's no surprise. 01:00:33.490 --> 01:00:37.800 align:middle line:90% 01:00:37.800 --> 01:00:40.660 align:middle line:84% It turns out that conditional probability, 01:00:40.660 --> 01:00:42.360 align:middle line:84% conditional expectations can still 01:00:42.360 --> 01:00:46.800 align:middle line:84% be very counterintuitive in ways that even 01:00:46.800 --> 01:00:49.440 align:middle line:84% math professors can be totally wrong 01:00:49.440 --> 01:00:52.120 align:middle line:90% and have the wrong intuition. 01:00:52.120 --> 01:00:56.400 align:middle line:84% So let me tell you about this amazing example that's 01:00:56.400 --> 01:01:00.280 align:middle line:90% due to my colleague upstairs. 01:01:00.280 --> 01:01:02.840 align:middle line:84% In fact, I saw him right before class. 01:01:02.840 --> 01:01:07.520 align:middle line:84% It's called Mossel's dice paradox. 01:01:07.520 --> 01:01:10.540 align:middle line:84% Elchanan Mossel is a very famous probabilist. 01:01:10.540 --> 01:01:14.040 align:middle line:84% And when he came up with this, his advisor told him, 01:01:14.040 --> 01:01:16.000 align:middle line:84% I know you've done a lot of great research 01:01:16.000 --> 01:01:18.000 align:middle line:84% and you've proved amazing things, 01:01:18.000 --> 01:01:20.370 align:middle line:84% but this will be what you're famous for. 01:01:20.370 --> 01:01:23.690 align:middle line:90% And he was very disappointed. 01:01:23.690 --> 01:01:25.890 align:middle line:90% So here is the experiment. 01:01:25.890 --> 01:01:31.360 align:middle line:84% You can throw a dice until you get a 6. 01:01:31.360 --> 01:01:35.730 align:middle line:90% 01:01:35.730 --> 01:01:39.450 align:middle line:84% And we're going to be interested in the number of rolls 01:01:39.450 --> 01:01:41.550 align:middle line:84% that you have to take in expectation. 01:01:41.550 --> 01:01:44.030 align:middle line:84% Now, if that were it, there would be no paradox. 01:01:44.030 --> 01:01:46.850 align:middle line:84% We already talked about that for the mean time to failure. 01:01:46.850 --> 01:01:52.450 align:middle line:84% Because if my probability of rolling a 6 is 1/6, 01:01:52.450 --> 01:01:54.750 align:middle line:84% I can look at the expected number of rolls, 01:01:54.750 --> 01:01:56.030 align:middle line:90% and that would just be 6. 01:01:56.030 --> 01:01:57.330 align:middle line:90% Fine. 01:01:57.330 --> 01:01:58.858 align:middle line:90% So nothing is hard here. 01:01:58.858 --> 01:02:00.650 align:middle line:84% But the experiment is you're going to throw 01:02:00.650 --> 01:02:03.650 align:middle line:90% a dice until you get a 6. 01:02:03.650 --> 01:02:08.250 align:middle line:84% But now we want to compute the conditional expectation. 01:02:08.250 --> 01:02:20.180 align:middle line:84% So conditioned on all the throws being even, 01:02:20.180 --> 01:02:23.500 align:middle line:84% what if I tell you that all the throws are even? 01:02:23.500 --> 01:02:31.410 align:middle line:84% What is the expected number of rolls? 01:02:31.410 --> 01:02:36.620 align:middle line:90% 01:02:36.620 --> 01:02:38.597 align:middle line:90% So very simple problem. 01:02:38.597 --> 01:02:40.680 align:middle line:84% It takes a little bit to wrap your head around it. 01:02:40.680 --> 01:02:43.300 align:middle line:84% But the way that this experiment goes, 01:02:43.300 --> 01:02:46.660 align:middle line:84% I'm sitting here throwing the die until I get a 6. 01:02:46.660 --> 01:02:48.640 align:middle line:84% And you're not watching me do the experiment. 01:02:48.640 --> 01:02:51.800 align:middle line:84% But then before I ask you anything about the experiment, 01:02:51.800 --> 01:02:55.200 align:middle line:84% I tell you, actually, while I was doing it, I only got evens. 01:02:55.200 --> 01:02:56.940 align:middle line:90% It was the darndest thing. 01:02:56.940 --> 01:03:00.120 align:middle line:84% And now telling you that I only got even throws, 01:03:00.120 --> 01:03:04.660 align:middle line:84% I'm asking you how that changes your expected number of rolls. 01:03:04.660 --> 01:03:09.260 align:middle line:84% So it turns out that there's a very natural way 01:03:09.260 --> 01:03:11.540 align:middle line:90% to try and answer this question. 01:03:11.540 --> 01:03:15.190 align:middle line:84% And almost everyone who sees this the first time 01:03:15.190 --> 01:03:16.850 align:middle line:90% gets it wrong the same way. 01:03:16.850 --> 01:03:19.030 align:middle line:90% It's amazing. 01:03:19.030 --> 01:03:23.070 align:middle line:84% So there's one argument that leads you to the wrong answer. 01:03:23.070 --> 01:03:33.270 align:middle line:84% So you'd claim, you'd think, that the answer is 3. 01:03:33.270 --> 01:03:35.130 align:middle line:84% Let me tell you why you might think that. 01:03:35.130 --> 01:03:37.910 align:middle line:84% So let's go through what we know. 01:03:37.910 --> 01:03:43.180 align:middle line:84% So we know that all rolls are 2, 4, 01:03:43.180 --> 01:03:48.330 align:middle line:84% or 6, because you just promised me that all the roles were even. 01:03:48.330 --> 01:03:49.930 align:middle line:84% So those are the only possibilities. 01:03:49.930 --> 01:03:53.150 align:middle line:84% Nothing else shows up when I roll the die. 01:03:53.150 --> 01:03:56.810 align:middle line:84% And now, if you're rolling a die and it's only got three sides, 01:03:56.810 --> 01:03:59.750 align:middle line:84% 2, 4, and 6, then how long do you 01:03:59.750 --> 01:04:02.270 align:middle line:84% have to go until you end up with your first 01:04:02.270 --> 01:04:04.430 align:middle line:90% 6 and stop the experiment? 01:04:04.430 --> 01:04:07.190 align:middle line:84% Well, 6 is 1/3 of the possibilities. 01:04:07.190 --> 01:04:11.310 align:middle line:84% So you'd expect just by this argument from this mean time 01:04:11.310 --> 01:04:18.170 align:middle line:84% to failure, that maybe the expected number of rolls is 1 01:04:18.170 --> 01:04:19.350 align:middle line:90% over 1/3. 01:04:19.350 --> 01:04:21.650 align:middle line:90% So namely 3. 01:04:21.650 --> 01:04:23.890 align:middle line:84% And this is actually completely wrong. 01:04:23.890 --> 01:04:26.650 align:middle line:84% So in fact, Elchanan was teaching over at Berkeley 01:04:26.650 --> 01:04:29.050 align:middle line:84% when he came up with this example, 01:04:29.050 --> 01:04:32.410 align:middle line:84% and he was doing a review for some probability class. 01:04:32.410 --> 01:04:35.307 align:middle line:84% And he wanted to make this argument that probability 01:04:35.307 --> 01:04:36.890 align:middle line:84% is very counterintuitive, and you just 01:04:36.890 --> 01:04:41.170 align:middle line:84% have to compute the darn algebraic expression to do it. 01:04:41.170 --> 01:04:43.230 align:middle line:84% And he came up with this random example. 01:04:43.230 --> 01:04:45.070 align:middle line:84% There is no special intuition behind it. 01:04:45.070 --> 01:04:46.930 align:middle line:90% He came up with it on the fly. 01:04:46.930 --> 01:04:49.290 align:middle line:84% And then he started doing the proof out, 01:04:49.290 --> 01:04:51.570 align:middle line:90% and he got the answer 3/2. 01:04:51.570 --> 01:04:54.330 align:middle line:84% And he was so convinced that somewhere in there 01:04:54.330 --> 01:04:58.090 align:middle line:84% was some algebraic mistake he'd made on the board. 01:04:58.090 --> 01:05:00.530 align:middle line:84% So the truth is, when you're speaking to a large audience, 01:05:00.530 --> 01:05:04.570 align:middle line:84% your IQ decreases by 20 points at least. 01:05:04.570 --> 01:05:07.730 align:middle line:84% And he told them that, don't worry, 01:05:07.730 --> 01:05:09.915 align:middle line:84% I'll tell you what's wrong with my computation, 01:05:09.915 --> 01:05:11.290 align:middle line:84% and I'll email the class and I'll 01:05:11.290 --> 01:05:14.700 align:middle line:84% tell you what I got wrong so that you're ready for your exam. 01:05:14.700 --> 01:05:16.400 align:middle line:90% He went home and he checked it. 01:05:16.400 --> 01:05:19.280 align:middle line:90% And the darnedest thing. 01:05:19.280 --> 01:05:20.920 align:middle line:90% The answer is actually 3/2. 01:05:20.920 --> 01:05:23.140 align:middle line:84% It's a very counterintuitive problem. 01:05:23.140 --> 01:05:25.280 align:middle line:84% In fact, I remember, I'll tell you-- 01:05:25.280 --> 01:05:26.520 align:middle line:90% I guess this is videotaped. 01:05:26.520 --> 01:05:28.820 align:middle line:84% I'm not sure how much I should say. 01:05:28.820 --> 01:05:32.880 align:middle line:84% But we have a mailing list among some of the three faculty, 01:05:32.880 --> 01:05:36.500 align:middle line:84% especially ones who teach some of the intro proof based 01:05:36.500 --> 01:05:38.060 align:middle line:90% math classes. 01:05:38.060 --> 01:05:39.820 align:middle line:84% And then when this thing came out, 01:05:39.820 --> 01:05:41.820 align:middle line:84% David Karger emailed me and told me, 01:05:41.820 --> 01:05:44.680 align:middle line:84% this would be a great example for 042. 01:05:44.680 --> 01:05:46.060 align:middle line:90% We should do this. 01:05:46.060 --> 01:05:48.340 align:middle line:84% And one faculty member who shall not be named 01:05:48.340 --> 01:05:51.860 align:middle line:84% was on the mailing list and did not believe the answer. 01:05:51.860 --> 01:05:53.200 align:middle line:90% It was the craziest thing. 01:05:53.200 --> 01:05:56.780 align:middle line:84% He was arguing with us back and forth on this thread I have. 01:05:56.780 --> 01:06:00.780 align:middle line:84% And he just was convinced that the answer was 3. 01:06:00.780 --> 01:06:03.040 align:middle line:84% And actually, should I cut the video for this? 01:06:03.040 --> 01:06:06.360 align:middle line:84% So let me tell you the other part of the story. 01:06:06.360 --> 01:06:10.350 align:middle line:84% So then he went on this debate for maybe a month or two, 01:06:10.350 --> 01:06:12.870 align:middle line:84% not believing the answers we gave him, even about this very 01:06:12.870 --> 01:06:14.470 align:middle line:90% basic statement. 01:06:14.470 --> 01:06:16.710 align:middle line:84% And then he finally came into our theory faculty 01:06:16.710 --> 01:06:19.320 align:middle line:84% lunch, which I have to go to after this, and he said, 01:06:19.320 --> 01:06:21.330 align:middle line:90% aha I understand what was wrong. 01:06:21.330 --> 01:06:23.130 align:middle line:84% You were all explaining it wrong. 01:06:23.130 --> 01:06:25.190 align:middle line:90% Let me tell you my explanation. 01:06:25.190 --> 01:06:28.430 align:middle line:84% So he started explaining it in this very convoluted way. 01:06:28.430 --> 01:06:30.830 align:middle line:84% And Peter was there with me at lunch. 01:06:30.830 --> 01:06:32.430 align:middle line:90% Peter was co-teaching. 01:06:32.430 --> 01:06:34.710 align:middle line:84% And Peter usually doesn't say that much in the faculty 01:06:34.710 --> 01:06:35.210 align:middle line:90% lunches. 01:06:35.210 --> 01:06:37.550 align:middle line:84% And he said, now I know why no one understands 01:06:37.550 --> 01:06:42.010 align:middle line:84% probability when you teach it, which is kind of amazing. 01:06:42.010 --> 01:06:45.070 align:middle line:84% So I'll scrub the rest of the names from the story. 01:06:45.070 --> 01:06:48.470 align:middle line:84% But we'll do some examples in class where we actually 01:06:48.470 --> 01:06:52.030 align:middle line:84% ask you to compute this, and you'll 01:06:52.030 --> 01:06:54.170 align:middle line:90% see some tricks for doing this. 01:06:54.170 --> 01:06:57.030 align:middle line:84% And it'll be a good exercise for some of your writing problems. 01:06:57.030 --> 01:06:59.230 align:middle line:84% So we'll stop there a bit early, and next time we 01:06:59.230 --> 01:07:02.350 align:middle line:84% will continue with inclusion, exclusion and other applications 01:07:02.350 --> 01:07:04.660 align:middle line:90% in probability. 01:07:04.660 --> 01:07:10.000 align:middle line:90%