WEBVTT 00:00:00.000 --> 00:00:01.920 align:middle line:90% [SQUEAKING] 00:00:01.920 --> 00:00:03.840 align:middle line:90% [RUSTLING] 00:00:03.840 --> 00:00:05.280 align:middle line:90% [CLICKING] 00:00:05.280 --> 00:00:11.040 align:middle line:90% 00:00:11.040 --> 00:00:12.820 align:middle line:84% PROFESSOR: Welcome back OpenCourseWare. 00:00:12.820 --> 00:00:17.360 align:middle line:84% So we're talking about random variables and expectation. 00:00:17.360 --> 00:00:19.840 align:middle line:84% And in fact, one theme that I want to mention 00:00:19.840 --> 00:00:22.760 align:middle line:84% is that a bunch of our probability lectures 00:00:22.760 --> 00:00:24.880 align:middle line:84% will be about how to get tools that 00:00:24.880 --> 00:00:29.260 align:middle line:84% give us a finer grained control of how random variables behave. 00:00:29.260 --> 00:00:30.760 align:middle line:84% At the end of today's lecture, we're 00:00:30.760 --> 00:00:32.020 align:middle line:90% going to talk about variance. 00:00:32.020 --> 00:00:33.395 align:middle line:84% And later on, we're going to talk 00:00:33.395 --> 00:00:35.120 align:middle line:90% about things like tail bounds. 00:00:35.120 --> 00:00:38.720 align:middle line:84% All right, so let me start off with the main motivating problem 00:00:38.720 --> 00:00:41.360 align:middle line:84% that we're going to spend the first part of class talking 00:00:41.360 --> 00:00:42.280 align:middle line:90% about. 00:00:42.280 --> 00:00:44.728 align:middle line:84% This is a challenging problem that usually, 00:00:44.728 --> 00:00:46.520 align:middle line:84% the way I would get students' attention was 00:00:46.520 --> 00:00:49.578 align:middle line:84% I would say that if you're going to do tech or finance 00:00:49.578 --> 00:00:51.120 align:middle line:84% interviews, this is the kind of thing 00:00:51.120 --> 00:00:53.360 align:middle line:84% that they ask you during the interviews. 00:00:53.360 --> 00:00:56.260 align:middle line:84% But the truth is that everyone knows that they ask this, 00:00:56.260 --> 00:00:58.520 align:middle line:84% so they no longer ask this anymore. 00:00:58.520 --> 00:01:01.170 align:middle line:84% But it's still a really fun problem to think about. 00:01:01.170 --> 00:01:03.770 align:middle line:84% So imagine we have n students in class, 00:01:03.770 --> 00:01:05.590 align:middle line:84% and they each turn in their psets. 00:01:05.590 --> 00:01:06.650 align:middle line:90% We grade them. 00:01:06.650 --> 00:01:09.090 align:middle line:84% And then, instead of figuring out who's who, 00:01:09.090 --> 00:01:12.770 align:middle line:84% we just randomly give out psets to different students. 00:01:12.770 --> 00:01:14.490 align:middle line:84% Well, one of the probability questions 00:01:14.490 --> 00:01:17.010 align:middle line:84% we can ask, which will be an excuse 00:01:17.010 --> 00:01:19.410 align:middle line:84% to learn a very important tool, as we'll 00:01:19.410 --> 00:01:24.210 align:middle line:84% be interested in what's the probability 00:01:24.210 --> 00:01:26.170 align:middle line:90% that we get everything wrong. 00:01:26.170 --> 00:01:30.170 align:middle line:84% So no one gets their correct pset. 00:01:30.170 --> 00:01:34.390 align:middle line:90% 00:01:34.390 --> 00:01:37.970 align:middle line:84% OK, so we're going to see a very important tool, 00:01:37.970 --> 00:01:41.850 align:middle line:84% called inclusion-exclusion, in order to answer this question. 00:01:41.850 --> 00:01:45.250 align:middle line:84% But let's build up towards it and get some intuition. 00:01:45.250 --> 00:01:46.850 align:middle line:84% So the first thing I want to do is 00:01:46.850 --> 00:01:49.850 align:middle line:84% I want to understand this probability of this very 00:01:49.850 --> 00:01:54.310 align:middle line:84% complicated thing by breaking it up into simpler events. 00:01:54.310 --> 00:01:55.810 align:middle line:84% So what I'm going to do is I'm going 00:01:55.810 --> 00:01:58.420 align:middle line:84% to define a bunch of helper events 00:01:58.420 --> 00:02:01.180 align:middle line:84% that will help me compute this probability. 00:02:01.180 --> 00:02:08.419 align:middle line:84% So I'm going to let A i denote the event that the i-th student 00:02:08.419 --> 00:02:10.900 align:middle line:90% gets their correct pset. 00:02:10.900 --> 00:02:15.340 align:middle line:90% 00:02:15.340 --> 00:02:17.960 align:middle line:84% So you can think about an event a few different ways. 00:02:17.960 --> 00:02:21.220 align:middle line:84% An event is literally a subset of the sample space. 00:02:21.220 --> 00:02:22.900 align:middle line:84% But that's not how I'm defining it here, 00:02:22.900 --> 00:02:26.260 align:middle line:84% because the sample space is gigantic to write down. 00:02:26.260 --> 00:02:28.140 align:middle line:84% So instead, I'm really defining it 00:02:28.140 --> 00:02:30.140 align:middle line:84% in words as a predicate that's applied 00:02:30.140 --> 00:02:31.860 align:middle line:90% to the different outcomes. 00:02:31.860 --> 00:02:34.180 align:middle line:84% So this is one event I care about. 00:02:34.180 --> 00:02:37.180 align:middle line:84% And then, the ultimate thing we care about 00:02:37.180 --> 00:02:42.360 align:middle line:84% is we want to understand the following probability. 00:02:42.360 --> 00:02:45.300 align:middle line:90% 00:02:45.300 --> 00:02:47.500 align:middle line:84% We want to look at the probability 00:02:47.500 --> 00:02:50.540 align:middle line:84% that A 1 does not happen, the first student 00:02:50.540 --> 00:02:53.100 align:middle line:84% does not get their correct pset back, 00:02:53.100 --> 00:02:56.720 align:middle line:84% the second student does not get their correct pset back, 00:02:56.720 --> 00:02:58.840 align:middle line:84% all the way down to the n-th student 00:02:58.840 --> 00:03:03.400 align:middle line:84% also does not get their correct pset back. 00:03:03.400 --> 00:03:05.460 align:middle line:84% Now, we can write this a different way, 00:03:05.460 --> 00:03:07.600 align:middle line:84% which will be more convenient for the way 00:03:07.600 --> 00:03:09.480 align:middle line:84% that we're going to think about this. 00:03:09.480 --> 00:03:13.640 align:middle line:84% This is the same thing as 1 minus the probability 00:03:13.640 --> 00:03:16.720 align:middle line:84% that the complement of this event happens. 00:03:16.720 --> 00:03:20.000 align:middle line:84% And what is the complement of the event that no student gets 00:03:20.000 --> 00:03:24.720 align:middle line:84% their pset correct is that at least one student gets 00:03:24.720 --> 00:03:27.080 align:middle line:90% their correct pset back? 00:03:27.080 --> 00:03:30.760 align:middle line:84% So in particular, we take the union of all of these events, 00:03:30.760 --> 00:03:33.080 align:middle line:84% either the first student gets the correct pset, 00:03:33.080 --> 00:03:36.600 align:middle line:84% or the second student, or the n-th student. 00:03:36.600 --> 00:03:40.160 align:middle line:84% So this is really the thing we want to compute today. 00:03:40.160 --> 00:03:44.040 align:middle line:84% So let's think about this in some special cases, which 00:03:44.040 --> 00:03:52.360 align:middle line:84% I claim we can always go back to our friend, the Venn diagram, 00:03:52.360 --> 00:03:55.890 align:middle line:84% to think about what's happening in this space. 00:03:55.890 --> 00:03:59.370 align:middle line:84% So let's write out a simple Venn diagram just in the case 00:03:59.370 --> 00:04:01.170 align:middle line:90% where n equals 2. 00:04:01.170 --> 00:04:03.650 align:middle line:90% So I have my sample space. 00:04:03.650 --> 00:04:06.810 align:middle line:90% I have my event A 1. 00:04:06.810 --> 00:04:10.010 align:middle line:90% I have my event A 2. 00:04:10.010 --> 00:04:13.450 align:middle line:84% And what I care about is the probability that I throw a dart 00:04:13.450 --> 00:04:16.050 align:middle line:84% and that it lands in the union of A 1 00:04:16.050 --> 00:04:21.329 align:middle line:84% and A 2, which is just this shaded region right here. 00:04:21.329 --> 00:04:23.850 align:middle line:84% And we can just use some basic set theory 00:04:23.850 --> 00:04:29.290 align:middle line:84% to write this, because the probability of A 1 union A 2, 00:04:29.290 --> 00:04:32.090 align:middle line:84% well, we can break it up into the area 00:04:32.090 --> 00:04:35.340 align:middle line:84% by each of these different pieces in the diagram. 00:04:35.340 --> 00:04:40.530 align:middle line:84% So we can write it out as the probability of A 1. 00:04:40.530 --> 00:04:43.930 align:middle line:84% We can add in the probability of A 2. 00:04:43.930 --> 00:04:46.110 align:middle line:84% Now, this is not a correct expression. 00:04:46.110 --> 00:04:47.670 align:middle line:84% So what's wrong with this expression? 00:04:47.670 --> 00:04:50.410 align:middle line:84% Why is the probability of A 1 union A 2 00:04:50.410 --> 00:04:53.740 align:middle line:84% not equal to the sum of their probabilities? 00:04:53.740 --> 00:04:54.540 align:middle line:90% Yes. 00:04:54.540 --> 00:04:55.900 align:middle line:84% AUDIENCE: The intersection is counted twice. 00:04:55.900 --> 00:04:56.692 align:middle line:90% PROFESSOR: Perfect. 00:04:56.692 --> 00:04:58.420 align:middle line:84% The intersection is counted twice. 00:04:58.420 --> 00:04:59.920 align:middle line:84% So here, what I'm going to do is I'm 00:04:59.920 --> 00:05:03.540 align:middle line:84% going to do a trick, which is, I'm just going to subtract off 00:05:03.540 --> 00:05:06.540 align:middle line:90% what I've overcounted. 00:05:06.540 --> 00:05:10.180 align:middle line:84% So this is the baby version of what inclusion-exclusion is. 00:05:10.180 --> 00:05:13.060 align:middle line:84% I want to get a handle on the union of these events. 00:05:13.060 --> 00:05:15.060 align:middle line:84% And I'm going to first overcount and then 00:05:15.060 --> 00:05:17.460 align:middle line:90% I'm going to correct my count. 00:05:17.460 --> 00:05:21.500 align:middle line:84% All right, so nothing too complicated happened here. 00:05:21.500 --> 00:05:25.100 align:middle line:84% And we can try the same type of strategy, but maybe in the n 00:05:25.100 --> 00:05:26.980 align:middle line:90% equals 3 case. 00:05:26.980 --> 00:05:29.700 align:middle line:84% So we can write out the probability 00:05:29.700 --> 00:05:36.140 align:middle line:90% that A 1 or A 2 or A 3 happen. 00:05:36.140 --> 00:05:37.900 align:middle line:84% And we can do the same type of trick, 00:05:37.900 --> 00:05:41.940 align:middle line:84% where we look at the sum of the probabilities. 00:05:41.940 --> 00:05:46.820 align:middle line:84% We add up the probability of A 1, the probability of A 2, 00:05:46.820 --> 00:05:50.670 align:middle line:90% the probability of A 3. 00:05:50.670 --> 00:05:52.250 align:middle line:84% But now we have the same problem, 00:05:52.250 --> 00:05:54.670 align:middle line:84% which is anything that happens in the intersection 00:05:54.670 --> 00:05:57.550 align:middle line:90% of these events is overcounted. 00:05:57.550 --> 00:05:59.710 align:middle line:84% So what I could try and do is subtract off 00:05:59.710 --> 00:06:01.590 align:middle line:90% pairwise differences. 00:06:01.590 --> 00:06:03.290 align:middle line:90% I could look at the probability. 00:06:03.290 --> 00:06:05.870 align:middle line:84% I could subtract off the probability A 1 00:06:05.870 --> 00:06:08.830 align:middle line:90% and A 2 both happen. 00:06:08.830 --> 00:06:14.670 align:middle line:84% I could subtract off the probability that A 1 and A 3 00:06:14.670 --> 00:06:16.030 align:middle line:90% both happen. 00:06:16.030 --> 00:06:18.830 align:middle line:84% And last but not least, I'll subtract off the probability 00:06:18.830 --> 00:06:22.590 align:middle line:90% that A 2 and A 3 happen. 00:06:22.590 --> 00:06:24.710 align:middle line:90% So this is still not correct. 00:06:24.710 --> 00:06:27.430 align:middle line:84% There's something wrong with this expression. 00:06:27.430 --> 00:06:31.550 align:middle line:84% Let's think about it again in terms of the Venn diagram. 00:06:31.550 --> 00:06:36.510 align:middle line:84% I can write out these three sets, A 1, A 2, and A 3. 00:06:36.510 --> 00:06:39.690 align:middle line:84% And if you think about what's happening in the Venn diagram, 00:06:39.690 --> 00:06:42.010 align:middle line:90% I've still miscounted. 00:06:42.010 --> 00:06:46.450 align:middle line:84% So when I look at the union of-- if I look at probability of A 1, 00:06:46.450 --> 00:06:49.080 align:middle line:84% the probability of A 2, the probability of A 3, 00:06:49.080 --> 00:06:54.160 align:middle line:84% all this region in their intersection is overcounted. 00:06:54.160 --> 00:06:57.740 align:middle line:84% And I'm trying to subtract off these pairwise intersections. 00:06:57.740 --> 00:07:01.280 align:middle line:84% So next, I subtract off this pairwise intersection, 00:07:01.280 --> 00:07:05.000 align:middle line:84% this pairwise intersection, and this pairwise intersection. 00:07:05.000 --> 00:07:09.240 align:middle line:84% But what's wrong with my formula now? 00:07:09.240 --> 00:07:11.160 align:middle line:90% Any ideas? 00:07:11.160 --> 00:07:15.440 align:middle line:84% So let's keep track pictorially of how the algebra connects. 00:07:15.440 --> 00:07:20.200 align:middle line:84% So I care about the union of A 1, A 2, and A 3. 00:07:20.200 --> 00:07:22.140 align:middle line:84% And I looked at the probability of A 1, 00:07:22.140 --> 00:07:24.440 align:middle line:84% that's everything in this disk, I 00:07:24.440 --> 00:07:26.280 align:middle line:84% looked at the probability of A 2, that's 00:07:26.280 --> 00:07:28.560 align:middle line:84% everything in this disk, and A 3, 00:07:28.560 --> 00:07:31.320 align:middle line:90% that's everything in this disk. 00:07:31.320 --> 00:07:34.240 align:middle line:84% That definitely overcounted the region in between. 00:07:34.240 --> 00:07:38.600 align:middle line:84% So what I did was I subtracted off this petal right here. 00:07:38.600 --> 00:07:42.040 align:middle line:84% And then I subtracted off this petal right here. 00:07:42.040 --> 00:07:45.920 align:middle line:84% And then I subtracted off this petal right here. 00:07:45.920 --> 00:07:47.500 align:middle line:90% So there's a problem. 00:07:47.500 --> 00:07:50.880 align:middle line:90% Anyone see it? 00:07:50.880 --> 00:07:51.380 align:middle line:90% Yeah. 00:07:51.380 --> 00:07:53.020 align:middle line:84% AUDIENCE: You subtracted the center. 00:07:53.020 --> 00:07:56.180 align:middle line:84% PROFESSOR: Yes, I've completely subtracted the center. 00:07:56.180 --> 00:07:57.700 align:middle line:90% That's exactly right. 00:07:57.700 --> 00:08:01.260 align:middle line:84% So, in fact, what I do is I add back in the center, 00:08:01.260 --> 00:08:04.140 align:middle line:84% and I'll get an actual correct expression. 00:08:04.140 --> 00:08:10.540 align:middle line:84% So I add in the probability of A 1 and A 2 and A 3. 00:08:10.540 --> 00:08:12.200 align:middle line:90% So that is a correct expression. 00:08:12.200 --> 00:08:14.140 align:middle line:84% These are two correct expressions for n 00:08:14.140 --> 00:08:16.300 align:middle line:90% equals 2 and n equals 3. 00:08:16.300 --> 00:08:18.980 align:middle line:84% And we can just use these expressions to figure out 00:08:18.980 --> 00:08:20.740 align:middle line:90% what the answer is. 00:08:20.740 --> 00:08:22.780 align:middle line:84% So let's go back to our definition 00:08:22.780 --> 00:08:24.700 align:middle line:90% of what these events are. 00:08:24.700 --> 00:08:26.880 align:middle line:84% So what is the probability of A 1? 00:08:26.880 --> 00:08:29.100 align:middle line:84% What's the probability that, if I have two students 00:08:29.100 --> 00:08:30.860 align:middle line:84% and I shuffle them randomly of who 00:08:30.860 --> 00:08:34.140 align:middle line:84% gets whose pset, that the first student gets 00:08:34.140 --> 00:08:36.020 align:middle line:90% the correct pset back. 00:08:36.020 --> 00:08:38.020 align:middle line:90% It's 1/2. 00:08:38.020 --> 00:08:39.740 align:middle line:90% So this is 1/2. 00:08:39.740 --> 00:08:41.700 align:middle line:90% This is 1/2, too. 00:08:41.700 --> 00:08:44.150 align:middle line:84% Now, just to make sure that everyone's awake, 00:08:44.150 --> 00:08:47.810 align:middle line:84% what's the probability that A 1 and A 2 happen? 00:08:47.810 --> 00:08:49.310 align:middle line:84% You have to be a little bit careful. 00:08:49.310 --> 00:08:49.810 align:middle line:90% Yeah? 00:08:49.810 --> 00:08:50.630 align:middle line:90% AUDIENCE: 1/4. 00:08:50.630 --> 00:08:51.630 align:middle line:90% PROFESSOR: It's not 1/4. 00:08:51.630 --> 00:08:55.312 align:middle line:90% 00:08:55.312 --> 00:08:57.270 align:middle line:84% Do you want to take a shot at why it's not 1/4? 00:08:57.270 --> 00:09:00.310 align:middle line:90% 00:09:00.310 --> 00:09:02.990 align:middle line:84% So let's think about what this event means. 00:09:02.990 --> 00:09:05.990 align:middle line:84% The first student gets their pset back, 00:09:05.990 --> 00:09:09.390 align:middle line:84% and the second student gets their pset back. 00:09:09.390 --> 00:09:11.230 align:middle line:84% So if you think about this, we could also 00:09:11.230 --> 00:09:13.950 align:middle line:84% write this in terms of conditional probability, 00:09:13.950 --> 00:09:17.470 align:middle line:84% the same way we talked about Bayes' rule last time. 00:09:17.470 --> 00:09:19.270 align:middle line:84% It's the same thing as the probability 00:09:19.270 --> 00:09:25.390 align:middle line:84% of A 1 times the probability of A 2 conditioned on A 1. 00:09:25.390 --> 00:09:27.550 align:middle line:84% See, once I tell you that A 1 has happened, 00:09:27.550 --> 00:09:32.590 align:middle line:84% which happens with 50/50 chance, then I now change my sample 00:09:32.590 --> 00:09:33.090 align:middle line:90% space. 00:09:33.090 --> 00:09:35.950 align:middle line:84% And I have to look at the conditional probability. 00:09:35.950 --> 00:09:40.910 align:middle line:84% So any thoughts on why it's not a 1/4? 00:09:40.910 --> 00:09:42.740 align:middle line:90% Not one? 00:09:42.740 --> 00:09:43.240 align:middle line:90% Yes. 00:09:43.240 --> 00:09:45.482 align:middle line:90% AUDIENCE: Even 1/2. 00:09:45.482 --> 00:09:46.940 align:middle line:84% PROFESSOR: I heard a 1/2 over here. 00:09:46.940 --> 00:09:49.040 align:middle line:90% So you want to-- yeah, perfect. 00:09:49.040 --> 00:09:50.500 align:middle line:90% OK, so that's exactly right. 00:09:50.500 --> 00:09:54.240 align:middle line:84% It's 1/2, because if we think about these in terms 00:09:54.240 --> 00:09:57.960 align:middle line:84% of their sample space, there are only four possibilities 00:09:57.960 --> 00:09:59.880 align:middle line:90% in this-- 00:09:59.880 --> 00:10:02.880 align:middle line:84% there are only two possibilities in the sample space. 00:10:02.880 --> 00:10:05.440 align:middle line:84% And once we've chosen that the first student gets 00:10:05.440 --> 00:10:08.220 align:middle line:84% their correct pset back, there's no choice 00:10:08.220 --> 00:10:10.760 align:middle line:84% but the second student had better get their correct pset 00:10:10.760 --> 00:10:12.080 align:middle line:90% back as well. 00:10:12.080 --> 00:10:14.440 align:middle line:84% So one of the things that makes computing 00:10:14.440 --> 00:10:17.360 align:middle line:84% this particular probability so nasty 00:10:17.360 --> 00:10:19.380 align:middle line:84% is that if these things were independent, 00:10:19.380 --> 00:10:22.240 align:middle line:84% life would be very easy, but they're not. 00:10:22.240 --> 00:10:23.800 align:middle line:84% See, when they're independent, I can 00:10:23.800 --> 00:10:26.608 align:middle line:84% multiply all of their individual probabilities. 00:10:26.608 --> 00:10:28.400 align:middle line:84% And we're seeing right here in this example 00:10:28.400 --> 00:10:31.480 align:middle line:90% that that's 100% false. 00:10:31.480 --> 00:10:35.320 align:middle line:84% But it turns out that it'll be morally true, at least when 00:10:35.320 --> 00:10:37.080 align:middle line:90% n becomes larger. 00:10:37.080 --> 00:10:39.680 align:middle line:84% So let's do this computation right here just to make 00:10:39.680 --> 00:10:41.490 align:middle line:90% sure we're on the same page. 00:10:41.490 --> 00:10:43.870 align:middle line:90% What's the probability of A 1? 00:10:43.870 --> 00:10:46.170 align:middle line:90% What's the probability of A 2? 00:10:46.170 --> 00:10:47.710 align:middle line:90% What's the probability of A 3? 00:10:47.710 --> 00:10:49.050 align:middle line:90% They're all 1/3. 00:10:49.050 --> 00:10:50.730 align:middle line:90% Now, let's be careful. 00:10:50.730 --> 00:10:55.330 align:middle line:84% What's the probability that A 1 and A 2 happen? 00:10:55.330 --> 00:10:57.350 align:middle line:90% So how large is my sample space? 00:10:57.350 --> 00:11:02.130 align:middle line:84% There are six possibilities in terms of who gets whose pset. 00:11:02.130 --> 00:11:06.250 align:middle line:84% And if A 1 happens, the first student gets the correct pset, 00:11:06.250 --> 00:11:10.430 align:middle line:84% the second student gets their correct pset, what can we say? 00:11:10.430 --> 00:11:13.170 align:middle line:90% 00:11:13.170 --> 00:11:14.520 align:middle line:90% What's the probability of this? 00:11:14.520 --> 00:11:17.050 align:middle line:90% 00:11:17.050 --> 00:11:19.368 align:middle line:84% AUDIENCE: So there's three options for the first choice 00:11:19.368 --> 00:11:20.410 align:middle line:90% if they want [INAUDIBLE]. 00:11:20.410 --> 00:11:21.035 align:middle line:90% PROFESSOR: Yes. 00:11:21.035 --> 00:11:23.510 align:middle line:84% AUDIENCE: So the second one, A 2, will have two choices? 00:11:23.510 --> 00:11:24.135 align:middle line:90% PROFESSOR: Yes. 00:11:24.135 --> 00:11:25.150 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:11:25.150 --> 00:11:26.290 align:middle line:90% 00:11:26.290 --> 00:11:28.470 align:middle line:84% PROFESSOR: OK, but what's the probability here? 00:11:28.470 --> 00:11:29.930 align:middle line:90% AUDIENCE: [INAUDIBLE]? 00:11:29.930 --> 00:11:31.198 align:middle line:90% PROFESSOR: Yeah. 00:11:31.198 --> 00:11:32.490 align:middle line:90% AUDIENCE: Would it just be 1/6? 00:11:32.490 --> 00:11:33.970 align:middle line:90% PROFESSOR: 1/6, perfect. 00:11:33.970 --> 00:11:36.910 align:middle line:84% So you gave a totally correct way to think about it. 00:11:36.910 --> 00:11:39.940 align:middle line:84% Another way to think about it is that there are six possibilities 00:11:39.940 --> 00:11:41.420 align:middle line:90% in the sample space. 00:11:41.420 --> 00:11:44.460 align:middle line:84% And when A 1 and A 2 happen, there's 00:11:44.460 --> 00:11:47.540 align:middle line:84% only one outcome that satisfies that condition, that's 00:11:47.540 --> 00:11:49.447 align:middle line:84% where everyone gets the correct pset, 00:11:49.447 --> 00:11:51.780 align:middle line:84% because if the first student does and the second student 00:11:51.780 --> 00:11:54.000 align:middle line:84% does, third student has no choice. 00:11:54.000 --> 00:11:55.660 align:middle line:90% They get it correctly too. 00:11:55.660 --> 00:11:59.520 align:middle line:84% So all of these other probabilities here are all 1/6. 00:11:59.520 --> 00:12:01.140 align:middle line:90% That's absolutely right. 00:12:01.140 --> 00:12:04.500 align:middle line:84% And that will give us that the total probability of the event 00:12:04.500 --> 00:12:06.380 align:middle line:90% is 2/3. 00:12:06.380 --> 00:12:10.220 align:middle line:84% OK, so one thing to emphasize is that these expressions I've 00:12:10.220 --> 00:12:13.900 align:middle line:84% written down in the n equals 2 and the n equals 3 case, 00:12:13.900 --> 00:12:17.600 align:middle line:84% these are valid not just in this particular problem, 00:12:17.600 --> 00:12:19.800 align:middle line:84% but they're true for any collection of events. 00:12:19.800 --> 00:12:22.140 align:middle line:84% I've made no assumptions whatsoever. 00:12:22.140 --> 00:12:24.133 align:middle line:84% It's just that if I had different events, 00:12:24.133 --> 00:12:25.800 align:middle line:84% they would have different probabilities, 00:12:25.800 --> 00:12:29.460 align:middle line:84% and I would have to compute all of the terms in this expression. 00:12:29.460 --> 00:12:31.400 align:middle line:90% All right, any questions so far? 00:12:31.400 --> 00:12:34.780 align:middle line:90% 00:12:34.780 --> 00:12:37.120 align:middle line:84% So I think you'll agree that we probably 00:12:37.120 --> 00:12:40.160 align:middle line:84% should stop at n equals 3, in terms of writing down 00:12:40.160 --> 00:12:43.420 align:middle line:84% one expression, and reasoning about the Venn diagram. 00:12:43.420 --> 00:12:46.360 align:middle line:84% These things just get bigger and more complicated. 00:12:46.360 --> 00:12:49.000 align:middle line:84% But it turns out that there's a closed form expression, 00:12:49.000 --> 00:12:50.680 align:middle line:84% without looking at the Venn diagram 00:12:50.680 --> 00:12:53.240 align:middle line:84% and reasoning about the individual slices 00:12:53.240 --> 00:12:55.520 align:middle line:84% and pieces of area, that's going to give us 00:12:55.520 --> 00:12:57.240 align:middle line:90% the right answer in general. 00:12:57.240 --> 00:12:59.320 align:middle line:84% So what I'm going to do is I'm going 00:12:59.320 --> 00:13:02.160 align:middle line:84% to guess what that formula is just 00:13:02.160 --> 00:13:05.400 align:middle line:84% by trying to extend the pattern that I've seen for the n equals 00:13:05.400 --> 00:13:07.680 align:middle line:90% 2 and n equals 3 case. 00:13:07.680 --> 00:13:09.240 align:middle line:90% That guess will be correct. 00:13:09.240 --> 00:13:11.040 align:middle line:90% And we'll check. 00:13:11.040 --> 00:13:12.600 align:middle line:84% We'll use that formula to compute 00:13:12.600 --> 00:13:14.840 align:middle line:84% what the answer to our challenge problem is. 00:13:14.840 --> 00:13:17.920 align:middle line:84% But then we'll go back and prove that formula. 00:13:17.920 --> 00:13:21.000 align:middle line:84% So that formula is not that hard to prove once you know 00:13:21.000 --> 00:13:23.680 align:middle line:90% what it ought to look like. 00:13:23.680 --> 00:13:25.560 align:middle line:84% So what I claim, and this is called 00:13:25.560 --> 00:13:29.360 align:middle line:84% the inclusion-exclusion formula, is 00:13:29.360 --> 00:13:35.490 align:middle line:84% that the probability of any union of events A 1, A 2, 00:13:35.490 --> 00:13:36.350 align:middle line:90% doesn't matter. 00:13:36.350 --> 00:13:39.850 align:middle line:84% It's not specific to our particular problem. 00:13:39.850 --> 00:13:43.650 align:middle line:84% We can write this out by over and then under and then 00:13:43.650 --> 00:13:44.550 align:middle line:90% overcounting. 00:13:44.550 --> 00:13:47.250 align:middle line:84% So we're going to go through a series of corrections 00:13:47.250 --> 00:13:49.410 align:middle line:84% the same way we did in the n equals 2 and the n 00:13:49.410 --> 00:13:51.050 align:middle line:90% equals 3 case. 00:13:51.050 --> 00:13:53.730 align:middle line:84% Our naive approximation is just going 00:13:53.730 --> 00:13:57.210 align:middle line:84% to be the sum over all of the events of the probability 00:13:57.210 --> 00:13:58.810 align:middle line:90% of the event. 00:13:58.810 --> 00:14:01.070 align:middle line:84% We know that that's not correct in general 00:14:01.070 --> 00:14:02.393 align:middle line:90% because we've over counted. 00:14:02.393 --> 00:14:03.810 align:middle line:84% So what we're going to do is we're 00:14:03.810 --> 00:14:06.810 align:middle line:84% going to subtract off the pairwise differences. 00:14:06.810 --> 00:14:09.810 align:middle line:84% We're going to take every pair of events, the same way we did 00:14:09.810 --> 00:14:13.410 align:middle line:84% in the n equals 3 case, and look at the probability 00:14:13.410 --> 00:14:19.850 align:middle line:84% of any pair of events happening, A i and A j. 00:14:19.850 --> 00:14:23.050 align:middle line:84% The next term in the expression will fix the fact 00:14:23.050 --> 00:14:24.290 align:middle line:90% that we've undercounted. 00:14:24.290 --> 00:14:28.170 align:middle line:84% We'll look over triples the sum over all i less than j 00:14:28.170 --> 00:14:36.460 align:middle line:84% less than k of the probability of their threewise intersection, 00:14:36.460 --> 00:14:37.660 align:middle line:90% and so on. 00:14:37.660 --> 00:14:41.180 align:middle line:84% And the last term in this expression is just minus 1 00:14:41.180 --> 00:14:45.900 align:middle line:84% to the n plus 1 times the probability 00:14:45.900 --> 00:14:47.370 align:middle line:90% of all of the events. 00:14:47.370 --> 00:14:51.580 align:middle line:90% 00:14:51.580 --> 00:14:55.397 align:middle line:84% So that's the inclusion-exclusion formula. 00:14:55.397 --> 00:14:57.480 align:middle line:84% It takes a little bit to wrap your head around it, 00:14:57.480 --> 00:14:58.900 align:middle line:90% but it's very natural. 00:14:58.900 --> 00:15:02.660 align:middle line:84% It's really just this approach of successively approximating 00:15:02.660 --> 00:15:06.500 align:middle line:84% what the probability of this is by over and then under and then 00:15:06.500 --> 00:15:09.540 align:middle line:84% over and undercounting until you hone in on what 00:15:09.540 --> 00:15:11.540 align:middle line:90% the correct probability is. 00:15:11.540 --> 00:15:15.460 align:middle line:84% And all of this holds in general for no assumptions. 00:15:15.460 --> 00:15:18.180 align:middle line:84% All right, so now it's not so obvious 00:15:18.180 --> 00:15:21.120 align:middle line:84% why this is actually helpful for our problem, 00:15:21.120 --> 00:15:24.620 align:middle line:84% because I took a simple-to-state problem, 00:15:24.620 --> 00:15:28.820 align:middle line:84% and I wrote down this behemoth of an expression right here. 00:15:28.820 --> 00:15:31.990 align:middle line:84% But what I claim is that all of the terms that 00:15:31.990 --> 00:15:35.150 align:middle line:84% happen in this expression, in our particular problem, 00:15:35.150 --> 00:15:37.310 align:middle line:84% turn out to be very easy to compute 00:15:37.310 --> 00:15:40.390 align:middle line:84% using the intuition we developed in the n equals 2 and n 00:15:40.390 --> 00:15:42.387 align:middle line:90% equals 3 case. 00:15:42.387 --> 00:15:43.970 align:middle line:84% So I'm going to ask you guys for help. 00:15:43.970 --> 00:15:45.345 align:middle line:84% And you guys are going to help me 00:15:45.345 --> 00:15:49.550 align:middle line:84% fill in what this looks like for our particular problem. 00:15:49.550 --> 00:15:53.030 align:middle line:84% So returning to our challenge problem, 00:15:53.030 --> 00:15:56.950 align:middle line:84% the one we started off with, well, 00:15:56.950 --> 00:16:01.190 align:middle line:84% in the general n case, what is the probability that 00:16:01.190 --> 00:16:04.710 align:middle line:90% event A i happens? 00:16:04.710 --> 00:16:06.225 align:middle line:90% Who can help me out? 00:16:06.225 --> 00:16:07.590 align:middle line:90% AUDIENCE: 1 over n 00:16:07.590 --> 00:16:09.990 align:middle line:90% PROFESSOR: 1 over n, perfect. 00:16:09.990 --> 00:16:13.070 align:middle line:84% 1 over n, because I can just imagine 00:16:13.070 --> 00:16:15.670 align:middle line:84% that I'm handing back that student's pset first 00:16:15.670 --> 00:16:17.450 align:middle line:90% and I selected randomly. 00:16:17.450 --> 00:16:21.390 align:middle line:84% So I have a 1 over n chance of getting them their correct pset. 00:16:21.390 --> 00:16:24.430 align:middle line:84% Now, we know, as we go to higher intersections, 00:16:24.430 --> 00:16:27.270 align:middle line:84% we have to be more careful, because that's 00:16:27.270 --> 00:16:30.600 align:middle line:84% where the dependence between these events 00:16:30.600 --> 00:16:33.920 align:middle line:84% kicks in so that we can't just naively multiply 00:16:33.920 --> 00:16:35.800 align:middle line:90% these probabilities. 00:16:35.800 --> 00:16:39.320 align:middle line:84% So the probability that A i and A j happen 00:16:39.320 --> 00:16:41.580 align:middle line:90% is not 1 over n squared. 00:16:41.580 --> 00:16:43.360 align:middle line:90% It's very close to that. 00:16:43.360 --> 00:16:46.680 align:middle line:84% So who can help me out and tell me what the answer is for this? 00:16:46.680 --> 00:16:49.960 align:middle line:84% And then we should see the pattern, hopefully. 00:16:49.960 --> 00:16:50.480 align:middle line:90% Yeah? 00:16:50.480 --> 00:16:52.680 align:middle line:84% AUDIENCE: 1 over n times n minus 1. 00:16:52.680 --> 00:16:54.680 align:middle line:84% PROFESSOR: Perfect, that's right. 00:16:54.680 --> 00:16:59.180 align:middle line:90% 1 over n times 1 over n minus 1. 00:16:59.180 --> 00:17:00.480 align:middle line:90% That's exactly right. 00:17:00.480 --> 00:17:03.320 align:middle line:84% The way to think about it is exactly in terms of the way 00:17:03.320 --> 00:17:06.079 align:middle line:84% I wrote it down with the conditional probabilities. 00:17:06.079 --> 00:17:08.800 align:middle line:84% If both of these events happen, well, I 00:17:08.800 --> 00:17:11.220 align:middle line:84% need student i to get the correct pset, 00:17:11.220 --> 00:17:13.319 align:middle line:84% student j to get the correct pset. 00:17:13.319 --> 00:17:16.000 align:middle line:84% So let's imagine the thought experiment that we hand back 00:17:16.000 --> 00:17:18.160 align:middle line:90% student i's pset first. 00:17:18.160 --> 00:17:21.000 align:middle line:84% They have 1 over n chance of getting the correct one. 00:17:21.000 --> 00:17:23.319 align:middle line:84% And now, when they get the correct one, 00:17:23.319 --> 00:17:27.400 align:middle line:84% we only have n minus 1 possibilities for the remaining 00:17:27.400 --> 00:17:28.260 align:middle line:90% pset. 00:17:28.260 --> 00:17:30.700 align:middle line:84% We know that student j's pset is still 00:17:30.700 --> 00:17:33.700 align:middle line:84% in that set because i did not get his pset. 00:17:33.700 --> 00:17:36.140 align:middle line:84% And so we have a 1 over n minus 1 chance 00:17:36.140 --> 00:17:39.260 align:middle line:84% of continuing so that we actually get both of their psets 00:17:39.260 --> 00:17:40.340 align:middle line:90% back. 00:17:40.340 --> 00:17:42.940 align:middle line:84% And just continuing the pattern, so the probability 00:17:42.940 --> 00:17:49.700 align:middle line:84% that we have a triple, A i and A j and A k 00:17:49.700 --> 00:17:53.180 align:middle line:84% is just going to be 1 over n times, 1 over n 00:17:53.180 --> 00:17:57.580 align:middle line:90% minus 1 times 1 over n minus 2. 00:17:57.580 --> 00:18:01.740 align:middle line:84% So now we're almost home free, because we 00:18:01.740 --> 00:18:05.260 align:middle line:84% can plug in all of these values that we've 00:18:05.260 --> 00:18:07.660 align:middle line:90% computed into our expression. 00:18:07.660 --> 00:18:11.020 align:middle line:84% But we're going to need a little bit of counting in order 00:18:11.020 --> 00:18:14.060 align:middle line:84% to actually simplify this expression, 00:18:14.060 --> 00:18:15.700 align:middle line:84% because one thing we need to know 00:18:15.700 --> 00:18:19.780 align:middle line:84% is, OK, all of these probability of A i intersect A j. 00:18:19.780 --> 00:18:21.320 align:middle line:90% We know what those values are. 00:18:21.320 --> 00:18:24.420 align:middle line:84% They're 1 over n times 1 over n minus 1. 00:18:24.420 --> 00:18:27.670 align:middle line:90% But how many of them are there? 00:18:27.670 --> 00:18:30.110 align:middle line:84% So that leads us to a very important concept 00:18:30.110 --> 00:18:32.150 align:middle line:84% that we're going to talk a ton about, which is 00:18:32.150 --> 00:18:35.790 align:middle line:90% called a binomial coefficient. 00:18:35.790 --> 00:18:48.110 align:middle line:84% So in particular, how many events of the form 00:18:48.110 --> 00:18:53.510 align:middle line:90% A i intersect A j are there? 00:18:53.510 --> 00:18:57.030 align:middle line:84% Each one of these possible pairwise intersections 00:18:57.030 --> 00:19:00.550 align:middle line:84% shows up in our inclusion-exclusion formula. 00:19:00.550 --> 00:19:03.670 align:middle line:84% And I'm going to tell you what the answer is. 00:19:03.670 --> 00:19:05.533 align:middle line:84% In fact, it's such an important quantity 00:19:05.533 --> 00:19:07.450 align:middle line:84% that it's going to come up all over the place. 00:19:07.450 --> 00:19:08.530 align:middle line:90% It has a special name. 00:19:08.530 --> 00:19:11.110 align:middle line:84% It's called the binomial coefficient. 00:19:11.110 --> 00:19:15.990 align:middle line:84% And it's defined as n times n minus 1 over 2. 00:19:15.990 --> 00:19:18.770 align:middle line:84% That's what we mean when we write this expression of n 00:19:18.770 --> 00:19:20.030 align:middle line:90% choose two. 00:19:20.030 --> 00:19:22.690 align:middle line:84% What this expression means, so when we look at it, 00:19:22.690 --> 00:19:26.520 align:middle line:84% we say this in words, n choose 2, 00:19:26.520 --> 00:19:29.560 align:middle line:84% is because I have n possibilities for what 00:19:29.560 --> 00:19:32.480 align:middle line:84% i is because it can be anything from 1 to n. 00:19:32.480 --> 00:19:36.000 align:middle line:84% I have n possibilities for what j is because it 00:19:36.000 --> 00:19:38.360 align:middle line:90% can be anything from 1 to n. 00:19:38.360 --> 00:19:39.960 align:middle line:84% And what I'm doing when I look at all 00:19:39.960 --> 00:19:43.080 align:middle line:84% of these events is I'm choosing two things out of them, 00:19:43.080 --> 00:19:45.760 align:middle line:90% a value for i and a value for j. 00:19:45.760 --> 00:19:48.880 align:middle line:84% But the key is making sure that you count this the right way, 00:19:48.880 --> 00:19:51.640 align:middle line:84% because a priori, I have n possibilities 00:19:51.640 --> 00:19:55.600 align:middle line:84% for the choice of i, and then I have n minus 1 possibilities 00:19:55.600 --> 00:19:58.520 align:middle line:84% for the choice of j, because it can't be the same as i. 00:19:58.520 --> 00:20:01.060 align:middle line:84% But the key is that I have this extra factor of 2, 00:20:01.060 --> 00:20:02.960 align:middle line:90% because I've overcounted. 00:20:02.960 --> 00:20:06.640 align:middle line:84% Because I only care about the unordered pair, i, j, 00:20:06.640 --> 00:20:09.040 align:middle line:84% I don't care about what order they have. 00:20:09.040 --> 00:20:13.320 align:middle line:84% So each pair of events happens two different ways, both for i 00:20:13.320 --> 00:20:17.240 align:middle line:84% being the first thing, j being the second and vice versa. 00:20:17.240 --> 00:20:19.480 align:middle line:84% All right, so now we're in good shape. 00:20:19.480 --> 00:20:26.490 align:middle line:84% In fact, more generally, if we look at how many terms 00:20:26.490 --> 00:20:33.770 align:middle line:84% there are for the r way intersection, what we can do 00:20:33.770 --> 00:20:38.570 align:middle line:84% is we can define the n choose r binomial coefficient, which 00:20:38.570 --> 00:20:42.690 align:middle line:84% is just defined as n times n minus 1 all the way down 00:20:42.690 --> 00:20:47.730 align:middle line:84% to n minus r plus 1 over r factorial. 00:20:47.730 --> 00:20:49.310 align:middle line:84% And it's the same type of intuition. 00:20:49.310 --> 00:20:52.970 align:middle line:84% If I'm trying to choose r values out of n possibilities, 00:20:52.970 --> 00:20:55.290 align:middle line:84% I have n possibilities for the first thing, n 00:20:55.290 --> 00:20:58.490 align:middle line:84% minus 1 all the way down to n minus r plus 1. 00:20:58.490 --> 00:21:00.115 align:middle line:84% But I have to divide by something 00:21:00.115 --> 00:21:01.490 align:middle line:84% to account for the fact that I've 00:21:01.490 --> 00:21:04.250 align:middle line:90% overcounted in this expression. 00:21:04.250 --> 00:21:07.850 align:middle line:84% So we'll talk much more about counting, actually, immediately 00:21:07.850 --> 00:21:08.750 align:middle line:90% after this lecture. 00:21:08.750 --> 00:21:11.050 align:middle line:84% That's what we'll be starting next week. 00:21:11.050 --> 00:21:13.770 align:middle line:84% But let's get down to business, and let's actually write out 00:21:13.770 --> 00:21:15.410 align:middle line:90% what our expression is. 00:21:15.410 --> 00:21:17.770 align:middle line:84% Then something beautiful is going to happen. 00:21:17.770 --> 00:21:22.940 align:middle line:84% So we care about this probability in the union of all 00:21:22.940 --> 00:21:24.300 align:middle line:90% the A i's. 00:21:24.300 --> 00:21:26.960 align:middle line:84% And if we work out what this expression is, 00:21:26.960 --> 00:21:30.580 align:middle line:90% well, I have n times 1 over n. 00:21:30.580 --> 00:21:33.500 align:middle line:84% That just comes from adding up all these probabilities 00:21:33.500 --> 00:21:34.400 align:middle line:90% of the A i's. 00:21:34.400 --> 00:21:37.300 align:middle line:84% There are n of them, and they're each 1 over n. 00:21:37.300 --> 00:21:39.460 align:middle line:90% And then I have my next term. 00:21:39.460 --> 00:21:42.240 align:middle line:90% My next term is minus-- 00:21:42.240 --> 00:21:45.620 align:middle line:84% I'll write it out as n times n minus 1 over 2. 00:21:45.620 --> 00:21:48.060 align:middle line:84% So there are n choose 2 terms there. 00:21:48.060 --> 00:21:53.780 align:middle line:84% We know that their probabilities are 1 over n times n minus 1. 00:21:53.780 --> 00:21:56.740 align:middle line:84% And let me write out just the next term and we'll stop there. 00:21:56.740 --> 00:22:01.900 align:middle line:84% We also get n choose 3, so n times n minus 1 times n minus 2 00:22:01.900 --> 00:22:04.860 align:middle line:90% over 6, namely 3 factorial. 00:22:04.860 --> 00:22:11.020 align:middle line:84% And this is all times 1 over n times n minus 1 times n minus 2. 00:22:11.020 --> 00:22:14.020 align:middle line:84% And this expression keeps going on the same way. 00:22:14.020 --> 00:22:16.820 align:middle line:90% I get a higher n choose r. 00:22:16.820 --> 00:22:19.590 align:middle line:90% I get more n's here. 00:22:19.590 --> 00:22:21.790 align:middle line:84% And the signs are alternating the same way 00:22:21.790 --> 00:22:25.110 align:middle line:84% they are in the inclusion-exclusion formula. 00:22:25.110 --> 00:22:28.710 align:middle line:84% And now you can see, I get a huge amount of cancellation. 00:22:28.710 --> 00:22:31.435 align:middle line:90% And things simplify. 00:22:31.435 --> 00:22:33.310 align:middle line:84% And I'm going to get something very beautiful 00:22:33.310 --> 00:22:34.630 align:middle line:90% at the end of the day. 00:22:34.630 --> 00:22:37.050 align:middle line:84% So I'm going to get that this overall probability, 00:22:37.050 --> 00:22:39.710 align:middle line:84% which we computed through inclusion-exclusion, 00:22:39.710 --> 00:22:41.710 align:middle line:90% is 1 minus-- 00:22:41.710 --> 00:22:45.110 align:middle line:84% I'll write it as 1 over 2 factorial suggestively-- 00:22:45.110 --> 00:22:51.850 align:middle line:84% plus 1 over 3 factorial minus 1 over 4 factorial, and so on. 00:22:51.850 --> 00:22:55.070 align:middle line:90% 00:22:55.070 --> 00:22:57.710 align:middle line:90% And now magic happens. 00:22:57.710 --> 00:23:02.030 align:middle line:84% So does this look familiar from calculus? 00:23:02.030 --> 00:23:02.710 align:middle line:90% Hopefully. 00:23:02.710 --> 00:23:05.230 align:middle line:90% 00:23:05.230 --> 00:23:09.570 align:middle line:84% Anyone brave enough to guess what this infinite sum is? 00:23:09.570 --> 00:23:13.010 align:middle line:90% 00:23:13.010 --> 00:23:13.510 align:middle line:90% Yeah? 00:23:13.510 --> 00:23:14.630 align:middle line:90% AUDIENCE: 1 over e. 00:23:14.630 --> 00:23:16.250 align:middle line:90% PROFESSOR: Perfect, 1 over e. 00:23:16.250 --> 00:23:18.210 align:middle line:90% That's exactly right. 00:23:18.210 --> 00:23:20.610 align:middle line:90% It's 1 over e. 00:23:20.610 --> 00:23:25.430 align:middle line:84% So this very simple-to-state question, the e pops out. 00:23:25.430 --> 00:23:27.250 align:middle line:90% It's a very beautiful answer. 00:23:27.250 --> 00:23:29.370 align:middle line:84% It involves all kinds of things that 00:23:29.370 --> 00:23:31.590 align:middle line:84% are going to show up in more detail in class. 00:23:31.590 --> 00:23:33.450 align:middle line:84% We have this basic building block 00:23:33.450 --> 00:23:36.290 align:middle line:84% and probability, the inclusion-exclusion formula. 00:23:36.290 --> 00:23:38.250 align:middle line:84% We have the starting point of counting, which 00:23:38.250 --> 00:23:39.850 align:middle line:90% is going to be a whole unit. 00:23:39.850 --> 00:23:43.430 align:middle line:84% And then we've connected it back to infinite series. 00:23:43.430 --> 00:23:47.650 align:middle line:84% And we're going to do a lot with generating functions later. 00:23:47.650 --> 00:23:49.890 align:middle line:84% I think it's also good to try and keep 00:23:49.890 --> 00:23:52.070 align:middle line:84% a mental model for what's going on here, 00:23:52.070 --> 00:23:54.890 align:middle line:84% because we did do a bunch of calculations. 00:23:54.890 --> 00:23:56.370 align:middle line:84% It turns out that there's actually 00:23:56.370 --> 00:24:00.130 align:middle line:90% a very simple heuristic. 00:24:00.130 --> 00:24:02.750 align:middle line:84% See, in the n equals 2 and n equals 3 case, 00:24:02.750 --> 00:24:04.210 align:middle line:84% remember, we had the subtlety when 00:24:04.210 --> 00:24:06.970 align:middle line:84% we were computing the intersections of events, 00:24:06.970 --> 00:24:08.750 align:middle line:84% that events were not independent. 00:24:08.750 --> 00:24:12.370 align:middle line:84% That's why that 1/2 was 1/2 and was not 1/4. 00:24:12.370 --> 00:24:13.850 align:middle line:84% But it turns out there's actually 00:24:13.850 --> 00:24:16.360 align:middle line:84% a simple heuristic that is not correct, 00:24:16.360 --> 00:24:20.065 align:middle line:90% but it works, so caution here. 00:24:20.065 --> 00:24:21.940 align:middle line:84% This is not a correct argument, but it's more 00:24:21.940 --> 00:24:24.530 align:middle line:84% of a back-of-the-envelope calculation. 00:24:24.530 --> 00:24:31.780 align:middle line:84% So the A i's, I claim they're almost independent. 00:24:31.780 --> 00:24:33.600 align:middle line:90% So they're not independent. 00:24:33.600 --> 00:24:36.700 align:middle line:84% We already saw that in the n equals 2 case. 00:24:36.700 --> 00:24:40.080 align:middle line:84% But what I claim is that as n gets larger and larger, 00:24:40.080 --> 00:24:42.340 align:middle line:90% then they do become independent. 00:24:42.340 --> 00:24:44.660 align:middle line:84% You can think about this intuitively. 00:24:44.660 --> 00:24:48.540 align:middle line:84% If I have an infinitely large class, the first person, 00:24:48.540 --> 00:24:52.060 align:middle line:84% if they get their pset back correctly or not, 00:24:52.060 --> 00:24:54.580 align:middle line:84% it really ought to not affect very much 00:24:54.580 --> 00:24:56.900 align:middle line:84% whether the second student does too, 00:24:56.900 --> 00:25:00.380 align:middle line:84% because I haven't really changed the denominator very much. 00:25:00.380 --> 00:25:03.100 align:middle line:84% So it turns out that, really, the dominant terms 00:25:03.100 --> 00:25:05.620 align:middle line:84% in this inclusion-exclusion formula 00:25:05.620 --> 00:25:09.260 align:middle line:84% all happen for smaller intersections of events, 00:25:09.260 --> 00:25:12.950 align:middle line:84% because these probabilities are becoming astronomically tiny. 00:25:12.950 --> 00:25:15.890 align:middle line:84% So wherever the action is really happening in this formula, 00:25:15.890 --> 00:25:18.630 align:middle line:84% things almost act like they're independent. 00:25:18.630 --> 00:25:21.070 align:middle line:84% So if you just pretend that they are, 00:25:21.070 --> 00:25:23.430 align:middle line:84% then you actually do get the right answer. 00:25:23.430 --> 00:25:27.510 align:middle line:84% So let's look at the probability that, 00:25:27.510 --> 00:25:34.890 align:middle line:84% not A 1 and not all the way up to A n, well, we can pretend-- 00:25:34.890 --> 00:25:36.610 align:middle line:90% and this is not literally true-- 00:25:36.610 --> 00:25:42.230 align:middle line:84% that this is the product of all of these probabilities. 00:25:42.230 --> 00:25:47.950 align:middle line:84% And each one of these things behaves like 1 minus 1 over n, 00:25:47.950 --> 00:25:52.430 align:middle line:84% because I'm very likely to not get my correct pset back. 00:25:52.430 --> 00:25:54.710 align:middle line:84% And I'm raising them to the n-th power. 00:25:54.710 --> 00:25:57.070 align:middle line:84% And this, in the limit, as n goes to infinity, 00:25:57.070 --> 00:26:00.470 align:middle line:90% really is 1 over E. 00:26:00.470 --> 00:26:03.630 align:middle line:84% OK, so inclusion-exclusion is actually 00:26:03.630 --> 00:26:05.710 align:middle line:84% a way to make some of these intuitions 00:26:05.710 --> 00:26:08.210 align:middle line:84% precise when you don't literally have independence, 00:26:08.210 --> 00:26:10.800 align:middle line:84% but you have something very close to it. 00:26:10.800 --> 00:26:12.440 align:middle line:90% So any questions? 00:26:12.440 --> 00:26:16.640 align:middle line:84% So far, we've just used the inclusion-exclusion formula. 00:26:16.640 --> 00:26:18.520 align:middle line:84% And what I still owe you is I owe you 00:26:18.520 --> 00:26:21.270 align:middle line:84% a proof of the inclusion-exclusion formula. 00:26:21.270 --> 00:26:25.120 align:middle line:90% 00:26:25.120 --> 00:26:26.640 align:middle line:90% Everyone following? 00:26:26.640 --> 00:26:27.240 align:middle line:90% Yeah? 00:26:27.240 --> 00:26:28.080 align:middle line:90% OK. 00:26:28.080 --> 00:26:30.700 align:middle line:84% All right, so let's prove this formula. 00:26:30.700 --> 00:26:32.160 align:middle line:84% It turns out that it's really not 00:26:32.160 --> 00:26:37.160 align:middle line:84% too hard to prove once you know what you're trying to prove. 00:26:37.160 --> 00:26:40.720 align:middle line:84% So let's take our inclusion-exclusion formula 00:26:40.720 --> 00:26:43.160 align:middle line:90% that we want to prove. 00:26:43.160 --> 00:26:45.780 align:middle line:84% And there are actually tons of different proofs of this fact. 00:26:45.780 --> 00:26:48.360 align:middle line:84% I'll give you a particularly simple one, 00:26:48.360 --> 00:26:50.860 align:middle line:84% which is we're just going to prove it by induction. 00:26:50.860 --> 00:26:59.440 align:middle line:90% 00:26:59.440 --> 00:27:04.520 align:middle line:84% So what do I have to do when I prove it by induction? 00:27:04.520 --> 00:27:07.160 align:middle line:90% I have to prove the base case. 00:27:07.160 --> 00:27:10.170 align:middle line:84% Actually, we already proved the base case 00:27:10.170 --> 00:27:12.570 align:middle line:84% because we already proved it pictorially, 00:27:12.570 --> 00:27:14.030 align:middle line:90% using the Venn diagram. 00:27:14.030 --> 00:27:14.930 align:middle line:90% So we're good. 00:27:14.930 --> 00:27:17.890 align:middle line:84% We proved it, for example, in the n equals 2 case. 00:27:17.890 --> 00:27:20.030 align:middle line:84% You can check it's true for n equals 1, 2. 00:27:20.030 --> 00:27:22.090 align:middle line:90% That's just a technicality. 00:27:22.090 --> 00:27:27.770 align:middle line:84% But now, what I want to do in the inductive step, 00:27:27.770 --> 00:27:36.730 align:middle line:84% I'm going to assume it's true for some value n. 00:27:36.730 --> 00:27:39.370 align:middle line:90% And what I want to prove-- 00:27:39.370 --> 00:27:46.490 align:middle line:84% I want to show that then it must be true for the next largest 00:27:46.490 --> 00:27:49.210 align:middle line:90% value, so for n plus 1. 00:27:49.210 --> 00:27:50.793 align:middle line:90% So that'll be my strategy. 00:27:50.793 --> 00:27:52.210 align:middle line:84% And all we're going to do is we're 00:27:52.210 --> 00:27:55.490 align:middle line:84% going to have to write down this mess of an expression 00:27:55.490 --> 00:27:57.530 align:middle line:90% and simplify it. 00:27:57.530 --> 00:27:59.790 align:middle line:90% So let me make my life easier. 00:27:59.790 --> 00:28:03.610 align:middle line:84% I'm just going to fix my expression right here. 00:28:03.610 --> 00:28:09.950 align:middle line:84% So let's write down the expression we want to show. 00:28:09.950 --> 00:28:12.490 align:middle line:84% We can change all these n's to n plus 1. 00:28:12.490 --> 00:28:15.870 align:middle line:84% So this is the sum from i equals 1 to n plus 1. 00:28:15.870 --> 00:28:18.710 align:middle line:84% These are all n plus 1 values, and so on. 00:28:18.710 --> 00:28:21.910 align:middle line:90% And this now becomes n plus 2. 00:28:21.910 --> 00:28:23.990 align:middle line:90% And this becomes n plus 1. 00:28:23.990 --> 00:28:26.510 align:middle line:84% So this is the formula that we want to show. 00:28:26.510 --> 00:28:31.310 align:middle line:84% This is what the expression would be at a larger value of n. 00:28:31.310 --> 00:28:34.190 align:middle line:84% And all I'm going to do is I'm going to collect terms 00:28:34.190 --> 00:28:36.670 align:middle line:84% in this expression so that I can appeal 00:28:36.670 --> 00:28:39.230 align:middle line:84% to what I know that the expression already holds 00:28:39.230 --> 00:28:40.950 align:middle line:90% for some smaller value of n. 00:28:40.950 --> 00:28:43.330 align:middle line:84% And then I'm going to see whatever is left over, 00:28:43.330 --> 00:28:45.070 align:middle line:84% what are the terms that are left over, 00:28:45.070 --> 00:28:47.270 align:middle line:84% and I'm going to have to interpret those. 00:28:47.270 --> 00:28:51.790 align:middle line:84% And we're going to connect it back to the n equals 2 case. 00:28:51.790 --> 00:28:55.190 align:middle line:84% So let's collect terms in this expression 00:28:55.190 --> 00:28:59.670 align:middle line:84% and see what's left over OK, so first, I'm 00:28:59.670 --> 00:29:04.430 align:middle line:84% going to let star denote this expression right here. 00:29:04.430 --> 00:29:12.040 align:middle line:84% This is the expression for n plus 1. 00:29:12.040 --> 00:29:17.280 align:middle line:84% And I'm going to group this term square into different terms. 00:29:17.280 --> 00:29:19.040 align:middle line:84% So let me start off with the terms, which 00:29:19.040 --> 00:29:21.120 align:middle line:84% I'm going to call triangle, that I'm just 00:29:21.120 --> 00:29:24.960 align:middle line:84% going to pull out my old inclusion-exclusion formula. 00:29:24.960 --> 00:29:28.480 align:middle line:84% So what if I look at the sum from i equals 1 to n 00:29:28.480 --> 00:29:31.440 align:middle line:84% of the probabilities of all these A i's? 00:29:31.440 --> 00:29:34.160 align:middle line:84% You can see that in my expression for n plus 1, 00:29:34.160 --> 00:29:38.940 align:middle line:84% I'm just removing the probability of A sub n plus 1. 00:29:38.940 --> 00:29:40.800 align:middle line:90% I'm just removing that term. 00:29:40.800 --> 00:29:44.880 align:middle line:84% And I'll only keep the terms that don't involve the n plus 1, 00:29:44.880 --> 00:29:45.760 align:middle line:90% OK? 00:29:45.760 --> 00:29:48.960 align:middle line:84% And then I have some terms here for this pairs. 00:29:48.960 --> 00:29:53.040 align:middle line:84% But my pairs only involve things up to n. 00:29:53.040 --> 00:29:58.120 align:middle line:84% So I'm going to look at the sum over all i less than j. 00:29:58.120 --> 00:30:01.480 align:middle line:84% But I'm going to put an upper bound that the n plus 1 term 00:30:01.480 --> 00:30:04.370 align:middle line:84% does not show up in this expression. 00:30:04.370 --> 00:30:09.410 align:middle line:84% And I'll have, again, the probability of the intersection, 00:30:09.410 --> 00:30:11.650 align:middle line:90% the pairwise intersection. 00:30:11.650 --> 00:30:14.330 align:middle line:84% And I can just continue this process here. 00:30:14.330 --> 00:30:17.030 align:middle line:84% So the terms that I've defined in triangle, 00:30:17.030 --> 00:30:19.310 align:middle line:84% they're not all the terms that are in square, 00:30:19.310 --> 00:30:21.650 align:middle line:90% but there are many of them. 00:30:21.650 --> 00:30:24.930 align:middle line:84% And now I just want to deal with what's left over. 00:30:24.930 --> 00:30:27.110 align:middle line:90% So I'll define one more thing. 00:30:27.110 --> 00:30:29.770 align:middle line:84% And then I can at least write down an expression. 00:30:29.770 --> 00:30:33.050 align:middle line:84% So let's look at almost all the terms 00:30:33.050 --> 00:30:35.090 align:middle line:90% that we've left off the table. 00:30:35.090 --> 00:30:37.890 align:middle line:84% Let's start with the pairwise intersections. 00:30:37.890 --> 00:30:42.330 align:middle line:84% So we can look from i equals 1 to n of the probability 00:30:42.330 --> 00:30:48.730 align:middle line:84% that A i happens and A n plus 1 happens, because all I'm doing 00:30:48.730 --> 00:30:51.650 align:middle line:84% is I'm looking at this expression right here. 00:30:51.650 --> 00:30:54.570 align:middle line:84% By putting an upper bound that I'm only considering things up 00:30:54.570 --> 00:30:58.010 align:middle line:84% to n, I've left off a whole bunch of terms. 00:30:58.010 --> 00:31:00.050 align:middle line:84% Let me look at all the pairs where i 00:31:00.050 --> 00:31:02.660 align:middle line:90% is something and j is n plus 1. 00:31:02.660 --> 00:31:05.040 align:middle line:84% Those are definitely terms I've left out. 00:31:05.040 --> 00:31:07.660 align:middle line:84% So I'm going to group them together in circle. 00:31:07.660 --> 00:31:11.700 align:middle line:84% And the same way I can look at the next term in circle, which 00:31:11.700 --> 00:31:18.540 align:middle line:84% is the sum of i less than j all between 1 and n 00:31:18.540 --> 00:31:21.820 align:middle line:84% of the probability of the threewise intersection, A i 00:31:21.820 --> 00:31:27.120 align:middle line:90% and A j and A n plus 1. 00:31:27.120 --> 00:31:30.820 align:middle line:84% All right, so this is just a whole bunch of bookkeeping. 00:31:30.820 --> 00:31:33.980 align:middle line:84% And let me write down the expression that I claim 00:31:33.980 --> 00:31:38.220 align:middle line:84% connects all of these different groupings of terms. 00:31:38.220 --> 00:31:42.060 align:middle line:84% At the end of the day, what I care about is this expression 00:31:42.060 --> 00:31:46.140 align:middle line:84% I have from inclusion-exclusion for n plus 1. 00:31:46.140 --> 00:31:47.580 align:middle line:90% That's square. 00:31:47.580 --> 00:31:50.140 align:middle line:84% And I claim that this is equal to all the terms 00:31:50.140 --> 00:31:56.940 align:middle line:84% that don't involve n plus 1 plus the probability of A n 00:31:56.940 --> 00:32:00.310 align:middle line:84% plus 1, that's not something that showed up in my circle 00:32:00.310 --> 00:32:05.390 align:middle line:84% expression, and then, minus circle. 00:32:05.390 --> 00:32:08.250 align:middle line:84% So you can see that I've actually flipped the signs here, 00:32:08.250 --> 00:32:11.210 align:middle line:84% because originally, when I had pairwise intersections, 00:32:11.210 --> 00:32:12.610 align:middle line:90% I had a minus sign. 00:32:12.610 --> 00:32:15.150 align:middle line:90% And here, I've put it as plus. 00:32:15.150 --> 00:32:17.750 align:middle line:84% But it'll be clear why I did this in a minute. 00:32:17.750 --> 00:32:20.250 align:middle line:84% So is everyone OK with this expression? 00:32:20.250 --> 00:32:22.910 align:middle line:84% I claim that we are basically home 00:32:22.910 --> 00:32:25.630 align:middle line:84% once we understand this grouping of terms. 00:32:25.630 --> 00:32:28.150 align:middle line:84% So now's a good time to ask questions? 00:32:28.150 --> 00:32:29.160 align:middle line:90% Does this make sense? 00:32:29.160 --> 00:32:31.670 align:middle line:90% 00:32:31.670 --> 00:32:33.030 align:middle line:90% Give me a nod yes. 00:32:33.030 --> 00:32:33.910 align:middle line:90% OK. 00:32:33.910 --> 00:32:36.310 align:middle line:84% All right, so let's put everything together now 00:32:36.310 --> 00:32:39.190 align:middle line:84% that we've done the painful work, which is writing down 00:32:39.190 --> 00:32:43.390 align:middle line:84% all these indices, and let's check that everything works out. 00:32:43.390 --> 00:32:46.710 align:middle line:84% So I need one more slightly clever thing. 00:32:46.710 --> 00:32:48.830 align:middle line:84% So what I'm going to do is I'm going to interpret 00:32:48.830 --> 00:32:50.630 align:middle line:90% each of these expressions. 00:32:50.630 --> 00:32:53.270 align:middle line:84% So first of all, triangle, I claim 00:32:53.270 --> 00:32:57.770 align:middle line:84% is equal to the probability of A 1 union A 00:32:57.770 --> 00:33:00.330 align:middle line:90% 2 all the way up to A n. 00:33:00.330 --> 00:33:03.130 align:middle line:90% Not A n plus 1, but A n. 00:33:03.130 --> 00:33:06.330 align:middle line:90% Why is that true? 00:33:06.330 --> 00:33:08.770 align:middle line:84% Just to make sure we're paying attention. 00:33:08.770 --> 00:33:10.770 align:middle line:90% Why is that true? 00:33:10.770 --> 00:33:13.130 align:middle line:84% Because we assumed it by induction. 00:33:13.130 --> 00:33:15.590 align:middle line:84% This is literally what we assume by induction. 00:33:15.590 --> 00:33:19.050 align:middle line:84% Remember, our strategy is to assume it's true up to n. 00:33:19.050 --> 00:33:22.290 align:middle line:84% And what I did was I pulled out all the terms in my n 00:33:22.290 --> 00:33:23.930 align:middle line:90% plus 1 expression. 00:33:23.930 --> 00:33:27.450 align:middle line:84% And actually, we get the same structure. 00:33:27.450 --> 00:33:31.330 align:middle line:84% And so we already know this by induction, perfect, OK? 00:33:31.330 --> 00:33:34.450 align:middle line:84% And now we need to be slightly clever, 00:33:34.450 --> 00:33:37.290 align:middle line:84% which is what I'm going to do is you can see that all 00:33:37.290 --> 00:33:40.970 align:middle line:84% of these events show up all over the place right here, 00:33:40.970 --> 00:33:45.450 align:middle line:84% these intersections between A i and A n plus 1. 00:33:45.450 --> 00:33:48.010 align:middle line:84% I'm going to call these new events. 00:33:48.010 --> 00:33:51.170 align:middle line:84% I'm going to define some helper events B i. 00:33:51.170 --> 00:33:55.780 align:middle line:84% So B i is defined as the event that A i happens and A n plus 1 00:33:55.780 --> 00:33:56.820 align:middle line:90% happens. 00:33:56.820 --> 00:33:58.700 align:middle line:90% Let's check our definition. 00:33:58.700 --> 00:34:04.200 align:middle line:84% I claim that this expression right here is b 1 intersect-- 00:34:04.200 --> 00:34:09.300 align:middle line:90% sorry, B i intersect B j. 00:34:09.300 --> 00:34:10.540 align:middle line:90% Why is that? 00:34:10.540 --> 00:34:14.440 align:middle line:84% Literally, the definition of B i is A i intersect A n plus 1. 00:34:14.440 --> 00:34:18.192 align:middle line:84% So A i intersect A n plus 1, intersect 00:34:18.192 --> 00:34:20.840 align:middle line:90% A j intersect A n plus 1. 00:34:20.840 --> 00:34:23.820 align:middle line:84% What happens when I intersect with A n plus 1 twice? 00:34:23.820 --> 00:34:25.780 align:middle line:84% It's the same thing as doing it once. 00:34:25.780 --> 00:34:28.860 align:middle line:84% So this is literally the pairwise intersection. 00:34:28.860 --> 00:34:31.620 align:middle line:84% And now, all of a sudden, we're in good shape, right? 00:34:31.620 --> 00:34:34.780 align:middle line:84% Because what I claim now is that circle 00:34:34.780 --> 00:34:37.780 align:middle line:84% has a very nice definition, which 00:34:37.780 --> 00:34:45.699 align:middle line:84% is that circle is the probability of B 1 union B 2 all 00:34:45.699 --> 00:34:49.659 align:middle line:90% the way up to union B n. 00:34:49.659 --> 00:34:51.520 align:middle line:84% That's literally the same structure. 00:34:51.520 --> 00:34:52.120 align:middle line:90% I have here. 00:34:52.120 --> 00:34:54.630 align:middle line:84% This is the same inclusion-exclusion formula 00:34:54.630 --> 00:34:55.710 align:middle line:90% had before. 00:34:55.710 --> 00:34:57.670 align:middle line:84% I just had to squint and figure out 00:34:57.670 --> 00:35:00.870 align:middle line:84% what are new events which this is expressing 00:35:00.870 --> 00:35:03.030 align:middle line:90% inclusion-exclusion on. 00:35:03.030 --> 00:35:06.310 align:middle line:84% So that's where the cleverness happened. 00:35:06.310 --> 00:35:10.830 align:middle line:84% And now I can remember what these B i's are. 00:35:10.830 --> 00:35:15.030 align:middle line:84% So this is the same thing I claim as the probability 00:35:15.030 --> 00:35:19.070 align:middle line:84% that A 1 or A 2, all the way up to 00:35:19.070 --> 00:35:25.910 align:middle line:84% or A n happens, and A n plus 1, because each one of these things 00:35:25.910 --> 00:35:29.710 align:middle line:84% just takes one of the A i's and intersects it with A n plus 1. 00:35:29.710 --> 00:35:34.070 align:middle line:84% So this is really just De Morgan's laws in disguise. 00:35:34.070 --> 00:35:35.590 align:middle line:90% So last piece of-- 00:35:35.590 --> 00:35:38.550 align:middle line:84% I won't even call this cleverness, right, 00:35:38.550 --> 00:35:42.590 align:middle line:84% which is, I'm going to call this event right here, C. 00:35:42.590 --> 00:35:45.670 align:middle line:84% The real event I care about is the union of all of the A i's 00:35:45.670 --> 00:35:47.830 align:middle line:90% from 1 to n plus 1. 00:35:47.830 --> 00:35:52.720 align:middle line:84% But let me look at C as just the union of from 1 to n. 00:35:52.720 --> 00:35:54.852 align:middle line:84% And let's write out what we know, 00:35:54.852 --> 00:35:58.840 align:middle line:84% and then we'll see that we've figured out our answer. 00:35:58.840 --> 00:36:02.200 align:middle line:84% So writing out, putting together all of this expression, 00:36:02.200 --> 00:36:04.560 align:middle line:90% we have that square-- 00:36:04.560 --> 00:36:07.040 align:middle line:84% well, we broke it up into different terms. 00:36:07.040 --> 00:36:08.960 align:middle line:84% This right here, this triangle is just 00:36:08.960 --> 00:36:12.200 align:middle line:84% the probability of the event, C. That's 00:36:12.200 --> 00:36:14.380 align:middle line:90% how we defined what C was. 00:36:14.380 --> 00:36:17.040 align:middle line:90% It was just the first n events. 00:36:17.040 --> 00:36:18.340 align:middle line:90% And then what do we have? 00:36:18.340 --> 00:36:24.560 align:middle line:84% We have the probability of A n plus 1. 00:36:24.560 --> 00:36:32.140 align:middle line:84% And then we subtract off the probability of C and A n plus 1. 00:36:32.140 --> 00:36:35.840 align:middle line:84% I just put everything we know together. 00:36:35.840 --> 00:36:38.120 align:middle line:90% So does this make sense? 00:36:38.120 --> 00:36:41.240 align:middle line:90% Why am I done? 00:36:41.240 --> 00:36:42.720 align:middle line:90% Yeah? 00:36:42.720 --> 00:36:44.820 align:middle line:90% This is the base case, perfect. 00:36:44.820 --> 00:36:47.000 align:middle line:90% This is the n equals 2 case. 00:36:47.000 --> 00:36:51.650 align:middle line:84% So by the n equals 2 case, we know 00:36:51.650 --> 00:36:58.250 align:middle line:84% that this is exactly equal to the probability of C union 00:36:58.250 --> 00:37:00.090 align:middle line:90% A n plus 1. 00:37:00.090 --> 00:37:01.690 align:middle line:84% And we just have to remember what 00:37:01.690 --> 00:37:08.450 align:middle line:84% C is, because C is the union of all of these A 1 up to A n. 00:37:08.450 --> 00:37:11.290 align:middle line:90% And then, we tack on A n plus 1. 00:37:11.290 --> 00:37:13.750 align:middle line:84% And we've proven inclusion-exclusion. 00:37:13.750 --> 00:37:15.810 align:middle line:90% So that's the proof. 00:37:15.810 --> 00:37:18.210 align:middle line:84% So it turns out that once you know what the expression is 00:37:18.210 --> 00:37:20.810 align:middle line:84% supposed to be, the rest of it is very natural, 00:37:20.810 --> 00:37:25.450 align:middle line:84% because this expression shows up in disguise in subparts 00:37:25.450 --> 00:37:28.210 align:middle line:90% of the expression we have. 00:37:28.210 --> 00:37:32.010 align:middle line:84% All right, so this was one of the longest proofs we've done. 00:37:32.010 --> 00:37:33.450 align:middle line:90% Are we good? 00:37:33.450 --> 00:37:34.490 align:middle line:90% Yeah? 00:37:34.490 --> 00:37:39.250 align:middle line:90% All right, in fact, I think-- 00:37:39.250 --> 00:37:43.890 align:middle line:84% yeah one of the other proofs we're 00:37:43.890 --> 00:37:46.530 align:middle line:84% going to do, maybe, I will be doing in three or four 00:37:46.530 --> 00:37:49.190 align:middle line:84% lectures time is the Chernoff bound, which 00:37:49.190 --> 00:37:51.390 align:middle line:84% will be so complicated that we're going to have 00:37:51.390 --> 00:37:53.370 align:middle line:90% you diagram it in recitation. 00:37:53.370 --> 00:37:54.510 align:middle line:90% So that'll be fun. 00:37:54.510 --> 00:37:57.190 align:middle line:84% But you should already start to think about not just verifying 00:37:57.190 --> 00:38:00.470 align:middle line:84% these things line by line, but why did I present it the way 00:38:00.470 --> 00:38:01.660 align:middle line:90% that I did? 00:38:01.660 --> 00:38:05.710 align:middle line:84% So I didn't literally have to show you the n equals 3 case. 00:38:05.710 --> 00:38:08.150 align:middle line:84% But we can see that that built up a lot of intuition 00:38:08.150 --> 00:38:10.350 align:middle line:84% for what the terms are doing, why they 00:38:10.350 --> 00:38:12.590 align:middle line:90% need to alternate in signs. 00:38:12.590 --> 00:38:16.430 align:middle line:84% And you can see some elements of at least how I thought about how 00:38:16.430 --> 00:38:18.870 align:middle line:90% to put these things together. 00:38:18.870 --> 00:38:26.110 align:middle line:84% OK, so now I want to tell you one other brief aside, 00:38:26.110 --> 00:38:29.060 align:middle line:84% and then we'll move on to our second topic for today. 00:38:29.060 --> 00:38:31.533 align:middle line:90% 00:38:31.533 --> 00:38:32.950 align:middle line:84% There is one thing that I told you 00:38:32.950 --> 00:38:37.070 align:middle line:84% about last time, which was when we 00:38:37.070 --> 00:38:41.630 align:middle line:84% talked about random variables, I talked about their expectation. 00:38:41.630 --> 00:38:44.030 align:middle line:84% And there was one key property that I told you 00:38:44.030 --> 00:38:49.560 align:middle line:84% about expectations, which is you add up two random variables, f 00:38:49.560 --> 00:38:53.000 align:middle line:84% and g, and then, the expectation of their sum 00:38:53.000 --> 00:38:55.360 align:middle line:84% is the sum of their expectations. 00:38:55.360 --> 00:38:57.420 align:middle line:84% So this was called linearity of expectation. 00:38:57.420 --> 00:38:58.680 align:middle line:90% I proved it for you. 00:38:58.680 --> 00:39:00.820 align:middle line:84% The proof was completely trivial. 00:39:00.820 --> 00:39:04.600 align:middle line:84% It was just rearranging things algebraically. 00:39:04.600 --> 00:39:09.160 align:middle line:84% But what I claim is that it's actually deeper than it sounds. 00:39:09.160 --> 00:39:11.020 align:middle line:84% So right now, what we went through, 00:39:11.020 --> 00:39:14.160 align:middle line:84% all of this work computing was the probability 00:39:14.160 --> 00:39:18.306 align:middle line:84% that no student gets their correct pset back. 00:39:18.306 --> 00:39:22.560 align:middle line:84% But another thing I could do through random variables 00:39:22.560 --> 00:39:31.720 align:middle line:84% is I could also ask, what about the expected number of students 00:39:31.720 --> 00:39:35.186 align:middle line:90% who get their correct pset back? 00:39:35.186 --> 00:39:37.140 align:middle line:90% So I had to do all of this work. 00:39:37.140 --> 00:39:39.140 align:middle line:84% I had to tell you about binomial coefficients, 00:39:39.140 --> 00:39:41.640 align:middle line:84% prove the inclusion-exclusion formula 00:39:41.640 --> 00:39:44.890 align:middle line:84% to answer a very similar sounding problem, 00:39:44.890 --> 00:39:47.730 align:middle line:84% but I claim that I have enough space left on the board 00:39:47.730 --> 00:39:51.875 align:middle line:84% right here to compute what this expression is, 00:39:51.875 --> 00:39:53.250 align:middle line:84% because the way that we can think 00:39:53.250 --> 00:39:56.770 align:middle line:84% about this random variable, the number right, really 00:39:56.770 --> 00:39:58.690 align:middle line:84% is the sum of a bunch of indicator 00:39:58.690 --> 00:40:00.730 align:middle line:90% variables for different events. 00:40:00.730 --> 00:40:05.010 align:middle line:84% I claim that this random variable is just the sum from i 00:40:05.010 --> 00:40:09.290 align:middle line:84% equals 1 to n of this indicator function of A i, 00:40:09.290 --> 00:40:13.570 align:middle line:84% whether the i-th student gets their correct pset back. 00:40:13.570 --> 00:40:15.950 align:middle line:84% Now, by linearity of expectation, 00:40:15.950 --> 00:40:18.130 align:middle line:84% this is the same thing as the sum from i 00:40:18.130 --> 00:40:24.730 align:middle line:84% equals 1 to n times the of the expectation of this indicator 00:40:24.730 --> 00:40:26.050 align:middle line:90% A i. 00:40:26.050 --> 00:40:29.010 align:middle line:84% So remember from last time that this indicator of an event 00:40:29.010 --> 00:40:31.450 align:middle line:84% is just the simple random variable that's 00:40:31.450 --> 00:40:35.250 align:middle line:84% 1 when the event happens and is 0 otherwise. 00:40:35.250 --> 00:40:38.450 align:middle line:84% So what is the expectation of the indicator 00:40:38.450 --> 00:40:40.050 align:middle line:84% of an event is just the probability 00:40:40.050 --> 00:40:42.290 align:middle line:90% that that event happens? 00:40:42.290 --> 00:40:45.660 align:middle line:84% So each of these things we already computed as 1 over n. 00:40:45.660 --> 00:40:50.420 align:middle line:84% So I'm going to get the sum from i equals 1 to n of 1 over n. 00:40:50.420 --> 00:40:52.460 align:middle line:90% And that's the same thing as 1. 00:40:52.460 --> 00:40:55.140 align:middle line:84% So the expected number of students you-- 00:40:55.140 --> 00:40:57.660 align:middle line:84% the number of students you expect to get the correct pset 00:40:57.660 --> 00:40:59.100 align:middle line:90% back is 1. 00:40:59.100 --> 00:41:00.820 align:middle line:84% So this is something important to keep 00:41:00.820 --> 00:41:03.860 align:middle line:84% track of is that it turns out that when you're dealing 00:41:03.860 --> 00:41:06.900 align:middle line:84% with expectations, you don't have to worry about all 00:41:06.900 --> 00:41:10.420 align:middle line:84% of the stuff about the dependency between the events, 00:41:10.420 --> 00:41:13.140 align:middle line:84% whether they're independent or not. 00:41:13.140 --> 00:41:16.220 align:middle line:84% And this is a really unique property of expectations. 00:41:16.220 --> 00:41:19.860 align:middle line:84% It won't be true when we get to other things like variance, 00:41:19.860 --> 00:41:21.740 align:middle line:84% but they'll still be helpful expressions 00:41:21.740 --> 00:41:25.980 align:middle line:84% we can do that can make our life easier for computing. 00:41:25.980 --> 00:41:28.780 align:middle line:90% OK, good? 00:41:28.780 --> 00:41:32.700 align:middle line:84% Let's go on to the second part of today's lecture, 00:41:32.700 --> 00:41:36.180 align:middle line:84% where what I want to do is I want to dig a bit more 00:41:36.180 --> 00:41:38.460 align:middle line:90% into random variables. 00:41:38.460 --> 00:41:41.710 align:middle line:84% We've talked about the expectation of random variables. 00:41:41.710 --> 00:41:44.110 align:middle line:84% And what I want to do is I want to introduce concepts 00:41:44.110 --> 00:41:46.330 align:middle line:84% like the variance of random variables. 00:41:46.330 --> 00:41:50.070 align:middle line:84% We're going to use these things extensively. 00:41:50.070 --> 00:41:54.895 align:middle line:84% So let me start off first with just a bit of a reminder 00:41:54.895 --> 00:41:56.270 align:middle line:84% about some of the cool things you 00:41:56.270 --> 00:41:58.690 align:middle line:90% can do with random variables. 00:41:58.690 --> 00:42:02.190 align:middle line:90% 00:42:02.190 --> 00:42:07.430 align:middle line:84% So last time, I told you this example of mean time to failure. 00:42:07.430 --> 00:42:10.350 align:middle line:84% That was one interesting illustration 00:42:10.350 --> 00:42:13.250 align:middle line:90% of the concept of expectation. 00:42:13.250 --> 00:42:15.450 align:middle line:84% If something is failing independently, 00:42:15.450 --> 00:42:18.310 align:middle line:84% like a computer, every day, what's 00:42:18.310 --> 00:42:21.310 align:middle line:84% the expected number of days that we 00:42:21.310 --> 00:42:25.390 align:middle line:84% expect to go before we actually have to replace that part? 00:42:25.390 --> 00:42:32.070 align:middle line:84% So remember, just as a reminder, we had this concept of mean time 00:42:32.070 --> 00:42:33.590 align:middle line:90% to failure. 00:42:33.590 --> 00:42:41.890 align:middle line:84% And if we had the CPU fails independently 00:42:41.890 --> 00:42:49.490 align:middle line:84% with probability p each day, then one of the quantities 00:42:49.490 --> 00:42:53.210 align:middle line:84% we cared about was the expectation of T, which 00:42:53.210 --> 00:42:55.270 align:middle line:84% is the failure time for the computer, 00:42:55.270 --> 00:42:58.610 align:middle line:84% so how often do we have to replace it. 00:42:58.610 --> 00:43:01.550 align:middle line:84% And we wrote out this expression for it that we simplified. 00:43:01.550 --> 00:43:04.890 align:middle line:84% And we showed that the answer was 1 over p, which makes sense, 00:43:04.890 --> 00:43:07.690 align:middle line:84% because if I have a coin that's fair, 00:43:07.690 --> 00:43:09.330 align:middle line:84% the expected number of flips until I 00:43:09.330 --> 00:43:11.610 align:middle line:90% get my first heads is two. 00:43:11.610 --> 00:43:16.170 align:middle line:84% And the smaller p is, the larger this expectation will be. 00:43:16.170 --> 00:43:18.410 align:middle line:84% So this was just an algebraic illustration 00:43:18.410 --> 00:43:22.970 align:middle line:84% of what we can do with the definition of expectation. 00:43:22.970 --> 00:43:25.890 align:middle line:84% I want to try a harder problem that builds off of mean time 00:43:25.890 --> 00:43:27.970 align:middle line:90% to failure. 00:43:27.970 --> 00:43:31.410 align:middle line:84% So let's consider the following experiment. 00:43:31.410 --> 00:43:34.770 align:middle line:84% What I'm going to do is I'm going to ask students 00:43:34.770 --> 00:43:39.250 align:middle line:84% in a huge class, OK, I'm going to ask them their birthday. 00:43:39.250 --> 00:43:43.220 align:middle line:90% 00:43:43.220 --> 00:43:51.340 align:middle line:84% And I want to know how many until we 00:43:51.340 --> 00:43:58.150 align:middle line:90% get all 365 days as answers. 00:43:58.150 --> 00:44:01.340 align:middle line:90% 00:44:01.340 --> 00:44:02.360 align:middle line:90% This is a leap year. 00:44:02.360 --> 00:44:04.840 align:middle line:84% So I guess we could ask a slightly different question 00:44:04.840 --> 00:44:05.860 align:middle line:90% this year. 00:44:05.860 --> 00:44:08.900 align:middle line:84% But we're going to assume that everyone's birthday is 00:44:08.900 --> 00:44:11.740 align:middle line:90% uniformly at random. 00:44:11.740 --> 00:44:15.000 align:middle line:84% And this is another very basic probability question. 00:44:15.000 --> 00:44:17.240 align:middle line:84% It sounds a bit like mean time to failure, 00:44:17.240 --> 00:44:19.180 align:middle line:84% but there's a twist on it, which is 00:44:19.180 --> 00:44:21.060 align:middle line:84% we don't care about one thing happening, 00:44:21.060 --> 00:44:23.940 align:middle line:84% like our first heads or the failure of a computer, 00:44:23.940 --> 00:44:25.460 align:middle line:84% we care about a coverage condition 00:44:25.460 --> 00:44:28.100 align:middle line:84% that we cover all of the different days. 00:44:28.100 --> 00:44:30.900 align:middle line:84% So a lot of times, you'll hear this colloquially referred 00:44:30.900 --> 00:44:33.900 align:middle line:84% to as the "coupon collector problem." 00:44:33.900 --> 00:44:37.310 align:middle line:84% We could ask this not with 365 as the number of things 00:44:37.310 --> 00:44:40.350 align:middle line:84% we want to cover, but some other parameter n. 00:44:40.350 --> 00:44:43.190 align:middle line:84% And you could imagine that every weekend you 00:44:43.190 --> 00:44:46.110 align:middle line:84% get a coupon in the paper, and it's uniformly at random 00:44:46.110 --> 00:44:47.790 align:middle line:90% for which coupon it is. 00:44:47.790 --> 00:44:49.790 align:middle line:84% And what you'd like to know is how many 00:44:49.790 --> 00:44:52.190 align:middle line:84% papers do you have to look at until you get 00:44:52.190 --> 00:44:53.883 align:middle line:90% all of the possible coupons. 00:44:53.883 --> 00:44:56.390 align:middle line:90% 00:44:56.390 --> 00:44:59.550 align:middle line:84% But already, we have some powerful tools 00:44:59.550 --> 00:45:01.670 align:middle line:84% for how to address this question. 00:45:01.670 --> 00:45:03.670 align:middle line:84% It's really not a straightforward question 00:45:03.670 --> 00:45:06.453 align:middle line:90% until you see the trick. 00:45:06.453 --> 00:45:07.870 align:middle line:84% You see, if you think about what I 00:45:07.870 --> 00:45:13.350 align:middle line:84% did in this example of computing the expected number of students 00:45:13.350 --> 00:45:16.990 align:middle line:84% who get their correct pset back, there was already something very 00:45:16.990 --> 00:45:19.870 align:middle line:84% powerful we had, which was that when 00:45:19.870 --> 00:45:22.450 align:middle line:84% I have a random variable that I care about, 00:45:22.450 --> 00:45:25.510 align:middle line:84% this random variable being the number of students 00:45:25.510 --> 00:45:28.510 align:middle line:84% who get the right answer, well, what I can do 00:45:28.510 --> 00:45:32.310 align:middle line:84% is I can decompose it into simpler building blocks that 00:45:32.310 --> 00:45:34.240 align:middle line:90% are easier to understand. 00:45:34.240 --> 00:45:36.800 align:middle line:84% So that's what I did there, because I broke up 00:45:36.800 --> 00:45:40.080 align:middle line:84% this random variable, which is the number right, 00:45:40.080 --> 00:45:42.280 align:middle line:84% into counting over all of these events 00:45:42.280 --> 00:45:44.408 align:middle line:84% just as a sum of indicator variables. 00:45:44.408 --> 00:45:46.700 align:middle line:84% Here, we're going to have to do the same type of trick, 00:45:46.700 --> 00:45:50.320 align:middle line:84% but we're going to do it in a more clever way. 00:45:50.320 --> 00:45:53.240 align:middle line:90% So same type of trick in action. 00:45:53.240 --> 00:45:56.600 align:middle line:84% So we're going to write T, which is our time where we collect all 00:45:56.600 --> 00:45:59.640 align:middle line:84% of these coupons or get all of these birthdays 00:45:59.640 --> 00:46:10.240 align:middle line:84% as a sum of simpler random variables. 00:46:10.240 --> 00:46:16.880 align:middle line:84% So what I'm going to do is I'm going to write T is T 0 plus T 1 00:46:16.880 --> 00:46:21.920 align:middle line:90% all the way up to T 364. 00:46:21.920 --> 00:46:27.200 align:middle line:84% And T i, just to be clear, is the number 00:46:27.200 --> 00:46:41.290 align:middle line:84% of steps between the i-th distinct and the i plus first. 00:46:41.290 --> 00:46:43.130 align:middle line:84% So what's going on is we have a progress 00:46:43.130 --> 00:46:46.970 align:middle line:84% measure of how many distinct birthdays we've seen. 00:46:46.970 --> 00:46:49.890 align:middle line:84% It doesn't matter to me what particular days they 00:46:49.890 --> 00:46:53.290 align:middle line:84% were, because everyone's birthday is uniformly at random. 00:46:53.290 --> 00:46:55.850 align:middle line:84% But I'm going to care about, in this progress 00:46:55.850 --> 00:47:00.330 align:middle line:84% measure, how much time do I have in between taking a step? 00:47:00.330 --> 00:47:02.010 align:middle line:84% So at the beginning, this might happen 00:47:02.010 --> 00:47:04.290 align:middle line:84% very quickly, because in the beginning, 00:47:04.290 --> 00:47:06.330 align:middle line:90% I've seen no birthdays. 00:47:06.330 --> 00:47:07.850 align:middle line:84% I ask a student, and I'm definitely 00:47:07.850 --> 00:47:10.690 align:middle line:84% going to cover a new day I haven't seen before. 00:47:10.690 --> 00:47:17.235 align:middle line:84% But towards the end, when I've covered all 364 out of 365 days, 00:47:17.235 --> 00:47:18.610 align:middle line:84% it's potentially going to take me 00:47:18.610 --> 00:47:23.010 align:middle line:84% a while in order to get that one remaining distinct birthday I 00:47:23.010 --> 00:47:28.850 align:middle line:84% haven't seen, to go from covering 364 to covering 365. 00:47:28.850 --> 00:47:30.990 align:middle line:90% So this is a big trick. 00:47:30.990 --> 00:47:33.570 align:middle line:84% Otherwise, it would not be at all obvious how to do this. 00:47:33.570 --> 00:47:34.910 align:middle line:90% Yes? 00:47:34.910 --> 00:47:38.310 align:middle line:84% AUDIENCE: Are they like, the number of students [INAUDIBLE]? 00:47:38.310 --> 00:47:39.390 align:middle line:90% PROFESSOR: Exactly. 00:47:39.390 --> 00:47:42.030 align:middle line:84% Sorry, number of steps meaning the number of students. 00:47:42.030 --> 00:47:43.510 align:middle line:90% So what this means is-- 00:47:43.510 --> 00:47:45.650 align:middle line:84% let's just break down what this expression means. 00:47:45.650 --> 00:47:47.390 align:middle line:90% So i is 0 here. 00:47:47.390 --> 00:47:50.590 align:middle line:84% So this would be the number of students 00:47:50.590 --> 00:47:54.670 align:middle line:84% I have to ask in order to get to the first distinct day. 00:47:54.670 --> 00:47:56.630 align:middle line:90% And that's clearly 1. 00:47:56.630 --> 00:47:59.490 align:middle line:84% But then, as I go later, it's the number of students I-- 00:47:59.490 --> 00:48:02.830 align:middle line:90% I just got my i-th new birthday. 00:48:02.830 --> 00:48:04.470 align:middle line:90% I've covered i days. 00:48:04.470 --> 00:48:06.530 align:middle line:84% I ask another student, ask another student, 00:48:06.530 --> 00:48:07.670 align:middle line:90% I ask another student. 00:48:07.670 --> 00:48:11.470 align:middle line:84% How long do I go until I get a new birthday out of that set? 00:48:11.470 --> 00:48:12.870 align:middle line:90% Any other questions? 00:48:12.870 --> 00:48:13.785 align:middle line:90% This is a big trick. 00:48:13.785 --> 00:48:15.910 align:middle line:84% It's hard to know that this is what you're supposed 00:48:15.910 --> 00:48:17.870 align:middle line:84% to do without seeing this trick before, 00:48:17.870 --> 00:48:21.950 align:middle line:84% but it's a trick we're going to use all the time. 00:48:21.950 --> 00:48:25.850 align:middle line:84% So now, by linearity of expectation, 00:48:25.850 --> 00:48:31.040 align:middle line:84% we know that the expectation of T is the sum from i equals 0 00:48:31.040 --> 00:48:37.640 align:middle line:84% to 364 of the expectation of all of these T i's. 00:48:37.640 --> 00:48:40.920 align:middle line:84% This is literally just using linearity of expectation. 00:48:40.920 --> 00:48:43.500 align:middle line:84% We know T is the sum of all of these T i's. 00:48:43.500 --> 00:48:47.680 align:middle line:84% That's how many students I have to ask until I 00:48:47.680 --> 00:48:49.920 align:middle line:90% cover all of the birthdays. 00:48:49.920 --> 00:48:53.440 align:middle line:84% And what I claim is that this expectation of T i 00:48:53.440 --> 00:48:56.640 align:middle line:90% is very easy to compute. 00:48:56.640 --> 00:48:58.400 align:middle line:84% So let's think about what happens 00:48:58.400 --> 00:49:02.440 align:middle line:90% with this expectation of T i. 00:49:02.440 --> 00:49:05.560 align:middle line:84% Let's draw a picture in case it's helpful. 00:49:05.560 --> 00:49:11.920 align:middle line:84% So we can think about all of the different birthdays. 00:49:11.920 --> 00:49:16.360 align:middle line:90% We have 365 of them. 00:49:16.360 --> 00:49:20.880 align:middle line:84% And we've seen i different birthdays so far. 00:49:20.880 --> 00:49:22.960 align:middle line:84% And we're asking how long until we 00:49:22.960 --> 00:49:25.400 align:middle line:90% see the next distinct birthday. 00:49:25.400 --> 00:49:27.930 align:middle line:84% So we've seen all of these different birthdays 00:49:27.930 --> 00:49:29.170 align:middle line:90% in this region. 00:49:29.170 --> 00:49:32.125 align:middle line:84% Because each dart I'm throwing is uniformly at random, 00:49:32.125 --> 00:49:34.250 align:middle line:84% I can just pretend that all the birthdays I've seen 00:49:34.250 --> 00:49:35.610 align:middle line:90% are all at the beginning. 00:49:35.610 --> 00:49:37.250 align:middle line:90% Nothing's going to change. 00:49:37.250 --> 00:49:38.850 align:middle line:84% And what I'm asking is how long, when 00:49:38.850 --> 00:49:42.730 align:middle line:84% I throw a dart that randomly hits one of these segments, 00:49:42.730 --> 00:49:46.730 align:middle line:84% until it actually is outside the set I've seen so far. 00:49:46.730 --> 00:49:50.290 align:middle line:84% So we know that this probability that any particular dart 00:49:50.290 --> 00:49:57.090 align:middle line:84% lands outside this set is just 365 minus i over 365. 00:49:57.090 --> 00:49:58.850 align:middle line:90% This makes intuitive sense. 00:49:58.850 --> 00:50:01.230 align:middle line:84% When i equals 0 and I haven't seen anything, 00:50:01.230 --> 00:50:03.650 align:middle line:84% I'm definitely going to get a new birthday. 00:50:03.650 --> 00:50:06.430 align:middle line:84% When I've seen all but one of the birthdays, 00:50:06.430 --> 00:50:09.610 align:middle line:90% I have a 1 over 365 chance. 00:50:09.610 --> 00:50:12.010 align:middle line:84% And so, now we're in good shape, because 00:50:12.010 --> 00:50:14.030 align:middle line:84% from the mean time to failure argument, 00:50:14.030 --> 00:50:18.210 align:middle line:84% we know exactly what this expectation of T is. 00:50:18.210 --> 00:50:24.320 align:middle line:84% It's equal to the sum of 1 over these P i's So in the beginning, 00:50:24.320 --> 00:50:27.620 align:middle line:90% we just have 365 over 365. 00:50:27.620 --> 00:50:32.580 align:middle line:84% Then we have 365 over 364 all the way down 00:50:32.580 --> 00:50:38.340 align:middle line:84% to adding up 365 over 1, which is 365. 00:50:38.340 --> 00:50:40.260 align:middle line:84% So it turns out that the answer for this, you 00:50:40.260 --> 00:50:46.660 align:middle line:84% can just compute it numerically, it's about 6.47% times 365. 00:50:46.660 --> 00:50:51.660 align:middle line:84% So I have to ask about 6 and 1/2 times as large a class as I 00:50:51.660 --> 00:50:55.100 align:middle line:84% would need in principle to cover all the distinct days. 00:50:55.100 --> 00:51:01.220 align:middle line:84% In fact, the same argument works even with 365 replaced by n. 00:51:01.220 --> 00:51:04.220 align:middle line:84% So more generally, for the coupon collector problem, 00:51:04.220 --> 00:51:09.460 align:middle line:84% you'll get n over n plus n over n minus 1 all the way down 00:51:09.460 --> 00:51:10.980 align:middle line:90% to adding n. 00:51:10.980 --> 00:51:18.140 align:middle line:84% That's the sum of i equals 1 to it's n times the sum of i 00:51:18.140 --> 00:51:21.140 align:middle line:90% equals 1 to n of 1 over n. 00:51:21.140 --> 00:51:23.510 align:middle line:84% And it turns out that this is equal to n 00:51:23.510 --> 00:51:25.890 align:middle line:90% times the n-th harmonic number. 00:51:25.890 --> 00:51:28.390 align:middle line:90% It behaves like n log n. 00:51:28.390 --> 00:51:32.150 align:middle line:84% So this gives you a very good back-of-the-envelope calculation 00:51:32.150 --> 00:51:36.630 align:middle line:84% for a lot of randomized things that this coupon collector 00:51:36.630 --> 00:51:38.950 align:middle line:90% problem shows up all the time. 00:51:38.950 --> 00:51:41.470 align:middle line:84% So basically, the 6.47 just comes 00:51:41.470 --> 00:51:45.880 align:middle line:84% from taking the natural log of the appropriate domain size. 00:51:45.880 --> 00:51:49.350 align:middle line:90% OK, any questions? 00:51:49.350 --> 00:51:50.750 align:middle line:90% Good? 00:51:50.750 --> 00:51:57.170 align:middle line:84% All right, so we've done a whole bunch of examples today. 00:51:57.170 --> 00:51:59.770 align:middle line:90% So we did the pset problem. 00:51:59.770 --> 00:52:01.830 align:middle line:84% We did this coupon collector problem. 00:52:01.830 --> 00:52:03.590 align:middle line:84% These are really important examples 00:52:03.590 --> 00:52:06.910 align:middle line:84% to build your probabilistic intuition. 00:52:06.910 --> 00:52:08.430 align:middle line:84% I want to tell you a little bit more 00:52:08.430 --> 00:52:11.250 align:middle line:84% about some powerful general tools. 00:52:11.250 --> 00:52:13.692 align:middle line:90% And then we'll call it a day. 00:52:13.692 --> 00:52:16.150 align:middle line:84% So we're going to do a little bit more theory, the same way 00:52:16.150 --> 00:52:18.750 align:middle line:84% that last time I told you about random variables 00:52:18.750 --> 00:52:22.970 align:middle line:84% and their expectation, I want to tell you about the variance 00:52:22.970 --> 00:52:26.770 align:middle line:84% and what happens when I take the expectation of products 00:52:26.770 --> 00:52:29.530 align:middle line:90% of random variables. 00:52:29.530 --> 00:52:33.570 align:middle line:84% And in fact, the first thing I'm going to do 00:52:33.570 --> 00:52:37.250 align:middle line:84% is, the same way that I defined the notion of two events 00:52:37.250 --> 00:52:39.410 align:middle line:84% being independent, I'm going to tell you 00:52:39.410 --> 00:52:44.170 align:middle line:84% what it means for two random variables to be independent. 00:52:44.170 --> 00:52:47.050 align:middle line:84% And then we'll see , when two random variables are 00:52:47.050 --> 00:52:51.330 align:middle line:84% independent, what kinds of computations become easier using 00:52:51.330 --> 00:52:51.830 align:middle line:90% that. 00:52:51.830 --> 00:52:54.370 align:middle line:90% 00:52:54.370 --> 00:52:56.490 align:middle line:84% So here's our key definition that we're 00:52:56.490 --> 00:53:00.170 align:middle line:84% going to explore for the rest of what's left in lecture. 00:53:00.170 --> 00:53:03.690 align:middle line:84% So remember, we have this notion of a random variable, which 00:53:03.690 --> 00:53:07.930 align:middle line:84% is just a function from the outcome space to a real value. 00:53:07.930 --> 00:53:11.570 align:middle line:84% And now, imagine I give you two random variables, f and g. 00:53:11.570 --> 00:53:13.450 align:middle line:84% Well, we're going to say that they're 00:53:13.450 --> 00:53:23.460 align:middle line:84% independent if the following events are independent. 00:53:23.460 --> 00:53:30.020 align:middle line:84% So let's look at all the x's where f of x equals alpha. 00:53:30.020 --> 00:53:37.980 align:middle line:84% And let's look at all of the x's where g of x equals some beta. 00:53:37.980 --> 00:53:41.260 align:middle line:84% All right, so a bunch of things are happening here. 00:53:41.260 --> 00:53:44.580 align:middle line:84% So I have a random variable, which remember, takes an outcome 00:53:44.580 --> 00:53:46.580 align:middle line:90% and maps it to a real value. 00:53:46.580 --> 00:53:48.740 align:middle line:84% So f and g are my two functions that 00:53:48.740 --> 00:53:50.980 align:middle line:90% map outcomes to real values. 00:53:50.980 --> 00:53:53.100 align:middle line:84% What I can do is I can use a random variable 00:53:53.100 --> 00:53:54.980 align:middle line:90% to define an event. 00:53:54.980 --> 00:53:57.020 align:middle line:84% So now I'm going the other direction. 00:53:57.020 --> 00:53:59.980 align:middle line:84% Before, what I told you about, was how to take an event 00:53:59.980 --> 00:54:01.860 align:middle line:90% and define a random variable. 00:54:01.860 --> 00:54:05.420 align:middle line:84% That took us to the notion of an indicator variable. 00:54:05.420 --> 00:54:08.100 align:middle line:84% And we used that in that computation. 00:54:08.100 --> 00:54:11.020 align:middle line:84% Now I'm going the other direction, which is I'm saying, 00:54:11.020 --> 00:54:12.880 align:middle line:84% imagine I give you a random variable 00:54:12.880 --> 00:54:16.220 align:middle line:84% and it takes on a whole bunch of discrete values. 00:54:16.220 --> 00:54:20.150 align:middle line:84% And what I'll do is I'll chop up the domain, the sample space, 00:54:20.150 --> 00:54:23.190 align:middle line:84% into the parts that correspond to different values 00:54:23.190 --> 00:54:24.950 align:middle line:90% of that random variable. 00:54:24.950 --> 00:54:26.330 align:middle line:84% So I'm going the other direction. 00:54:26.330 --> 00:54:28.638 align:middle line:84% I'm taking a random variable and defining an event, 00:54:28.638 --> 00:54:30.430 align:middle line:84% because at the end of the day, this is just 00:54:30.430 --> 00:54:32.950 align:middle line:90% a subset of the outcome space. 00:54:32.950 --> 00:54:35.270 align:middle line:84% So this is an event that's defined 00:54:35.270 --> 00:54:37.110 align:middle line:90% based on the choice of alpha. 00:54:37.110 --> 00:54:39.990 align:middle line:84% I get a different event for different choices of alpha. 00:54:39.990 --> 00:54:42.270 align:middle line:84% This is an event that's defined based 00:54:42.270 --> 00:54:44.670 align:middle line:90% on the choice of beta and g. 00:54:44.670 --> 00:54:47.430 align:middle line:84% And what I want is that I want these two events are 00:54:47.430 --> 00:54:50.790 align:middle line:84% independent, which is just the same definition of independence 00:54:50.790 --> 00:54:52.150 align:middle line:90% we had before. 00:54:52.150 --> 00:54:56.110 align:middle line:84% I want the probability of the intersection of those two events 00:54:56.110 --> 00:54:59.110 align:middle line:84% to be equal to the product of their probabilities. 00:54:59.110 --> 00:55:02.870 align:middle line:84% That's literally how we define independence of events. 00:55:02.870 --> 00:55:05.310 align:middle line:84% And what it means for random variables to be independent 00:55:05.310 --> 00:55:07.870 align:middle line:84% is this more stringent thing, which 00:55:07.870 --> 00:55:11.510 align:middle line:84% is I want that for any choice of alpha and beta 00:55:11.510 --> 00:55:14.110 align:middle line:84% that induces some choice of an event, one 00:55:14.110 --> 00:55:16.360 align:middle line:84% for one random variable, one for the other, 00:55:16.360 --> 00:55:19.000 align:middle line:84% I want every choice of alpha and beta 00:55:19.000 --> 00:55:22.280 align:middle line:84% to define two events that are independent. 00:55:22.280 --> 00:55:23.960 align:middle line:90% Does this make sense? 00:55:23.960 --> 00:55:25.880 align:middle line:84% So this is a more complicated definition 00:55:25.880 --> 00:55:29.520 align:middle line:84% than we had before because we have one quantifier here. 00:55:29.520 --> 00:55:32.892 align:middle line:84% So checking independence for events is very easy. 00:55:32.892 --> 00:55:34.600 align:middle line:84% You just compute a bunch of probabilities 00:55:34.600 --> 00:55:36.960 align:middle line:84% and check that the equation holds. 00:55:36.960 --> 00:55:39.560 align:middle line:84% Now, computing independence for random variables 00:55:39.560 --> 00:55:41.720 align:middle line:84% is trickier, because in principle, you 00:55:41.720 --> 00:55:44.960 align:middle line:84% have to worry about every possible value of alpha and beta 00:55:44.960 --> 00:55:49.920 align:middle line:84% and compute all of those probabilities, OK? 00:55:49.920 --> 00:55:52.530 align:middle line:84% Nevertheless, it's a very powerful notion. 00:55:52.530 --> 00:55:55.520 align:middle line:90% 00:55:55.520 --> 00:55:57.520 align:middle line:84% One thing you can do is let's just 00:55:57.520 --> 00:55:59.310 align:middle line:90% do a simple consistency check. 00:55:59.310 --> 00:56:03.200 align:middle line:90% 00:56:03.200 --> 00:56:05.940 align:middle line:84% Sorry for going back and forth and back and forth. 00:56:05.940 --> 00:56:13.990 align:middle line:84% But we can look at the indicator of two different events, 00:56:13.990 --> 00:56:16.130 align:middle line:84% A and B. Those are random variables. 00:56:16.130 --> 00:56:19.690 align:middle line:84% What does it mean for those functions to be independent? 00:56:19.690 --> 00:56:22.510 align:middle line:84% I claim that those functions, those random variables, 00:56:22.510 --> 00:56:26.050 align:middle line:84% are independent if and only if their events are. 00:56:26.050 --> 00:56:29.510 align:middle line:84% So this just means that our definition makes sense, 00:56:29.510 --> 00:56:33.010 align:middle line:84% because we're thinking of random variables as a generalization 00:56:33.010 --> 00:56:36.790 align:middle line:84% of events because I can have not just 0 1 values, 00:56:36.790 --> 00:56:38.970 align:middle line:90% but I can have any real values. 00:56:38.970 --> 00:56:41.930 align:middle line:84% But my definition better at least recover 00:56:41.930 --> 00:56:44.550 align:middle line:84% what my definition was in the case of events. 00:56:44.550 --> 00:56:45.650 align:middle line:90% And that's true. 00:56:45.650 --> 00:56:49.210 align:middle line:84% You can check that if you create two random variables based 00:56:49.210 --> 00:56:52.810 align:middle line:84% on the indicator functions of two different events, 00:56:52.810 --> 00:56:55.130 align:middle line:84% that they will meet this condition if and only 00:56:55.130 --> 00:56:58.330 align:middle line:84% if the corresponding events meet that condition too, because you 00:56:58.330 --> 00:57:02.070 align:middle line:84% really only have to check alpha and beta being 0 and 1. 00:57:02.070 --> 00:57:04.530 align:middle line:84% And you can check that everything works out. 00:57:04.530 --> 00:57:05.350 align:middle line:90% So that's good. 00:57:05.350 --> 00:57:10.250 align:middle line:84% So we have a reasonable notion of independence. 00:57:10.250 --> 00:57:12.590 align:middle line:84% What I want to work up today is some idea 00:57:12.590 --> 00:57:14.380 align:middle line:90% of why this is powerful. 00:57:14.380 --> 00:57:17.710 align:middle line:90% 00:57:17.710 --> 00:57:19.230 align:middle line:84% And I want to compare and contrast 00:57:19.230 --> 00:57:21.470 align:middle line:84% this to linearity of expectation that we've already 00:57:21.470 --> 00:57:23.830 align:middle line:90% used to great effect. 00:57:23.830 --> 00:57:28.830 align:middle line:84% So here's the key lemma, which we'll prove. 00:57:28.830 --> 00:57:34.010 align:middle line:84% The proof is pretty boring, but it's a very powerful lemma. 00:57:34.010 --> 00:57:37.150 align:middle line:84% The proof is really just about massaging the definitions 00:57:37.150 --> 00:57:40.150 align:middle line:84% the same way we had for linearity of expectation. 00:57:40.150 --> 00:57:43.670 align:middle line:84% What I claim is that if we have two random variables, f and g, 00:57:43.670 --> 00:57:47.350 align:middle line:84% so let's say f and g are independent random variables, 00:57:47.350 --> 00:57:52.790 align:middle line:84% then, what I claim is that the expectation of f 00:57:52.790 --> 00:57:58.110 align:middle line:84% of x times g of x is equal to the product 00:57:58.110 --> 00:57:59.650 align:middle line:90% of their expectation. 00:57:59.650 --> 00:58:05.370 align:middle line:90% 00:58:05.370 --> 00:58:07.470 align:middle line:84% So that's the lemma I want to prove. 00:58:07.470 --> 00:58:10.280 align:middle line:84% And I promise you the proof is very easy. 00:58:10.280 --> 00:58:12.680 align:middle line:84% But this is still a very powerful lemma. 00:58:12.680 --> 00:58:15.680 align:middle line:84% So usually, when I'm trying to learn something new, 00:58:15.680 --> 00:58:17.760 align:middle line:84% I try and connect it back to something 00:58:17.760 --> 00:58:21.080 align:middle line:90% that I already understand. 00:58:21.080 --> 00:58:23.760 align:middle line:84% So let's think about the special case, again, 00:58:23.760 --> 00:58:28.520 align:middle line:84% where these random variables are indicators of events. 00:58:28.520 --> 00:58:30.580 align:middle line:84% This is an indicator of an event A. 00:58:30.580 --> 00:58:34.120 align:middle line:84% This is an indicator of an event B. 00:58:34.120 --> 00:58:38.040 align:middle line:84% So what would the expectation of the indicator of event A 00:58:38.040 --> 00:58:38.700 align:middle line:90% look like? 00:58:38.700 --> 00:58:41.600 align:middle line:84% It would be the probability of the event A. 00:58:41.600 --> 00:58:44.120 align:middle line:84% What would the expectation of the indicator function 00:58:44.120 --> 00:58:46.400 align:middle line:90% of the other event look like? 00:58:46.400 --> 00:58:48.400 align:middle line:84% It would be the probability of the other event. 00:58:48.400 --> 00:58:50.960 align:middle line:84% So this is the product of their probabilities. 00:58:50.960 --> 00:58:53.440 align:middle line:84% What is that supposed to be equal to? 00:58:53.440 --> 00:58:57.480 align:middle line:84% What happens if I take two indicator functions of two 00:58:57.480 --> 00:58:59.240 align:middle line:90% different events, A and B? 00:58:59.240 --> 00:59:01.280 align:middle line:90% What do I get? 00:59:01.280 --> 00:59:02.020 align:middle line:90% What is that? 00:59:02.020 --> 00:59:04.360 align:middle line:84% That's a whole bunch of English to parse. 00:59:04.360 --> 00:59:08.850 align:middle line:84% I take the indicator of two different events, A and B, 00:59:08.850 --> 00:59:13.130 align:middle line:84% and I multiply the indicator random variables. 00:59:13.130 --> 00:59:14.070 align:middle line:90% What is that? 00:59:14.070 --> 00:59:15.370 align:middle line:84% Can anyone describe that to me in English? 00:59:15.370 --> 00:59:15.870 align:middle line:90% Yeah. 00:59:15.870 --> 00:59:17.200 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:59:17.200 --> 00:59:18.490 align:middle line:90% PROFESSOR: Yes, exactly. 00:59:18.490 --> 00:59:21.090 align:middle line:84% So the product of them is literally 00:59:21.090 --> 00:59:23.570 align:middle line:84% the indicator of the intersection of the events. 00:59:23.570 --> 00:59:25.870 align:middle line:84% And then, the expectation of that, as you said, 00:59:25.870 --> 00:59:28.210 align:middle line:84% is the probability that both of them happen. 00:59:28.210 --> 00:59:32.650 align:middle line:84% So this recovers something we already know, which literally, 00:59:32.650 --> 00:59:37.370 align:middle line:84% this is the definition of what independence is for events. 00:59:37.370 --> 00:59:39.130 align:middle line:84% So this is just a generalization, 00:59:39.130 --> 00:59:42.650 align:middle line:84% building a more general theory of what's going on. 00:59:42.650 --> 00:59:46.426 align:middle line:84% All right, so let's prove this thing. 00:59:46.426 --> 00:59:48.670 align:middle line:90% Ah, that's tedious. 00:59:48.670 --> 00:59:50.170 align:middle line:84% Let's just write out the definitions 00:59:50.170 --> 00:59:52.210 align:middle line:90% and see what happens. 00:59:52.210 --> 00:59:56.970 align:middle line:84% So I've got the expectation of f of x and g of x. 00:59:56.970 --> 00:59:59.270 align:middle line:84% Well, whatever this mess is inside, 00:59:59.270 --> 01:00:00.650 align:middle line:90% that's a random variable. 01:00:00.650 --> 01:00:02.730 align:middle line:84% It takes in an outcome x and it spits 01:00:02.730 --> 01:00:05.740 align:middle line:84% out a real value that's given by the product of the values of f 01:00:05.740 --> 01:00:06.780 align:middle line:90% and g. 01:00:06.780 --> 01:00:08.820 align:middle line:84% So I can take its expectation just 01:00:08.820 --> 01:00:13.780 align:middle line:84% by literally using the definition of the expectation. 01:00:13.780 --> 01:00:17.780 align:middle line:84% It's the sum over all x's in the sample space of p 01:00:17.780 --> 01:00:20.140 align:middle line:84% of x, the probability of that event, 01:00:20.140 --> 01:00:23.352 align:middle line:90% times f of x times g of x. 01:00:23.352 --> 01:00:24.560 align:middle line:90% I haven't done anything here. 01:00:24.560 --> 01:00:26.900 align:middle line:84% I've just literally used the definition. 01:00:26.900 --> 01:00:29.440 align:middle line:84% Whatever this monstrosity is, it's a random variable. 01:00:29.440 --> 01:00:33.060 align:middle line:84% And this is the definition of what its expectation is. 01:00:33.060 --> 01:00:34.700 align:middle line:84% And now what we're going to do is, 01:00:34.700 --> 01:00:37.980 align:middle line:84% here's the one clever part, is we're going to break up 01:00:37.980 --> 01:00:42.140 align:middle line:84% the sum into different pieces in a way that corresponds you know 01:00:42.140 --> 01:00:45.520 align:middle line:84% clearly, we have to use this definition of independence, 01:00:45.520 --> 01:00:47.340 align:middle line:90% otherwise, the thing is false. 01:00:47.340 --> 01:00:49.020 align:middle line:84% And independence is fundamentally 01:00:49.020 --> 01:00:52.660 align:middle line:84% about breaking up the sample space into different pieces. 01:00:52.660 --> 01:00:54.580 align:middle line:84% So I'm going to break up that sum 01:00:54.580 --> 01:00:57.780 align:middle line:84% into different pieces mimicking the way that I've broken it up 01:00:57.780 --> 01:00:59.420 align:middle line:90% in this definition. 01:00:59.420 --> 01:01:01.420 align:middle line:84% So literally, the way that I'm going to do that, 01:01:01.420 --> 01:01:04.290 align:middle line:84% is it's going to be the sum over alpha and beta-- 01:01:04.290 --> 01:01:06.790 align:middle line:90% and that should sound familiar-- 01:01:06.790 --> 01:01:09.990 align:middle line:84% and then the sum over all x that are 01:01:09.990 --> 01:01:13.510 align:middle line:84% in the sample space, but moreover, where f of x 01:01:13.510 --> 01:01:17.830 align:middle line:84% is equal to alpha and g of x is equal to beta. 01:01:17.830 --> 01:01:20.590 align:middle line:84% So I'm just breaking this sum up into different pieces, 01:01:20.590 --> 01:01:22.430 align:middle line:90% different contributions. 01:01:22.430 --> 01:01:24.470 align:middle line:90% And then I have the same thing. 01:01:24.470 --> 01:01:27.830 align:middle line:84% I have the probability of that outcome. 01:01:27.830 --> 01:01:28.750 align:middle line:90% I have f of x. 01:01:28.750 --> 01:01:30.190 align:middle line:90% I have g of x. 01:01:30.190 --> 01:01:32.350 align:middle line:84% Except, because I've broken this event up, 01:01:32.350 --> 01:01:34.830 align:middle line:84% I know what f of x is and I know what g of x is. 01:01:34.830 --> 01:01:37.230 align:middle line:84% They're alpha and beta, respectively. 01:01:37.230 --> 01:01:42.750 align:middle line:84% So this is the same thing as alpha times beta times 01:01:42.750 --> 01:01:48.630 align:middle line:90% the probability of x. 01:01:48.630 --> 01:01:51.230 align:middle line:90% So now we're in good shape. 01:01:51.230 --> 01:01:54.730 align:middle line:84% And we're just going to have to rearrange what the sum is. 01:01:54.730 --> 01:01:59.390 align:middle line:90% 01:01:59.390 --> 01:02:03.860 align:middle line:84% So let's massage the sum until it stares us in the face, 01:02:03.860 --> 01:02:05.880 align:middle line:90% how to appeal to independents. 01:02:05.880 --> 01:02:12.560 align:middle line:84% So this sum is the same thing as sum over alpha 01:02:12.560 --> 01:02:16.960 align:middle line:90% and beta times alpha-- 01:02:16.960 --> 01:02:23.600 align:middle line:84% of alpha times beta times the probability that f of x 01:02:23.600 --> 01:02:30.280 align:middle line:84% is equal to alpha and g of x is equal to beta. 01:02:30.280 --> 01:02:32.640 align:middle line:90% So what have I done here? 01:02:32.640 --> 01:02:34.800 align:middle line:84% This alpha and beta, I can actually 01:02:34.800 --> 01:02:37.440 align:middle line:90% just pull it outside the sum. 01:02:37.440 --> 01:02:40.440 align:middle line:84% And when I pull this alpha and beta outside the sum, 01:02:40.440 --> 01:02:44.120 align:middle line:84% I'm left with the sum of all these probabilities 01:02:44.120 --> 01:02:46.320 align:middle line:90% of these atomic outcomes. 01:02:46.320 --> 01:02:49.140 align:middle line:84% And this is really just an event. 01:02:49.140 --> 01:02:52.720 align:middle line:84% It's the event that f of x equals alpha and g of x 01:02:52.720 --> 01:02:53.920 align:middle line:90% equals beta. 01:02:53.920 --> 01:02:57.800 align:middle line:84% So I get the probability of that event. 01:02:57.800 --> 01:03:00.620 align:middle line:84% And now I can appeal to independence, 01:03:00.620 --> 01:03:04.420 align:middle line:84% because this is the same thing as the sum over alpha and beta, 01:03:04.420 --> 01:03:06.620 align:middle line:90% again, alpha times beta. 01:03:06.620 --> 01:03:09.220 align:middle line:84% But now I can break up these probabilities, 01:03:09.220 --> 01:03:13.940 align:middle line:84% f of x equals alpha times the probability that g of x 01:03:13.940 --> 01:03:15.420 align:middle line:90% equals beta. 01:03:15.420 --> 01:03:18.100 align:middle line:84% That's what it means for the two events 01:03:18.100 --> 01:03:20.620 align:middle line:84% that I've defined here to be independent is literally 01:03:20.620 --> 01:03:22.900 align:middle line:84% that the probability of their intersection 01:03:22.900 --> 01:03:26.380 align:middle line:84% is the product of their probabilities. 01:03:26.380 --> 01:03:29.660 align:middle line:84% And so now I'm in good shape, because what I can do 01:03:29.660 --> 01:03:32.520 align:middle line:84% is I can turn this into two different sums, 01:03:32.520 --> 01:03:34.640 align:middle line:84% the sum over alpha and the sum over beta. 01:03:34.640 --> 01:03:37.660 align:middle line:84% And I can pull out terms that don't depend on beta. 01:03:37.660 --> 01:03:41.100 align:middle line:84% And I claim that this is the same thing as the sum over alpha 01:03:41.100 --> 01:03:45.140 align:middle line:84% of alpha times the probability that f of x 01:03:45.140 --> 01:03:51.100 align:middle line:84% equals alpha all times the sum over beta of beta times 01:03:51.100 --> 01:03:55.380 align:middle line:84% the probability that g of x equals beta. 01:03:55.380 --> 01:03:57.950 align:middle line:84% So I really haven't done much other than figure 01:03:57.950 --> 01:04:01.110 align:middle line:84% out this way to break up the sample space so that I 01:04:01.110 --> 01:04:03.270 align:middle line:90% can appeal to my definition. 01:04:03.270 --> 01:04:05.710 align:middle line:84% But the rest of it is just algebra. 01:04:05.710 --> 01:04:07.620 align:middle line:84% So what is this expression right here? 01:04:07.620 --> 01:04:10.910 align:middle line:90% 01:04:10.910 --> 01:04:12.110 align:middle line:90% Any ideas? 01:04:12.110 --> 01:04:14.290 align:middle line:84% This wasn't quite my original definition, 01:04:14.290 --> 01:04:18.270 align:middle line:84% but hopefully, you should see that it's the same. 01:04:18.270 --> 01:04:20.230 align:middle line:84% What property of the random variable f 01:04:20.230 --> 01:04:22.830 align:middle line:90% is that expression computing? 01:04:22.830 --> 01:04:23.710 align:middle line:90% Yeah. 01:04:23.710 --> 01:04:25.430 align:middle line:90% Expectation, that's right. 01:04:25.430 --> 01:04:27.350 align:middle line:84% So my original definition for expectation 01:04:27.350 --> 01:04:29.725 align:middle line:84% was the sum over x's in the sample space. 01:04:29.725 --> 01:04:31.350 align:middle line:84% Here, I'm doing the same thing, but I'm 01:04:31.350 --> 01:04:35.790 align:middle line:84% collecting all the x's together that have f of x equal alpha 01:04:35.790 --> 01:04:38.230 align:middle line:90% and arranging it that way. 01:04:38.230 --> 01:04:40.750 align:middle line:84% So this is exactly just another definition 01:04:40.750 --> 01:04:43.910 align:middle line:90% for the expectation of f of x. 01:04:43.910 --> 01:04:50.350 align:middle line:84% And likewise, this right here is the expectation of g of x. 01:04:50.350 --> 01:04:51.810 align:middle line:90% Awesome, so we're done. 01:04:51.810 --> 01:04:56.000 align:middle line:90% 01:04:56.000 --> 01:04:56.980 align:middle line:90% Any questions? 01:04:56.980 --> 01:05:02.080 align:middle line:90% 01:05:02.080 --> 01:05:06.785 align:middle line:84% OK, let's do a diagnostic question. 01:05:06.785 --> 01:05:08.160 align:middle line:84% This is definitely the way I like 01:05:08.160 --> 01:05:10.480 align:middle line:90% to think about these things. 01:05:10.480 --> 01:05:12.070 align:middle line:90% I just proved something for you. 01:05:12.070 --> 01:05:14.648 align:middle line:90% 01:05:14.648 --> 01:05:17.520 align:middle line:84% Learning about proof-based math, especially learning 01:05:17.520 --> 01:05:19.880 align:middle line:84% about how to write it, isn't just 01:05:19.880 --> 01:05:23.480 align:middle line:84% about verifying line by line what I do on the board. 01:05:23.480 --> 01:05:25.720 align:middle line:84% It's about trying to understand why 01:05:25.720 --> 01:05:28.720 align:middle line:84% I organized my proof that way, and also, 01:05:28.720 --> 01:05:30.950 align:middle line:84% where I use the different assumptions. 01:05:30.950 --> 01:05:33.260 align:middle line:84% So the critical thing here for this lemma, 01:05:33.260 --> 01:05:36.000 align:middle line:84% which is different than linearity of expectation-- 01:05:36.000 --> 01:05:39.920 align:middle line:84% see, linearity of expectation, I used nothing. 01:05:39.920 --> 01:05:41.497 align:middle line:90% For this lemma, I use something. 01:05:41.497 --> 01:05:43.080 align:middle line:84% I use something very strong, which was 01:05:43.080 --> 01:05:44.787 align:middle line:90% that f and g are independent. 01:05:44.787 --> 01:05:47.120 align:middle line:84% And we talked about how it's a bit of a pain in the neck 01:05:47.120 --> 01:05:49.880 align:middle line:84% to actually verify independence, because you have to check it 01:05:49.880 --> 01:05:52.880 align:middle line:90% over all alpha and beta. 01:05:52.880 --> 01:05:58.450 align:middle line:84% Did I really need that property of independence? 01:05:58.450 --> 01:06:02.610 align:middle line:84% So obviously, your intuition is that I did use it crucially. 01:06:02.610 --> 01:06:11.610 align:middle line:84% So can you think of an example where 01:06:11.610 --> 01:06:14.850 align:middle line:84% we have two random variables, f and g, 01:06:14.850 --> 01:06:20.490 align:middle line:84% but f and g are not independent, and where this lemma fails, 01:06:20.490 --> 01:06:23.530 align:middle line:84% where the expectation of the product 01:06:23.530 --> 01:06:29.062 align:middle line:84% is not equal to the product of their expectations? 01:06:29.062 --> 01:06:34.570 align:middle line:90% 01:06:34.570 --> 01:06:38.810 align:middle line:84% So if you're able to push back all of the symbols 01:06:38.810 --> 01:06:40.350 align:middle line:84% and think about this intuitively, 01:06:40.350 --> 01:06:43.310 align:middle line:90% this would be very easy. 01:06:43.310 --> 01:06:44.560 align:middle line:90% But it's not always that easy. 01:06:44.560 --> 01:06:47.850 align:middle line:90% 01:06:47.850 --> 01:06:49.970 align:middle line:84% Is anyone bold enough to think of an example 01:06:49.970 --> 01:06:54.820 align:middle line:84% where I have two f and g's, two random variables that violate 01:06:54.820 --> 01:06:56.040 align:middle line:90% my assumptions of the lemma? 01:06:56.040 --> 01:06:57.740 align:middle line:90% They are not independent. 01:06:57.740 --> 01:07:01.460 align:middle line:84% And the conclusion of my lemma is false, too? 01:07:01.460 --> 01:07:02.140 align:middle line:90% Yeah. 01:07:02.140 --> 01:07:05.060 align:middle line:84% AUDIENCE: Maybe some indicator variables of just [INAUDIBLE]. 01:07:05.060 --> 01:07:07.300 align:middle line:90% PROFESSOR: Yes, perfect. 01:07:07.300 --> 01:07:10.380 align:middle line:84% Let me raise you one, which is, let's just 01:07:10.380 --> 01:07:14.740 align:middle line:84% define f to be the indicator function of some event A, 01:07:14.740 --> 01:07:19.620 align:middle line:84% and let's let G be the indicator function of the complement of A. 01:07:19.620 --> 01:07:21.300 align:middle line:90% That would do the trick, right? 01:07:21.300 --> 01:07:23.900 align:middle line:84% So maybe these both have some nonzero probability 01:07:23.900 --> 01:07:24.940 align:middle line:90% of happening. 01:07:24.940 --> 01:07:27.540 align:middle line:84% The product of their probabilities is nonzero. 01:07:27.540 --> 01:07:30.100 align:middle line:90% But what's the product of them? 01:07:30.100 --> 01:07:31.700 align:middle line:84% Well, that's the indicator function 01:07:31.700 --> 01:07:33.080 align:middle line:90% of both events happening. 01:07:33.080 --> 01:07:36.040 align:middle line:84% And no event and its complement can both happen. 01:07:36.040 --> 01:07:37.060 align:middle line:90% So we're done. 01:07:37.060 --> 01:07:38.360 align:middle line:90% So these are not independent. 01:07:38.360 --> 01:07:40.140 align:middle line:90% And the statement is false. 01:07:40.140 --> 01:07:43.120 align:middle line:84% So especially as we go to more and more general abstraction, 01:07:43.120 --> 01:07:45.580 align:middle line:84% it's always good to keep in some mental model that helps 01:07:45.580 --> 01:07:49.220 align:middle line:90% you navigate what's going on. 01:07:49.220 --> 01:07:51.880 align:middle line:84% So we've only got a few minutes left, which is perfect 01:07:51.880 --> 01:07:56.800 align:middle line:84% because I want to introduce you to one last concept which 01:07:56.800 --> 01:07:58.160 align:middle line:90% is also powerful. 01:07:58.160 --> 01:08:00.280 align:middle line:84% This is the thing I've been promising you, 01:08:00.280 --> 01:08:02.340 align:middle line:90% which is the notion of variance. 01:08:02.340 --> 01:08:04.760 align:middle line:84% So that's our last topic for today. 01:08:04.760 --> 01:08:07.600 align:middle line:84% And I'll tell you one basic but very powerful 01:08:07.600 --> 01:08:11.120 align:middle line:84% property of variance that we're going 01:08:11.120 --> 01:08:15.940 align:middle line:84% to see at length in the sequel, when we get back to probability, 01:08:15.940 --> 01:08:17.800 align:middle line:90% when we talk about tail bounds. 01:08:17.800 --> 01:08:20.319 align:middle line:84% And we can already see some of the intuition. 01:08:20.319 --> 01:08:23.600 align:middle line:84% So first, let me just define what the variance is. 01:08:23.600 --> 01:08:25.880 align:middle line:84% It's another property of a random variable. 01:08:25.880 --> 01:08:31.200 align:middle line:84% So the variance of a random variable 01:08:31.200 --> 01:08:33.160 align:middle line:90% is the following quantity. 01:08:33.160 --> 01:08:35.520 align:middle line:90% We write it as var of f. 01:08:35.520 --> 01:08:43.439 align:middle line:84% It's the expectation of f of x minus the expectation of f 01:08:43.439 --> 01:08:47.080 align:middle line:90% of x squared. 01:08:47.080 --> 01:08:49.810 align:middle line:84% All right, so a bunch of things are happening here. 01:08:49.810 --> 01:08:52.670 align:middle line:84% So this, whatever it is, it's a scalar. 01:08:52.670 --> 01:08:54.770 align:middle line:84% And it's whatever the average value 01:08:54.770 --> 01:08:56.529 align:middle line:90% is for the random variable. 01:08:56.529 --> 01:08:58.410 align:middle line:84% I'm creating a new random variable that 01:08:58.410 --> 01:09:01.090 align:middle line:90% shifts the average down to 0. 01:09:01.090 --> 01:09:04.130 align:middle line:84% So the average of this new random variable is 0. 01:09:04.130 --> 01:09:05.609 align:middle line:84% And when I square this thing, I'm 01:09:05.609 --> 01:09:08.130 align:middle line:90% measuring how spread out it is. 01:09:08.130 --> 01:09:09.950 align:middle line:84% So the variance, as the name suggests, 01:09:09.950 --> 01:09:13.290 align:middle line:84% is how much you expect the random variable to vary. 01:09:13.290 --> 01:09:17.170 align:middle line:84% It's a very coarse grained way to define how 01:09:17.170 --> 01:09:19.649 align:middle line:90% much a random variable varies. 01:09:19.649 --> 01:09:21.770 align:middle line:84% And we'll develop a lot of finer grained tools 01:09:21.770 --> 01:09:23.853 align:middle line:84% that will give us a better understanding of what's 01:09:23.853 --> 01:09:24.410 align:middle line:90% going on. 01:09:24.410 --> 01:09:27.120 align:middle line:84% But already, we'll be able to use this to great effect. 01:09:27.120 --> 01:09:29.850 align:middle line:90% 01:09:29.850 --> 01:09:36.770 align:middle line:84% So that was my informal claim is that it measures how spread 01:09:36.770 --> 01:09:42.390 align:middle line:90% out A random variable is. 01:09:42.390 --> 01:09:45.410 align:middle line:90% 01:09:45.410 --> 01:09:51.620 align:middle line:84% Let me give you a basic fact, which is easy to show. 01:09:51.620 --> 01:09:54.860 align:middle line:84% So first of all, I claim that the variance of f 01:09:54.860 --> 01:09:56.040 align:middle line:90% is non-negative. 01:09:56.040 --> 01:10:00.740 align:middle line:84% That's true for any random variable. 01:10:00.740 --> 01:10:03.420 align:middle line:84% Can anyone argue why this is true so 01:10:03.420 --> 01:10:06.020 align:middle line:90% that I don't have to prove it? 01:10:06.020 --> 01:10:07.560 align:middle line:84% There should just be a one sentence. 01:10:07.560 --> 01:10:08.060 align:middle line:90% Yeah. 01:10:08.060 --> 01:10:09.660 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:10:09.660 --> 01:10:11.380 align:middle line:84% PROFESSOR: That's right, that's right. 01:10:11.380 --> 01:10:13.260 align:middle line:90% Let's just unpack what this is. 01:10:13.260 --> 01:10:16.300 align:middle line:84% It's the expectation of a random variable. 01:10:16.300 --> 01:10:18.740 align:middle line:84% So what we do is we take a weighted average, according 01:10:18.740 --> 01:10:22.220 align:middle line:84% to the probabilities in the sample space, of the values 01:10:22.220 --> 01:10:24.100 align:middle line:84% that this random variable takes on. 01:10:24.100 --> 01:10:25.740 align:middle line:84% But the random variable I care about 01:10:25.740 --> 01:10:27.340 align:middle line:90% is this thing that's really-- 01:10:27.340 --> 01:10:29.900 align:middle line:84% it's the outcome of squaring the thing. 01:10:29.900 --> 01:10:31.480 align:middle line:90% So it's always non-negative. 01:10:31.480 --> 01:10:33.740 align:middle line:84% The weighted average of a bunch of non-negative things 01:10:33.740 --> 01:10:35.860 align:middle line:90% had better be non-negative. 01:10:35.860 --> 01:10:39.180 align:middle line:90% In fact, let me raise you one. 01:10:39.180 --> 01:10:41.620 align:middle line:84% I claim that not only is that statement true 01:10:41.620 --> 01:10:44.220 align:middle line:84% and your explanation is perfect, I 01:10:44.220 --> 01:10:47.870 align:middle line:84% claim that when the variance of a function is 0, 01:10:47.870 --> 01:10:49.710 align:middle line:84% there's something very strong we can 01:10:49.710 --> 01:10:52.590 align:middle line:90% say about the random variable. 01:10:52.590 --> 01:10:56.270 align:middle line:84% So what happens when the variance of a random variable 01:10:56.270 --> 01:10:56.810 align:middle line:90% is 0? 01:10:56.810 --> 01:10:57.630 align:middle line:90% Yes. 01:10:57.630 --> 01:10:59.550 align:middle line:90% It's constant, that's right. 01:10:59.550 --> 01:11:01.020 align:middle line:90% So f is constant value. 01:11:01.020 --> 01:11:03.670 align:middle line:90% 01:11:03.670 --> 01:11:07.430 align:middle line:84% So it doesn't matter what point you give me, 01:11:07.430 --> 01:11:09.470 align:middle line:84% what outcome in the sample space, 01:11:09.470 --> 01:11:12.810 align:middle line:84% there's always one fixed value and it only takes on that value. 01:11:12.810 --> 01:11:14.750 align:middle line:90% That's exactly right. 01:11:14.750 --> 01:11:18.350 align:middle line:84% Now, there's one other alternative formula 01:11:18.350 --> 01:11:22.270 align:middle line:84% for the variance that's sometimes helpful. 01:11:22.270 --> 01:11:25.370 align:middle line:84% In some cases, one is a better starting point than the other. 01:11:25.370 --> 01:11:26.910 align:middle line:90% They're both equivalent. 01:11:26.910 --> 01:11:29.670 align:middle line:84% But it pays to just keep both in mind and use 01:11:29.670 --> 01:11:31.670 align:middle line:90% whichever one is easier. 01:11:31.670 --> 01:11:33.870 align:middle line:84% So let me give you another alternative expression 01:11:33.870 --> 01:11:34.810 align:middle line:90% for the variance. 01:11:34.810 --> 01:11:37.750 align:middle line:84% And then let me tell you a powerful property 01:11:37.750 --> 01:11:43.800 align:middle line:84% of the variance that kicks in when we have independence. 01:11:43.800 --> 01:11:49.200 align:middle line:84% So an alternative-- completely alternative formula 01:11:49.200 --> 01:11:56.200 align:middle line:84% is that the variance of f is equal to the expectation of f 01:11:56.200 --> 01:12:04.080 align:middle line:84% of x squared minus the expectation of f of x, 01:12:04.080 --> 01:12:08.100 align:middle line:84% but with the square on the outside. 01:12:08.100 --> 01:12:17.760 align:middle line:84% And this happens because, from our first expression, when 01:12:17.760 --> 01:12:22.320 align:middle line:84% we expand out the square, we'll get that the variance of f 01:12:22.320 --> 01:12:27.400 align:middle line:84% is equal to the expectation of f of x quantity squared 01:12:27.400 --> 01:12:33.760 align:middle line:84% minus 2 times the value of the function times the expectation 01:12:33.760 --> 01:12:41.140 align:middle line:84% of the function, and then, plus the expectation of f 01:12:41.140 --> 01:12:44.260 align:middle line:90% of x squared. 01:12:44.260 --> 01:12:47.460 align:middle line:84% And now we can appeal to linearity of expectation. 01:12:47.460 --> 01:12:49.380 align:middle line:90% And life becomes good. 01:12:49.380 --> 01:12:51.540 align:middle line:84% So we have the expectation of f of x 01:12:51.540 --> 01:12:53.660 align:middle line:90% with the square on the inside. 01:12:53.660 --> 01:12:56.120 align:middle line:84% Well, this part right here is not a random variable. 01:12:56.120 --> 01:12:57.300 align:middle line:90% That's a scalar. 01:12:57.300 --> 01:13:00.340 align:middle line:84% So it really just pulls outside of the expectation 01:13:00.340 --> 01:13:01.420 align:middle line:90% in a nice way. 01:13:01.420 --> 01:13:04.300 align:middle line:90% And we get another E of f of x. 01:13:04.300 --> 01:13:06.980 align:middle line:84% So we get minus 2 copies of the expectation 01:13:06.980 --> 01:13:09.740 align:middle line:84% of f of x squared with the square on the outside. 01:13:09.740 --> 01:13:12.180 align:middle line:90% We add one more copy back in. 01:13:12.180 --> 01:13:15.300 align:middle line:84% And in total, we're just subtracting 1 copy 01:13:15.300 --> 01:13:19.700 align:middle line:90% with the square on the outside. 01:13:19.700 --> 01:13:23.740 align:middle line:84% So this just uses the properties of the expectation and linearity 01:13:23.740 --> 01:13:25.060 align:middle line:90% of expectation. 01:13:25.060 --> 01:13:27.880 align:middle line:84% But this gives us our two different definitions. 01:13:27.880 --> 01:13:31.060 align:middle line:90% 01:13:31.060 --> 01:13:35.940 align:middle line:84% And now we can say the last property 01:13:35.940 --> 01:13:37.780 align:middle line:84% that we're going to show for today. 01:13:37.780 --> 01:13:40.410 align:middle line:84% And I'll tell you why this property is so important. 01:13:40.410 --> 01:13:47.990 align:middle line:90% 01:13:47.990 --> 01:13:51.670 align:middle line:84% So I claim that if we have two functions, f and g, 01:13:51.670 --> 01:13:58.692 align:middle line:84% that are independent, then the variance of their sum 01:13:58.692 --> 01:14:01.770 align:middle line:84% is equal to the sum of their variances. 01:14:01.770 --> 01:14:05.670 align:middle line:90% 01:14:05.670 --> 01:14:10.230 align:middle line:84% So this is definitely not true in general. 01:14:10.230 --> 01:14:12.690 align:middle line:84% This looks a bit like linearity of expectation. 01:14:12.690 --> 01:14:16.270 align:middle line:84% But it's really very different, because linearity of expectation 01:14:16.270 --> 01:14:17.990 align:middle line:90% held always. 01:14:17.990 --> 01:14:20.930 align:middle line:84% This only holds when f and g are independent. 01:14:20.930 --> 01:14:24.310 align:middle line:90% 01:14:24.310 --> 01:14:25.890 align:middle line:90% So you can prove this. 01:14:25.890 --> 01:14:27.830 align:middle line:84% Let's just do a quick proof, and then I'll 01:14:27.830 --> 01:14:30.350 align:middle line:84% tell you about some cool applications. 01:14:30.350 --> 01:14:40.280 align:middle line:84% So by definition-- well, the variance of f of x plus g of x-- 01:14:40.280 --> 01:14:43.680 align:middle line:84% well, we can write it out from our first definition. 01:14:43.680 --> 01:14:46.150 align:middle line:90% We add up f of x plus g of x. 01:14:46.150 --> 01:14:51.480 align:middle line:90% 01:14:51.480 --> 01:14:54.258 align:middle line:84% We subtract off their expectation. 01:14:54.258 --> 01:14:55.800 align:middle line:84% And then we're going to regroup terms 01:14:55.800 --> 01:14:57.560 align:middle line:90% and we'll see what happens. 01:14:57.560 --> 01:15:00.480 align:middle line:84% So we subtract off the expectation 01:15:00.480 --> 01:15:05.760 align:middle line:84% of their sum, f of x plus g of x. 01:15:05.760 --> 01:15:09.166 align:middle line:84% And we take the square of this entire thing. 01:15:09.166 --> 01:15:11.880 align:middle line:90% That's literally the definition. 01:15:11.880 --> 01:15:14.400 align:middle line:84% And now what we can do is we can appeal 01:15:14.400 --> 01:15:19.640 align:middle line:90% to linearity of expectation. 01:15:19.640 --> 01:15:24.040 align:middle line:84% And we'll get that this entire thing is equal to-- 01:15:24.040 --> 01:15:26.320 align:middle line:84% well, let's group together terms. 01:15:26.320 --> 01:15:31.200 align:middle line:84% It'll be the expectation of f of x 01:15:31.200 --> 01:15:39.890 align:middle line:84% minus the expectation of f of x plus g 01:15:39.890 --> 01:15:45.730 align:middle line:84% of x minus its expectation of g of x. 01:15:45.730 --> 01:15:48.090 align:middle line:84% And this entire thing is squared. 01:15:48.090 --> 01:15:50.090 align:middle line:84% So all I'm doing is, remember, the square 01:15:50.090 --> 01:15:52.210 align:middle line:84% is happening outside all this mess. 01:15:52.210 --> 01:15:54.270 align:middle line:84% I'm pushing the expectation through. 01:15:54.270 --> 01:15:56.090 align:middle line:84% So this is the expectation of f of x 01:15:56.090 --> 01:15:58.170 align:middle line:90% plus the expectation of g of x. 01:15:58.170 --> 01:16:02.410 align:middle line:84% I'm pulling one term with f and one term with g. 01:16:02.410 --> 01:16:05.308 align:middle line:84% But now what I can do is I can expand the square. 01:16:05.308 --> 01:16:06.850 align:middle line:84% And if I expand the square, I'm going 01:16:06.850 --> 01:16:08.630 align:middle line:90% to get this quantity squared. 01:16:08.630 --> 01:16:11.650 align:middle line:84% That's literally the definition of the variance of f. 01:16:11.650 --> 01:16:13.510 align:middle line:84% I'm going to get this quantity squared. 01:16:13.510 --> 01:16:15.330 align:middle line:90% That's the variance of g. 01:16:15.330 --> 01:16:17.810 align:middle line:84% And I'm going to get cross terms. 01:16:17.810 --> 01:16:20.010 align:middle line:84% But now, the key thing is that cross term 01:16:20.010 --> 01:16:22.890 align:middle line:90% is 0 because of independence. 01:16:22.890 --> 01:16:26.530 align:middle line:84% I have the expectation of this times this. 01:16:26.530 --> 01:16:28.930 align:middle line:84% Because these things are independent, 01:16:28.930 --> 01:16:31.370 align:middle line:84% it's the product of their expectations. 01:16:31.370 --> 01:16:34.140 align:middle line:90% And each individual thing is 0. 01:16:34.140 --> 01:16:36.400 align:middle line:84% So that's really the last part of the argument. 01:16:36.400 --> 01:16:38.686 align:middle line:84% I can write that out a bit more precisely. 01:16:38.686 --> 01:16:41.660 align:middle line:90% 01:16:41.660 --> 01:16:45.180 align:middle line:84% So the key thing is really what happens with the cross term. 01:16:45.180 --> 01:16:47.500 align:middle line:84% So when we get this cross term that 01:16:47.500 --> 01:16:52.740 align:middle line:84% looks like 2 times the expectation of f of x minus E 01:16:52.740 --> 01:17:04.020 align:middle line:84% of f of x times g of x minus the expectation of g of x, then 01:17:04.020 --> 01:17:06.780 align:middle line:84% what's going to happen is by independence, 01:17:06.780 --> 01:17:11.020 align:middle line:84% we can write this out as the product of their expectations. 01:17:11.020 --> 01:17:15.740 align:middle line:90% 01:17:15.740 --> 01:17:18.700 align:middle line:84% And this is all times the same thing, the expectation of g 01:17:18.700 --> 01:17:23.940 align:middle line:90% of x minus its own expectation. 01:17:23.940 --> 01:17:26.020 align:middle line:84% And now we're in good shape, because literally, 01:17:26.020 --> 01:17:29.300 align:middle line:84% by linearity of expectation, this is 0. 01:17:29.300 --> 01:17:31.180 align:middle line:84% It's important to not lose track of what 01:17:31.180 --> 01:17:32.540 align:middle line:90% these different things mean. 01:17:32.540 --> 01:17:35.240 align:middle line:84% This term right here is just the scalar shift. 01:17:35.240 --> 01:17:37.880 align:middle line:84% And what's going on is I'm just shifting 01:17:37.880 --> 01:17:40.458 align:middle line:84% so that the mean of my random variable is 0. 01:17:40.458 --> 01:17:42.000 align:middle line:84% And the fact that they're independent 01:17:42.000 --> 01:17:43.540 align:middle line:84% allows me to write it as a product. 01:17:43.540 --> 01:17:45.673 align:middle line:90% So that's exactly why I get 0. 01:17:45.673 --> 01:17:47.340 align:middle line:84% So we can put together all these pieces, 01:17:47.340 --> 01:17:48.840 align:middle line:84% and we have this beautiful statement 01:17:48.840 --> 01:17:51.960 align:middle line:84% that the sum of the variances for two 01:17:51.960 --> 01:17:54.760 align:middle line:84% independent random variables adds when we add up 01:17:54.760 --> 01:17:56.520 align:middle line:90% the two random variables. 01:17:56.520 --> 01:18:00.040 align:middle line:84% So I'll leave you with a closing example, 01:18:00.040 --> 01:18:04.360 align:middle line:84% which we're going to talk about in much more detail. 01:18:04.360 --> 01:18:06.160 align:middle line:84% There's a super-important example 01:18:06.160 --> 01:18:09.840 align:middle line:84% from this that's becoming very relevant these days, 01:18:09.840 --> 01:18:11.720 align:middle line:90% which is about polling. 01:18:11.720 --> 01:18:15.400 align:middle line:84% So we might have some particular events 01:18:15.400 --> 01:18:17.720 align:middle line:84% that we care about, which are things 01:18:17.720 --> 01:18:21.280 align:middle line:84% like the probability any particular person votes 01:18:21.280 --> 01:18:22.880 align:middle line:90% one way or the other. 01:18:22.880 --> 01:18:26.080 align:middle line:84% And we care about something like what the sum of those events 01:18:26.080 --> 01:18:29.320 align:middle line:84% are going to be because we want to forecast elections. 01:18:29.320 --> 01:18:31.610 align:middle line:84% So it turns out that the way that you can do this, 01:18:31.610 --> 01:18:35.170 align:middle line:84% at least in principle, is if you pick a random person 01:18:35.170 --> 01:18:38.350 align:middle line:84% and ask them who they're going to vote for, well, 01:18:38.350 --> 01:18:42.610 align:middle line:84% that'll give you some estimate of what the underlying baseline 01:18:42.610 --> 01:18:44.050 align:middle line:90% probability is. 01:18:44.050 --> 01:18:47.530 align:middle line:84% And then you can do this not for one person, but for many people. 01:18:47.530 --> 01:18:49.310 align:middle line:84% So when you look at that estimator, 01:18:49.310 --> 01:18:53.170 align:middle line:84% it ends up being the sum of independent random variables. 01:18:53.170 --> 01:18:56.050 align:middle line:84% And the fact that that converges very quickly 01:18:56.050 --> 01:18:59.010 align:middle line:84% to the true average for the random variables, 01:18:59.010 --> 01:19:01.770 align:middle line:84% as we'll see, really comes out of this fact 01:19:01.770 --> 01:19:04.890 align:middle line:84% that the variance adds instead of increasing 01:19:04.890 --> 01:19:06.410 align:middle line:90% by a larger quantity. 01:19:06.410 --> 01:19:09.090 align:middle line:84% So we'll talk a lot more about that in later lectures 01:19:09.090 --> 01:19:11.538 align:middle line:84% when we talk about things called concentration bounds. 01:19:11.538 --> 01:19:13.330 align:middle line:84% But I just want to give you a hint of where 01:19:13.330 --> 01:19:15.010 align:middle line:90% we're going with these tools. 01:19:15.010 --> 01:19:17.810 align:middle line:84% And we'll practice a bunch of explicit computations 01:19:17.810 --> 01:19:19.330 align:middle line:84% of variance to get a handle first 01:19:19.330 --> 01:19:21.370 align:middle line:90% before we go to tail bounds. 01:19:21.370 --> 01:19:23.600 align:middle line:90% So we'll stop there. 01:19:23.600 --> 01:19:29.000 align:middle line:90%