1 00:00:00,810 --> 00:00:03,520 We will now go through two examples of convergence in 2 00:00:03,520 --> 00:00:04,590 probability. 3 00:00:04,590 --> 00:00:07,370 Our first example is quite trivial. 4 00:00:07,370 --> 00:00:11,090 We're dealing with a sequence of random variables Yn that 5 00:00:11,090 --> 00:00:12,220 are discrete. 6 00:00:12,220 --> 00:00:15,680 Most of the probability is concentrated at 0. 7 00:00:15,680 --> 00:00:19,690 But there is also a small probability of a large value. 8 00:00:19,690 --> 00:00:22,040 Because the bulk of the probability mass is 9 00:00:22,040 --> 00:00:25,540 concentrated at 0, it is a good guess that this sequence 10 00:00:25,540 --> 00:00:27,070 converges to 0. 11 00:00:27,070 --> 00:00:29,380 Do we have, indeed, convergence in 12 00:00:29,380 --> 00:00:30,760 probability to 0? 13 00:00:30,760 --> 00:00:32,710 We need to check the definition. 14 00:00:32,710 --> 00:00:36,930 So we fix some epsilon, which is a positive number. 15 00:00:36,930 --> 00:00:40,550 And we look at the probability of the event that our random 16 00:00:40,550 --> 00:00:46,360 variable is epsilon or more away than what we think is the 17 00:00:46,360 --> 00:00:48,460 limit of that sequence. 18 00:00:48,460 --> 00:00:50,370 We look at that probability. 19 00:00:50,370 --> 00:00:54,590 And in this example, it is equal to 1 over n, which goes 20 00:00:54,590 --> 00:00:57,820 to 0 as n goes to infinity. 21 00:00:57,820 --> 00:01:01,990 And this verifies that, indeed, in this example, Yn 22 00:01:01,990 --> 00:01:07,065 converges to 0, as n goes to infinity in probability. 23 00:01:10,330 --> 00:01:12,900 Now, we make the following observation. 24 00:01:12,900 --> 00:01:16,220 If we are to calculate the expected value of this random 25 00:01:16,220 --> 00:01:19,390 variable, what we get is the following. 26 00:01:19,390 --> 00:01:23,060 We get a value of 0 with this probability, no contribution 27 00:01:23,060 --> 00:01:24,280 to the expectation. 28 00:01:24,280 --> 00:01:27,970 But we also get a value of n squared with 29 00:01:27,970 --> 00:01:30,490 probability 1 over n. 30 00:01:30,490 --> 00:01:34,630 And so the expected value is equal to n, which, actually, 31 00:01:34,630 --> 00:01:38,440 goes to infinity, as n goes to infinity. 32 00:01:38,440 --> 00:01:43,120 So we have a situation where the sequence of the random 33 00:01:43,120 --> 00:01:45,220 variables converges to 0. 34 00:01:45,220 --> 00:01:48,090 But the expectation does not converge to 0. 35 00:01:48,090 --> 00:01:50,120 In fact, it goes to infinity. 36 00:01:50,120 --> 00:01:52,920 And this example serves to make the point that 37 00:01:52,920 --> 00:01:56,509 convergence in probability does not imply convergence of 38 00:01:56,509 --> 00:01:57,890 expectations. 39 00:01:57,890 --> 00:02:01,940 The reason is that convergence in probability has to do with 40 00:02:01,940 --> 00:02:04,360 the bulk of the distribution. 41 00:02:04,360 --> 00:02:08,259 It only cares that the tail of the distribution has small 42 00:02:08,259 --> 00:02:09,340 probability. 43 00:02:09,340 --> 00:02:11,980 On the other hand, the expectation is highly 44 00:02:11,980 --> 00:02:15,410 sensitive to the tail of the distribution. 45 00:02:15,410 --> 00:02:19,150 It might be that the tail only has a small probability. 46 00:02:19,150 --> 00:02:21,900 But if that probability is assigned to a very large 47 00:02:21,900 --> 00:02:26,140 value, then the expectation will be strongly affected and 48 00:02:26,140 --> 00:02:29,020 can be quite different from the limit 49 00:02:29,020 --> 00:02:32,270 of the random variable. 50 00:02:32,270 --> 00:02:35,570 Our second example is going to be less trivial and more 51 00:02:35,570 --> 00:02:36,860 interesting. 52 00:02:36,860 --> 00:02:40,290 Consider random variables that are independent and 53 00:02:40,290 --> 00:02:44,050 identically distributed and whose common distribution is 54 00:02:44,050 --> 00:02:47,350 uniform on the unit interval, so that the 55 00:02:47,350 --> 00:02:50,160 PDF takes this form. 56 00:02:50,160 --> 00:02:55,150 Are these random variables convergent to something? 57 00:02:55,150 --> 00:02:56,480 The answer is no. 58 00:02:56,480 --> 00:03:00,980 And the reason is that as i increases, the distribution 59 00:03:00,980 --> 00:03:02,020 does not change. 60 00:03:02,020 --> 00:03:04,380 And it does not to get concentrated 61 00:03:04,380 --> 00:03:06,010 around a certain number. 62 00:03:06,010 --> 00:03:09,130 The distribution remains spread out over the entire 63 00:03:09,130 --> 00:03:10,960 unit interval. 64 00:03:10,960 --> 00:03:14,330 But let us look now at some related random variables. 65 00:03:14,330 --> 00:03:19,940 Let us define Yn to be the minimum of the first n of the 66 00:03:19,940 --> 00:03:21,530 X's that we get. 67 00:03:21,530 --> 00:03:25,540 So if n is equal to 4, and we obtain these four values, Yn 68 00:03:25,540 --> 00:03:28,100 would be equal to this value. 69 00:03:28,100 --> 00:03:33,560 Notice that if we draw more values, then the new values 70 00:03:33,560 --> 00:03:36,810 might be above the minimum, in which case the minimum does 71 00:03:36,810 --> 00:03:37,890 not change. 72 00:03:37,890 --> 00:03:41,610 But we might also get a value that's below the minimum, in 73 00:03:41,610 --> 00:03:44,100 which case the minimum moves down. 74 00:03:44,100 --> 00:03:48,400 The only thing that can happen is that the minimum goes down. 75 00:03:48,400 --> 00:03:49,960 It cannot go up. 76 00:03:49,960 --> 00:03:52,680 And this gives us this inequality. 77 00:03:52,680 --> 00:03:56,420 So the random variables Yn tends to go down. 78 00:03:56,420 --> 00:03:58,720 How far down? 79 00:03:58,720 --> 00:04:03,640 If n is very large, we expect that we will obtain some X's 80 00:04:03,640 --> 00:04:07,570 whose value happens to be very close to 0, which means that 81 00:04:07,570 --> 00:04:12,580 Yn will go down to values that are very close to 0. 82 00:04:12,580 --> 00:04:16,010 And this leads us to conjecture that, perhaps, Yn 83 00:04:16,010 --> 00:04:18,390 does converge to 0. 84 00:04:18,390 --> 00:04:21,209 This is always the first step when dealing with convergence 85 00:04:21,209 --> 00:04:22,029 in probability. 86 00:04:22,029 --> 00:04:25,940 The first step is to make an educated guess about what the 87 00:04:25,940 --> 00:04:27,580 limit might be. 88 00:04:27,580 --> 00:04:30,610 And then we want to verify that this is, indeed, the 89 00:04:30,610 --> 00:04:32,150 correct limit. 90 00:04:32,150 --> 00:04:37,380 To verify that, what we do is we fix some positive epsilon. 91 00:04:37,380 --> 00:04:41,640 And we look for the probability that the distance 92 00:04:41,640 --> 00:04:46,090 of the random variable Yn from the conjectured limit has a 93 00:04:46,090 --> 00:04:49,610 magnitude that's larger than or equal to epsilon. 94 00:04:49,610 --> 00:04:52,630 And what we need to show is that this quantity converges 95 00:04:52,630 --> 00:04:58,320 to 0 as n goes to infinity, no matter what epsilon is. 96 00:04:58,320 --> 00:05:01,940 Now, because Yn is a non-negative random variable, 97 00:05:01,940 --> 00:05:05,270 this is the same as the probability that Yn is larger 98 00:05:05,270 --> 00:05:07,330 than or equal to epsilon. 99 00:05:07,330 --> 00:05:10,210 Now, let us distinguish between two cases. 100 00:05:10,210 --> 00:05:13,870 If epsilon is bigger than 1, we're asking for the 101 00:05:13,870 --> 00:05:18,190 probability that Yn is larger than or equal to a certain 102 00:05:18,190 --> 00:05:20,460 number epsilon that's out there. 103 00:05:20,460 --> 00:05:22,610 But this probability is 0. 104 00:05:22,610 --> 00:05:25,690 There's no way that the minimum of these uniforms will 105 00:05:25,690 --> 00:05:29,000 take a value that's larger than some epsilon that's 106 00:05:29,000 --> 00:05:30,660 larger than 1. 107 00:05:30,660 --> 00:05:34,150 So in that case, this quantity is equal to zero. 108 00:05:34,150 --> 00:05:35,380 And we are done. 109 00:05:35,380 --> 00:05:38,530 But we need to check that this quantity becomes small no 110 00:05:38,530 --> 00:05:40,290 matter what epsilon is. 111 00:05:40,290 --> 00:05:44,360 So now, let us consider taking a small epsilon that is some 112 00:05:44,360 --> 00:05:47,260 number that's less than or equal to 1. 113 00:05:47,260 --> 00:05:51,180 For that case, let us continue with the calculation. 114 00:05:51,180 --> 00:05:56,300 The minimum is going to be at least epsilon, if, and only 115 00:05:56,300 --> 00:05:59,490 if, all of the random variables 116 00:05:59,490 --> 00:06:01,160 are at least epsilon. 117 00:06:04,210 --> 00:06:07,950 So this is an equivalent way of writing this particular 118 00:06:07,950 --> 00:06:09,300 event here. 119 00:06:09,300 --> 00:06:12,470 Now, because of independence, this is the product of the 120 00:06:12,470 --> 00:06:17,080 probabilities that each one of the random variables is larger 121 00:06:17,080 --> 00:06:20,370 than or equal to epsilon. 122 00:06:20,370 --> 00:06:23,470 The probability that X1 is larger than or equal to 123 00:06:23,470 --> 00:06:26,150 epsilon can be found as follows. 124 00:06:26,150 --> 00:06:29,420 If we have here epsilon, the probability of being larger 125 00:06:29,420 --> 00:06:31,540 than or equal to epsilon is the probability 126 00:06:31,540 --> 00:06:32,990 of this event here. 127 00:06:32,990 --> 00:06:35,020 So it's the area of this rectangle. 128 00:06:35,020 --> 00:06:38,060 The base of that rectangle is 1 minus epsilon. 129 00:06:38,060 --> 00:06:42,409 And so we obtain 1 minus epsilon for this first term. 130 00:06:42,409 --> 00:06:45,180 But because the Xi's are identically distributed, all 131 00:06:45,180 --> 00:06:48,280 the other terms that we multiply are also the same. 132 00:06:48,280 --> 00:06:52,940 And so the answer is this expression here. 133 00:06:52,940 --> 00:06:55,890 Now, epsilon is a positive number. 134 00:06:55,890 --> 00:06:58,920 So 1 minus epsilon is strictly less than 1. 135 00:06:58,920 --> 00:07:03,010 And when we take higher powers of a number that's less than 136 00:07:03,010 --> 00:07:07,500 1, we obtain something that converges to 0 137 00:07:07,500 --> 00:07:09,030 as n goes to infinity. 138 00:07:09,030 --> 00:07:11,690 And that's what we needed to verify. 139 00:07:11,690 --> 00:07:16,090 Since this is the case for any epsilon, we conclude that the 140 00:07:16,090 --> 00:07:22,370 random variables Yn converge to zero in the sense that we 141 00:07:22,370 --> 00:07:24,135 have defined, in probability. 142 00:07:27,270 --> 00:07:31,170 Generalizing from this example, when we want to show 143 00:07:31,170 --> 00:07:34,850 convergence in probability, the first step is to make a 144 00:07:34,850 --> 00:07:38,540 guess as to what is the value that the 145 00:07:38,540 --> 00:07:40,530 sequence converges to. 146 00:07:40,530 --> 00:07:44,260 In this example, that value was equal to 0. 147 00:07:44,260 --> 00:07:48,240 Once we have made that conjecture, then we write down 148 00:07:48,240 --> 00:07:51,540 an expression for the probability of being epsilon 149 00:07:51,540 --> 00:07:54,690 away from the conjectured limit. 150 00:07:54,690 --> 00:07:58,280 And then we calculate that probability either exactly, as 151 00:07:58,280 --> 00:07:59,475 in this example. 152 00:07:59,475 --> 00:08:02,500 Or we try to bound it somehow and still show 153 00:08:02,500 --> 00:08:03,750 that it goes to 0.