1
00:00:00,530 --> 00:00:02,960
The following content is
provided under a Creative
2
00:00:02,960 --> 00:00:04,370
Commons license.
3
00:00:04,370 --> 00:00:07,410
Your support will help MIT
OpenCourseWare continue to
4
00:00:07,410 --> 00:00:11,060
offer high quality educational
resources for free.
5
00:00:11,060 --> 00:00:13,960
To make a donation or view
additional materials from
6
00:00:13,960 --> 00:00:19,790
hundreds of MIT courses, visit
MIT OpenCourseWare at
7
00:00:19,790 --> 00:00:21,040
ocw.mit.edu.
8
00:00:22,600 --> 00:00:27,030
PROFESSOR: OK, today we are
moving on with renewal
9
00:00:27,030 --> 00:00:39,150
processes, and we're going to
finish up talking a little
10
00:00:39,150 --> 00:00:44,080
more about residual life, and
we're going to generalize it
11
00:00:44,080 --> 00:00:49,440
to time averages for arbitrary
renewal processes.
12
00:00:49,440 --> 00:00:53,590
And then finally we're going
into the topic of stopping
13
00:00:53,590 --> 00:00:56,660
trials, stopping times, optional
stopping times--
14
00:00:56,660 --> 00:00:59,290
people call them different
things--
15
00:00:59,290 --> 00:01:00,540
and Wald's equality.
16
00:01:03,660 --> 00:01:07,140
This is, for some reason or
other, normally a very tricky
17
00:01:07,140 --> 00:01:09,840
thing to understand.
18
00:01:09,840 --> 00:01:15,140
I think finally after many
years I understand it.
19
00:01:15,140 --> 00:01:18,700
And I hope that I can make it
easy for you to understand it,
20
00:01:18,700 --> 00:01:23,940
so some of the confusions that
often come in won't come in.
21
00:01:23,940 --> 00:01:27,120
And then we're going to end up
talking about stopping when
22
00:01:27,120 --> 00:01:33,590
you're ahead, which is a
gambling strategy which says
23
00:01:33,590 --> 00:01:37,940
if you're playing a fair game
you play until you're $1.00
24
00:01:37,940 --> 00:01:40,440
ahead, and then you stop.
25
00:01:40,440 --> 00:01:42,670
And you show that with
probability 1 you will
26
00:01:42,670 --> 00:01:46,700
eventually become $1.00 ahead,
so you have a way to beat the
27
00:01:46,700 --> 00:01:50,100
bank, or beat the casino.
28
00:01:50,100 --> 00:01:53,500
But of course casinos don't give
you even odds, but you
29
00:01:53,500 --> 00:01:58,550
have a way to win any time
you get even odds.
30
00:01:58,550 --> 00:02:00,510
We'll find that there's
something wrong with that, but
31
00:02:00,510 --> 00:02:06,600
it is an interesting topic
which illustrates Wald's
32
00:02:06,600 --> 00:02:09,729
equality in an interesting
way.
33
00:02:09,729 --> 00:02:12,930
So let's start out by reviewing
a little bit.
34
00:02:17,270 --> 00:02:21,630
We first talked last time
about convergence with
35
00:02:21,630 --> 00:02:24,880
probability 1, and we weren't
talking about the strong law
36
00:02:24,880 --> 00:02:28,500
then, we were just talking about
a sequence of random
37
00:02:28,500 --> 00:02:32,760
variables and what it means
for them to converge with
38
00:02:32,760 --> 00:02:34,480
probability 1.
39
00:02:34,480 --> 00:02:40,440
And the theorem is that if you
have a sequence of random
40
00:02:40,440 --> 00:02:46,260
variables, z sub n, they
converge to some number alpha.
41
00:02:46,260 --> 00:02:48,540
This is slightly different than
the theorem we stated
42
00:02:48,540 --> 00:02:53,640
last time, but it's trivially
the same.
43
00:02:53,640 --> 00:02:56,810
In other words, it's the
probability of the set of
44
00:02:56,810 --> 00:02:58,000
sample points.
45
00:02:58,000 --> 00:03:01,760
A sample point, now, is
something which runs from time
46
00:03:01,760 --> 00:03:03,800
0 to time infinity.
47
00:03:03,800 --> 00:03:07,670
It's something that covers the
whole process, covers anything
48
00:03:07,670 --> 00:03:09,670
else you might want to
talk about also.
49
00:03:09,670 --> 00:03:15,460
It's what you do when you go
from a real situation to a
50
00:03:15,460 --> 00:03:19,610
model, and mathematically when
you go to a model you put in
51
00:03:19,610 --> 00:03:20,670
everything you're going to be
52
00:03:20,670 --> 00:03:23,270
interested in at the beginning.
53
00:03:23,270 --> 00:03:30,260
So, it converges to some alpha
probability 1 if the set of
54
00:03:30,260 --> 00:03:34,520
sequences for which the
statement holds has
55
00:03:34,520 --> 00:03:40,350
probability 1, and this
statement is that the sample
56
00:03:40,350 --> 00:03:43,050
paths converge to alpha.
57
00:03:43,050 --> 00:03:46,300
The set of sample paths has
converged to alpha as
58
00:03:46,300 --> 00:03:47,280
probability 1.
59
00:03:47,280 --> 00:03:50,640
So, it's a statement not like
most of the statements in
60
00:03:50,640 --> 00:03:54,200
probability where you start
out talking about finite
61
00:03:54,200 --> 00:03:56,750
length, talking about
probabilities, and
62
00:03:56,750 --> 00:03:59,110
expectation, and so forth.
63
00:03:59,110 --> 00:04:02,260
And then you go to the limit
with these things you're used
64
00:04:02,260 --> 00:04:05,220
to talking about, which are
easier to deal with.
65
00:04:05,220 --> 00:04:09,060
Here it's a statement
about sample paths
66
00:04:09,060 --> 00:04:11,280
over the whole sequence.
67
00:04:11,280 --> 00:04:14,580
You break them into
two categories.
68
00:04:14,580 --> 00:04:17,480
Implicitly we have shown that
when you break them into two
69
00:04:17,480 --> 00:04:22,220
categories those two categories
in fact have
70
00:04:22,220 --> 00:04:23,060
probabilities.
71
00:04:23,060 --> 00:04:28,540
They are events, and one of
them has probability 1.
72
00:04:28,540 --> 00:04:34,750
For renewal processes, if we
have a renewal process we can
73
00:04:34,750 --> 00:04:40,030
use this theorem to talk about
the sample averages.
74
00:04:40,030 --> 00:04:45,970
In other words, in a renewal
process we have a sequence of
75
00:04:45,970 --> 00:04:51,800
IID random variables, x1,
x2, x3, and so forth.
76
00:04:51,800 --> 00:04:56,410
We have the sample sums, the sum
of the time 1, the sum of
77
00:04:56,410 --> 00:05:01,920
the time 2, sum of the
time 3, and so forth.
78
00:05:01,920 --> 00:05:06,330
And the strong law of large
numbers, which comes pretty
79
00:05:06,330 --> 00:05:10,540
much directly out of this
convergence of probability 1
80
00:05:10,540 --> 00:05:16,190
theorem, says that the
probability is the set of
81
00:05:16,190 --> 00:05:24,530
sample points for which the
limit of the sample average,
82
00:05:24,530 --> 00:05:26,990
namely sn of omega over n.
83
00:05:26,990 --> 00:05:31,500
You take the sample average
over all lengths for one
84
00:05:31,500 --> 00:05:32,850
sample point.
85
00:05:32,850 --> 00:05:35,380
That goes to a limit, that's
this convergence with
86
00:05:35,380 --> 00:05:40,180
probability 1, the probability
that you get convergence is
87
00:05:40,180 --> 00:05:45,740
equal to 1, which is what with
probability 1 means.
88
00:05:56,710 --> 00:05:58,480
Then we use the theorem
on the top.
89
00:06:03,700 --> 00:06:05,360
Let's reset a little bit.
90
00:06:05,360 --> 00:06:08,690
This theorem on the top is more
than just a theorem about
91
00:06:08,690 --> 00:06:12,280
convergence with
probability 1.
92
00:06:12,280 --> 00:06:15,400
It's the reason why convergence
with probability 1
93
00:06:15,400 --> 00:06:17,110
is so useful.
94
00:06:17,110 --> 00:06:22,400
It says if you have some
function f of x, that function
95
00:06:22,400 --> 00:06:27,950
is continuous at this
point alpha.
96
00:06:27,950 --> 00:06:32,020
You have this limit here, then
the probability of the set of
97
00:06:32,020 --> 00:06:36,440
omega for which the function
applied to each of these
98
00:06:36,440 --> 00:06:44,318
sample points as equal
to alpha is also one.
99
00:06:44,318 --> 00:06:46,640
AUDIENCE: [INAUDIBLE].
100
00:06:46,640 --> 00:06:48,230
PROFESSOR: It should
be f of alpha.
101
00:06:58,680 --> 00:06:59,940
Yes, absolutely.
102
00:06:59,940 --> 00:07:01,190
Sorry about that.
103
00:07:11,110 --> 00:07:14,340
This is f of alpha
right there.
104
00:07:14,340 --> 00:07:16,610
So that probability
is equal to 1.
105
00:07:19,540 --> 00:07:22,130
I'm finally getting my bearings
back on this.
106
00:07:25,910 --> 00:07:31,550
So if I use this theorem here,
where I use f of x as equal to
107
00:07:31,550 --> 00:07:37,810
1 over x, then what it says for
a renewal process, which
108
00:07:37,810 --> 00:07:44,160
has inter-renewals x1, x2, and
so forth, with a inter-renewal
109
00:07:44,160 --> 00:07:49,920
process the expected
inter-renewal time, which
110
00:07:49,920 --> 00:07:52,830
might be infinite in general,
but it has to
111
00:07:52,830 --> 00:07:54,730
be bigger than 0.
112
00:07:54,730 --> 00:08:02,960
It has to be bigger than 0
because renewals in 0 time are
113
00:08:02,960 --> 00:08:05,890
not possible by definition.
114
00:08:05,890 --> 00:08:10,670
And you've done a problem in the
homework, or you will do a
115
00:08:10,670 --> 00:08:13,930
problem in the homework, whether
you've done it or not,
116
00:08:13,930 --> 00:08:19,042
I don't know, but when you do
it, it will in fact show that
117
00:08:19,042 --> 00:08:23,390
the expected value of x has
to be greater than 0.
118
00:08:23,390 --> 00:08:26,930
We're also assuming that it's
less than infinity.
119
00:08:26,930 --> 00:08:35,039
So, at that point we can use
this theorem which says that
120
00:08:35,039 --> 00:08:41,260
when you look at, not sn of
omega over n, but the inverse
121
00:08:41,260 --> 00:08:45,850
of that, n over sn of omega, the
limit of that is going to
122
00:08:45,850 --> 00:08:52,730
be equal to f of x-bar which
is 1 over x-bar.
123
00:08:52,730 --> 00:08:56,220
So we have this limit here
which is equal to 1 with
124
00:08:56,220 --> 00:08:58,230
probability 1.
125
00:08:58,230 --> 00:09:03,610
This is almost the strong law
for renewal processes, and the
126
00:09:03,610 --> 00:09:10,560
reason why it's not takes a
couple of pages in the text.
127
00:09:10,560 --> 00:09:15,040
What I'm going to try to argue
here is that the couple of
128
00:09:15,040 --> 00:09:24,880
pages, while you need to do
mathematically, you can see
129
00:09:24,880 --> 00:09:27,240
why when you do it you
can see what answer
130
00:09:27,240 --> 00:09:28,440
you're going to get.
131
00:09:28,440 --> 00:09:30,290
And the argument is
the following.
132
00:09:30,290 --> 00:09:32,910
We talked about this last
time a little bit.
133
00:09:32,910 --> 00:09:38,420
If you take an arbitrary time t
then you look at n of t for
134
00:09:38,420 --> 00:09:42,860
that value of t, that's the
number of arrivals you've had
135
00:09:42,860 --> 00:09:46,370
in this renewal process
up until time t.
136
00:09:46,370 --> 00:09:50,460
And then you compare this point
down here, which is the
137
00:09:50,460 --> 00:09:55,280
time of the last arrival before
time t, and this point
138
00:09:55,280 --> 00:10:00,320
here, which is the time of the
next arrival after time t, and
139
00:10:00,320 --> 00:10:04,390
you look at the slopes
here, this slope here
140
00:10:04,390 --> 00:10:07,090
is n of t over t.
141
00:10:07,090 --> 00:10:08,520
That's the thing we're
interested in.
142
00:10:15,380 --> 00:10:18,520
That expression there is the
time average of the number of
143
00:10:18,520 --> 00:10:23,830
renewals within time t, and what
we're interested in is
144
00:10:23,830 --> 00:10:27,210
what this becomes as
t goes to infinity.
145
00:10:27,210 --> 00:10:31,920
That's squeezed between this
slope here, which is n of t
146
00:10:31,920 --> 00:10:35,240
over s sub n of t.
147
00:10:35,240 --> 00:10:38,760
This is s sub n of t.
148
00:10:38,760 --> 00:10:45,420
And this point here is n of t,
and this lower band here,
149
00:10:45,420 --> 00:10:50,680
which is n of t divided by
s sub n of t plus 1.
150
00:10:50,680 --> 00:10:57,480
Now, all this argument is
devoted to proving two things,
151
00:10:57,480 --> 00:11:01,040
which are almost obvious and
therefore I'm just going to
152
00:11:01,040 --> 00:11:03,350
wave my hands about it.
153
00:11:03,350 --> 00:11:08,450
What this says is when t gets
very, very large, n of t
154
00:11:08,450 --> 00:11:10,140
becomes very, very large also.
155
00:11:12,770 --> 00:11:16,830
When n of t becomes very, very
large, the difference between
156
00:11:16,830 --> 00:11:20,250
this slope and this slope
doesn't make a hill of beans
157
00:11:20,250 --> 00:11:23,670
worth of difference, and
therefore those two, that
158
00:11:23,670 --> 00:11:27,320
upper band and that lower
band become the same.
159
00:11:27,320 --> 00:11:33,180
So with the argument as t gets
large, n of t gets large, and
160
00:11:33,180 --> 00:11:36,450
these two slopes become the
same, you suddenly have the
161
00:11:36,450 --> 00:11:44,900
statement that if you have
a renewal process and the
162
00:11:44,900 --> 00:11:48,830
inter-renewal time is less than
infinity, it has to be
163
00:11:48,830 --> 00:11:50,520
greater than 0.
164
00:11:50,520 --> 00:11:57,950
Then on a sample path basis the
sample paths, the number
165
00:11:57,950 --> 00:12:04,960
of arrivals per unit time,
converges to 1 over x-bar.
166
00:12:04,960 --> 00:12:09,370
That's what this previous
picture was saying to you.
167
00:12:09,370 --> 00:12:19,210
It was saying that n over t over
t, which is the number of
168
00:12:19,210 --> 00:12:24,300
arrivals per unit time t, as t
becomes very, very large on a
169
00:12:24,300 --> 00:12:28,170
sample path basis that becomes
equal to 1 over x-bar.
170
00:12:33,700 --> 00:12:38,190
So, it says that the rate of
renewals over the infinite
171
00:12:38,190 --> 00:12:43,330
time horizon is 1 over x-bar
with probability 1.
172
00:12:43,330 --> 00:12:46,530
And what probability 1 statement
again is saying that
173
00:12:46,530 --> 00:12:50,090
the set of sample paths for
which this is true has
174
00:12:50,090 --> 00:12:51,960
probability 1.
175
00:12:51,960 --> 00:12:55,430
This also implies the weak law
for renewals, which says that
176
00:12:55,430 --> 00:12:59,420
the limit as t approaches
infinity of the probability
177
00:12:59,420 --> 00:13:03,970
that n of t over t minus 1 over
x-bar is greater than
178
00:13:03,970 --> 00:13:07,520
epsilon goes to 0.
179
00:13:07,520 --> 00:13:10,180
This, surprisingly enough,
is a statement that
180
00:13:10,180 --> 00:13:12,460
you hardly ever see.
181
00:13:12,460 --> 00:13:16,330
You see a bunch of other renewal
theorems, which this
182
00:13:16,330 --> 00:13:20,180
theorem up here is probably
the most important.
183
00:13:20,180 --> 00:13:23,500
There's something called an
elementary renewal theorem,
184
00:13:23,500 --> 00:13:26,610
which is sort of a trivial
statement but
185
00:13:26,610 --> 00:13:28,860
kind of hard to prove.
186
00:13:28,860 --> 00:13:31,840
And there's something called
Blackwell's theorem, which
187
00:13:31,840 --> 00:13:35,760
talks about a local period of
time after time gets very,
188
00:13:35,760 --> 00:13:36,400
very large.
189
00:13:36,400 --> 00:13:40,640
It's talking about the number
of renewals in some tiny
190
00:13:40,640 --> 00:13:46,700
interval and it's proving things
about stationarity over
191
00:13:46,700 --> 00:13:48,830
that local time interval.
192
00:13:48,830 --> 00:13:53,830
And we'll talk about that later,
but this is sort of a
193
00:13:53,830 --> 00:14:00,000
nice result, and that's
what it is.
194
00:14:00,000 --> 00:14:03,370
Let's review residual
life a little bit.
195
00:14:03,370 --> 00:14:09,870
The residual life of a renewal
process at time t, the
196
00:14:09,870 --> 00:14:14,560
residual life is the time you
have to wait until the next
197
00:14:14,560 --> 00:14:17,480
renewal, until the
next arrival.
198
00:14:17,480 --> 00:14:21,990
So residual life means,
how long does it take?
199
00:14:21,990 --> 00:14:24,710
If you think of these renewals
as something that comes along
200
00:14:24,710 --> 00:14:30,370
and kills everybody, then this
residual life is the amount of
201
00:14:30,370 --> 00:14:33,230
life that people still
have left.
202
00:14:33,230 --> 00:14:37,770
So, if you look at what that
is for a particular sample
203
00:14:37,770 --> 00:14:41,080
path, I have a particular
sample path drawn here,
204
00:14:41,080 --> 00:14:44,080
completely arbitrary.
205
00:14:44,080 --> 00:14:48,490
When I look at the first arrival
time here, s1 of
206
00:14:48,490 --> 00:14:52,410
omega, and I look at all of
the time before that, the
207
00:14:52,410 --> 00:14:57,350
amount of time you have to
wait until this arrival.
208
00:14:57,350 --> 00:15:01,650
Remember this is one
sample point.
209
00:15:04,480 --> 00:15:08,690
We're not thinking in terms of
an observer who comes in and
210
00:15:08,690 --> 00:15:11,410
doesn't know what's going
to happen in the future.
211
00:15:11,410 --> 00:15:12,990
We're looking at one
sample point.
212
00:15:12,990 --> 00:15:14,920
We know the whole
sample point.
213
00:15:14,920 --> 00:15:18,356
And what we're doing we know
this, we know this, we know
214
00:15:18,356 --> 00:15:24,610
this, and all we're doing here
saying for each time t along
215
00:15:24,610 --> 00:15:29,120
this infinite path we're just
plotting how long it is until
216
00:15:29,120 --> 00:15:30,620
the next arrival.
217
00:15:30,620 --> 00:15:34,260
And that's just a line with
slope minus 1 until the next
218
00:15:34,260 --> 00:15:35,660
arrival occurs.
219
00:15:35,660 --> 00:15:38,860
After the next arrival and you
start waiting for the arrival
220
00:15:38,860 --> 00:15:41,640
after that, after that
when you wait for the
221
00:15:41,640 --> 00:15:43,220
arrival after that.
222
00:15:43,220 --> 00:15:49,570
So on a sample path basis, the
residual life is just this
223
00:15:49,570 --> 00:15:55,580
sequence of isosceles
triangles here.
224
00:15:55,580 --> 00:16:01,490
So we looked at that, we said,
if we look at the time from n
225
00:16:01,490 --> 00:16:07,950
equals 1 up until the number
of arrivals at time t, the
226
00:16:07,950 --> 00:16:13,240
number of triangles we
have is n of t omega.
227
00:16:13,240 --> 00:16:20,130
The size of the area of each
triangle is xi squared over 2.
228
00:16:24,830 --> 00:16:28,610
The size of each triangle is xi
squared over 2, and when we
229
00:16:28,610 --> 00:16:31,800
get all done we want to take the
average of it so we divide
230
00:16:31,800 --> 00:16:33,620
each of these by t.
231
00:16:33,620 --> 00:16:37,570
This quantity here is less than
or equal to the integral
232
00:16:37,570 --> 00:16:43,750
of y of t over t, and that's
less than or equal to the same
233
00:16:43,750 --> 00:16:48,700
sum where if we look at a
particular time of t this sum
234
00:16:48,700 --> 00:16:52,500
here is summing everything
up to this point.
235
00:16:52,500 --> 00:16:57,500
This sum here is summing
everything up to this point.
236
00:16:57,500 --> 00:17:01,710
And as t goes to infinity, this
one little triangle in
237
00:17:01,710 --> 00:17:05,410
here, even though this is the
biggest triangle there, it
238
00:17:05,410 --> 00:17:07,010
still doesn't make
any difference.
239
00:17:07,010 --> 00:17:09,790
As t goes to infinity,
that washes out.
240
00:17:09,790 --> 00:17:12,410
You have to show that, of
course, but that's what we
241
00:17:12,410 --> 00:17:13,660
showed last time.
242
00:17:16,079 --> 00:17:22,119
When we go to the limit we find
that this time average
243
00:17:22,119 --> 00:17:26,450
residual life is equal to the
expected value of x squared
244
00:17:26,450 --> 00:17:29,150
over twice the main effect.
245
00:17:29,150 --> 00:17:31,980
Now, this can't be infinite.
246
00:17:31,980 --> 00:17:38,710
You can have a finite expected
in a renewal time and you can
247
00:17:38,710 --> 00:17:42,500
still have an infinite
second moment.
248
00:17:42,500 --> 00:17:49,570
If you look at the example we
talked about last time where
249
00:17:49,570 --> 00:17:55,250
the inter-renewal time at two
possible values, either
250
00:17:55,250 --> 00:17:59,100
epsilon or 1 over epsilon, in
other words, it was either
251
00:17:59,100 --> 00:18:04,280
enormous or it was very, very
small, what we found out there
252
00:18:04,280 --> 00:18:07,090
was that some of these
inter-renewal intervals, a
253
00:18:07,090 --> 00:18:10,540
very small fraction of them, but
a very small fraction of
254
00:18:10,540 --> 00:18:14,570
the inter-renewal intervals
were enormously large.
255
00:18:14,570 --> 00:18:24,150
And because they were enormously
large, the time
256
00:18:24,150 --> 00:18:29,510
average residual life turned
out to be enormous also.
257
00:18:29,510 --> 00:18:32,000
You think of what happens
as epsilon
258
00:18:32,000 --> 00:18:35,100
get smaller and smaller.
259
00:18:35,100 --> 00:18:40,600
You can see intuitively why it
makes sense that when you have
260
00:18:40,600 --> 00:18:46,910
humongously long inter-renewal
times, and you have this x
261
00:18:46,910 --> 00:18:52,240
squared that occurs, because of
this triangle here, I think
262
00:18:52,240 --> 00:18:57,150
is possible to see why this
quantity here can't become
263
00:18:57,150 --> 00:19:03,010
infinite if you had this
situation of a very long
264
00:19:03,010 --> 00:19:06,070
tailed distributions for
the inter-renewal time.
265
00:19:06,070 --> 00:19:12,320
You have enormously long
residual waits then, and you
266
00:19:12,320 --> 00:19:19,240
have them with high probability,
because if you
267
00:19:19,240 --> 00:19:26,060
come into this process at some
arbitrary time you're somewhat
268
00:19:26,060 --> 00:19:29,770
more likely to wind up in one of
these large intervals than
269
00:19:29,770 --> 00:19:33,500
one of the small intervals, and
that's what causes all the
270
00:19:33,500 --> 00:19:34,750
trouble here.
271
00:19:38,250 --> 00:19:40,600
There are similar
examples here.
272
00:19:40,600 --> 00:19:47,360
You can look at the age of a
process, z of t is defined as
273
00:19:47,360 --> 00:19:53,160
the interval between t and
the previous arrival.
274
00:19:53,160 --> 00:19:59,240
So, if we look at sum time t,
the age at that point is the
275
00:19:59,240 --> 00:20:03,090
amount of time back to the
previous arrival that goes up
276
00:20:03,090 --> 00:20:07,760
like a triangle, drops as soon
as we get the next arrival.
277
00:20:07,760 --> 00:20:08,940
It's exactly the same.
278
00:20:08,940 --> 00:20:11,850
It's just the same thing
looking at it backwards
279
00:20:11,850 --> 00:20:17,910
instead of forwards, so the
answer is the same also.
280
00:20:17,910 --> 00:20:22,240
If you look at something called
duration, the question
281
00:20:22,240 --> 00:20:27,760
here is at a particular
time t.
282
00:20:27,760 --> 00:20:31,390
If we take the difference
between the next arrival and
283
00:20:31,390 --> 00:20:36,260
the previous arrival, that's
called the duration of time t.
284
00:20:36,260 --> 00:20:37,650
How big is that?
285
00:20:37,650 --> 00:20:41,590
Well, obviously this is the same
problem as that, and the
286
00:20:41,590 --> 00:20:45,270
residual life and in fact,
that's exactly the sum of the
287
00:20:45,270 --> 00:20:47,760
residual life and the age.
288
00:20:47,760 --> 00:20:51,990
So, it's not surprising that
the time average--
289
00:20:51,990 --> 00:20:54,180
oh my.
290
00:20:54,180 --> 00:20:55,990
There should be a
2 down there.
291
00:21:31,340 --> 00:21:35,200
So it's exactly the same
situation as we had before
292
00:21:35,200 --> 00:21:38,290
when we were talking about
residual life.
293
00:21:41,250 --> 00:21:44,380
Now we'd like to generalize
this, and I hope the
294
00:21:44,380 --> 00:21:49,860
generalization is almost
obvious at this point.
295
00:21:49,860 --> 00:21:55,650
These three quantities here are
all examples of assigning
296
00:21:55,650 --> 00:21:58,430
rewards to renewal processes.
297
00:21:58,430 --> 00:22:04,040
And the reward at any time t,
when I'm trying to do this, is
298
00:22:04,040 --> 00:22:07,890
restricted to be a function of
the inter-renewal period
299
00:22:07,890 --> 00:22:08,860
containing t.
300
00:22:08,860 --> 00:22:14,110
In other words, when you try to
define what the reward is
301
00:22:14,110 --> 00:22:17,940
at a given time t, it's a
function only of what's going
302
00:22:17,940 --> 00:22:21,650
on in a renewal period
containing t.
303
00:22:24,290 --> 00:22:27,860
Now, in its simplest
form, I want to go
304
00:22:27,860 --> 00:22:29,780
make this even simpler.
305
00:22:29,780 --> 00:22:35,050
In its simplest form r of t is
restricted to be a function of
306
00:22:35,050 --> 00:22:39,020
the age of time t and the
duration of time t.
307
00:22:39,020 --> 00:22:42,620
In other words, you try to say
what's the reward in a given
308
00:22:42,620 --> 00:22:45,430
time t, and it's the
same as these three
309
00:22:45,430 --> 00:22:47,280
examples we looked at.
310
00:22:47,280 --> 00:22:52,550
It's some function of how far
back do you have to go to the
311
00:22:52,550 --> 00:22:56,590
previous arrival, and how far
forward do you have to go on
312
00:22:56,590 --> 00:22:59,040
until the next arrival.
313
00:22:59,040 --> 00:23:02,610
You can make this a function if
you want to of any of the
314
00:23:02,610 --> 00:23:07,260
three quantities residual
life, age, and duration.
315
00:23:07,260 --> 00:23:11,590
It seems to be intuitively a
little simpler to talk about
316
00:23:11,590 --> 00:23:16,342
age and duration as opposed
to the other quantities.
317
00:23:16,342 --> 00:23:20,730
So the time average for sample
path of r of t is found by
318
00:23:20,730 --> 00:23:23,790
analogy to residual life.
319
00:23:23,790 --> 00:23:26,100
That's the way I'm going
to do it here in class.
320
00:23:26,100 --> 00:23:28,140
The notes do a little
more careful job
321
00:23:28,140 --> 00:23:30,610
than just by analogy.
322
00:23:30,610 --> 00:23:34,310
But you start with the n-th
inter-renewal interval and you
323
00:23:34,310 --> 00:23:39,920
say, what is the aggregate
reward I get in the n-th
324
00:23:39,920 --> 00:23:40,790
inter-renewal interval?
325
00:23:40,790 --> 00:23:43,810
It's a random variable,
obviously.
326
00:23:43,810 --> 00:23:54,730
And what it is is the integral
from the previous arrival up
327
00:23:54,730 --> 00:23:59,190
to the next arrival of
r of t and omega dt.
328
00:24:05,130 --> 00:24:11,000
This is very strange, because
when we looked at this before
329
00:24:11,000 --> 00:24:15,750
we talked about n of t and we
talked about s sub n of t plus
330
00:24:15,750 --> 00:24:22,720
1 and s sub n of t, and suddenly
n of t plus 1 has
331
00:24:22,720 --> 00:24:28,690
turned into n, and n of t has
turned into n minus 1.
332
00:24:28,690 --> 00:24:31,210
What has happened?
333
00:24:31,210 --> 00:24:34,280
Well, this gives you a clue.
334
00:24:34,280 --> 00:24:37,730
And it's partly how we
define the intervals.
335
00:24:37,730 --> 00:24:44,010
Interval 1 goes from 0 to s1 and
z of t equals t, interval
336
00:24:44,010 --> 00:24:48,590
n z of t is t minus
s sub n minus 1.
337
00:24:48,590 --> 00:24:53,240
Let me go back and show you the
original figure here and I
338
00:24:53,240 --> 00:24:54,595
think it will make it clear.
339
00:25:06,090 --> 00:25:11,010
Now, the first arrival comes
at this time here.
340
00:25:11,010 --> 00:25:15,560
The first interval we're talking
about goes from 0 to
341
00:25:15,560 --> 00:25:17,530
time s1 of omega.
342
00:25:21,080 --> 00:25:25,000
What we're interested in, in
this first interval going from
343
00:25:25,000 --> 00:25:31,000
0 to s1 of omega, is this
interarrival time, which is
344
00:25:31,000 --> 00:25:32,850
the first interarrival time.
345
00:25:32,850 --> 00:25:36,840
This first interval is connected
to x1 and it goes
346
00:25:36,840 --> 00:25:42,250
from s sub 0, which is
just at 0 up to s1.
347
00:25:42,250 --> 00:25:47,890
The second arrival time
goes from s1 up to s2.
348
00:25:47,890 --> 00:25:53,400
When we look at n of t, n of
t is a sum t here in this
349
00:25:53,400 --> 00:25:57,510
interval here, and this is s.
350
00:26:03,760 --> 00:26:05,790
Am I going to get totally
confused writing
351
00:26:05,790 --> 00:26:07,905
this or aren't I?
352
00:26:19,600 --> 00:26:22,120
I think I might get totally
confused, so I'm not going to
353
00:26:22,120 --> 00:26:23,370
try to write it.
354
00:26:29,000 --> 00:26:30,950
Because when I try to write
it I'm trying to make the
355
00:26:30,950 --> 00:26:35,470
comparison between n of t and
n of t plus 1, which are the
356
00:26:35,470 --> 00:26:39,080
things up here.
357
00:26:39,080 --> 00:26:44,100
But the quantities here with the
renewal periods are, this
358
00:26:44,100 --> 00:26:47,150
is the first inter-renewal
period, this is the second
359
00:26:47,150 --> 00:26:48,730
inter-renewal period.
360
00:26:48,730 --> 00:26:54,910
The second inter-renewal period
goes from s sub 2 back
361
00:26:54,910 --> 00:26:56,280
to s sub 1.
362
00:26:56,280 --> 00:27:02,570
The n-th goes from s sub n
back to s sub n minus 1.
363
00:27:02,570 --> 00:27:05,190
So that's just the way it is
when you count this way.
364
00:27:05,190 --> 00:27:09,040
If you count it the other way
you would get even more
365
00:27:09,040 --> 00:27:14,030
confused because then in the
interval x sub n the
366
00:27:14,030 --> 00:27:17,580
inter-renewal time would
be x sub n plus 1 and
367
00:27:17,580 --> 00:27:19,090
that would be awful.
368
00:27:19,090 --> 00:27:21,460
So, this is the way
we have to do it.
369
00:27:30,310 --> 00:27:33,770
Let's look at what happens
with the expected
370
00:27:33,770 --> 00:27:37,195
inter-renewal time, then,
as a time average.
371
00:27:42,400 --> 00:27:45,990
And you can think of this as
a being for a sample time.
372
00:27:45,990 --> 00:27:49,700
When you have an expression
which is valid for a sample
373
00:27:49,700 --> 00:27:53,810
time, it's also valid for a
random variable, because it
374
00:27:53,810 --> 00:27:58,080
maps every sample point
into some value.
375
00:27:58,080 --> 00:28:01,620
So, I'll just call it here r sub
n, which is the amount of
376
00:28:01,620 --> 00:28:07,410
reward that you pick up in the
n-th inter-renewal period, so
377
00:28:07,410 --> 00:28:14,830
it's the integral from the n
minus first arrival up to the
378
00:28:14,830 --> 00:28:19,460
n-th arrival, and we're
integrating over r of t over
379
00:28:19,460 --> 00:28:21,010
this whole interval.
380
00:28:21,010 --> 00:28:28,250
So r of t is a function of the
age and of the duration.
381
00:28:28,250 --> 00:28:37,940
The age is t minus s sub n minus
1, and the duration is
382
00:28:37,940 --> 00:28:39,660
just x of n.
383
00:28:39,660 --> 00:28:43,410
Now that's a little weird also
because before we were talking
384
00:28:43,410 --> 00:28:47,710
about the duration as being
statistically very, very
385
00:28:47,710 --> 00:28:52,260
different from the inter-renewal
period.
386
00:28:52,260 --> 00:28:54,200
But it wasn't.
387
00:28:54,200 --> 00:29:04,310
If you look on a sample path
basis, this duration of the
388
00:29:04,310 --> 00:29:10,680
n-th inter-renewal period is
exactly equal to x sub n of
389
00:29:10,680 --> 00:29:19,110
omega, which is, for that
particular sample path, it's
390
00:29:19,110 --> 00:29:22,700
the value of that inter-renewal
interval.
391
00:29:22,700 --> 00:29:26,950
So when we integrate this
quantity now, we want to
392
00:29:26,950 --> 00:29:29,460
integrate it over dt.
393
00:29:29,460 --> 00:29:30,460
What do we get?
394
00:29:30,460 --> 00:29:36,210
Well, we've gotten rid of this
t here, we just have the n-th
395
00:29:36,210 --> 00:29:37,750
duration here.
396
00:29:37,750 --> 00:29:44,010
The only t appears over here,
and this is r of a difference
397
00:29:44,010 --> 00:29:47,370
between t and s sub n minus 1.
398
00:29:47,370 --> 00:29:51,510
So, we want to do a change of
variables, we want to let z be
399
00:29:51,510 --> 00:29:55,570
t minus s sub n minus 1,
and then we integrate z
400
00:29:55,570 --> 00:29:58,480
from 0 to x sub n.
401
00:29:58,480 --> 00:30:02,230
And now just imagine that you
put omegas in all of these
402
00:30:02,230 --> 00:30:05,220
things to let you see that
you're actually dealing with
403
00:30:05,220 --> 00:30:08,900
one sample function, and when
you leave the omegas out
404
00:30:08,900 --> 00:30:11,760
you're just dealing with the
random variables that arise
405
00:30:11,760 --> 00:30:14,580
because of doing that.
406
00:30:14,580 --> 00:30:23,290
So when we do this integration,
what we get is
407
00:30:23,290 --> 00:30:29,090
the integral of r of z with this
fixed x sub n integrated
408
00:30:29,090 --> 00:30:30,280
against dz.
409
00:30:30,280 --> 00:30:33,810
This is a function only of the
random variable x sub n.
410
00:30:33,810 --> 00:30:35,060
Strange.
411
00:30:35,060 --> 00:30:37,490
That's the expected
value of r sub n.
412
00:30:40,340 --> 00:30:43,290
This unfortunately can't be
reduced any further, this is
413
00:30:43,290 --> 00:30:45,040
just what it is.
414
00:30:45,040 --> 00:30:50,480
You have to find out what is
the integral of r of zx
415
00:30:50,480 --> 00:30:55,300
integrated over z, and then we
have to take the expected
416
00:30:55,300 --> 00:30:57,830
value over x.
417
00:30:57,830 --> 00:31:00,950
If the expectation exists, you
can write this in a simple
418
00:31:00,950 --> 00:31:09,370
way, because this quantity
here integrated
419
00:31:09,370 --> 00:31:11,140
is just r sub n.
420
00:31:11,140 --> 00:31:15,880
And then you take the expected
value of r sub n, and you
421
00:31:15,880 --> 00:31:18,590
divide by x-bar down here.
422
00:31:21,950 --> 00:31:27,770
This quantity here is just the
expected value of the integral
423
00:31:27,770 --> 00:31:30,240
of r of zx dz.
424
00:31:30,240 --> 00:31:32,980
And that integral there
is just the expected
425
00:31:32,980 --> 00:31:36,410
value of r sub n.
426
00:31:36,410 --> 00:31:40,020
So, if you look over the entire
sample path from 0 to
427
00:31:40,020 --> 00:31:49,940
infinity, what you get then is
that the time average reward
428
00:31:49,940 --> 00:31:53,560
is just equal to the expected
value of r sub n
429
00:31:53,560 --> 00:31:55,000
divided by x squared.
430
00:31:59,060 --> 00:32:00,270
Oops.
431
00:32:00,270 --> 00:32:01,520
Lost the example.
432
00:32:03,970 --> 00:32:08,580
Suppose we want to find the
k-th moment of the age.
433
00:32:08,580 --> 00:32:10,090
Simple thing to try to find.
434
00:32:12,930 --> 00:32:16,670
I want to do that because
it's simple.
435
00:32:16,670 --> 00:32:20,560
If you're looking at the k-th
moment of the age looked at as
436
00:32:20,560 --> 00:32:29,130
a time average, what it is is
the reward at time t, it's the
437
00:32:29,130 --> 00:32:32,490
age at time t to
the k-th power.
438
00:32:32,490 --> 00:32:38,640
We want to take the age to the
k-th power at time t, and then
439
00:32:38,640 --> 00:32:43,550
integrate over rt and divide
by the final value
440
00:32:43,550 --> 00:32:45,990
and go to the limit.
441
00:32:45,990 --> 00:32:50,460
So the expected value of
this k-th moment over 1
442
00:32:50,460 --> 00:32:56,620
inter-renewal period is this
integral here, z to the k dz
443
00:32:56,620 --> 00:33:01,160
times dF sub x of x.
444
00:33:01,160 --> 00:33:06,060
The only place x comes in this
integral is in the final point
445
00:33:06,060 --> 00:33:09,780
of the integration where we
are integrating up to the
446
00:33:09,780 --> 00:33:14,650
point x, at which the duration
ends, and then we're taking
447
00:33:14,650 --> 00:33:16,250
the expected value of this.
448
00:33:16,250 --> 00:33:20,420
So, we integrate this, the
integral of the z to the k
449
00:33:20,420 --> 00:33:29,080
from 0 to x is just x to the
k plus 1 divided by k.
450
00:33:29,080 --> 00:33:31,200
So when you take that integral
and you look at
451
00:33:31,200 --> 00:33:33,280
this, what is this?
452
00:33:33,280 --> 00:33:37,620
It's the 1 over k times the
expected value of the random
453
00:33:37,620 --> 00:33:40,710
variable, x to the k
plus first power.
454
00:33:40,710 --> 00:33:46,660
That's just this taking the
expected value of it, which is
455
00:33:46,660 --> 00:33:49,490
you take the expected value
by integrating times
456
00:33:49,490 --> 00:33:50,980
the s sub x of x.
457
00:33:50,980 --> 00:33:51,520
Yes?
458
00:33:51,520 --> 00:33:53,440
AUDIENCE: [INAUDIBLE]
459
00:33:53,440 --> 00:33:54,690
divide by k plus 1?
460
00:33:59,160 --> 00:34:00,410
PROFESSOR: Yes.
461
00:34:03,180 --> 00:34:04,480
I'm sorry.
462
00:34:07,050 --> 00:34:11,719
My evil twin brother is speaking
this morning, and
463
00:34:11,719 --> 00:34:14,560
this should be a
k plus 1 here.
464
00:34:14,560 --> 00:34:18,670
This should be a k plus 1, and
this should be a k plus 1.
465
00:34:18,670 --> 00:34:21,670
And if you look at the example
of the first moment, which is
466
00:34:21,670 --> 00:34:24,840
the first thing we looked at
today, namely finding the
467
00:34:24,840 --> 00:34:29,840
expected value of the age, what
we find out is that it's
468
00:34:29,840 --> 00:34:37,199
expected value of x squared
divided by 2 times x-bar.
469
00:34:37,199 --> 00:34:41,960
So with k equal to 1 we get
the expected value of x
470
00:34:41,960 --> 00:34:46,070
squared over 2 times x-bar.
471
00:34:46,070 --> 00:34:48,690
It's even worse because I left
a 2 out when we were talking
472
00:34:48,690 --> 00:34:51,690
about the expected
value of age.
473
00:34:51,690 --> 00:34:55,659
And to make things even worse,
when I did this I went back to
474
00:34:55,659 --> 00:34:59,590
check whether it was the right
answer, and of course the two
475
00:34:59,590 --> 00:35:01,590
mistakes cancelled
each other out.
476
00:35:01,590 --> 00:35:03,250
So, sorry about that.
477
00:35:07,150 --> 00:35:10,800
The thing I really wanted to
talk about today is stopping
478
00:35:10,800 --> 00:35:14,960
trials for stochastic processes,
because that's a
479
00:35:14,960 --> 00:35:21,220
topic which has always caused
confusion in this class.
480
00:35:21,220 --> 00:35:24,600
If you look at any of the
textbooks on stochastic
481
00:35:24,600 --> 00:35:28,130
processes, it causes even
more confusion.
482
00:35:28,130 --> 00:35:32,890
People talk about it, and then
they talk about it again, and
483
00:35:32,890 --> 00:35:36,410
they say what we said before
wasn't right, and then they
484
00:35:36,410 --> 00:35:39,110
talk about it again, they say
what we said before that
485
00:35:39,110 --> 00:35:41,710
wasn't right, and it goes
on and on and on.
486
00:35:41,710 --> 00:35:43,660
And you never know
what's going on.
487
00:35:47,920 --> 00:35:52,570
Very often instead of looking
at an entire stochastic
488
00:35:52,570 --> 00:35:55,900
process over the infinite
interval, you'd like to look
489
00:35:55,900 --> 00:35:57,380
at a finite interval.
490
00:35:57,380 --> 00:36:00,490
You'd like to look at the
interval going from 0 up to
491
00:36:00,490 --> 00:36:02,130
some time t.
492
00:36:02,130 --> 00:36:09,280
But in almost a majority of
those cases, the time t that
493
00:36:09,280 --> 00:36:13,430
you want to look at is not
some fixed time but some
494
00:36:13,430 --> 00:36:18,990
random time which depends on
something which is happening.
495
00:36:18,990 --> 00:36:26,680
And when you want to look at
what's going on here, it's
496
00:36:26,680 --> 00:36:31,560
always tricky to visualize this
because what you have is
497
00:36:31,560 --> 00:36:34,600
t becomes a random variable.
498
00:36:34,600 --> 00:36:39,730
That random variable is a
function of the sample values
499
00:36:39,730 --> 00:36:43,390
that you have up until time t.
500
00:36:43,390 --> 00:36:47,560
But that seems like a circular
argument, t a random variable
501
00:36:47,560 --> 00:36:50,110
and it depends on t.
502
00:36:52,620 --> 00:36:55,640
Things are not supposed to
depend on themselves, because
503
00:36:55,640 --> 00:37:00,060
when they depend on themselves
you have trouble understanding
504
00:37:00,060 --> 00:37:01,930
what's going on.
505
00:37:01,930 --> 00:37:05,450
So, we will sort that out.
506
00:37:05,450 --> 00:37:07,470
In sorting it out we're
only going to look at
507
00:37:07,470 --> 00:37:09,610
discrete-time processes.
508
00:37:09,610 --> 00:37:13,830
So, we'll only look at sequences
of random variables.
509
00:37:13,830 --> 00:37:17,670
We will not assume that these
random variables are IID for
510
00:37:17,670 --> 00:37:22,070
this argument here, because we
want to look at a much wider
511
00:37:22,070 --> 00:37:22,950
range of things.
512
00:37:22,950 --> 00:37:28,180
So we have some arbitrary
sequence of random variables,
513
00:37:28,180 --> 00:37:33,090
and the idea is you want to
sit there looking at this
514
00:37:33,090 --> 00:37:40,800
evolution of the sequence of
random variables, and in fact,
515
00:37:40,800 --> 00:37:44,630
you want to sit there looking
at the sample
516
00:37:44,630 --> 00:37:46,150
path of this evolution.
517
00:37:46,150 --> 00:37:49,830
You want to look at x1, and you
want to look at x2, and
518
00:37:49,830 --> 00:37:52,830
you want to look at
x3, and so forth.
519
00:37:52,830 --> 00:37:56,410
And at each point along the line
in terms of what you've
520
00:37:56,410 --> 00:37:59,160
seen already, you want to decide
whether you're going to
521
00:37:59,160 --> 00:38:02,370
stop or whether you're going
to continue at that point.
522
00:38:10,380 --> 00:38:14,670
And if at that point you develop
a rule for what you're
523
00:38:14,670 --> 00:38:19,380
going to do with each point you
call that a stopping rule,
524
00:38:19,380 --> 00:38:22,240
so that you have a rule that
tells you when you want to
525
00:38:22,240 --> 00:38:24,750
stop looking at these things.
526
00:38:24,750 --> 00:38:29,540
Let me give you some idea of
the generality of this.
527
00:38:29,540 --> 00:38:34,080
Often when we look at queuing
systems we are going to have
528
00:38:34,080 --> 00:38:40,310
an arrival process where
arrivals keep coming in,
529
00:38:40,310 --> 00:38:45,040
things get serviced one after
the other, and one of the very
530
00:38:45,040 --> 00:38:49,670
interesting things is whether
the system ever empties out or
531
00:38:49,670 --> 00:38:53,140
whether the queue just keeps
building up forever.
532
00:38:53,140 --> 00:38:58,160
Well, an interesting stopping
rule then is when the queue
533
00:38:58,160 --> 00:39:00,480
empties out.
534
00:39:00,480 --> 00:39:04,430
We will see that an even more
interesting stopping rule is
535
00:39:04,430 --> 00:39:09,310
starting with some arbitrary
first arrival at time 0.
536
00:39:09,310 --> 00:39:14,600
Let's look for the time until
another arrival occurs which
537
00:39:14,600 --> 00:39:19,580
starts another busy period.
538
00:39:19,580 --> 00:39:24,180
These stopping times, then, are
going to be critical in
539
00:39:24,180 --> 00:39:27,030
analyzing things like
queuing systems.
540
00:39:27,030 --> 00:39:31,080
They're also going to become
critical in trying to analyze
541
00:39:31,080 --> 00:39:34,890
these renewal systems that we're
already talking about.
542
00:39:34,890 --> 00:39:40,260
They will become critical to
trying to take a time average
543
00:39:40,260 --> 00:39:45,920
view about renewal processes
instead of a, not a time
544
00:39:45,920 --> 00:39:49,690
average, but an ensemble average
viewpoint looking at
545
00:39:49,690 --> 00:39:54,020
particular times t as opposed
to taking a time average.
546
00:39:54,020 --> 00:39:56,180
So we're going to look
at these things in
547
00:39:56,180 --> 00:39:57,430
many different ways.
548
00:40:02,070 --> 00:40:06,920
In order to do that, since
you're looking at a sample
549
00:40:06,920 --> 00:40:11,840
function, you're observing
it on each
550
00:40:11,840 --> 00:40:14,910
arrival of this process.
551
00:40:23,460 --> 00:40:27,080
At each observation of a sample
value of a random
552
00:40:27,080 --> 00:40:30,730
variable, you decide whether
you want to stop or not.
553
00:40:30,730 --> 00:40:37,220
So a sensible way to deal with
this is to look over all time,
554
00:40:37,220 --> 00:40:43,290
define a random variable J,
which describes when this
555
00:40:43,290 --> 00:40:44,830
sequence is to be stopped.
556
00:40:44,830 --> 00:40:49,470
So for each sample value x1 of
omega, x2 of omega, x3 of
557
00:40:49,470 --> 00:40:53,690
omega, and so forth, if your
rule is to stop the first time
558
00:40:53,690 --> 00:41:00,630
you see a sample value which
is equal to 0, then J is a
559
00:41:00,630 --> 00:41:05,790
integer random variable whose
value is the first n at which
560
00:41:05,790 --> 00:41:10,450
is x of n is equal to 0 and
many other examples.
561
00:41:10,450 --> 00:41:16,190
So we try a 1 then, x1
of omega is observed.
562
00:41:16,190 --> 00:41:20,120
A decision is made based
on x1 of omega
563
00:41:20,120 --> 00:41:23,560
whether or not to stop.
564
00:41:23,560 --> 00:41:27,550
If we stop, J of
omega equals 1.
565
00:41:27,550 --> 00:41:31,910
If we don't stop, J of omega is
bigger than 1, but we don't
566
00:41:31,910 --> 00:41:33,510
know what it is yet.
567
00:41:33,510 --> 00:41:40,440
At trial two, if we haven't
stopped yet, you observe x2 of
568
00:41:40,440 --> 00:41:44,710
omega, the second random
variable, second sample value,
569
00:41:44,710 --> 00:41:49,240
you make a decision again based
on x1 of omega and x2 of
570
00:41:49,240 --> 00:41:51,770
omega whether or not to stop.
571
00:41:51,770 --> 00:41:54,790
If you stop, J of omega
is equal to 2.
572
00:41:54,790 --> 00:42:00,090
So you can visualize doing this
on each trial, you don't
573
00:42:00,090 --> 00:42:02,790
have to worry about whether
you've already stopped, you
574
00:42:02,790 --> 00:42:06,370
just have a rule that says, I
want to stop at this point if
575
00:42:06,370 --> 00:42:08,240
I haven't stopped before.
576
00:42:08,240 --> 00:42:14,925
So the stopping event at time
n is that you stop either if
577
00:42:14,925 --> 00:42:20,780
your rule tells you to stop
or you stop before.
578
00:42:20,780 --> 00:42:23,990
And if you stop before then
you're obviously stopped.
579
00:42:23,990 --> 00:42:27,700
At each trial n if stopping has
not yet occurred x of n is
580
00:42:27,700 --> 00:42:32,600
observed and the decision based
on x1 to xn is made.
581
00:42:32,600 --> 00:42:37,060
If you stop, then J omega
is equal to n.
582
00:42:37,060 --> 00:42:45,410
So for each sample path J of n
is the time or the trial at
583
00:42:45,410 --> 00:42:46,660
which you stop.
584
00:42:50,850 --> 00:42:54,890
So, we're going to define a
stopping trial, or stopping
585
00:42:54,890 --> 00:42:59,980
time, J for a sequence
of random variables.
586
00:42:59,980 --> 00:43:03,610
You're going to define this to
be a positive integer-valued
587
00:43:03,610 --> 00:43:08,130
random variable that has to be
positive, because if you're
588
00:43:08,130 --> 00:43:11,290
going to stop before you observe
anything that's not a
589
00:43:11,290 --> 00:43:13,430
very interesting thing.
590
00:43:13,430 --> 00:43:17,190
So you always observe at least
one thing, then you decide
591
00:43:17,190 --> 00:43:20,120
whether you want to stop or
you proceed, so forth.
592
00:43:20,120 --> 00:43:22,690
So it's a positive
integer-valued random
593
00:43:22,690 --> 00:43:30,900
variable, you always stop at an
integer trial, and for each
594
00:43:30,900 --> 00:43:36,000
n greater than or equal to 1 the
indicator random variable
595
00:43:36,000 --> 00:43:42,520
the indicator of the event J
equals n is a function of what
596
00:43:42,520 --> 00:43:44,835
you have observed up
until that point.
597
00:43:47,880 --> 00:43:50,850
This part is a really
critical part of it.
598
00:43:50,850 --> 00:43:56,700
The decision to stop at a time n
has to be a function only of
599
00:43:56,700 --> 00:43:59,010
what you have already
observed.
600
00:43:59,010 --> 00:44:03,290
You're not allowed as an
observer to peek ahead a
601
00:44:03,290 --> 00:44:06,900
little bit and then decide on
the basis of what you see in
602
00:44:06,900 --> 00:44:12,310
the future whether you want to
stop at this time or not.
603
00:44:12,310 --> 00:44:15,570
One example in the notes is that
you're playing poker with
604
00:44:15,570 --> 00:44:23,280
somebody, and you make a bet
and the other person wins.
605
00:44:23,280 --> 00:44:26,150
You make the bet on the basis
of what you've seen in your
606
00:44:26,150 --> 00:44:30,210
cards so far, you don't make
your bet on the basis of your
607
00:44:30,210 --> 00:44:34,410
observation of what the other
player has, and the other
608
00:44:34,410 --> 00:44:39,890
player will get very angry if
you try to base your decision
609
00:44:39,890 --> 00:44:45,600
later on what the person you're
playing with has.
610
00:44:45,600 --> 00:44:47,590
So you can't do that
sort of thing.
611
00:44:47,590 --> 00:44:51,490
You can't peek ahead when
you're doing this.
612
00:44:51,490 --> 00:44:54,890
You could design random
variables where, in fact, you
613
00:44:54,890 --> 00:44:59,920
can look ahead, but the times
where you use what people call
614
00:44:59,920 --> 00:45:05,510
stopping trials are situations
in which you stop based on
615
00:45:05,510 --> 00:45:07,990
what has already happened.
616
00:45:07,990 --> 00:45:11,380
You can generalize this, and we
will generalize it later,
617
00:45:11,380 --> 00:45:14,380
where you can stop on the
basis of other random
618
00:45:14,380 --> 00:45:19,350
variables, which also evolve in
time, but you stop on the
619
00:45:19,350 --> 00:45:25,730
basis of what has already
happened up until time n.
620
00:45:25,730 --> 00:45:29,230
We're going to visualize
conducting successive trials
621
00:45:29,230 --> 00:45:33,850
until sum n in which the event
J equals n occurs.
622
00:45:33,850 --> 00:45:36,520
Further trials then cease.
623
00:45:36,520 --> 00:45:39,680
It is simpler conceptually
to visualize stopping the
624
00:45:39,680 --> 00:45:43,720
observation of trials after
the stopping trial, but
625
00:45:43,720 --> 00:45:45,530
continuing to conduct trials.
626
00:45:45,530 --> 00:45:49,310
In other words, if we start
talking about a stopping
627
00:45:49,310 --> 00:45:55,100
process and we say we stop on
the n-th trial, but we're
628
00:45:55,100 --> 00:45:58,020
forbidden to even talk
about the n plus
629
00:45:58,020 --> 00:45:59,700
first random variable.
630
00:45:59,700 --> 00:46:05,890
Now the n plus first random
variable occurs on some sample
631
00:46:05,890 --> 00:46:09,520
paths but doesn't occur
on other sample paths.
632
00:46:09,520 --> 00:46:11,960
I don't know how to deal with
that, and neither do you.
633
00:46:14,800 --> 00:46:17,800
And what that means is the way
we're going to visualize these
634
00:46:17,800 --> 00:46:24,200
processes is physically they
continue forever, but we just
635
00:46:24,200 --> 00:46:26,540
stop observing them at
a certain point.
636
00:46:26,540 --> 00:46:30,840
So, this stopping rule is the
time at which we stop
637
00:46:30,840 --> 00:46:35,090
observing as opposed to the
time at which the random
638
00:46:35,090 --> 00:46:50,470
variable ceases existing, if we
define the random variable,
639
00:46:50,470 --> 00:46:55,430
it has a value for all sample
values, and sample values
640
00:46:55,430 --> 00:46:59,200
involve all of these paths.
641
00:46:59,200 --> 00:47:01,940
So you sort of have to
define it that way.
642
00:47:01,940 --> 00:47:07,550
One thing you would like to do
is in many of these situations
643
00:47:07,550 --> 00:47:11,040
you wind up never stopping.
644
00:47:11,040 --> 00:47:15,360
And the trouble is, when you're
investigating stopping
645
00:47:15,360 --> 00:47:20,110
rules, stopping trials, you
usually don't know ahead of
646
00:47:20,110 --> 00:47:24,530
time whether you're ever
going to stop or not.
647
00:47:24,530 --> 00:47:27,660
And because of that if you
don't know whether you're
648
00:47:27,660 --> 00:47:29,310
going to stop or not,
you can't call
649
00:47:29,310 --> 00:47:30,810
it a stopping trial.
650
00:47:30,810 --> 00:47:35,990
So, what one normally does is
to say that J is a possibly
651
00:47:35,990 --> 00:47:38,240
defective random variable.
652
00:47:38,240 --> 00:47:44,400
And if it's possibly defective
you mean that all sample
653
00:47:44,400 --> 00:47:49,700
points are mapped into either
finite J or perhaps J equals
654
00:47:49,700 --> 00:47:52,840
infinity, which means
that you never stop.
655
00:47:52,840 --> 00:47:57,660
But you still have the same
condition that we have here,
656
00:47:57,660 --> 00:48:02,290
that you stop on the basis
of x1, x of n.
657
00:48:02,290 --> 00:48:05,400
And you also have the condition,
which isn't quite
658
00:48:05,400 --> 00:48:08,310
clear here, but it's clear
from the fact that J is a
659
00:48:08,310 --> 00:48:17,060
random variable, that the events
J equals 1, J equals 2,
660
00:48:17,060 --> 00:48:20,840
J equals 3, are all disjoint.
661
00:48:20,840 --> 00:48:24,940
So you can't stop twice, you
have to just stop once.
662
00:48:24,940 --> 00:48:30,140
And once you stop you
can't start again,
663
00:48:30,140 --> 00:48:34,020
Now does everybody understand
what is stopping trial or a
664
00:48:34,020 --> 00:48:35,740
stopping rule is
at this point?
665
00:48:39,300 --> 00:48:44,770
If you don't understand it I
think the trouble is you won't
666
00:48:44,770 --> 00:48:47,730
realize you don't understand it
until you start seeing some
667
00:48:47,730 --> 00:48:52,540
of these examples where strange
things are going on.
668
00:48:52,540 --> 00:48:55,790
I guess we have to just
go ahead and see
669
00:48:55,790 --> 00:48:58,460
what happens then.
670
00:48:58,460 --> 00:49:00,300
So the examples, which
I'm not going to
671
00:49:00,300 --> 00:49:02,730
develop in any detail.
672
00:49:02,730 --> 00:49:07,160
A gambler goes to a casino and
he gambles until he's broke.
673
00:49:07,160 --> 00:49:10,480
So, he goes in with a finite
amount of capital.
674
00:49:10,480 --> 00:49:14,480
He has some particular system
that he's playing by.
675
00:49:14,480 --> 00:49:18,040
If he's lucky he gets to play
for a long, long time, he gets
676
00:49:18,040 --> 00:49:22,030
a lot of amusement as he loses
his money, and if he's unlucky
677
00:49:22,030 --> 00:49:24,600
he loses very quickly.
678
00:49:24,600 --> 00:49:28,490
If you look at the odds in
casinos and you apply the
679
00:49:28,490 --> 00:49:31,000
strong law of large numbers,
you realize that with
680
00:49:31,000 --> 00:49:34,480
probability 1 you get wiped
out eventually.
681
00:49:34,480 --> 00:49:37,560
Because the odds are
not in your favor.
682
00:49:37,560 --> 00:49:41,200
Casinos wouldn't be built if the
odds were in your favor.
683
00:49:41,200 --> 00:49:44,410
Another example, just
flip a coin until 10
684
00:49:44,410 --> 00:49:46,500
successive heads appear.
685
00:49:46,500 --> 00:49:48,000
10 heads in a row.
686
00:49:48,000 --> 00:49:50,260
That's a rather unlikely
event.
687
00:49:50,260 --> 00:49:53,470
You're going to be able to
figure out very easily from
688
00:49:53,470 --> 00:49:57,170
this theory what's the expected
amount of time until
689
00:49:57,170 --> 00:49:59,730
10 successive heads appear.
690
00:49:59,730 --> 00:50:02,801
It's a very easy problem.
691
00:50:02,801 --> 00:50:08,070
A more important problem, this
test and hypothesis with
692
00:50:08,070 --> 00:50:11,310
repeated trials.
693
00:50:11,310 --> 00:50:13,730
So, you have the hypothesis
that a certain kind of
694
00:50:13,730 --> 00:50:18,710
treatment will make patients
well, and you want to know
695
00:50:18,710 --> 00:50:22,580
whether to use this treatment.
696
00:50:22,580 --> 00:50:27,820
So you test it on mice, or
people, or what have you.
697
00:50:27,820 --> 00:50:29,420
If you think it's pretty
safe, you start
698
00:50:29,420 --> 00:50:32,070
testing it on people.
699
00:50:32,070 --> 00:50:35,015
And how many tests
do you make?
700
00:50:37,950 --> 00:50:41,320
Well, you originally think that
as a scientist you should
701
00:50:41,320 --> 00:50:45,240
plan an experiment ahead of
time, and you should say, I'm
702
00:50:45,240 --> 00:50:47,810
going to do 1,000 tests.
703
00:50:47,810 --> 00:50:51,180
But if you're doing experiments
on people and you
704
00:50:51,180 --> 00:50:55,700
find out that the first 10
patients die, you're going to
705
00:50:55,700 --> 00:50:59,380
stop the experiment
at that point.
706
00:50:59,380 --> 00:51:03,990
And if you find out that the
first 100 patients all live,
707
00:51:03,990 --> 00:51:05,505
well, you might continue
the experiment.
708
00:51:05,505 --> 00:51:08,430
But you're going to publish
the results at that point
709
00:51:08,430 --> 00:51:11,230
because you're going to try to
get the FDA to approve this
710
00:51:11,230 --> 00:51:17,080
drug, or this operation,
or this what have you.
711
00:51:17,080 --> 00:51:21,730
So if you view this stopping
trial as the time at which
712
00:51:21,730 --> 00:51:26,410
you're going to try to publish
something, then, again, if you
713
00:51:26,410 --> 00:51:30,510
have any sense, you are going
to perform experiments, look
714
00:51:30,510 --> 00:51:34,420
at the experiments as you go,
and decide what you're going
715
00:51:34,420 --> 00:51:37,700
to do as a function of what
you've already seen.
716
00:51:37,700 --> 00:51:38,860
I mean, that's the way that all
717
00:51:38,860 --> 00:51:40,480
intelligent people operate.
718
00:51:43,200 --> 00:51:46,620
So if science says that's not
a good way to operate,
719
00:51:46,620 --> 00:51:48,000
something's wrong
with science.
720
00:51:48,000 --> 00:51:50,290
But fortunately science
doesn't say that.
721
00:51:50,290 --> 00:51:52,170
Science allows you to do this.
722
00:51:52,170 --> 00:51:55,217
AUDIENCE: So not every J is
potentially defective?
723
00:51:55,217 --> 00:51:56,675
You can come up with
examples of
724
00:51:56,675 --> 00:51:57,656
where it would be defective.
725
00:51:57,656 --> 00:52:00,300
But not every one is necessarily
[INAUDIBLE].
726
00:52:03,470 --> 00:52:03,966
PROFESSOR: Yes.
727
00:52:03,966 --> 00:52:11,830
If a random variable is
defective it means there's
728
00:52:11,830 --> 00:52:15,600
some probability that J is
going to be infinite, but
729
00:52:15,600 --> 00:52:19,710
there's also presumably some
probability that J is equal to
730
00:52:19,710 --> 00:52:23,070
1, a probability J is equal
to 2, and so forth.
731
00:52:23,070 --> 00:52:27,050
So we have a distribution
function for J, which instead
732
00:52:27,050 --> 00:52:31,870
of going up to 1 and stopping
goes up to some smaller value
733
00:52:31,870 --> 00:52:35,202
and stops, and then it doesn't
go any further.
734
00:52:35,202 --> 00:52:36,194
AUDIENCE: Not every
[INAUDIBLE].
735
00:52:36,194 --> 00:52:38,550
Some of these are definitely
going to [INAUDIBLE].
736
00:52:38,550 --> 00:52:39,800
PROFESSOR: Yes.
737
00:52:43,180 --> 00:52:50,200
So this testing hypotheses is
really what triggered Abraham
738
00:52:50,200 --> 00:52:55,210
Wald to start trying to
understand these situations,
739
00:52:55,210 --> 00:52:59,090
because he found out relatively
quickly that it
740
00:52:59,090 --> 00:53:02,990
wasn't trivial to understand
them, and all sorts of crazy
741
00:53:02,990 --> 00:53:04,350
things were happening.
742
00:53:04,350 --> 00:53:09,080
So he spent a lot of his time
studying what happened in
743
00:53:09,080 --> 00:53:10,130
these cases.
744
00:53:10,130 --> 00:53:11,930
He called it sequential
analysis.
745
00:53:11,930 --> 00:53:13,520
You might have heard the word.
746
00:53:13,520 --> 00:53:18,930
And sequential analysis is
exactly the same idea.
747
00:53:18,930 --> 00:53:25,620
It's looking at analyzing
experiments as you go in time,
748
00:53:25,620 --> 00:53:28,660
and either stopping or doing
something else or changing the
749
00:53:28,660 --> 00:53:33,360
experiment or what have you,
depending on what happens.
750
00:53:33,360 --> 00:53:36,700
Another thing is observe
successive renewals in a
751
00:53:36,700 --> 00:53:40,750
renewal process until
s sub n is greater
752
00:53:40,750 --> 00:53:43,760
than or equal to 100.
753
00:53:43,760 --> 00:53:47,820
Now, this particular thing is
one of the things that we're
754
00:53:47,820 --> 00:53:51,230
going to use in studying renewal
processes, it's why
755
00:53:51,230 --> 00:53:54,740
we're studying this
topic right now.
756
00:53:54,740 --> 00:53:57,270
Well, we might study it now
anyway, because we're going to
757
00:53:57,270 --> 00:54:01,460
use it in lots of other places,
but it's also why you
758
00:54:01,460 --> 00:54:07,920
have to be very careful about
talking about stopping time as
759
00:54:07,920 --> 00:54:10,310
opposed to stopping trials.
760
00:54:10,310 --> 00:54:16,910
Because if you stop an
experiment at the arrival in a
761
00:54:16,910 --> 00:54:26,080
renewal process at which that
renewal occurs after time 100,
762
00:54:26,080 --> 00:54:29,140
you don't know when you're
going to stop.
763
00:54:29,140 --> 00:54:33,420
It might be a very long time
after 100 before that first
764
00:54:33,420 --> 00:54:38,190
arrival after time 100 occurs,
it might be a very short time.
765
00:54:38,190 --> 00:54:41,390
So it's not a stopping time that
you're defining, it's a
766
00:54:41,390 --> 00:54:43,120
stopping trial.
767
00:54:43,120 --> 00:54:46,560
You are defining a rule which
tells you at which trial
768
00:54:46,560 --> 00:54:50,090
you're going to stop as opposed
to a rule that tells
769
00:54:50,090 --> 00:54:53,690
you in which time you're going
to stop, and we'll see this as
770
00:54:53,690 --> 00:54:54,960
we go on in this.
771
00:54:58,250 --> 00:55:02,480
Suppose the random variables in
this process we're looking
772
00:55:02,480 --> 00:55:07,970
at have a finite number of
possible sample values.
773
00:55:07,970 --> 00:55:13,200
Then any possibly defective
stopping trial, and a stopping
774
00:55:13,200 --> 00:55:15,720
trial is a set of rules for
when you're going to stop
775
00:55:15,720 --> 00:55:22,350
observing things, any stopping
trial can be represented as a
776
00:55:22,350 --> 00:55:24,280
rooted tree.
777
00:55:24,280 --> 00:55:26,970
If you don't know what a rooted
tree is, a rooted tree
778
00:55:26,970 --> 00:55:28,080
is what I've drawn here.
779
00:55:28,080 --> 00:55:30,740
So, there's no confusion here.
780
00:55:30,740 --> 00:55:33,670
A rooted tree is something that
starts on the left hand
781
00:55:33,670 --> 00:55:35,740
side and it grows for a while.
782
00:55:35,740 --> 00:55:36,910
It's like an ordinary tree.
783
00:55:36,910 --> 00:55:40,420
It has a root, and it has
branches that go up.
784
00:55:43,580 --> 00:55:48,530
But any possibly defective
stopping trial can be
785
00:55:48,530 --> 00:55:52,250
represented as a rooted tree
where the trial at which each
786
00:55:52,250 --> 00:55:55,320
sample path stops
is represented
787
00:55:55,320 --> 00:55:58,400
by a terminal node.
788
00:55:58,400 --> 00:56:00,780
So the example here I want
to use is the same
789
00:56:00,780 --> 00:56:03,360
example as in the text.
790
00:56:03,360 --> 00:56:06,210
I was trying to get another
example, which will be more
791
00:56:06,210 --> 00:56:07,440
interesting.
792
00:56:07,440 --> 00:56:11,490
The trouble is with these trees
they get very big very
793
00:56:11,490 --> 00:56:17,840
quickly, and therefore this is
not really a practical way of
794
00:56:17,840 --> 00:56:19,730
analyzing these problems.
795
00:56:19,730 --> 00:56:23,340
I think it's a nice conceptual
way of analyzing.
796
00:56:23,340 --> 00:56:28,240
So, the experiment is x is a
binary random variable, and
797
00:56:28,240 --> 00:56:33,090
stopping occurs when the pattern
1, 0 first occurs.
798
00:56:33,090 --> 00:56:36,030
You can certainly look at any
other pattern you want to, any
799
00:56:36,030 --> 00:56:37,520
much longer pattern.
800
00:56:37,520 --> 00:56:42,010
But what I want to do here is to
illustrate what the tree is
801
00:56:42,010 --> 00:56:46,220
that corresponds to this
stopping rule.
802
00:56:46,220 --> 00:56:53,680
So if the first random variable
has a value 1 and
803
00:56:53,680 --> 00:56:59,230
then the second one has a value
0, you observe 1, 0, and
804
00:56:59,230 --> 00:57:01,530
at that point the experiment
is over.
805
00:57:01,530 --> 00:57:04,430
You've seen the pattern
1, 0, and you stop.
806
00:57:04,430 --> 00:57:06,440
So there's a big circle there.
807
00:57:06,440 --> 00:57:11,040
If you observe 1 followed by a
1 followed by 0, again you
808
00:57:11,040 --> 00:57:15,830
stop 1, 1, 1, followed by a
0 you stop, and so forth.
809
00:57:15,830 --> 00:57:20,380
If you observe a 0 and then you
observe a 1 and then you
810
00:57:20,380 --> 00:57:24,020
observe a 0, that's the first
time at which you've seen the
811
00:57:24,020 --> 00:57:27,200
pattern 1, 0 so you
stop there.
812
00:57:27,200 --> 00:57:31,470
0, 1, 1, 0, you stop there,
and so forth.
813
00:57:31,470 --> 00:57:34,280
So you get this kind of
ladder structure.
814
00:57:34,280 --> 00:57:37,180
The point that I want to make
is not that this particular
815
00:57:37,180 --> 00:57:39,150
structure is very interesting.
816
00:57:39,150 --> 00:57:42,800
It's that whatever kind of rule
you decide to use for
817
00:57:42,800 --> 00:57:46,340
stopping, you can express
it in this way.
818
00:57:46,340 --> 00:57:50,090
You can express the points at
which you stop by big circles
819
00:57:50,090 --> 00:57:53,600
and the points at which you
don't stop as intermediate
820
00:57:53,600 --> 00:57:55,350
nodes where you keep on going.
821
00:57:58,940 --> 00:58:03,010
I think in some sense this takes
the picture which you
822
00:58:03,010 --> 00:58:07,860
have of sample values, of sample
functions with the rule
823
00:58:07,860 --> 00:58:12,050
that each sample value, which
is often the easiest way to
824
00:58:12,050 --> 00:58:18,280
express stopping rules, but in
terms of random variables it
825
00:58:18,280 --> 00:58:21,550
doesn't always give you
the right story.
826
00:58:21,550 --> 00:58:26,760
This gives you a story in terms
of all possible choices
827
00:58:26,760 --> 00:58:28,790
of the random variables.
828
00:58:28,790 --> 00:58:32,420
The first toss has to be
either a 1 or a 0.
829
00:58:32,420 --> 00:58:36,630
The second toss has to be a 1
or a 0, but if it's a 0 you
830
00:58:36,630 --> 00:58:38,580
stop and don't go any further.
831
00:58:38,580 --> 00:58:41,020
If it's a 1 you continue
further.
832
00:58:41,020 --> 00:58:43,570
So you keep continuing
along here, you can
833
00:58:43,570 --> 00:58:47,000
continue forever here.
834
00:58:47,000 --> 00:58:50,580
If you want to analyze
this, it has a
835
00:58:50,580 --> 00:58:53,310
rather peculiar analysis.
836
00:58:53,310 --> 00:58:57,870
How long does it take until
the first 1, 0 occurs?
837
00:58:57,870 --> 00:59:05,230
Well, if you start out with a 1
this experiment lasts until
838
00:59:05,230 --> 00:59:07,920
the first 0 appears.
839
00:59:07,920 --> 00:59:10,930
Because if the 0 doesn't appear
right away another 1
840
00:59:10,930 --> 00:59:13,430
appears, so you keep
going along this
841
00:59:13,430 --> 00:59:16,870
path until a 0 appears.
842
00:59:16,870 --> 00:59:21,180
If you see a 0 first and then
you see a 1, you're back in
843
00:59:21,180 --> 00:59:24,160
the same situation again.
844
00:59:24,160 --> 00:59:26,370
You proceed until a 0 occurs.
845
00:59:26,370 --> 00:59:31,540
What this is really saying is
the time until you stop here
846
00:59:31,540 --> 00:59:37,690
consists of an arbitrary number,
possibly 0 of 0's,
847
00:59:37,690 --> 00:59:42,420
followed by an arbitrary number,
possibly 0 of 1's,
848
00:59:42,420 --> 00:59:45,260
followed by a single 0.
849
00:59:45,260 --> 00:59:48,500
So the only patterns you can
have here which would lead to
850
00:59:48,500 --> 00:59:53,670
these terminal nodes are some
number of 0's followed by some
851
00:59:53,670 --> 00:59:56,320
number of 1's, followed
by a final 0.
852
01:00:00,210 --> 01:00:04,280
But the example is not
important, so that just tells
853
01:00:04,280 --> 01:00:06,630
you what it means.
854
01:00:06,630 --> 01:00:11,700
With all of this we finally
have Wald's equality.
855
01:00:11,700 --> 01:00:18,140
And Wald's equality, it looks
like a strange thing.
856
01:00:18,140 --> 01:00:23,880
I spent so much time talking
about stopping rules because
857
01:00:23,880 --> 01:00:26,760
all the complexity lies
in the stopping rules.
858
01:00:26,760 --> 01:00:30,790
Wald's equality itself is a very
simple thing after you
859
01:00:30,790 --> 01:00:32,010
understand that.
860
01:00:32,010 --> 01:00:37,440
What Wald's equality says is,
let x sub n be a sequence of
861
01:00:37,440 --> 01:00:42,420
IID random variables, each
would mean x-bar.
862
01:00:42,420 --> 01:00:43,760
This is the kind of
thing we've been
863
01:00:43,760 --> 01:00:45,320
talking about a lot.
864
01:00:45,320 --> 01:00:49,740
If J is a stopping trial for x
sub n, n greater than or equal
865
01:00:49,740 --> 01:00:53,050
to 1, in other words, if it
satisfies that definition
866
01:00:53,050 --> 01:00:57,230
where it never peeks ahead, and
if the expected value of J
867
01:00:57,230 --> 01:01:02,695
is less than infinity, then the
sum s sub J equals x1, x2,
868
01:01:02,695 --> 01:01:07,520
up to x sub J, the amount that
the gambler has accumulated
869
01:01:07,520 --> 01:01:08,840
before the gambler stops.
870
01:01:11,730 --> 01:01:16,500
The sum of the stopping trial
satisfied expected value of
871
01:01:16,500 --> 01:01:21,120
the gain as equal to the
expected value of x times the
872
01:01:21,120 --> 01:01:24,610
expected number of
times you play.
873
01:01:24,610 --> 01:01:29,820
It sort of says that even when
you use these stopping rules
874
01:01:29,820 --> 01:01:33,080
to decide when to
stop, there's no
875
01:01:33,080 --> 01:01:34,330
way to beat the house.
876
01:01:41,760 --> 01:01:46,830
If you're playing a fair game,
your expected gain is equal to
877
01:01:46,830 --> 01:01:50,920
the expected gain for trial
times the expected number of
878
01:01:50,920 --> 01:01:53,210
times you play.
879
01:01:53,210 --> 01:01:55,375
And there's no way you
can get around that.
880
01:02:01,230 --> 01:02:03,430
Note that this is more
than a [? poof ?]
881
01:02:03,430 --> 01:02:04,670
and less than a proof.
882
01:02:04,670 --> 01:02:07,530
I'll explain why it's more
than a [? poof ?]
883
01:02:07,530 --> 01:02:11,630
and less than a proof
as we go.
884
01:02:11,630 --> 01:02:19,060
s sub J, the random variable s
sub J, it's equal to x sub 1
885
01:02:19,060 --> 01:02:21,630
times the indicator function
of j greater
886
01:02:21,630 --> 01:02:24,070
than or equal to 1.
887
01:02:24,070 --> 01:02:27,860
In other words, well, the
indicator function J greater
888
01:02:27,860 --> 01:02:31,900
than or equal to 1 is
just universally 1.
889
01:02:31,900 --> 01:02:35,630
x sub 2 times the indicator
function of J greater than or
890
01:02:35,630 --> 01:02:36,350
equal to 2.
891
01:02:36,350 --> 01:02:41,210
In other words, s sub J includes
x sub 2 if the
892
01:02:41,210 --> 01:02:45,490
experiment proceeds long enough
that you take the
893
01:02:45,490 --> 01:02:46,820
second trial.
894
01:02:46,820 --> 01:02:54,480
In other words, this quantity
here, is 1 minus the
895
01:02:54,480 --> 01:02:58,980
probability that you stop at
the end of the first trial.
896
01:02:58,980 --> 01:03:05,370
You continue here, each x
sub n is included if the
897
01:03:05,370 --> 01:03:09,920
experiment has not been
stopped before time n.
898
01:03:09,920 --> 01:03:15,360
So the experiments continue
under the event that the
899
01:03:15,360 --> 01:03:19,770
stopping time was greater than
or equal to the J. So the
900
01:03:19,770 --> 01:03:25,180
expected value of s of J is
equal to the expected value of
901
01:03:25,180 --> 01:03:30,520
this sum here, of x sub n times
the indicator function
902
01:03:30,520 --> 01:03:34,840
of J greater than or equal to
n, which is equal to the sum
903
01:03:34,840 --> 01:03:38,730
over n of the expected value
of x sub n times this
904
01:03:38,730 --> 01:03:39,980
indicator function.
905
01:03:43,100 --> 01:03:46,280
By this time I hope that most
of you are at least a little
906
01:03:46,280 --> 01:03:52,610
bit sensitive to interchanging
expectations and sums.
907
01:03:52,610 --> 01:03:56,730
And the notes and the problem
sets deal with that.
908
01:03:56,730 --> 01:03:59,400
That's the part of the proof
that I don't want to talk
909
01:03:59,400 --> 01:04:02,410
about here, because it isn't
really very interesting.
910
01:04:02,410 --> 01:04:08,610
It's just one of these typical
tedious analysis things.
911
01:04:08,610 --> 01:04:10,730
Most of the time
this is valid.
912
01:04:10,730 --> 01:04:14,470
We will see an example where
it isn't, so you'll see
913
01:04:14,470 --> 01:04:16,630
what's going on.
914
01:04:16,630 --> 01:04:20,060
The essence of the proof,
however, the interesting part
915
01:04:20,060 --> 01:04:25,690
of the proof is to show that xn
and this indicator function
916
01:04:25,690 --> 01:04:28,630
are independent random
variables.
917
01:04:28,630 --> 01:04:34,140
So you can think of that as a
separate limit, that the n-th
918
01:04:34,140 --> 01:04:40,020
trial and the indicator function
of J greater than or
919
01:04:40,020 --> 01:04:43,930
equal to n are independent
of each other.
920
01:04:43,930 --> 01:04:47,430
This seems a little bit weird,
and it seems a little bit
921
01:04:47,430 --> 01:04:51,770
weird because this includes
the event that
922
01:04:51,770 --> 01:04:54,030
J is equal to n.
923
01:04:54,030 --> 01:04:56,840
And the event that J is equal
to n is something that's
924
01:04:56,840 --> 01:05:05,040
decided on the basis of the
previous arrivals, or the
925
01:05:05,040 --> 01:05:07,440
previous what have you.
926
01:05:07,440 --> 01:05:11,130
It's highly influenced
by x sub n.
927
01:05:11,130 --> 01:05:14,950
So, x sub n is not independent
of the indicator
928
01:05:14,950 --> 01:05:19,240
function of J equals n.
929
01:05:19,240 --> 01:05:22,760
So this is strange.
930
01:05:22,760 --> 01:05:24,440
But we'll see how
this works out.
931
01:05:27,750 --> 01:05:35,810
Now, if we want to show that
x sub n and this indicator
932
01:05:35,810 --> 01:05:41,300
function are independent, the
thing that you do is note that
933
01:05:41,300 --> 01:05:46,220
the indicator function of J
greater than or equal to n,
934
01:05:46,220 --> 01:05:50,130
this is the set of events in
which J is greater than or
935
01:05:50,130 --> 01:05:51,240
equal to n.
936
01:05:51,240 --> 01:05:53,910
What's the complement
of that event?
937
01:05:53,910 --> 01:05:57,730
The complement of the event J
greater than or equal to n is
938
01:05:57,730 --> 01:06:00,230
the event J less than n.
939
01:06:00,230 --> 01:06:04,280
So this indicator function
is 1 minus
940
01:06:04,280 --> 01:06:06,110
this indicator function.
941
01:06:06,110 --> 01:06:09,890
J less than n is the complement
of the event J
942
01:06:09,890 --> 01:06:11,500
greater than or equal to n.
943
01:06:11,500 --> 01:06:14,140
When you perform the experiment
either this
944
01:06:14,140 --> 01:06:19,020
happens, in which case this is
1, or this happens, in which
945
01:06:19,020 --> 01:06:24,760
case this is 1, and if
this is 1, this is 0.
946
01:06:24,760 --> 01:06:27,450
If this is 1, this is 0.
947
01:06:27,450 --> 01:06:32,020
Also, the indicator function
and J less than n is a
948
01:06:32,020 --> 01:06:37,370
function of x1, x2, up
to x sub n minus 1.
949
01:06:37,370 --> 01:06:42,190
Let me spell that out a little
more because it looks a little
950
01:06:42,190 --> 01:06:45,020
strange the way it is.
951
01:06:45,020 --> 01:06:56,300
i of J less than n is equal to
i if the indicator random
952
01:06:56,300 --> 01:07:01,090
variable for J equals 1.
953
01:07:03,950 --> 01:07:08,640
Union with the indicator
random variable
954
01:07:08,640 --> 01:07:10,700
for J equals 2.
955
01:07:10,700 --> 01:07:19,910
Union indicator function
for J equals n minus 1.
956
01:07:19,910 --> 01:07:22,190
These are all disjoint events.
957
01:07:22,190 --> 01:07:24,590
You can't stop at two
different times.
958
01:07:24,590 --> 01:07:29,280
So if you stop before time n you
either stop at time 1, or
959
01:07:29,280 --> 01:07:34,710
you stop at time 2, or you
stop at time n minus 1.
960
01:07:37,540 --> 01:07:39,685
This event is independent.
961
01:07:43,740 --> 01:07:49,616
Well not independent, it
depends on x sub 1.
962
01:07:53,160 --> 01:07:58,550
This event depends
on x1 and x2.
963
01:07:58,550 --> 01:08:07,040
This event depends on x1
up to x sub n minus 1.
964
01:08:07,040 --> 01:08:12,320
OK, so this is what I mean when
I say that the event J
965
01:08:12,320 --> 01:08:18,550
less than n is determined by
x1 up to x of n minus 1,
966
01:08:18,550 --> 01:08:23,060
because each of these sub events
is determined by a sub
967
01:08:23,060 --> 01:08:24,390
event up there.
968
01:08:29,310 --> 01:08:33,939
Since we're looking at the
situation now where the x is
969
01:08:33,939 --> 01:08:44,510
our IID and this event is a
function of x1 x of n minus 1,
970
01:08:44,510 --> 01:08:47,090
it just depends on those
random variables.
971
01:08:47,090 --> 01:08:53,310
These random variables are
independent of x sub n so i of
972
01:08:53,310 --> 01:09:00,840
J less than n is independent of
x sub n and thus x sub n is
973
01:09:00,840 --> 01:09:05,359
independent of J greater
than or equal to n.
974
01:09:05,359 --> 01:09:08,350
Now, as I said, this is very
surprising, and it's very
975
01:09:08,350 --> 01:09:16,800
surprising because this event,
J greater than or equal to n,
976
01:09:16,800 --> 01:09:20,540
typically depends very
heavily on x sub n.
977
01:09:20,540 --> 01:09:24,930
It depends on J equals n
plus 1, and so forth.
978
01:09:24,930 --> 01:09:28,319
So it's a little paradoxical
and the resolution of the
979
01:09:28,319 --> 01:09:33,149
paradox is that given that J is
greater than or equal to n,
980
01:09:33,149 --> 01:09:38,990
in other words, given that you
haven't stopped before time n,
981
01:09:38,990 --> 01:09:43,779
the time at which you stop
is very dependent on the
982
01:09:43,779 --> 01:09:46,170
observations that you make.
983
01:09:46,170 --> 01:09:51,260
But whether you stopped before
time n or whether you haven't
984
01:09:51,260 --> 01:09:54,040
stopped before time n is
independent of x sub n.
985
01:09:57,760 --> 01:10:01,680
So it really is this
conditioning here that makes
986
01:10:01,680 --> 01:10:03,570
this a confusing issue.
987
01:10:03,570 --> 01:10:08,730
But whether or not J greater
than n occurs depends only on
988
01:10:08,730 --> 01:10:12,330
x1 up to x sub n minus 1.
989
01:10:12,330 --> 01:10:17,060
And with that it's easy enough
to finish the proof, it's just
990
01:10:17,060 --> 01:10:18,920
writing a few equations.
991
01:10:18,920 --> 01:10:23,320
The expected value of s sub J is
equal to the sum over n and
992
01:10:23,320 --> 01:10:27,440
the expected value of x sub n
times the indicator function
993
01:10:27,440 --> 01:10:30,290
of J greater than
or equal to n.
994
01:10:30,290 --> 01:10:33,590
We've just shown that these
are independent.
995
01:10:33,590 --> 01:10:36,870
This random variable is
independent of this random
996
01:10:36,870 --> 01:10:40,150
variable, that's independent of
the event J greater than or
997
01:10:40,150 --> 01:10:41,940
equal to n, so it's independent
998
01:10:41,940 --> 01:10:43,890
of the random variable.
999
01:10:43,890 --> 01:10:47,550
The indicator function is J
greater than or equal to n, so
1000
01:10:47,550 --> 01:10:49,700
we can write this way.
1001
01:10:49,700 --> 01:10:53,140
Now, when we have this
expression each of the x sub
1002
01:10:53,140 --> 01:10:58,650
n's are IID, so all of these are
the same quantity x-bar.
1003
01:10:58,650 --> 01:11:02,160
So since they're all the same
quantity x-bar we can just
1004
01:11:02,160 --> 01:11:04,550
take it outside of
the summation.
1005
01:11:04,550 --> 01:11:09,805
So we have x-bar times the sum
of the expected value of i of
1006
01:11:09,805 --> 01:11:13,100
J greater than or equal to n.
1007
01:11:13,100 --> 01:11:17,480
Now, the expected value of this
indicator function, this
1008
01:11:17,480 --> 01:11:22,800
indicator function is 1 when J
is greater than or equal to 1,
1009
01:11:22,800 --> 01:11:24,610
and it's 0 otherwise.
1010
01:11:24,610 --> 01:11:29,450
So the expected value of this
is just a probability that J
1011
01:11:29,450 --> 01:11:31,940
is greater than or equal to n.
1012
01:11:31,940 --> 01:11:38,100
So, this is equal to the
expected value of x times the
1013
01:11:38,100 --> 01:11:41,980
sum of the probabilities
that J is greater
1014
01:11:41,980 --> 01:11:43,230
than or equal to n.
1015
01:11:48,040 --> 01:11:51,560
This is really adding the
elements of the complimentary
1016
01:11:51,560 --> 01:11:55,770
distribution function for J, and
that gives us the expected
1017
01:11:55,770 --> 01:12:02,290
value of J. If finite or not
finite gives us this anyway.
1018
01:12:02,290 --> 01:12:06,515
So, this really is a proof
except for that one step.
1019
01:12:15,050 --> 01:12:20,700
Except for this interchange of
expectation and summation here
1020
01:12:20,700 --> 01:12:29,480
which is, well, we will get an
idea of what that has to do
1021
01:12:29,480 --> 01:12:30,970
with anything.
1022
01:12:30,970 --> 01:12:33,570
So let's look at an example.
1023
01:12:33,570 --> 01:12:36,270
Stop when you're ahead.
1024
01:12:36,270 --> 01:12:41,360
And stop when you're ahead
is a strategy for
1025
01:12:41,360 --> 01:12:45,180
gambling with coins.
1026
01:12:45,180 --> 01:12:47,440
You toss a coin with
probability of
1027
01:12:47,440 --> 01:12:48,980
heads equal to p.
1028
01:12:52,280 --> 01:12:55,020
And we want to look at all the
different cases for p, whether
1029
01:12:55,020 --> 01:12:59,860
p is fair or biased in your
favor or biased in the other
1030
01:12:59,860 --> 01:13:02,250
person's favor.
1031
01:13:02,250 --> 01:13:05,090
And the rule is that you
stop whenever you're
1032
01:13:05,090 --> 01:13:07,290
winnings reach one.
1033
01:13:07,290 --> 01:13:11,590
So you keep on dumbly gambling,
and gambling, and
1034
01:13:11,590 --> 01:13:15,150
gambling, until finally you get
to the point where you're
1035
01:13:15,150 --> 01:13:18,580
one ahead and when you're one
ahead you sigh a sigh of
1036
01:13:18,580 --> 01:13:23,140
relief and say, now my wife
won't be angry at me, or my
1037
01:13:23,140 --> 01:13:28,400
husband won't divorce me, or
something of that sort.
1038
01:13:28,400 --> 01:13:33,400
And you walk away swearing
never to gamble again.
1039
01:13:33,400 --> 01:13:37,990
Except, when you look at
this you say, aha!
1040
01:13:37,990 --> 01:13:42,400
If the game is fair, I am sure
to win, and after I win I
1041
01:13:42,400 --> 01:13:45,110
might as well play again,
because I'm sure to win again,
1042
01:13:45,110 --> 01:13:48,270
and I'm sure to win again, and
I'm sure to win again, and so
1043
01:13:48,270 --> 01:13:50,700
forth, which is all
very strange.
1044
01:13:50,700 --> 01:13:53,870
And we'll see why that
is in a little bit.
1045
01:13:53,870 --> 01:13:57,210
But first let's look at
the case where p is
1046
01:13:57,210 --> 01:13:59,480
greater than 1/2.
1047
01:13:59,480 --> 01:14:03,340
In other words, you have a
loaded coin and you've talked
1048
01:14:03,340 --> 01:14:08,270
somebody else, some poor sucker,
into playing with you,
1049
01:14:08,270 --> 01:14:10,650
and you've decided you're going
to stop the first time
1050
01:14:10,650 --> 01:14:12,420
you're ahead.
1051
01:14:12,420 --> 01:14:18,490
And because p is greater than
1/2, the expected value of
1052
01:14:18,490 --> 01:14:23,590
your gain each time you play
the game is greater than 0.
1053
01:14:23,590 --> 01:14:27,820
You're applying IID random
variables so that eventually
1054
01:14:27,820 --> 01:14:30,500
your winnings, if you kept
on playing forever,
1055
01:14:30,500 --> 01:14:32,360
would become humongous.
1056
01:14:32,360 --> 01:14:37,030
So at some point you have to be
1 ahead in this process of
1057
01:14:37,030 --> 01:14:40,300
getting to the point where
you're arbitrarily large
1058
01:14:40,300 --> 01:14:41,370
amounts ahead.
1059
01:14:41,370 --> 01:14:47,190
So with probability 1, you will
eventually become ahead.
1060
01:14:47,190 --> 01:14:52,670
We would like to know since we
know that you're going to be
1061
01:14:52,670 --> 01:14:56,030
ahead at some point, how
long has it going to
1062
01:14:56,030 --> 01:14:59,410
take you to get ahead?
1063
01:14:59,410 --> 01:15:02,320
We know that j has to be a
random variable in this case,
1064
01:15:02,320 --> 01:15:07,200
because we know you have to win
with probability 1 s sub J
1065
01:15:07,200 --> 01:15:09,820
equals 1 with probability 1.
1066
01:15:09,820 --> 01:15:15,190
Namely, your winnings up to the
time J. At the time J your
1067
01:15:15,190 --> 01:15:16,710
winnings are 1.
1068
01:15:16,710 --> 01:15:21,210
So s sub J is equal to 1, the
expected value of s sub J is
1069
01:15:21,210 --> 01:15:28,330
equal to 1, and Wald says that
the expected value of J, the
1070
01:15:28,330 --> 01:15:34,060
expected time you have to play,
is just 1 over x-bar.
1071
01:15:37,590 --> 01:15:38,840
Isn't that neat?
1072
01:15:41,470 --> 01:15:44,800
Which says it's 1
over 2p minus 1.
1073
01:15:44,800 --> 01:15:52,520
The expected value of x is p
plus minus 1 times 1 minus p
1074
01:15:52,520 --> 01:15:56,450
and if you look at that it's 2p
minus 1, which is positive
1075
01:15:56,450 --> 01:15:57,790
when p is bigger than 1/2.
1076
01:16:00,650 --> 01:16:06,200
After all the abstraction of
Wald's equality it's nice to
1077
01:16:06,200 --> 01:16:10,020
look at this and solve it in a
different way to see if we get
1078
01:16:10,020 --> 01:16:10,940
the same answer.
1079
01:16:10,940 --> 01:16:12,190
So we'll do that.
1080
01:16:20,430 --> 01:16:26,350
If J is equal to 1, that means
that on the first flip of the
1081
01:16:26,350 --> 01:16:31,780
coin you've got heads, you
became 1 up and you stopped.
1082
01:16:31,780 --> 01:16:36,440
So the experiment was over
at time 1 if the
1083
01:16:36,440 --> 01:16:39,070
first toss was a head.
1084
01:16:39,070 --> 01:16:43,350
So if the first toss was a tail,
on the other hand, what
1085
01:16:43,350 --> 01:16:48,030
has to happen in order
for you to win?
1086
01:16:48,030 --> 01:16:51,920
Well, you're sitting there at
minus 1 and you're going to
1087
01:16:51,920 --> 01:16:55,900
stop when you get to plus 1.
1088
01:16:55,900 --> 01:16:59,670
Every time you move up one or
down one, which means if
1089
01:16:59,670 --> 01:17:03,320
you're ever going to get from
minus 1 up to plus 1, you've
1090
01:17:03,320 --> 01:17:07,410
got to go through 0 in
order to get there.
1091
01:17:07,410 --> 01:17:11,410
So let's look at the expected
time that it takes to get to 0
1092
01:17:11,410 --> 01:17:14,820
for the first time, and then
we'll look at the expected
1093
01:17:14,820 --> 01:17:20,350
time that it takes to get from
0 to 1 for the first time.
1094
01:17:20,350 --> 01:17:24,960
Well, the expected time that it
takes to get from minus 1
1095
01:17:24,960 --> 01:17:29,560
to 0 is exactly the same as
the expected time that it
1096
01:17:29,560 --> 01:17:31,980
takes to get from 0 to 1.
1097
01:17:31,980 --> 01:17:39,700
So, the equation that you then
have is the expected value of
1098
01:17:39,700 --> 01:17:42,690
J is going to be 1.
1099
01:17:42,690 --> 01:17:46,930
This is the first step you have
to take anyway, and with
1100
01:17:46,930 --> 01:17:52,260
probability 1 minus p you went
to minus 1, and if you went to
1101
01:17:52,260 --> 01:17:57,965
minus 1 you still have J-bar
left to go to get to 0 again,
1102
01:17:57,965 --> 01:18:06,960
and another J-bar to get to 1,
so J-bar is equal to 1 plus
1103
01:18:06,960 --> 01:18:10,940
failing the first time it
takes 2J bar to get
1104
01:18:10,940 --> 01:18:12,720
back up to plus 1.
1105
01:18:12,720 --> 01:18:19,870
If you analyze this equation,
J-bar is equal to 1 plus
1106
01:18:19,870 --> 01:18:26,490
2J-bar minus 2Jp-bar, you work
that out and it's just 1 over
1107
01:18:26,490 --> 01:18:27,980
2p minus 1.
1108
01:18:27,980 --> 01:18:31,850
So fortunately we get the same
answer as we got with the
1109
01:18:31,850 --> 01:18:33,100
Wald's equality.
1110
01:18:36,580 --> 01:18:39,010
Let's go one again.
1111
01:18:39,010 --> 01:18:43,880
Let's look at what happens if
you go to the casino and you
1112
01:18:43,880 --> 01:18:50,130
play a game where you
win $1 or lose $1.
1113
01:18:50,130 --> 01:18:55,350
But the probability in casinos
is somehow always tilted to be
1114
01:18:55,350 --> 01:18:56,600
a little less.
1115
01:19:00,790 --> 01:19:06,080
If you play for red or black in
roulette, what happens is
1116
01:19:06,080 --> 01:19:11,440
that red or black are equally
likely, but there's the 0 and
1117
01:19:11,440 --> 01:19:15,090
double 0 where the house
always wins.
1118
01:19:15,090 --> 01:19:18,470
So your probability of winning
is always just a little less
1119
01:19:18,470 --> 01:19:23,990
than 1/2, so that's the
situation we have here.
1120
01:19:23,990 --> 01:19:27,450
It's still possible
to win and stop.
1121
01:19:27,450 --> 01:19:32,610
Like if you win on the
first toss, then
1122
01:19:32,610 --> 01:19:34,880
you're going to stop.
1123
01:19:34,880 --> 01:19:38,340
So J equals 1 with
probability p.
1124
01:19:38,340 --> 01:19:42,780
J equals 3 if you lose,
and then you win,
1125
01:19:42,780 --> 01:19:43,930
and then you win.
1126
01:19:43,930 --> 01:19:48,660
So J equals 3 with probability
p squared times 1 minus b.
1127
01:19:48,660 --> 01:19:52,680
And there are lots of other
situations you can look at J
1128
01:19:52,680 --> 01:19:57,150
equals 5 and see when you win
then, and you can count all
1129
01:19:57,150 --> 01:19:58,990
these things up.
1130
01:19:58,990 --> 01:20:04,730
Let's try to find a simple
way of counting them up.
1131
01:20:04,730 --> 01:20:08,880
Let's let theta be the
probability that J is less
1132
01:20:08,880 --> 01:20:10,960
than infinity.
1133
01:20:10,960 --> 01:20:13,480
In other words, theta is the
probability that you're ever
1134
01:20:13,480 --> 01:20:15,010
going to stop.
1135
01:20:15,010 --> 01:20:18,610
Sometimes in this game you won't
ever stop because you're
1136
01:20:18,610 --> 01:20:21,820
going to lose for while, and
as soon as you lose a fair
1137
01:20:21,820 --> 01:20:26,090
amount you're very unlikely to
ever recover because you're
1138
01:20:26,090 --> 01:20:27,640
drifting South all the time.
1139
01:20:40,970 --> 01:20:43,580
Since you're drifting South,
theta is going to
1140
01:20:43,580 --> 01:20:46,750
be less than 1.
1141
01:20:46,750 --> 01:20:49,050
So how do we analyze this?
1142
01:20:49,050 --> 01:20:53,980
Same way as before, given that
J is greater than 1, in other
1143
01:20:53,980 --> 01:20:57,970
words, given that you lose on
the first trial, the event J
1144
01:20:57,970 --> 01:21:04,310
less than infinity requires that
at some time you go from
1145
01:21:04,310 --> 01:21:06,580
minus 1 back up to 1.
1146
01:21:06,580 --> 01:21:12,170
In other words, there's some
time m at which s sub m minus
1147
01:21:12,170 --> 01:21:14,330
s sub 1 is equal to 1.
1148
01:21:14,330 --> 01:21:17,760
s sub m is the time at which
you first get from minus 1
1149
01:21:17,760 --> 01:21:21,860
back up to 0, and then there's
some time that it takes to get
1150
01:21:21,860 --> 01:21:25,530
from 0 up to 1, and
there's some
1151
01:21:25,530 --> 01:21:27,820
probability that ever happened.
1152
01:21:27,820 --> 01:21:33,400
The probability that you ever
reach 0 is equal to theta, the
1153
01:21:33,400 --> 01:21:38,720
probability you ever get from 0
up to plus 1 is theta again.
1154
01:21:38,720 --> 01:21:42,740
So theta is equal to p, which
is the probability that you
1155
01:21:42,740 --> 01:21:46,640
win right away, plus probability
you lose on the
1156
01:21:46,640 --> 01:21:53,990
first toss, and you ever get
from minus 1 to 0 and you ever
1157
01:21:53,990 --> 01:21:55,650
get from 0 to 1.
1158
01:21:55,650 --> 01:21:59,500
So theta equals p plus 1 minus
p times theta squared.
1159
01:21:59,500 --> 01:22:01,010
There are two solutions
to this.
1160
01:22:04,420 --> 01:22:09,120
One is that theta it is p over
1 minus p and the other is
1161
01:22:09,120 --> 01:22:13,230
theta equals 1, which we
know is impossible.
1162
01:22:13,230 --> 01:22:16,610
And thus J is defective,
and Wald's equation is
1163
01:22:16,610 --> 01:22:19,240
inapplicable.
1164
01:22:19,240 --> 01:22:23,600
But the solution that we have,
which is useful to know, is
1165
01:22:23,600 --> 01:22:28,970
that your probability of ever
winning is p over 1 minus p.
1166
01:22:28,970 --> 01:22:30,220
Nice to know.
1167
01:22:33,130 --> 01:22:37,280
That's what happens when
you go to a casino.
1168
01:22:37,280 --> 01:22:41,570
Finally, let's consider the
interesting case which is p
1169
01:22:41,570 --> 01:22:43,470
equals 1/2.
1170
01:22:43,470 --> 01:22:47,850
The limit as p approaches 1/2
from below, probability that J
1171
01:22:47,850 --> 01:23:02,840
less than infinity goes from
p over 1 minus p goes to 1.
1172
01:23:02,840 --> 01:23:07,830
In other words, in a fair game,
and this is not obvious,
1173
01:23:07,830 --> 01:23:11,350
there's probability 1 that you
will eventually get to the
1174
01:23:11,350 --> 01:23:16,270
point where s sub
n is equal to 1.
1175
01:23:16,270 --> 01:23:23,550
You can sort of see this if you
view this in terms of the
1176
01:23:23,550 --> 01:23:24,840
central limit theorem.
1177
01:23:27,850 --> 01:23:30,910
And we looked pretty hard at the
central limit theorem in
1178
01:23:30,910 --> 01:23:33,910
the case of these binary
experiments.
1179
01:23:33,910 --> 01:23:42,170
As n gets bigger and bigger, the
standard deviation of this
1180
01:23:42,170 --> 01:23:45,720
grid grows as the square
root of n.
1181
01:23:45,720 --> 01:23:48,900
Now the standard deviation is
growing as the square root of
1182
01:23:48,900 --> 01:23:53,675
n and over a period of n
you're wobbling around.
1183
01:23:56,560 --> 01:24:00,050
You can see that it doesn't
make any sense to have the
1184
01:24:00,050 --> 01:24:03,630
standard deviation growing with
the square root of n if
1185
01:24:03,630 --> 01:24:06,820
you don't at some point go from
the positive half down to
1186
01:24:06,820 --> 01:24:09,150
the negative half.
1187
01:24:09,150 --> 01:24:12,350
If you always stay in the
positive half after a certain
1188
01:24:12,350 --> 01:24:16,920
point, then that much time later
you're always going to
1189
01:24:16,920 --> 01:24:22,070
stay in the same half again, and
that doesn't work in terms
1190
01:24:22,070 --> 01:24:24,600
of the square root of n.
1191
01:24:24,600 --> 01:24:28,850
So you can convince yourself on
the basis of a very tedious
1192
01:24:28,850 --> 01:24:31,670
argument what this very simple
argument shows you.
1193
01:24:35,010 --> 01:24:38,430
And as soon as we start studying
Markov chains with
1194
01:24:38,430 --> 01:24:41,710
uncountably infinite number of
states, we're going to analyze
1195
01:24:41,710 --> 01:24:43,470
this case again.
1196
01:24:43,470 --> 01:24:46,720
In other words, this is a very
interesting situation because
1197
01:24:46,720 --> 01:24:48,750
something very interesting
happens here and we're going
1198
01:24:48,750 --> 01:24:51,880
to analyze it at least
three different ways.
1199
01:24:51,880 --> 01:24:54,560
So if you don't believe
this way, you will
1200
01:24:54,560 --> 01:24:57,780
believe the other way.
1201
01:24:57,780 --> 01:25:04,351
And what happens as p approaches
1/2 from above you
1202
01:25:04,351 --> 01:25:08,040
know the expected value of
J approaches infinity.
1203
01:25:08,040 --> 01:25:12,020
So what's going to happen at p
equals 1/2 as the expected
1204
01:25:12,020 --> 01:25:17,500
time for you to win goes to
infinity, the probability of
1205
01:25:17,500 --> 01:25:23,460
your winning goes to 1, and
Wald's equality does not hold
1206
01:25:23,460 --> 01:25:28,800
because the expected value of x
is equal to 0, and you look
1207
01:25:28,800 --> 01:25:33,050
at 0 times infinity and it
doesn't tell you anything.
1208
01:25:33,050 --> 01:25:39,520
The expected value of s sub J s
sub n at every value of n is
1209
01:25:39,520 --> 01:25:44,610
0, but s sub n when the
experiment ends is equal to 1.
1210
01:25:44,610 --> 01:25:49,760
So there's something very funny
going on here and if you
1211
01:25:49,760 --> 01:25:52,800
think in practical terms though,
it takes an infinite
1212
01:25:52,800 --> 01:25:58,250
time to make your $1.00 and more
important, it requires
1213
01:25:58,250 --> 01:26:00,610
access to an infinite capital.
1214
01:26:00,610 --> 01:26:04,570
We will see that if you limit
your capital, if you limit how
1215
01:26:04,570 --> 01:26:09,050
far South you can go, then in
fact you don't win with
1216
01:26:09,050 --> 01:26:13,720
probability 1, because you
can go there also.
1217
01:26:13,720 --> 01:26:16,500
Next time we will apply this
to renewal processes.