WEBVTT
00:00:00.530 --> 00:00:02.960
The following content is
provided under a Creative
00:00:02.960 --> 00:00:04.370
Commons license.
00:00:04.370 --> 00:00:07.410
Your support will help MIT
OpenCourseWare continue to
00:00:07.410 --> 00:00:11.060
offer high quality educational
resources for free.
00:00:11.060 --> 00:00:13.960
To make a donation or view
additional materials from
00:00:13.960 --> 00:00:19.790
hundreds of MIT courses, visit
MIT OpenCourseWare at
00:00:19.790 --> 00:00:21.040
ocw.mit.edu.
00:00:22.600 --> 00:00:27.030
PROFESSOR: OK, today we are
moving on with renewal
00:00:27.030 --> 00:00:39.150
processes, and we're going to
finish up talking a little
00:00:39.150 --> 00:00:44.080
more about residual life, and
we're going to generalize it
00:00:44.080 --> 00:00:49.440
to time averages for arbitrary
renewal processes.
00:00:49.440 --> 00:00:53.590
And then finally we're going
into the topic of stopping
00:00:53.590 --> 00:00:56.660
trials, stopping times, optional
stopping times--
00:00:56.660 --> 00:00:59.290
people call them different
things--
00:00:59.290 --> 00:01:00.540
and Wald's equality.
00:01:03.660 --> 00:01:07.140
This is, for some reason or
other, normally a very tricky
00:01:07.140 --> 00:01:09.840
thing to understand.
00:01:09.840 --> 00:01:15.140
I think finally after many
years I understand it.
00:01:15.140 --> 00:01:18.700
And I hope that I can make it
easy for you to understand it,
00:01:18.700 --> 00:01:23.940
so some of the confusions that
often come in won't come in.
00:01:23.940 --> 00:01:27.120
And then we're going to end up
talking about stopping when
00:01:27.120 --> 00:01:33.590
you're ahead, which is a
gambling strategy which says
00:01:33.590 --> 00:01:37.940
if you're playing a fair game
you play until you're $1.00
00:01:37.940 --> 00:01:40.440
ahead, and then you stop.
00:01:40.440 --> 00:01:42.670
And you show that with
probability 1 you will
00:01:42.670 --> 00:01:46.700
eventually become $1.00 ahead,
so you have a way to beat the
00:01:46.700 --> 00:01:50.100
bank, or beat the casino.
00:01:50.100 --> 00:01:53.500
But of course casinos don't give
you even odds, but you
00:01:53.500 --> 00:01:58.550
have a way to win any time
you get even odds.
00:01:58.550 --> 00:02:00.510
We'll find that there's
something wrong with that, but
00:02:00.510 --> 00:02:06.600
it is an interesting topic
which illustrates Wald's
00:02:06.600 --> 00:02:09.729
equality in an interesting
way.
00:02:09.729 --> 00:02:12.930
So let's start out by reviewing
a little bit.
00:02:17.270 --> 00:02:21.630
We first talked last time
about convergence with
00:02:21.630 --> 00:02:24.880
probability 1, and we weren't
talking about the strong law
00:02:24.880 --> 00:02:28.500
then, we were just talking about
a sequence of random
00:02:28.500 --> 00:02:32.760
variables and what it means
for them to converge with
00:02:32.760 --> 00:02:34.480
probability 1.
00:02:34.480 --> 00:02:40.440
And the theorem is that if you
have a sequence of random
00:02:40.440 --> 00:02:46.260
variables, z sub n, they
converge to some number alpha.
00:02:46.260 --> 00:02:48.540
This is slightly different than
the theorem we stated
00:02:48.540 --> 00:02:53.640
last time, but it's trivially
the same.
00:02:53.640 --> 00:02:56.810
In other words, it's the
probability of the set of
00:02:56.810 --> 00:02:58.000
sample points.
00:02:58.000 --> 00:03:01.760
A sample point, now, is
something which runs from time
00:03:01.760 --> 00:03:03.800
0 to time infinity.
00:03:03.800 --> 00:03:07.670
It's something that covers the
whole process, covers anything
00:03:07.670 --> 00:03:09.670
else you might want to
talk about also.
00:03:09.670 --> 00:03:15.460
It's what you do when you go
from a real situation to a
00:03:15.460 --> 00:03:19.610
model, and mathematically when
you go to a model you put in
00:03:19.610 --> 00:03:20.670
everything you're going to be
00:03:20.670 --> 00:03:23.270
interested in at the beginning.
00:03:23.270 --> 00:03:30.260
So, it converges to some alpha
probability 1 if the set of
00:03:30.260 --> 00:03:34.520
sequences for which the
statement holds has
00:03:34.520 --> 00:03:40.350
probability 1, and this
statement is that the sample
00:03:40.350 --> 00:03:43.050
paths converge to alpha.
00:03:43.050 --> 00:03:46.300
The set of sample paths has
converged to alpha as
00:03:46.300 --> 00:03:47.280
probability 1.
00:03:47.280 --> 00:03:50.640
So, it's a statement not like
most of the statements in
00:03:50.640 --> 00:03:54.200
probability where you start
out talking about finite
00:03:54.200 --> 00:03:56.750
length, talking about
probabilities, and
00:03:56.750 --> 00:03:59.110
expectation, and so forth.
00:03:59.110 --> 00:04:02.260
And then you go to the limit
with these things you're used
00:04:02.260 --> 00:04:05.220
to talking about, which are
easier to deal with.
00:04:05.220 --> 00:04:09.060
Here it's a statement
about sample paths
00:04:09.060 --> 00:04:11.280
over the whole sequence.
00:04:11.280 --> 00:04:14.580
You break them into
two categories.
00:04:14.580 --> 00:04:17.480
Implicitly we have shown that
when you break them into two
00:04:17.480 --> 00:04:22.220
categories those two categories
in fact have
00:04:22.220 --> 00:04:23.060
probabilities.
00:04:23.060 --> 00:04:28.540
They are events, and one of
them has probability 1.
00:04:28.540 --> 00:04:34.750
For renewal processes, if we
have a renewal process we can
00:04:34.750 --> 00:04:40.030
use this theorem to talk about
the sample averages.
00:04:40.030 --> 00:04:45.970
In other words, in a renewal
process we have a sequence of
00:04:45.970 --> 00:04:51.800
IID random variables, x1,
x2, x3, and so forth.
00:04:51.800 --> 00:04:56.410
We have the sample sums, the sum
of the time 1, the sum of
00:04:56.410 --> 00:05:01.920
the time 2, sum of the
time 3, and so forth.
00:05:01.920 --> 00:05:06.330
And the strong law of large
numbers, which comes pretty
00:05:06.330 --> 00:05:10.540
much directly out of this
convergence of probability 1
00:05:10.540 --> 00:05:16.190
theorem, says that the
probability is the set of
00:05:16.190 --> 00:05:24.530
sample points for which the
limit of the sample average,
00:05:24.530 --> 00:05:26.990
namely sn of omega over n.
00:05:26.990 --> 00:05:31.500
You take the sample average
over all lengths for one
00:05:31.500 --> 00:05:32.850
sample point.
00:05:32.850 --> 00:05:35.380
That goes to a limit, that's
this convergence with
00:05:35.380 --> 00:05:40.180
probability 1, the probability
that you get convergence is
00:05:40.180 --> 00:05:45.740
equal to 1, which is what with
probability 1 means.
00:05:56.710 --> 00:05:58.480
Then we use the theorem
on the top.
00:06:03.700 --> 00:06:05.360
Let's reset a little bit.
00:06:05.360 --> 00:06:08.690
This theorem on the top is more
than just a theorem about
00:06:08.690 --> 00:06:12.280
convergence with
probability 1.
00:06:12.280 --> 00:06:15.400
It's the reason why convergence
with probability 1
00:06:15.400 --> 00:06:17.110
is so useful.
00:06:17.110 --> 00:06:22.400
It says if you have some
function f of x, that function
00:06:22.400 --> 00:06:27.950
is continuous at this
point alpha.
00:06:27.950 --> 00:06:32.020
You have this limit here, then
the probability of the set of
00:06:32.020 --> 00:06:36.440
omega for which the function
applied to each of these
00:06:36.440 --> 00:06:44.318
sample points as equal
to alpha is also one.
00:06:44.318 --> 00:06:46.640
AUDIENCE: [INAUDIBLE].
00:06:46.640 --> 00:06:48.230
PROFESSOR: It should
be f of alpha.
00:06:58.680 --> 00:06:59.940
Yes, absolutely.
00:06:59.940 --> 00:07:01.190
Sorry about that.
00:07:11.110 --> 00:07:14.340
This is f of alpha
right there.
00:07:14.340 --> 00:07:16.610
So that probability
is equal to 1.
00:07:19.540 --> 00:07:22.130
I'm finally getting my bearings
back on this.
00:07:25.910 --> 00:07:31.550
So if I use this theorem here,
where I use f of x as equal to
00:07:31.550 --> 00:07:37.810
1 over x, then what it says for
a renewal process, which
00:07:37.810 --> 00:07:44.160
has inter-renewals x1, x2, and
so forth, with a inter-renewal
00:07:44.160 --> 00:07:49.920
process the expected
inter-renewal time, which
00:07:49.920 --> 00:07:52.830
might be infinite in general,
but it has to
00:07:52.830 --> 00:07:54.730
be bigger than 0.
00:07:54.730 --> 00:08:02.960
It has to be bigger than 0
because renewals in 0 time are
00:08:02.960 --> 00:08:05.890
not possible by definition.
00:08:05.890 --> 00:08:10.670
And you've done a problem in the
homework, or you will do a
00:08:10.670 --> 00:08:13.930
problem in the homework, whether
you've done it or not,
00:08:13.930 --> 00:08:19.042
I don't know, but when you do
it, it will in fact show that
00:08:19.042 --> 00:08:23.390
the expected value of x has
to be greater than 0.
00:08:23.390 --> 00:08:26.930
We're also assuming that it's
less than infinity.
00:08:26.930 --> 00:08:35.039
So, at that point we can use
this theorem which says that
00:08:35.039 --> 00:08:41.260
when you look at, not sn of
omega over n, but the inverse
00:08:41.260 --> 00:08:45.850
of that, n over sn of omega, the
limit of that is going to
00:08:45.850 --> 00:08:52.730
be equal to f of x-bar which
is 1 over x-bar.
00:08:52.730 --> 00:08:56.220
So we have this limit here
which is equal to 1 with
00:08:56.220 --> 00:08:58.230
probability 1.
00:08:58.230 --> 00:09:03.610
This is almost the strong law
for renewal processes, and the
00:09:03.610 --> 00:09:10.560
reason why it's not takes a
couple of pages in the text.
00:09:10.560 --> 00:09:15.040
What I'm going to try to argue
here is that the couple of
00:09:15.040 --> 00:09:24.880
pages, while you need to do
mathematically, you can see
00:09:24.880 --> 00:09:27.240
why when you do it you
can see what answer
00:09:27.240 --> 00:09:28.440
you're going to get.
00:09:28.440 --> 00:09:30.290
And the argument is
the following.
00:09:30.290 --> 00:09:32.910
We talked about this last
time a little bit.
00:09:32.910 --> 00:09:38.420
If you take an arbitrary time t
then you look at n of t for
00:09:38.420 --> 00:09:42.860
that value of t, that's the
number of arrivals you've had
00:09:42.860 --> 00:09:46.370
in this renewal process
up until time t.
00:09:46.370 --> 00:09:50.460
And then you compare this point
down here, which is the
00:09:50.460 --> 00:09:55.280
time of the last arrival before
time t, and this point
00:09:55.280 --> 00:10:00.320
here, which is the time of the
next arrival after time t, and
00:10:00.320 --> 00:10:04.390
you look at the slopes
here, this slope here
00:10:04.390 --> 00:10:07.090
is n of t over t.
00:10:07.090 --> 00:10:08.520
That's the thing we're
interested in.
00:10:15.380 --> 00:10:18.520
That expression there is the
time average of the number of
00:10:18.520 --> 00:10:23.830
renewals within time t, and what
we're interested in is
00:10:23.830 --> 00:10:27.210
what this becomes as
t goes to infinity.
00:10:27.210 --> 00:10:31.920
That's squeezed between this
slope here, which is n of t
00:10:31.920 --> 00:10:35.240
over s sub n of t.
00:10:35.240 --> 00:10:38.760
This is s sub n of t.
00:10:38.760 --> 00:10:45.420
And this point here is n of t,
and this lower band here,
00:10:45.420 --> 00:10:50.680
which is n of t divided by
s sub n of t plus 1.
00:10:50.680 --> 00:10:57.480
Now, all this argument is
devoted to proving two things,
00:10:57.480 --> 00:11:01.040
which are almost obvious and
therefore I'm just going to
00:11:01.040 --> 00:11:03.350
wave my hands about it.
00:11:03.350 --> 00:11:08.450
What this says is when t gets
very, very large, n of t
00:11:08.450 --> 00:11:10.140
becomes very, very large also.
00:11:12.770 --> 00:11:16.830
When n of t becomes very, very
large, the difference between
00:11:16.830 --> 00:11:20.250
this slope and this slope
doesn't make a hill of beans
00:11:20.250 --> 00:11:23.670
worth of difference, and
therefore those two, that
00:11:23.670 --> 00:11:27.320
upper band and that lower
band become the same.
00:11:27.320 --> 00:11:33.180
So with the argument as t gets
large, n of t gets large, and
00:11:33.180 --> 00:11:36.450
these two slopes become the
same, you suddenly have the
00:11:36.450 --> 00:11:44.900
statement that if you have
a renewal process and the
00:11:44.900 --> 00:11:48.830
inter-renewal time is less than
infinity, it has to be
00:11:48.830 --> 00:11:50.520
greater than 0.
00:11:50.520 --> 00:11:57.950
Then on a sample path basis the
sample paths, the number
00:11:57.950 --> 00:12:04.960
of arrivals per unit time,
converges to 1 over x-bar.
00:12:04.960 --> 00:12:09.370
That's what this previous
picture was saying to you.
00:12:09.370 --> 00:12:19.210
It was saying that n over t over
t, which is the number of
00:12:19.210 --> 00:12:24.300
arrivals per unit time t, as t
becomes very, very large on a
00:12:24.300 --> 00:12:28.170
sample path basis that becomes
equal to 1 over x-bar.
00:12:33.700 --> 00:12:38.190
So, it says that the rate of
renewals over the infinite
00:12:38.190 --> 00:12:43.330
time horizon is 1 over x-bar
with probability 1.
00:12:43.330 --> 00:12:46.530
And what probability 1 statement
again is saying that
00:12:46.530 --> 00:12:50.090
the set of sample paths for
which this is true has
00:12:50.090 --> 00:12:51.960
probability 1.
00:12:51.960 --> 00:12:55.430
This also implies the weak law
for renewals, which says that
00:12:55.430 --> 00:12:59.420
the limit as t approaches
infinity of the probability
00:12:59.420 --> 00:13:03.970
that n of t over t minus 1 over
x-bar is greater than
00:13:03.970 --> 00:13:07.520
epsilon goes to 0.
00:13:07.520 --> 00:13:10.180
This, surprisingly enough,
is a statement that
00:13:10.180 --> 00:13:12.460
you hardly ever see.
00:13:12.460 --> 00:13:16.330
You see a bunch of other renewal
theorems, which this
00:13:16.330 --> 00:13:20.180
theorem up here is probably
the most important.
00:13:20.180 --> 00:13:23.500
There's something called an
elementary renewal theorem,
00:13:23.500 --> 00:13:26.610
which is sort of a trivial
statement but
00:13:26.610 --> 00:13:28.860
kind of hard to prove.
00:13:28.860 --> 00:13:31.840
And there's something called
Blackwell's theorem, which
00:13:31.840 --> 00:13:35.760
talks about a local period of
time after time gets very,
00:13:35.760 --> 00:13:36.400
very large.
00:13:36.400 --> 00:13:40.640
It's talking about the number
of renewals in some tiny
00:13:40.640 --> 00:13:46.700
interval and it's proving things
about stationarity over
00:13:46.700 --> 00:13:48.830
that local time interval.
00:13:48.830 --> 00:13:53.830
And we'll talk about that later,
but this is sort of a
00:13:53.830 --> 00:14:00.000
nice result, and that's
what it is.
00:14:00.000 --> 00:14:03.370
Let's review residual
life a little bit.
00:14:03.370 --> 00:14:09.870
The residual life of a renewal
process at time t, the
00:14:09.870 --> 00:14:14.560
residual life is the time you
have to wait until the next
00:14:14.560 --> 00:14:17.480
renewal, until the
next arrival.
00:14:17.480 --> 00:14:21.990
So residual life means,
how long does it take?
00:14:21.990 --> 00:14:24.710
If you think of these renewals
as something that comes along
00:14:24.710 --> 00:14:30.370
and kills everybody, then this
residual life is the amount of
00:14:30.370 --> 00:14:33.230
life that people still
have left.
00:14:33.230 --> 00:14:37.770
So, if you look at what that
is for a particular sample
00:14:37.770 --> 00:14:41.080
path, I have a particular
sample path drawn here,
00:14:41.080 --> 00:14:44.080
completely arbitrary.
00:14:44.080 --> 00:14:48.490
When I look at the first arrival
time here, s1 of
00:14:48.490 --> 00:14:52.410
omega, and I look at all of
the time before that, the
00:14:52.410 --> 00:14:57.350
amount of time you have to
wait until this arrival.
00:14:57.350 --> 00:15:01.650
Remember this is one
sample point.
00:15:04.480 --> 00:15:08.690
We're not thinking in terms of
an observer who comes in and
00:15:08.690 --> 00:15:11.410
doesn't know what's going
to happen in the future.
00:15:11.410 --> 00:15:12.990
We're looking at one
sample point.
00:15:12.990 --> 00:15:14.920
We know the whole
sample point.
00:15:14.920 --> 00:15:18.356
And what we're doing we know
this, we know this, we know
00:15:18.356 --> 00:15:24.610
this, and all we're doing here
saying for each time t along
00:15:24.610 --> 00:15:29.120
this infinite path we're just
plotting how long it is until
00:15:29.120 --> 00:15:30.620
the next arrival.
00:15:30.620 --> 00:15:34.260
And that's just a line with
slope minus 1 until the next
00:15:34.260 --> 00:15:35.660
arrival occurs.
00:15:35.660 --> 00:15:38.860
After the next arrival and you
start waiting for the arrival
00:15:38.860 --> 00:15:41.640
after that, after that
when you wait for the
00:15:41.640 --> 00:15:43.220
arrival after that.
00:15:43.220 --> 00:15:49.570
So on a sample path basis, the
residual life is just this
00:15:49.570 --> 00:15:55.580
sequence of isosceles
triangles here.
00:15:55.580 --> 00:16:01.490
So we looked at that, we said,
if we look at the time from n
00:16:01.490 --> 00:16:07.950
equals 1 up until the number
of arrivals at time t, the
00:16:07.950 --> 00:16:13.240
number of triangles we
have is n of t omega.
00:16:13.240 --> 00:16:20.130
The size of the area of each
triangle is xi squared over 2.
00:16:24.830 --> 00:16:28.610
The size of each triangle is xi
squared over 2, and when we
00:16:28.610 --> 00:16:31.800
get all done we want to take the
average of it so we divide
00:16:31.800 --> 00:16:33.620
each of these by t.
00:16:33.620 --> 00:16:37.570
This quantity here is less than
or equal to the integral
00:16:37.570 --> 00:16:43.750
of y of t over t, and that's
less than or equal to the same
00:16:43.750 --> 00:16:48.700
sum where if we look at a
particular time of t this sum
00:16:48.700 --> 00:16:52.500
here is summing everything
up to this point.
00:16:52.500 --> 00:16:57.500
This sum here is summing
everything up to this point.
00:16:57.500 --> 00:17:01.710
And as t goes to infinity, this
one little triangle in
00:17:01.710 --> 00:17:05.410
here, even though this is the
biggest triangle there, it
00:17:05.410 --> 00:17:07.010
still doesn't make
any difference.
00:17:07.010 --> 00:17:09.790
As t goes to infinity,
that washes out.
00:17:09.790 --> 00:17:12.410
You have to show that, of
course, but that's what we
00:17:12.410 --> 00:17:13.660
showed last time.
00:17:16.079 --> 00:17:22.119
When we go to the limit we find
that this time average
00:17:22.119 --> 00:17:26.450
residual life is equal to the
expected value of x squared
00:17:26.450 --> 00:17:29.150
over twice the main effect.
00:17:29.150 --> 00:17:31.980
Now, this can't be infinite.
00:17:31.980 --> 00:17:38.710
You can have a finite expected
in a renewal time and you can
00:17:38.710 --> 00:17:42.500
still have an infinite
second moment.
00:17:42.500 --> 00:17:49.570
If you look at the example we
talked about last time where
00:17:49.570 --> 00:17:55.250
the inter-renewal time at two
possible values, either
00:17:55.250 --> 00:17:59.100
epsilon or 1 over epsilon, in
other words, it was either
00:17:59.100 --> 00:18:04.280
enormous or it was very, very
small, what we found out there
00:18:04.280 --> 00:18:07.090
was that some of these
inter-renewal intervals, a
00:18:07.090 --> 00:18:10.540
very small fraction of them, but
a very small fraction of
00:18:10.540 --> 00:18:14.570
the inter-renewal intervals
were enormously large.
00:18:14.570 --> 00:18:24.150
And because they were enormously
large, the time
00:18:24.150 --> 00:18:29.510
average residual life turned
out to be enormous also.
00:18:29.510 --> 00:18:32.000
You think of what happens
as epsilon
00:18:32.000 --> 00:18:35.100
get smaller and smaller.
00:18:35.100 --> 00:18:40.600
You can see intuitively why it
makes sense that when you have
00:18:40.600 --> 00:18:46.910
humongously long inter-renewal
times, and you have this x
00:18:46.910 --> 00:18:52.240
squared that occurs, because of
this triangle here, I think
00:18:52.240 --> 00:18:57.150
is possible to see why this
quantity here can't become
00:18:57.150 --> 00:19:03.010
infinite if you had this
situation of a very long
00:19:03.010 --> 00:19:06.070
tailed distributions for
the inter-renewal time.
00:19:06.070 --> 00:19:12.320
You have enormously long
residual waits then, and you
00:19:12.320 --> 00:19:19.240
have them with high probability,
because if you
00:19:19.240 --> 00:19:26.060
come into this process at some
arbitrary time you're somewhat
00:19:26.060 --> 00:19:29.770
more likely to wind up in one of
these large intervals than
00:19:29.770 --> 00:19:33.500
one of the small intervals, and
that's what causes all the
00:19:33.500 --> 00:19:34.750
trouble here.
00:19:38.250 --> 00:19:40.600
There are similar
examples here.
00:19:40.600 --> 00:19:47.360
You can look at the age of a
process, z of t is defined as
00:19:47.360 --> 00:19:53.160
the interval between t and
the previous arrival.
00:19:53.160 --> 00:19:59.240
So, if we look at sum time t,
the age at that point is the
00:19:59.240 --> 00:20:03.090
amount of time back to the
previous arrival that goes up
00:20:03.090 --> 00:20:07.760
like a triangle, drops as soon
as we get the next arrival.
00:20:07.760 --> 00:20:08.940
It's exactly the same.
00:20:08.940 --> 00:20:11.850
It's just the same thing
looking at it backwards
00:20:11.850 --> 00:20:17.910
instead of forwards, so the
answer is the same also.
00:20:17.910 --> 00:20:22.240
If you look at something called
duration, the question
00:20:22.240 --> 00:20:27.760
here is at a particular
time t.
00:20:27.760 --> 00:20:31.390
If we take the difference
between the next arrival and
00:20:31.390 --> 00:20:36.260
the previous arrival, that's
called the duration of time t.
00:20:36.260 --> 00:20:37.650
How big is that?
00:20:37.650 --> 00:20:41.590
Well, obviously this is the same
problem as that, and the
00:20:41.590 --> 00:20:45.270
residual life and in fact,
that's exactly the sum of the
00:20:45.270 --> 00:20:47.760
residual life and the age.
00:20:47.760 --> 00:20:51.990
So, it's not surprising that
the time average--
00:20:51.990 --> 00:20:54.180
oh my.
00:20:54.180 --> 00:20:55.990
There should be a
2 down there.
00:21:31.340 --> 00:21:35.200
So it's exactly the same
situation as we had before
00:21:35.200 --> 00:21:38.290
when we were talking about
residual life.
00:21:41.250 --> 00:21:44.380
Now we'd like to generalize
this, and I hope the
00:21:44.380 --> 00:21:49.860
generalization is almost
obvious at this point.
00:21:49.860 --> 00:21:55.650
These three quantities here are
all examples of assigning
00:21:55.650 --> 00:21:58.430
rewards to renewal processes.
00:21:58.430 --> 00:22:04.040
And the reward at any time t,
when I'm trying to do this, is
00:22:04.040 --> 00:22:07.890
restricted to be a function of
the inter-renewal period
00:22:07.890 --> 00:22:08.860
containing t.
00:22:08.860 --> 00:22:14.110
In other words, when you try to
define what the reward is
00:22:14.110 --> 00:22:17.940
at a given time t, it's a
function only of what's going
00:22:17.940 --> 00:22:21.650
on in a renewal period
containing t.
00:22:24.290 --> 00:22:27.860
Now, in its simplest
form, I want to go
00:22:27.860 --> 00:22:29.780
make this even simpler.
00:22:29.780 --> 00:22:35.050
In its simplest form r of t is
restricted to be a function of
00:22:35.050 --> 00:22:39.020
the age of time t and the
duration of time t.
00:22:39.020 --> 00:22:42.620
In other words, you try to say
what's the reward in a given
00:22:42.620 --> 00:22:45.430
time t, and it's the
same as these three
00:22:45.430 --> 00:22:47.280
examples we looked at.
00:22:47.280 --> 00:22:52.550
It's some function of how far
back do you have to go to the
00:22:52.550 --> 00:22:56.590
previous arrival, and how far
forward do you have to go on
00:22:56.590 --> 00:22:59.040
until the next arrival.
00:22:59.040 --> 00:23:02.610
You can make this a function if
you want to of any of the
00:23:02.610 --> 00:23:07.260
three quantities residual
life, age, and duration.
00:23:07.260 --> 00:23:11.590
It seems to be intuitively a
little simpler to talk about
00:23:11.590 --> 00:23:16.342
age and duration as opposed
to the other quantities.
00:23:16.342 --> 00:23:20.730
So the time average for sample
path of r of t is found by
00:23:20.730 --> 00:23:23.790
analogy to residual life.
00:23:23.790 --> 00:23:26.100
That's the way I'm going
to do it here in class.
00:23:26.100 --> 00:23:28.140
The notes do a little
more careful job
00:23:28.140 --> 00:23:30.610
than just by analogy.
00:23:30.610 --> 00:23:34.310
But you start with the n-th
inter-renewal interval and you
00:23:34.310 --> 00:23:39.920
say, what is the aggregate
reward I get in the n-th
00:23:39.920 --> 00:23:40.790
inter-renewal interval?
00:23:40.790 --> 00:23:43.810
It's a random variable,
obviously.
00:23:43.810 --> 00:23:54.730
And what it is is the integral
from the previous arrival up
00:23:54.730 --> 00:23:59.190
to the next arrival of
r of t and omega dt.
00:24:05.130 --> 00:24:11.000
This is very strange, because
when we looked at this before
00:24:11.000 --> 00:24:15.750
we talked about n of t and we
talked about s sub n of t plus
00:24:15.750 --> 00:24:22.720
1 and s sub n of t, and suddenly
n of t plus 1 has
00:24:22.720 --> 00:24:28.690
turned into n, and n of t has
turned into n minus 1.
00:24:28.690 --> 00:24:31.210
What has happened?
00:24:31.210 --> 00:24:34.280
Well, this gives you a clue.
00:24:34.280 --> 00:24:37.730
And it's partly how we
define the intervals.
00:24:37.730 --> 00:24:44.010
Interval 1 goes from 0 to s1 and
z of t equals t, interval
00:24:44.010 --> 00:24:48.590
n z of t is t minus
s sub n minus 1.
00:24:48.590 --> 00:24:53.240
Let me go back and show you the
original figure here and I
00:24:53.240 --> 00:24:54.595
think it will make it clear.
00:25:06.090 --> 00:25:11.010
Now, the first arrival comes
at this time here.
00:25:11.010 --> 00:25:15.560
The first interval we're talking
about goes from 0 to
00:25:15.560 --> 00:25:17.530
time s1 of omega.
00:25:21.080 --> 00:25:25.000
What we're interested in, in
this first interval going from
00:25:25.000 --> 00:25:31.000
0 to s1 of omega, is this
interarrival time, which is
00:25:31.000 --> 00:25:32.850
the first interarrival time.
00:25:32.850 --> 00:25:36.840
This first interval is connected
to x1 and it goes
00:25:36.840 --> 00:25:42.250
from s sub 0, which is
just at 0 up to s1.
00:25:42.250 --> 00:25:47.890
The second arrival time
goes from s1 up to s2.
00:25:47.890 --> 00:25:53.400
When we look at n of t, n of
t is a sum t here in this
00:25:53.400 --> 00:25:57.510
interval here, and this is s.
00:26:03.760 --> 00:26:05.790
Am I going to get totally
confused writing
00:26:05.790 --> 00:26:07.905
this or aren't I?
00:26:19.600 --> 00:26:22.120
I think I might get totally
confused, so I'm not going to
00:26:22.120 --> 00:26:23.370
try to write it.
00:26:29.000 --> 00:26:30.950
Because when I try to write
it I'm trying to make the
00:26:30.950 --> 00:26:35.470
comparison between n of t and
n of t plus 1, which are the
00:26:35.470 --> 00:26:39.080
things up here.
00:26:39.080 --> 00:26:44.100
But the quantities here with the
renewal periods are, this
00:26:44.100 --> 00:26:47.150
is the first inter-renewal
period, this is the second
00:26:47.150 --> 00:26:48.730
inter-renewal period.
00:26:48.730 --> 00:26:54.910
The second inter-renewal period
goes from s sub 2 back
00:26:54.910 --> 00:26:56.280
to s sub 1.
00:26:56.280 --> 00:27:02.570
The n-th goes from s sub n
back to s sub n minus 1.
00:27:02.570 --> 00:27:05.190
So that's just the way it is
when you count this way.
00:27:05.190 --> 00:27:09.040
If you count it the other way
you would get even more
00:27:09.040 --> 00:27:14.030
confused because then in the
interval x sub n the
00:27:14.030 --> 00:27:17.580
inter-renewal time would
be x sub n plus 1 and
00:27:17.580 --> 00:27:19.090
that would be awful.
00:27:19.090 --> 00:27:21.460
So, this is the way
we have to do it.
00:27:30.310 --> 00:27:33.770
Let's look at what happens
with the expected
00:27:33.770 --> 00:27:37.195
inter-renewal time, then,
as a time average.
00:27:42.400 --> 00:27:45.990
And you can think of this as
a being for a sample time.
00:27:45.990 --> 00:27:49.700
When you have an expression
which is valid for a sample
00:27:49.700 --> 00:27:53.810
time, it's also valid for a
random variable, because it
00:27:53.810 --> 00:27:58.080
maps every sample point
into some value.
00:27:58.080 --> 00:28:01.620
So, I'll just call it here r sub
n, which is the amount of
00:28:01.620 --> 00:28:07.410
reward that you pick up in the
n-th inter-renewal period, so
00:28:07.410 --> 00:28:14.830
it's the integral from the n
minus first arrival up to the
00:28:14.830 --> 00:28:19.460
n-th arrival, and we're
integrating over r of t over
00:28:19.460 --> 00:28:21.010
this whole interval.
00:28:21.010 --> 00:28:28.250
So r of t is a function of the
age and of the duration.
00:28:28.250 --> 00:28:37.940
The age is t minus s sub n minus
1, and the duration is
00:28:37.940 --> 00:28:39.660
just x of n.
00:28:39.660 --> 00:28:43.410
Now that's a little weird also
because before we were talking
00:28:43.410 --> 00:28:47.710
about the duration as being
statistically very, very
00:28:47.710 --> 00:28:52.260
different from the inter-renewal
period.
00:28:52.260 --> 00:28:54.200
But it wasn't.
00:28:54.200 --> 00:29:04.310
If you look on a sample path
basis, this duration of the
00:29:04.310 --> 00:29:10.680
n-th inter-renewal period is
exactly equal to x sub n of
00:29:10.680 --> 00:29:19.110
omega, which is, for that
particular sample path, it's
00:29:19.110 --> 00:29:22.700
the value of that inter-renewal
interval.
00:29:22.700 --> 00:29:26.950
So when we integrate this
quantity now, we want to
00:29:26.950 --> 00:29:29.460
integrate it over dt.
00:29:29.460 --> 00:29:30.460
What do we get?
00:29:30.460 --> 00:29:36.210
Well, we've gotten rid of this
t here, we just have the n-th
00:29:36.210 --> 00:29:37.750
duration here.
00:29:37.750 --> 00:29:44.010
The only t appears over here,
and this is r of a difference
00:29:44.010 --> 00:29:47.370
between t and s sub n minus 1.
00:29:47.370 --> 00:29:51.510
So, we want to do a change of
variables, we want to let z be
00:29:51.510 --> 00:29:55.570
t minus s sub n minus 1,
and then we integrate z
00:29:55.570 --> 00:29:58.480
from 0 to x sub n.
00:29:58.480 --> 00:30:02.230
And now just imagine that you
put omegas in all of these
00:30:02.230 --> 00:30:05.220
things to let you see that
you're actually dealing with
00:30:05.220 --> 00:30:08.900
one sample function, and when
you leave the omegas out
00:30:08.900 --> 00:30:11.760
you're just dealing with the
random variables that arise
00:30:11.760 --> 00:30:14.580
because of doing that.
00:30:14.580 --> 00:30:23.290
So when we do this integration,
what we get is
00:30:23.290 --> 00:30:29.090
the integral of r of z with this
fixed x sub n integrated
00:30:29.090 --> 00:30:30.280
against dz.
00:30:30.280 --> 00:30:33.810
This is a function only of the
random variable x sub n.
00:30:33.810 --> 00:30:35.060
Strange.
00:30:35.060 --> 00:30:37.490
That's the expected
value of r sub n.
00:30:40.340 --> 00:30:43.290
This unfortunately can't be
reduced any further, this is
00:30:43.290 --> 00:30:45.040
just what it is.
00:30:45.040 --> 00:30:50.480
You have to find out what is
the integral of r of zx
00:30:50.480 --> 00:30:55.300
integrated over z, and then we
have to take the expected
00:30:55.300 --> 00:30:57.830
value over x.
00:30:57.830 --> 00:31:00.950
If the expectation exists, you
can write this in a simple
00:31:00.950 --> 00:31:09.370
way, because this quantity
here integrated
00:31:09.370 --> 00:31:11.140
is just r sub n.
00:31:11.140 --> 00:31:15.880
And then you take the expected
value of r sub n, and you
00:31:15.880 --> 00:31:18.590
divide by x-bar down here.
00:31:21.950 --> 00:31:27.770
This quantity here is just the
expected value of the integral
00:31:27.770 --> 00:31:30.240
of r of zx dz.
00:31:30.240 --> 00:31:32.980
And that integral there
is just the expected
00:31:32.980 --> 00:31:36.410
value of r sub n.
00:31:36.410 --> 00:31:40.020
So, if you look over the entire
sample path from 0 to
00:31:40.020 --> 00:31:49.940
infinity, what you get then is
that the time average reward
00:31:49.940 --> 00:31:53.560
is just equal to the expected
value of r sub n
00:31:53.560 --> 00:31:55.000
divided by x squared.
00:31:59.060 --> 00:32:00.270
Oops.
00:32:00.270 --> 00:32:01.520
Lost the example.
00:32:03.970 --> 00:32:08.580
Suppose we want to find the
k-th moment of the age.
00:32:08.580 --> 00:32:10.090
Simple thing to try to find.
00:32:12.930 --> 00:32:16.670
I want to do that because
it's simple.
00:32:16.670 --> 00:32:20.560
If you're looking at the k-th
moment of the age looked at as
00:32:20.560 --> 00:32:29.130
a time average, what it is is
the reward at time t, it's the
00:32:29.130 --> 00:32:32.490
age at time t to
the k-th power.
00:32:32.490 --> 00:32:38.640
We want to take the age to the
k-th power at time t, and then
00:32:38.640 --> 00:32:43.550
integrate over rt and divide
by the final value
00:32:43.550 --> 00:32:45.990
and go to the limit.
00:32:45.990 --> 00:32:50.460
So the expected value of
this k-th moment over 1
00:32:50.460 --> 00:32:56.620
inter-renewal period is this
integral here, z to the k dz
00:32:56.620 --> 00:33:01.160
times dF sub x of x.
00:33:01.160 --> 00:33:06.060
The only place x comes in this
integral is in the final point
00:33:06.060 --> 00:33:09.780
of the integration where we
are integrating up to the
00:33:09.780 --> 00:33:14.650
point x, at which the duration
ends, and then we're taking
00:33:14.650 --> 00:33:16.250
the expected value of this.
00:33:16.250 --> 00:33:20.420
So, we integrate this, the
integral of the z to the k
00:33:20.420 --> 00:33:29.080
from 0 to x is just x to the
k plus 1 divided by k.
00:33:29.080 --> 00:33:31.200
So when you take that integral
and you look at
00:33:31.200 --> 00:33:33.280
this, what is this?
00:33:33.280 --> 00:33:37.620
It's the 1 over k times the
expected value of the random
00:33:37.620 --> 00:33:40.710
variable, x to the k
plus first power.
00:33:40.710 --> 00:33:46.660
That's just this taking the
expected value of it, which is
00:33:46.660 --> 00:33:49.490
you take the expected value
by integrating times
00:33:49.490 --> 00:33:50.980
the s sub x of x.
00:33:50.980 --> 00:33:51.520
Yes?
00:33:51.520 --> 00:33:53.440
AUDIENCE: [INAUDIBLE]
00:33:53.440 --> 00:33:54.690
divide by k plus 1?
00:33:59.160 --> 00:34:00.410
PROFESSOR: Yes.
00:34:03.180 --> 00:34:04.480
I'm sorry.
00:34:07.050 --> 00:34:11.719
My evil twin brother is speaking
this morning, and
00:34:11.719 --> 00:34:14.560
this should be a
k plus 1 here.
00:34:14.560 --> 00:34:18.670
This should be a k plus 1, and
this should be a k plus 1.
00:34:18.670 --> 00:34:21.670
And if you look at the example
of the first moment, which is
00:34:21.670 --> 00:34:24.840
the first thing we looked at
today, namely finding the
00:34:24.840 --> 00:34:29.840
expected value of the age, what
we find out is that it's
00:34:29.840 --> 00:34:37.199
expected value of x squared
divided by 2 times x-bar.
00:34:37.199 --> 00:34:41.960
So with k equal to 1 we get
the expected value of x
00:34:41.960 --> 00:34:46.070
squared over 2 times x-bar.
00:34:46.070 --> 00:34:48.690
It's even worse because I left
a 2 out when we were talking
00:34:48.690 --> 00:34:51.690
about the expected
value of age.
00:34:51.690 --> 00:34:55.659
And to make things even worse,
when I did this I went back to
00:34:55.659 --> 00:34:59.590
check whether it was the right
answer, and of course the two
00:34:59.590 --> 00:35:01.590
mistakes cancelled
each other out.
00:35:01.590 --> 00:35:03.250
So, sorry about that.
00:35:07.150 --> 00:35:10.800
The thing I really wanted to
talk about today is stopping
00:35:10.800 --> 00:35:14.960
trials for stochastic processes,
because that's a
00:35:14.960 --> 00:35:21.220
topic which has always caused
confusion in this class.
00:35:21.220 --> 00:35:24.600
If you look at any of the
textbooks on stochastic
00:35:24.600 --> 00:35:28.130
processes, it causes even
more confusion.
00:35:28.130 --> 00:35:32.890
People talk about it, and then
they talk about it again, and
00:35:32.890 --> 00:35:36.410
they say what we said before
wasn't right, and then they
00:35:36.410 --> 00:35:39.110
talk about it again, they say
what we said before that
00:35:39.110 --> 00:35:41.710
wasn't right, and it goes
on and on and on.
00:35:41.710 --> 00:35:43.660
And you never know
what's going on.
00:35:47.920 --> 00:35:52.570
Very often instead of looking
at an entire stochastic
00:35:52.570 --> 00:35:55.900
process over the infinite
interval, you'd like to look
00:35:55.900 --> 00:35:57.380
at a finite interval.
00:35:57.380 --> 00:36:00.490
You'd like to look at the
interval going from 0 up to
00:36:00.490 --> 00:36:02.130
some time t.
00:36:02.130 --> 00:36:09.280
But in almost a majority of
those cases, the time t that
00:36:09.280 --> 00:36:13.430
you want to look at is not
some fixed time but some
00:36:13.430 --> 00:36:18.990
random time which depends on
something which is happening.
00:36:18.990 --> 00:36:26.680
And when you want to look at
what's going on here, it's
00:36:26.680 --> 00:36:31.560
always tricky to visualize this
because what you have is
00:36:31.560 --> 00:36:34.600
t becomes a random variable.
00:36:34.600 --> 00:36:39.730
That random variable is a
function of the sample values
00:36:39.730 --> 00:36:43.390
that you have up until time t.
00:36:43.390 --> 00:36:47.560
But that seems like a circular
argument, t a random variable
00:36:47.560 --> 00:36:50.110
and it depends on t.
00:36:52.620 --> 00:36:55.640
Things are not supposed to
depend on themselves, because
00:36:55.640 --> 00:37:00.060
when they depend on themselves
you have trouble understanding
00:37:00.060 --> 00:37:01.930
what's going on.
00:37:01.930 --> 00:37:05.450
So, we will sort that out.
00:37:05.450 --> 00:37:07.470
In sorting it out we're
only going to look at
00:37:07.470 --> 00:37:09.610
discrete-time processes.
00:37:09.610 --> 00:37:13.830
So, we'll only look at sequences
of random variables.
00:37:13.830 --> 00:37:17.670
We will not assume that these
random variables are IID for
00:37:17.670 --> 00:37:22.070
this argument here, because we
want to look at a much wider
00:37:22.070 --> 00:37:22.950
range of things.
00:37:22.950 --> 00:37:28.180
So we have some arbitrary
sequence of random variables,
00:37:28.180 --> 00:37:33.090
and the idea is you want to
sit there looking at this
00:37:33.090 --> 00:37:40.800
evolution of the sequence of
random variables, and in fact,
00:37:40.800 --> 00:37:44.630
you want to sit there looking
at the sample
00:37:44.630 --> 00:37:46.150
path of this evolution.
00:37:46.150 --> 00:37:49.830
You want to look at x1, and you
want to look at x2, and
00:37:49.830 --> 00:37:52.830
you want to look at
x3, and so forth.
00:37:52.830 --> 00:37:56.410
And at each point along the line
in terms of what you've
00:37:56.410 --> 00:37:59.160
seen already, you want to decide
whether you're going to
00:37:59.160 --> 00:38:02.370
stop or whether you're going
to continue at that point.
00:38:10.380 --> 00:38:14.670
And if at that point you develop
a rule for what you're
00:38:14.670 --> 00:38:19.380
going to do with each point you
call that a stopping rule,
00:38:19.380 --> 00:38:22.240
so that you have a rule that
tells you when you want to
00:38:22.240 --> 00:38:24.750
stop looking at these things.
00:38:24.750 --> 00:38:29.540
Let me give you some idea of
the generality of this.
00:38:29.540 --> 00:38:34.080
Often when we look at queuing
systems we are going to have
00:38:34.080 --> 00:38:40.310
an arrival process where
arrivals keep coming in,
00:38:40.310 --> 00:38:45.040
things get serviced one after
the other, and one of the very
00:38:45.040 --> 00:38:49.670
interesting things is whether
the system ever empties out or
00:38:49.670 --> 00:38:53.140
whether the queue just keeps
building up forever.
00:38:53.140 --> 00:38:58.160
Well, an interesting stopping
rule then is when the queue
00:38:58.160 --> 00:39:00.480
empties out.
00:39:00.480 --> 00:39:04.430
We will see that an even more
interesting stopping rule is
00:39:04.430 --> 00:39:09.310
starting with some arbitrary
first arrival at time 0.
00:39:09.310 --> 00:39:14.600
Let's look for the time until
another arrival occurs which
00:39:14.600 --> 00:39:19.580
starts another busy period.
00:39:19.580 --> 00:39:24.180
These stopping times, then, are
going to be critical in
00:39:24.180 --> 00:39:27.030
analyzing things like
queuing systems.
00:39:27.030 --> 00:39:31.080
They're also going to become
critical in trying to analyze
00:39:31.080 --> 00:39:34.890
these renewal systems that we're
already talking about.
00:39:34.890 --> 00:39:40.260
They will become critical to
trying to take a time average
00:39:40.260 --> 00:39:45.920
view about renewal processes
instead of a, not a time
00:39:45.920 --> 00:39:49.690
average, but an ensemble average
viewpoint looking at
00:39:49.690 --> 00:39:54.020
particular times t as opposed
to taking a time average.
00:39:54.020 --> 00:39:56.180
So we're going to look
at these things in
00:39:56.180 --> 00:39:57.430
many different ways.
00:40:02.070 --> 00:40:06.920
In order to do that, since
you're looking at a sample
00:40:06.920 --> 00:40:11.840
function, you're observing
it on each
00:40:11.840 --> 00:40:14.910
arrival of this process.
00:40:23.460 --> 00:40:27.080
At each observation of a sample
value of a random
00:40:27.080 --> 00:40:30.730
variable, you decide whether
you want to stop or not.
00:40:30.730 --> 00:40:37.220
So a sensible way to deal with
this is to look over all time,
00:40:37.220 --> 00:40:43.290
define a random variable J,
which describes when this
00:40:43.290 --> 00:40:44.830
sequence is to be stopped.
00:40:44.830 --> 00:40:49.470
So for each sample value x1 of
omega, x2 of omega, x3 of
00:40:49.470 --> 00:40:53.690
omega, and so forth, if your
rule is to stop the first time
00:40:53.690 --> 00:41:00.630
you see a sample value which
is equal to 0, then J is a
00:41:00.630 --> 00:41:05.790
integer random variable whose
value is the first n at which
00:41:05.790 --> 00:41:10.450
is x of n is equal to 0 and
many other examples.
00:41:10.450 --> 00:41:16.190
So we try a 1 then, x1
of omega is observed.
00:41:16.190 --> 00:41:20.120
A decision is made based
on x1 of omega
00:41:20.120 --> 00:41:23.560
whether or not to stop.
00:41:23.560 --> 00:41:27.550
If we stop, J of
omega equals 1.
00:41:27.550 --> 00:41:31.910
If we don't stop, J of omega is
bigger than 1, but we don't
00:41:31.910 --> 00:41:33.510
know what it is yet.
00:41:33.510 --> 00:41:40.440
At trial two, if we haven't
stopped yet, you observe x2 of
00:41:40.440 --> 00:41:44.710
omega, the second random
variable, second sample value,
00:41:44.710 --> 00:41:49.240
you make a decision again based
on x1 of omega and x2 of
00:41:49.240 --> 00:41:51.770
omega whether or not to stop.
00:41:51.770 --> 00:41:54.790
If you stop, J of omega
is equal to 2.
00:41:54.790 --> 00:42:00.090
So you can visualize doing this
on each trial, you don't
00:42:00.090 --> 00:42:02.790
have to worry about whether
you've already stopped, you
00:42:02.790 --> 00:42:06.370
just have a rule that says, I
want to stop at this point if
00:42:06.370 --> 00:42:08.240
I haven't stopped before.
00:42:08.240 --> 00:42:14.925
So the stopping event at time
n is that you stop either if
00:42:14.925 --> 00:42:20.780
your rule tells you to stop
or you stop before.
00:42:20.780 --> 00:42:23.990
And if you stop before then
you're obviously stopped.
00:42:23.990 --> 00:42:27.700
At each trial n if stopping has
not yet occurred x of n is
00:42:27.700 --> 00:42:32.600
observed and the decision based
on x1 to xn is made.
00:42:32.600 --> 00:42:37.060
If you stop, then J omega
is equal to n.
00:42:37.060 --> 00:42:45.410
So for each sample path J of n
is the time or the trial at
00:42:45.410 --> 00:42:46.660
which you stop.
00:42:50.850 --> 00:42:54.890
So, we're going to define a
stopping trial, or stopping
00:42:54.890 --> 00:42:59.980
time, J for a sequence
of random variables.
00:42:59.980 --> 00:43:03.610
You're going to define this to
be a positive integer-valued
00:43:03.610 --> 00:43:08.130
random variable that has to be
positive, because if you're
00:43:08.130 --> 00:43:11.290
going to stop before you observe
anything that's not a
00:43:11.290 --> 00:43:13.430
very interesting thing.
00:43:13.430 --> 00:43:17.190
So you always observe at least
one thing, then you decide
00:43:17.190 --> 00:43:20.120
whether you want to stop or
you proceed, so forth.
00:43:20.120 --> 00:43:22.690
So it's a positive
integer-valued random
00:43:22.690 --> 00:43:30.900
variable, you always stop at an
integer trial, and for each
00:43:30.900 --> 00:43:36.000
n greater than or equal to 1 the
indicator random variable
00:43:36.000 --> 00:43:42.520
the indicator of the event J
equals n is a function of what
00:43:42.520 --> 00:43:44.835
you have observed up
until that point.
00:43:47.880 --> 00:43:50.850
This part is a really
critical part of it.
00:43:50.850 --> 00:43:56.700
The decision to stop at a time n
has to be a function only of
00:43:56.700 --> 00:43:59.010
what you have already
observed.
00:43:59.010 --> 00:44:03.290
You're not allowed as an
observer to peek ahead a
00:44:03.290 --> 00:44:06.900
little bit and then decide on
the basis of what you see in
00:44:06.900 --> 00:44:12.310
the future whether you want to
stop at this time or not.
00:44:12.310 --> 00:44:15.570
One example in the notes is that
you're playing poker with
00:44:15.570 --> 00:44:23.280
somebody, and you make a bet
and the other person wins.
00:44:23.280 --> 00:44:26.150
You make the bet on the basis
of what you've seen in your
00:44:26.150 --> 00:44:30.210
cards so far, you don't make
your bet on the basis of your
00:44:30.210 --> 00:44:34.410
observation of what the other
player has, and the other
00:44:34.410 --> 00:44:39.890
player will get very angry if
you try to base your decision
00:44:39.890 --> 00:44:45.600
later on what the person you're
playing with has.
00:44:45.600 --> 00:44:47.590
So you can't do that
sort of thing.
00:44:47.590 --> 00:44:51.490
You can't peek ahead when
you're doing this.
00:44:51.490 --> 00:44:54.890
You could design random
variables where, in fact, you
00:44:54.890 --> 00:44:59.920
can look ahead, but the times
where you use what people call
00:44:59.920 --> 00:45:05.510
stopping trials are situations
in which you stop based on
00:45:05.510 --> 00:45:07.990
what has already happened.
00:45:07.990 --> 00:45:11.380
You can generalize this, and we
will generalize it later,
00:45:11.380 --> 00:45:14.380
where you can stop on the
basis of other random
00:45:14.380 --> 00:45:19.350
variables, which also evolve in
time, but you stop on the
00:45:19.350 --> 00:45:25.730
basis of what has already
happened up until time n.
00:45:25.730 --> 00:45:29.230
We're going to visualize
conducting successive trials
00:45:29.230 --> 00:45:33.850
until sum n in which the event
J equals n occurs.
00:45:33.850 --> 00:45:36.520
Further trials then cease.
00:45:36.520 --> 00:45:39.680
It is simpler conceptually
to visualize stopping the
00:45:39.680 --> 00:45:43.720
observation of trials after
the stopping trial, but
00:45:43.720 --> 00:45:45.530
continuing to conduct trials.
00:45:45.530 --> 00:45:49.310
In other words, if we start
talking about a stopping
00:45:49.310 --> 00:45:55.100
process and we say we stop on
the n-th trial, but we're
00:45:55.100 --> 00:45:58.020
forbidden to even talk
about the n plus
00:45:58.020 --> 00:45:59.700
first random variable.
00:45:59.700 --> 00:46:05.890
Now the n plus first random
variable occurs on some sample
00:46:05.890 --> 00:46:09.520
paths but doesn't occur
on other sample paths.
00:46:09.520 --> 00:46:11.960
I don't know how to deal with
that, and neither do you.
00:46:14.800 --> 00:46:17.800
And what that means is the way
we're going to visualize these
00:46:17.800 --> 00:46:24.200
processes is physically they
continue forever, but we just
00:46:24.200 --> 00:46:26.540
stop observing them at
a certain point.
00:46:26.540 --> 00:46:30.840
So, this stopping rule is the
time at which we stop
00:46:30.840 --> 00:46:35.090
observing as opposed to the
time at which the random
00:46:35.090 --> 00:46:50.470
variable ceases existing, if we
define the random variable,
00:46:50.470 --> 00:46:55.430
it has a value for all sample
values, and sample values
00:46:55.430 --> 00:46:59.200
involve all of these paths.
00:46:59.200 --> 00:47:01.940
So you sort of have to
define it that way.
00:47:01.940 --> 00:47:07.550
One thing you would like to do
is in many of these situations
00:47:07.550 --> 00:47:11.040
you wind up never stopping.
00:47:11.040 --> 00:47:15.360
And the trouble is, when you're
investigating stopping
00:47:15.360 --> 00:47:20.110
rules, stopping trials, you
usually don't know ahead of
00:47:20.110 --> 00:47:24.530
time whether you're ever
going to stop or not.
00:47:24.530 --> 00:47:27.660
And because of that if you
don't know whether you're
00:47:27.660 --> 00:47:29.310
going to stop or not,
you can't call
00:47:29.310 --> 00:47:30.810
it a stopping trial.
00:47:30.810 --> 00:47:35.990
So, what one normally does is
to say that J is a possibly
00:47:35.990 --> 00:47:38.240
defective random variable.
00:47:38.240 --> 00:47:44.400
And if it's possibly defective
you mean that all sample
00:47:44.400 --> 00:47:49.700
points are mapped into either
finite J or perhaps J equals
00:47:49.700 --> 00:47:52.840
infinity, which means
that you never stop.
00:47:52.840 --> 00:47:57.660
But you still have the same
condition that we have here,
00:47:57.660 --> 00:48:02.290
that you stop on the basis
of x1, x of n.
00:48:02.290 --> 00:48:05.400
And you also have the condition,
which isn't quite
00:48:05.400 --> 00:48:08.310
clear here, but it's clear
from the fact that J is a
00:48:08.310 --> 00:48:17.060
random variable, that the events
J equals 1, J equals 2,
00:48:17.060 --> 00:48:20.840
J equals 3, are all disjoint.
00:48:20.840 --> 00:48:24.940
So you can't stop twice, you
have to just stop once.
00:48:24.940 --> 00:48:30.140
And once you stop you
can't start again,
00:48:30.140 --> 00:48:34.020
Now does everybody understand
what is stopping trial or a
00:48:34.020 --> 00:48:35.740
stopping rule is
at this point?
00:48:39.300 --> 00:48:44.770
If you don't understand it I
think the trouble is you won't
00:48:44.770 --> 00:48:47.730
realize you don't understand it
until you start seeing some
00:48:47.730 --> 00:48:52.540
of these examples where strange
things are going on.
00:48:52.540 --> 00:48:55.790
I guess we have to just
go ahead and see
00:48:55.790 --> 00:48:58.460
what happens then.
00:48:58.460 --> 00:49:00.300
So the examples, which
I'm not going to
00:49:00.300 --> 00:49:02.730
develop in any detail.
00:49:02.730 --> 00:49:07.160
A gambler goes to a casino and
he gambles until he's broke.
00:49:07.160 --> 00:49:10.480
So, he goes in with a finite
amount of capital.
00:49:10.480 --> 00:49:14.480
He has some particular system
that he's playing by.
00:49:14.480 --> 00:49:18.040
If he's lucky he gets to play
for a long, long time, he gets
00:49:18.040 --> 00:49:22.030
a lot of amusement as he loses
his money, and if he's unlucky
00:49:22.030 --> 00:49:24.600
he loses very quickly.
00:49:24.600 --> 00:49:28.490
If you look at the odds in
casinos and you apply the
00:49:28.490 --> 00:49:31.000
strong law of large numbers,
you realize that with
00:49:31.000 --> 00:49:34.480
probability 1 you get wiped
out eventually.
00:49:34.480 --> 00:49:37.560
Because the odds are
not in your favor.
00:49:37.560 --> 00:49:41.200
Casinos wouldn't be built if the
odds were in your favor.
00:49:41.200 --> 00:49:44.410
Another example, just
flip a coin until 10
00:49:44.410 --> 00:49:46.500
successive heads appear.
00:49:46.500 --> 00:49:48.000
10 heads in a row.
00:49:48.000 --> 00:49:50.260
That's a rather unlikely
event.
00:49:50.260 --> 00:49:53.470
You're going to be able to
figure out very easily from
00:49:53.470 --> 00:49:57.170
this theory what's the expected
amount of time until
00:49:57.170 --> 00:49:59.730
10 successive heads appear.
00:49:59.730 --> 00:50:02.801
It's a very easy problem.
00:50:02.801 --> 00:50:08.070
A more important problem, this
test and hypothesis with
00:50:08.070 --> 00:50:11.310
repeated trials.
00:50:11.310 --> 00:50:13.730
So, you have the hypothesis
that a certain kind of
00:50:13.730 --> 00:50:18.710
treatment will make patients
well, and you want to know
00:50:18.710 --> 00:50:22.580
whether to use this treatment.
00:50:22.580 --> 00:50:27.820
So you test it on mice, or
people, or what have you.
00:50:27.820 --> 00:50:29.420
If you think it's pretty
safe, you start
00:50:29.420 --> 00:50:32.070
testing it on people.
00:50:32.070 --> 00:50:35.015
And how many tests
do you make?
00:50:37.950 --> 00:50:41.320
Well, you originally think that
as a scientist you should
00:50:41.320 --> 00:50:45.240
plan an experiment ahead of
time, and you should say, I'm
00:50:45.240 --> 00:50:47.810
going to do 1,000 tests.
00:50:47.810 --> 00:50:51.180
But if you're doing experiments
on people and you
00:50:51.180 --> 00:50:55.700
find out that the first 10
patients die, you're going to
00:50:55.700 --> 00:50:59.380
stop the experiment
at that point.
00:50:59.380 --> 00:51:03.990
And if you find out that the
first 100 patients all live,
00:51:03.990 --> 00:51:05.505
well, you might continue
the experiment.
00:51:05.505 --> 00:51:08.430
But you're going to publish
the results at that point
00:51:08.430 --> 00:51:11.230
because you're going to try to
get the FDA to approve this
00:51:11.230 --> 00:51:17.080
drug, or this operation,
or this what have you.
00:51:17.080 --> 00:51:21.730
So if you view this stopping
trial as the time at which
00:51:21.730 --> 00:51:26.410
you're going to try to publish
something, then, again, if you
00:51:26.410 --> 00:51:30.510
have any sense, you are going
to perform experiments, look
00:51:30.510 --> 00:51:34.420
at the experiments as you go,
and decide what you're going
00:51:34.420 --> 00:51:37.700
to do as a function of what
you've already seen.
00:51:37.700 --> 00:51:38.860
I mean, that's the way that all
00:51:38.860 --> 00:51:40.480
intelligent people operate.
00:51:43.200 --> 00:51:46.620
So if science says that's not
a good way to operate,
00:51:46.620 --> 00:51:48.000
something's wrong
with science.
00:51:48.000 --> 00:51:50.290
But fortunately science
doesn't say that.
00:51:50.290 --> 00:51:52.170
Science allows you to do this.
00:51:52.170 --> 00:51:55.217
AUDIENCE: So not every J is
potentially defective?
00:51:55.217 --> 00:51:56.675
You can come up with
examples of
00:51:56.675 --> 00:51:57.656
where it would be defective.
00:51:57.656 --> 00:52:00.300
But not every one is necessarily
[INAUDIBLE].
00:52:03.470 --> 00:52:03.966
PROFESSOR: Yes.
00:52:03.966 --> 00:52:11.830
If a random variable is
defective it means there's
00:52:11.830 --> 00:52:15.600
some probability that J is
going to be infinite, but
00:52:15.600 --> 00:52:19.710
there's also presumably some
probability that J is equal to
00:52:19.710 --> 00:52:23.070
1, a probability J is equal
to 2, and so forth.
00:52:23.070 --> 00:52:27.050
So we have a distribution
function for J, which instead
00:52:27.050 --> 00:52:31.870
of going up to 1 and stopping
goes up to some smaller value
00:52:31.870 --> 00:52:35.202
and stops, and then it doesn't
go any further.
00:52:35.202 --> 00:52:36.194
AUDIENCE: Not every
[INAUDIBLE].
00:52:36.194 --> 00:52:38.550
Some of these are definitely
going to [INAUDIBLE].
00:52:38.550 --> 00:52:39.800
PROFESSOR: Yes.
00:52:43.180 --> 00:52:50.200
So this testing hypotheses is
really what triggered Abraham
00:52:50.200 --> 00:52:55.210
Wald to start trying to
understand these situations,
00:52:55.210 --> 00:52:59.090
because he found out relatively
quickly that it
00:52:59.090 --> 00:53:02.990
wasn't trivial to understand
them, and all sorts of crazy
00:53:02.990 --> 00:53:04.350
things were happening.
00:53:04.350 --> 00:53:09.080
So he spent a lot of his time
studying what happened in
00:53:09.080 --> 00:53:10.130
these cases.
00:53:10.130 --> 00:53:11.930
He called it sequential
analysis.
00:53:11.930 --> 00:53:13.520
You might have heard the word.
00:53:13.520 --> 00:53:18.930
And sequential analysis is
exactly the same idea.
00:53:18.930 --> 00:53:25.620
It's looking at analyzing
experiments as you go in time,
00:53:25.620 --> 00:53:28.660
and either stopping or doing
something else or changing the
00:53:28.660 --> 00:53:33.360
experiment or what have you,
depending on what happens.
00:53:33.360 --> 00:53:36.700
Another thing is observe
successive renewals in a
00:53:36.700 --> 00:53:40.750
renewal process until
s sub n is greater
00:53:40.750 --> 00:53:43.760
than or equal to 100.
00:53:43.760 --> 00:53:47.820
Now, this particular thing is
one of the things that we're
00:53:47.820 --> 00:53:51.230
going to use in studying renewal
processes, it's why
00:53:51.230 --> 00:53:54.740
we're studying this
topic right now.
00:53:54.740 --> 00:53:57.270
Well, we might study it now
anyway, because we're going to
00:53:57.270 --> 00:54:01.460
use it in lots of other places,
but it's also why you
00:54:01.460 --> 00:54:07.920
have to be very careful about
talking about stopping time as
00:54:07.920 --> 00:54:10.310
opposed to stopping trials.
00:54:10.310 --> 00:54:16.910
Because if you stop an
experiment at the arrival in a
00:54:16.910 --> 00:54:26.080
renewal process at which that
renewal occurs after time 100,
00:54:26.080 --> 00:54:29.140
you don't know when you're
going to stop.
00:54:29.140 --> 00:54:33.420
It might be a very long time
after 100 before that first
00:54:33.420 --> 00:54:38.190
arrival after time 100 occurs,
it might be a very short time.
00:54:38.190 --> 00:54:41.390
So it's not a stopping time that
you're defining, it's a
00:54:41.390 --> 00:54:43.120
stopping trial.
00:54:43.120 --> 00:54:46.560
You are defining a rule which
tells you at which trial
00:54:46.560 --> 00:54:50.090
you're going to stop as opposed
to a rule that tells
00:54:50.090 --> 00:54:53.690
you in which time you're going
to stop, and we'll see this as
00:54:53.690 --> 00:54:54.960
we go on in this.
00:54:58.250 --> 00:55:02.480
Suppose the random variables in
this process we're looking
00:55:02.480 --> 00:55:07.970
at have a finite number of
possible sample values.
00:55:07.970 --> 00:55:13.200
Then any possibly defective
stopping trial, and a stopping
00:55:13.200 --> 00:55:15.720
trial is a set of rules for
when you're going to stop
00:55:15.720 --> 00:55:22.350
observing things, any stopping
trial can be represented as a
00:55:22.350 --> 00:55:24.280
rooted tree.
00:55:24.280 --> 00:55:26.970
If you don't know what a rooted
tree is, a rooted tree
00:55:26.970 --> 00:55:28.080
is what I've drawn here.
00:55:28.080 --> 00:55:30.740
So, there's no confusion here.
00:55:30.740 --> 00:55:33.670
A rooted tree is something that
starts on the left hand
00:55:33.670 --> 00:55:35.740
side and it grows for a while.
00:55:35.740 --> 00:55:36.910
It's like an ordinary tree.
00:55:36.910 --> 00:55:40.420
It has a root, and it has
branches that go up.
00:55:43.580 --> 00:55:48.530
But any possibly defective
stopping trial can be
00:55:48.530 --> 00:55:52.250
represented as a rooted tree
where the trial at which each
00:55:52.250 --> 00:55:55.320
sample path stops
is represented
00:55:55.320 --> 00:55:58.400
by a terminal node.
00:55:58.400 --> 00:56:00.780
So the example here I want
to use is the same
00:56:00.780 --> 00:56:03.360
example as in the text.
00:56:03.360 --> 00:56:06.210
I was trying to get another
example, which will be more
00:56:06.210 --> 00:56:07.440
interesting.
00:56:07.440 --> 00:56:11.490
The trouble is with these trees
they get very big very
00:56:11.490 --> 00:56:17.840
quickly, and therefore this is
not really a practical way of
00:56:17.840 --> 00:56:19.730
analyzing these problems.
00:56:19.730 --> 00:56:23.340
I think it's a nice conceptual
way of analyzing.
00:56:23.340 --> 00:56:28.240
So, the experiment is x is a
binary random variable, and
00:56:28.240 --> 00:56:33.090
stopping occurs when the pattern
1, 0 first occurs.
00:56:33.090 --> 00:56:36.030
You can certainly look at any
other pattern you want to, any
00:56:36.030 --> 00:56:37.520
much longer pattern.
00:56:37.520 --> 00:56:42.010
But what I want to do here is to
illustrate what the tree is
00:56:42.010 --> 00:56:46.220
that corresponds to this
stopping rule.
00:56:46.220 --> 00:56:53.680
So if the first random variable
has a value 1 and
00:56:53.680 --> 00:56:59.230
then the second one has a value
0, you observe 1, 0, and
00:56:59.230 --> 00:57:01.530
at that point the experiment
is over.
00:57:01.530 --> 00:57:04.430
You've seen the pattern
1, 0, and you stop.
00:57:04.430 --> 00:57:06.440
So there's a big circle there.
00:57:06.440 --> 00:57:11.040
If you observe 1 followed by a
1 followed by 0, again you
00:57:11.040 --> 00:57:15.830
stop 1, 1, 1, followed by a
0 you stop, and so forth.
00:57:15.830 --> 00:57:20.380
If you observe a 0 and then you
observe a 1 and then you
00:57:20.380 --> 00:57:24.020
observe a 0, that's the first
time at which you've seen the
00:57:24.020 --> 00:57:27.200
pattern 1, 0 so you
stop there.
00:57:27.200 --> 00:57:31.470
0, 1, 1, 0, you stop there,
and so forth.
00:57:31.470 --> 00:57:34.280
So you get this kind of
ladder structure.
00:57:34.280 --> 00:57:37.180
The point that I want to make
is not that this particular
00:57:37.180 --> 00:57:39.150
structure is very interesting.
00:57:39.150 --> 00:57:42.800
It's that whatever kind of rule
you decide to use for
00:57:42.800 --> 00:57:46.340
stopping, you can express
it in this way.
00:57:46.340 --> 00:57:50.090
You can express the points at
which you stop by big circles
00:57:50.090 --> 00:57:53.600
and the points at which you
don't stop as intermediate
00:57:53.600 --> 00:57:55.350
nodes where you keep on going.
00:57:58.940 --> 00:58:03.010
I think in some sense this takes
the picture which you
00:58:03.010 --> 00:58:07.860
have of sample values, of sample
functions with the rule
00:58:07.860 --> 00:58:12.050
that each sample value, which
is often the easiest way to
00:58:12.050 --> 00:58:18.280
express stopping rules, but in
terms of random variables it
00:58:18.280 --> 00:58:21.550
doesn't always give you
the right story.
00:58:21.550 --> 00:58:26.760
This gives you a story in terms
of all possible choices
00:58:26.760 --> 00:58:28.790
of the random variables.
00:58:28.790 --> 00:58:32.420
The first toss has to be
either a 1 or a 0.
00:58:32.420 --> 00:58:36.630
The second toss has to be a 1
or a 0, but if it's a 0 you
00:58:36.630 --> 00:58:38.580
stop and don't go any further.
00:58:38.580 --> 00:58:41.020
If it's a 1 you continue
further.
00:58:41.020 --> 00:58:43.570
So you keep continuing
along here, you can
00:58:43.570 --> 00:58:47.000
continue forever here.
00:58:47.000 --> 00:58:50.580
If you want to analyze
this, it has a
00:58:50.580 --> 00:58:53.310
rather peculiar analysis.
00:58:53.310 --> 00:58:57.870
How long does it take until
the first 1, 0 occurs?
00:58:57.870 --> 00:59:05.230
Well, if you start out with a 1
this experiment lasts until
00:59:05.230 --> 00:59:07.920
the first 0 appears.
00:59:07.920 --> 00:59:10.930
Because if the 0 doesn't appear
right away another 1
00:59:10.930 --> 00:59:13.430
appears, so you keep
going along this
00:59:13.430 --> 00:59:16.870
path until a 0 appears.
00:59:16.870 --> 00:59:21.180
If you see a 0 first and then
you see a 1, you're back in
00:59:21.180 --> 00:59:24.160
the same situation again.
00:59:24.160 --> 00:59:26.370
You proceed until a 0 occurs.
00:59:26.370 --> 00:59:31.540
What this is really saying is
the time until you stop here
00:59:31.540 --> 00:59:37.690
consists of an arbitrary number,
possibly 0 of 0's,
00:59:37.690 --> 00:59:42.420
followed by an arbitrary number,
possibly 0 of 1's,
00:59:42.420 --> 00:59:45.260
followed by a single 0.
00:59:45.260 --> 00:59:48.500
So the only patterns you can
have here which would lead to
00:59:48.500 --> 00:59:53.670
these terminal nodes are some
number of 0's followed by some
00:59:53.670 --> 00:59:56.320
number of 1's, followed
by a final 0.
01:00:00.210 --> 01:00:04.280
But the example is not
important, so that just tells
01:00:04.280 --> 01:00:06.630
you what it means.
01:00:06.630 --> 01:00:11.700
With all of this we finally
have Wald's equality.
01:00:11.700 --> 01:00:18.140
And Wald's equality, it looks
like a strange thing.
01:00:18.140 --> 01:00:23.880
I spent so much time talking
about stopping rules because
01:00:23.880 --> 01:00:26.760
all the complexity lies
in the stopping rules.
01:00:26.760 --> 01:00:30.790
Wald's equality itself is a very
simple thing after you
01:00:30.790 --> 01:00:32.010
understand that.
01:00:32.010 --> 01:00:37.440
What Wald's equality says is,
let x sub n be a sequence of
01:00:37.440 --> 01:00:42.420
IID random variables, each
would mean x-bar.
01:00:42.420 --> 01:00:43.760
This is the kind of
thing we've been
01:00:43.760 --> 01:00:45.320
talking about a lot.
01:00:45.320 --> 01:00:49.740
If J is a stopping trial for x
sub n, n greater than or equal
01:00:49.740 --> 01:00:53.050
to 1, in other words, if it
satisfies that definition
01:00:53.050 --> 01:00:57.230
where it never peeks ahead, and
if the expected value of J
01:00:57.230 --> 01:01:02.695
is less than infinity, then the
sum s sub J equals x1, x2,
01:01:02.695 --> 01:01:07.520
up to x sub J, the amount that
the gambler has accumulated
01:01:07.520 --> 01:01:08.840
before the gambler stops.
01:01:11.730 --> 01:01:16.500
The sum of the stopping trial
satisfied expected value of
01:01:16.500 --> 01:01:21.120
the gain as equal to the
expected value of x times the
01:01:21.120 --> 01:01:24.610
expected number of
times you play.
01:01:24.610 --> 01:01:29.820
It sort of says that even when
you use these stopping rules
01:01:29.820 --> 01:01:33.080
to decide when to
stop, there's no
01:01:33.080 --> 01:01:34.330
way to beat the house.
01:01:41.760 --> 01:01:46.830
If you're playing a fair game,
your expected gain is equal to
01:01:46.830 --> 01:01:50.920
the expected gain for trial
times the expected number of
01:01:50.920 --> 01:01:53.210
times you play.
01:01:53.210 --> 01:01:55.375
And there's no way you
can get around that.
01:02:01.230 --> 01:02:03.430
Note that this is more
than a [? poof ?]
01:02:03.430 --> 01:02:04.670
and less than a proof.
01:02:04.670 --> 01:02:07.530
I'll explain why it's more
than a [? poof ?]
01:02:07.530 --> 01:02:11.630
and less than a proof
as we go.
01:02:11.630 --> 01:02:19.060
s sub J, the random variable s
sub J, it's equal to x sub 1
01:02:19.060 --> 01:02:21.630
times the indicator function
of j greater
01:02:21.630 --> 01:02:24.070
than or equal to 1.
01:02:24.070 --> 01:02:27.860
In other words, well, the
indicator function J greater
01:02:27.860 --> 01:02:31.900
than or equal to 1 is
just universally 1.
01:02:31.900 --> 01:02:35.630
x sub 2 times the indicator
function of J greater than or
01:02:35.630 --> 01:02:36.350
equal to 2.
01:02:36.350 --> 01:02:41.210
In other words, s sub J includes
x sub 2 if the
01:02:41.210 --> 01:02:45.490
experiment proceeds long enough
that you take the
01:02:45.490 --> 01:02:46.820
second trial.
01:02:46.820 --> 01:02:54.480
In other words, this quantity
here, is 1 minus the
01:02:54.480 --> 01:02:58.980
probability that you stop at
the end of the first trial.
01:02:58.980 --> 01:03:05.370
You continue here, each x
sub n is included if the
01:03:05.370 --> 01:03:09.920
experiment has not been
stopped before time n.
01:03:09.920 --> 01:03:15.360
So the experiments continue
under the event that the
01:03:15.360 --> 01:03:19.770
stopping time was greater than
or equal to the J. So the
01:03:19.770 --> 01:03:25.180
expected value of s of J is
equal to the expected value of
01:03:25.180 --> 01:03:30.520
this sum here, of x sub n times
the indicator function
01:03:30.520 --> 01:03:34.840
of J greater than or equal to
n, which is equal to the sum
01:03:34.840 --> 01:03:38.730
over n of the expected value
of x sub n times this
01:03:38.730 --> 01:03:39.980
indicator function.
01:03:43.100 --> 01:03:46.280
By this time I hope that most
of you are at least a little
01:03:46.280 --> 01:03:52.610
bit sensitive to interchanging
expectations and sums.
01:03:52.610 --> 01:03:56.730
And the notes and the problem
sets deal with that.
01:03:56.730 --> 01:03:59.400
That's the part of the proof
that I don't want to talk
01:03:59.400 --> 01:04:02.410
about here, because it isn't
really very interesting.
01:04:02.410 --> 01:04:08.610
It's just one of these typical
tedious analysis things.
01:04:08.610 --> 01:04:10.730
Most of the time
this is valid.
01:04:10.730 --> 01:04:14.470
We will see an example where
it isn't, so you'll see
01:04:14.470 --> 01:04:16.630
what's going on.
01:04:16.630 --> 01:04:20.060
The essence of the proof,
however, the interesting part
01:04:20.060 --> 01:04:25.690
of the proof is to show that xn
and this indicator function
01:04:25.690 --> 01:04:28.630
are independent random
variables.
01:04:28.630 --> 01:04:34.140
So you can think of that as a
separate limit, that the n-th
01:04:34.140 --> 01:04:40.020
trial and the indicator function
of J greater than or
01:04:40.020 --> 01:04:43.930
equal to n are independent
of each other.
01:04:43.930 --> 01:04:47.430
This seems a little bit weird,
and it seems a little bit
01:04:47.430 --> 01:04:51.770
weird because this includes
the event that
01:04:51.770 --> 01:04:54.030
J is equal to n.
01:04:54.030 --> 01:04:56.840
And the event that J is equal
to n is something that's
01:04:56.840 --> 01:05:05.040
decided on the basis of the
previous arrivals, or the
01:05:05.040 --> 01:05:07.440
previous what have you.
01:05:07.440 --> 01:05:11.130
It's highly influenced
by x sub n.
01:05:11.130 --> 01:05:14.950
So, x sub n is not independent
of the indicator
01:05:14.950 --> 01:05:19.240
function of J equals n.
01:05:19.240 --> 01:05:22.760
So this is strange.
01:05:22.760 --> 01:05:24.440
But we'll see how
this works out.
01:05:27.750 --> 01:05:35.810
Now, if we want to show that
x sub n and this indicator
01:05:35.810 --> 01:05:41.300
function are independent, the
thing that you do is note that
01:05:41.300 --> 01:05:46.220
the indicator function of J
greater than or equal to n,
01:05:46.220 --> 01:05:50.130
this is the set of events in
which J is greater than or
01:05:50.130 --> 01:05:51.240
equal to n.
01:05:51.240 --> 01:05:53.910
What's the complement
of that event?
01:05:53.910 --> 01:05:57.730
The complement of the event J
greater than or equal to n is
01:05:57.730 --> 01:06:00.230
the event J less than n.
01:06:00.230 --> 01:06:04.280
So this indicator function
is 1 minus
01:06:04.280 --> 01:06:06.110
this indicator function.
01:06:06.110 --> 01:06:09.890
J less than n is the complement
of the event J
01:06:09.890 --> 01:06:11.500
greater than or equal to n.
01:06:11.500 --> 01:06:14.140
When you perform the experiment
either this
01:06:14.140 --> 01:06:19.020
happens, in which case this is
1, or this happens, in which
01:06:19.020 --> 01:06:24.760
case this is 1, and if
this is 1, this is 0.
01:06:24.760 --> 01:06:27.450
If this is 1, this is 0.
01:06:27.450 --> 01:06:32.020
Also, the indicator function
and J less than n is a
01:06:32.020 --> 01:06:37.370
function of x1, x2, up
to x sub n minus 1.
01:06:37.370 --> 01:06:42.190
Let me spell that out a little
more because it looks a little
01:06:42.190 --> 01:06:45.020
strange the way it is.
01:06:45.020 --> 01:06:56.300
i of J less than n is equal to
i if the indicator random
01:06:56.300 --> 01:07:01.090
variable for J equals 1.
01:07:03.950 --> 01:07:08.640
Union with the indicator
random variable
01:07:08.640 --> 01:07:10.700
for J equals 2.
01:07:10.700 --> 01:07:19.910
Union indicator function
for J equals n minus 1.
01:07:19.910 --> 01:07:22.190
These are all disjoint events.
01:07:22.190 --> 01:07:24.590
You can't stop at two
different times.
01:07:24.590 --> 01:07:29.280
So if you stop before time n you
either stop at time 1, or
01:07:29.280 --> 01:07:34.710
you stop at time 2, or you
stop at time n minus 1.
01:07:37.540 --> 01:07:39.685
This event is independent.
01:07:43.740 --> 01:07:49.616
Well not independent, it
depends on x sub 1.
01:07:53.160 --> 01:07:58.550
This event depends
on x1 and x2.
01:07:58.550 --> 01:08:07.040
This event depends on x1
up to x sub n minus 1.
01:08:07.040 --> 01:08:12.320
OK, so this is what I mean when
I say that the event J
01:08:12.320 --> 01:08:18.550
less than n is determined by
x1 up to x of n minus 1,
01:08:18.550 --> 01:08:23.060
because each of these sub events
is determined by a sub
01:08:23.060 --> 01:08:24.390
event up there.
01:08:29.310 --> 01:08:33.939
Since we're looking at the
situation now where the x is
01:08:33.939 --> 01:08:44.510
our IID and this event is a
function of x1 x of n minus 1,
01:08:44.510 --> 01:08:47.090
it just depends on those
random variables.
01:08:47.090 --> 01:08:53.310
These random variables are
independent of x sub n so i of
01:08:53.310 --> 01:09:00.840
J less than n is independent of
x sub n and thus x sub n is
01:09:00.840 --> 01:09:05.359
independent of J greater
than or equal to n.
01:09:05.359 --> 01:09:08.350
Now, as I said, this is very
surprising, and it's very
01:09:08.350 --> 01:09:16.800
surprising because this event,
J greater than or equal to n,
01:09:16.800 --> 01:09:20.540
typically depends very
heavily on x sub n.
01:09:20.540 --> 01:09:24.930
It depends on J equals n
plus 1, and so forth.
01:09:24.930 --> 01:09:28.319
So it's a little paradoxical
and the resolution of the
01:09:28.319 --> 01:09:33.149
paradox is that given that J is
greater than or equal to n,
01:09:33.149 --> 01:09:38.990
in other words, given that you
haven't stopped before time n,
01:09:38.990 --> 01:09:43.779
the time at which you stop
is very dependent on the
01:09:43.779 --> 01:09:46.170
observations that you make.
01:09:46.170 --> 01:09:51.260
But whether you stopped before
time n or whether you haven't
01:09:51.260 --> 01:09:54.040
stopped before time n is
independent of x sub n.
01:09:57.760 --> 01:10:01.680
So it really is this
conditioning here that makes
01:10:01.680 --> 01:10:03.570
this a confusing issue.
01:10:03.570 --> 01:10:08.730
But whether or not J greater
than n occurs depends only on
01:10:08.730 --> 01:10:12.330
x1 up to x sub n minus 1.
01:10:12.330 --> 01:10:17.060
And with that it's easy enough
to finish the proof, it's just
01:10:17.060 --> 01:10:18.920
writing a few equations.
01:10:18.920 --> 01:10:23.320
The expected value of s sub J is
equal to the sum over n and
01:10:23.320 --> 01:10:27.440
the expected value of x sub n
times the indicator function
01:10:27.440 --> 01:10:30.290
of J greater than
or equal to n.
01:10:30.290 --> 01:10:33.590
We've just shown that these
are independent.
01:10:33.590 --> 01:10:36.870
This random variable is
independent of this random
01:10:36.870 --> 01:10:40.150
variable, that's independent of
the event J greater than or
01:10:40.150 --> 01:10:41.940
equal to n, so it's independent
01:10:41.940 --> 01:10:43.890
of the random variable.
01:10:43.890 --> 01:10:47.550
The indicator function is J
greater than or equal to n, so
01:10:47.550 --> 01:10:49.700
we can write this way.
01:10:49.700 --> 01:10:53.140
Now, when we have this
expression each of the x sub
01:10:53.140 --> 01:10:58.650
n's are IID, so all of these are
the same quantity x-bar.
01:10:58.650 --> 01:11:02.160
So since they're all the same
quantity x-bar we can just
01:11:02.160 --> 01:11:04.550
take it outside of
the summation.
01:11:04.550 --> 01:11:09.805
So we have x-bar times the sum
of the expected value of i of
01:11:09.805 --> 01:11:13.100
J greater than or equal to n.
01:11:13.100 --> 01:11:17.480
Now, the expected value of this
indicator function, this
01:11:17.480 --> 01:11:22.800
indicator function is 1 when J
is greater than or equal to 1,
01:11:22.800 --> 01:11:24.610
and it's 0 otherwise.
01:11:24.610 --> 01:11:29.450
So the expected value of this
is just a probability that J
01:11:29.450 --> 01:11:31.940
is greater than or equal to n.
01:11:31.940 --> 01:11:38.100
So, this is equal to the
expected value of x times the
01:11:38.100 --> 01:11:41.980
sum of the probabilities
that J is greater
01:11:41.980 --> 01:11:43.230
than or equal to n.
01:11:48.040 --> 01:11:51.560
This is really adding the
elements of the complimentary
01:11:51.560 --> 01:11:55.770
distribution function for J, and
that gives us the expected
01:11:55.770 --> 01:12:02.290
value of J. If finite or not
finite gives us this anyway.
01:12:02.290 --> 01:12:06.515
So, this really is a proof
except for that one step.
01:12:15.050 --> 01:12:20.700
Except for this interchange of
expectation and summation here
01:12:20.700 --> 01:12:29.480
which is, well, we will get an
idea of what that has to do
01:12:29.480 --> 01:12:30.970
with anything.
01:12:30.970 --> 01:12:33.570
So let's look at an example.
01:12:33.570 --> 01:12:36.270
Stop when you're ahead.
01:12:36.270 --> 01:12:41.360
And stop when you're ahead
is a strategy for
01:12:41.360 --> 01:12:45.180
gambling with coins.
01:12:45.180 --> 01:12:47.440
You toss a coin with
probability of
01:12:47.440 --> 01:12:48.980
heads equal to p.
01:12:52.280 --> 01:12:55.020
And we want to look at all the
different cases for p, whether
01:12:55.020 --> 01:12:59.860
p is fair or biased in your
favor or biased in the other
01:12:59.860 --> 01:13:02.250
person's favor.
01:13:02.250 --> 01:13:05.090
And the rule is that you
stop whenever you're
01:13:05.090 --> 01:13:07.290
winnings reach one.
01:13:07.290 --> 01:13:11.590
So you keep on dumbly gambling,
and gambling, and
01:13:11.590 --> 01:13:15.150
gambling, until finally you get
to the point where you're
01:13:15.150 --> 01:13:18.580
one ahead and when you're one
ahead you sigh a sigh of
01:13:18.580 --> 01:13:23.140
relief and say, now my wife
won't be angry at me, or my
01:13:23.140 --> 01:13:28.400
husband won't divorce me, or
something of that sort.
01:13:28.400 --> 01:13:33.400
And you walk away swearing
never to gamble again.
01:13:33.400 --> 01:13:37.990
Except, when you look at
this you say, aha!
01:13:37.990 --> 01:13:42.400
If the game is fair, I am sure
to win, and after I win I
01:13:42.400 --> 01:13:45.110
might as well play again,
because I'm sure to win again,
01:13:45.110 --> 01:13:48.270
and I'm sure to win again, and
I'm sure to win again, and so
01:13:48.270 --> 01:13:50.700
forth, which is all
very strange.
01:13:50.700 --> 01:13:53.870
And we'll see why that
is in a little bit.
01:13:53.870 --> 01:13:57.210
But first let's look at
the case where p is
01:13:57.210 --> 01:13:59.480
greater than 1/2.
01:13:59.480 --> 01:14:03.340
In other words, you have a
loaded coin and you've talked
01:14:03.340 --> 01:14:08.270
somebody else, some poor sucker,
into playing with you,
01:14:08.270 --> 01:14:10.650
and you've decided you're going
to stop the first time
01:14:10.650 --> 01:14:12.420
you're ahead.
01:14:12.420 --> 01:14:18.490
And because p is greater than
1/2, the expected value of
01:14:18.490 --> 01:14:23.590
your gain each time you play
the game is greater than 0.
01:14:23.590 --> 01:14:27.820
You're applying IID random
variables so that eventually
01:14:27.820 --> 01:14:30.500
your winnings, if you kept
on playing forever,
01:14:30.500 --> 01:14:32.360
would become humongous.
01:14:32.360 --> 01:14:37.030
So at some point you have to be
1 ahead in this process of
01:14:37.030 --> 01:14:40.300
getting to the point where
you're arbitrarily large
01:14:40.300 --> 01:14:41.370
amounts ahead.
01:14:41.370 --> 01:14:47.190
So with probability 1, you will
eventually become ahead.
01:14:47.190 --> 01:14:52.670
We would like to know since we
know that you're going to be
01:14:52.670 --> 01:14:56.030
ahead at some point, how
long has it going to
01:14:56.030 --> 01:14:59.410
take you to get ahead?
01:14:59.410 --> 01:15:02.320
We know that j has to be a
random variable in this case,
01:15:02.320 --> 01:15:07.200
because we know you have to win
with probability 1 s sub J
01:15:07.200 --> 01:15:09.820
equals 1 with probability 1.
01:15:09.820 --> 01:15:15.190
Namely, your winnings up to the
time J. At the time J your
01:15:15.190 --> 01:15:16.710
winnings are 1.
01:15:16.710 --> 01:15:21.210
So s sub J is equal to 1, the
expected value of s sub J is
01:15:21.210 --> 01:15:28.330
equal to 1, and Wald says that
the expected value of J, the
01:15:28.330 --> 01:15:34.060
expected time you have to play,
is just 1 over x-bar.
01:15:37.590 --> 01:15:38.840
Isn't that neat?
01:15:41.470 --> 01:15:44.800
Which says it's 1
over 2p minus 1.
01:15:44.800 --> 01:15:52.520
The expected value of x is p
plus minus 1 times 1 minus p
01:15:52.520 --> 01:15:56.450
and if you look at that it's 2p
minus 1, which is positive
01:15:56.450 --> 01:15:57.790
when p is bigger than 1/2.
01:16:00.650 --> 01:16:06.200
After all the abstraction of
Wald's equality it's nice to
01:16:06.200 --> 01:16:10.020
look at this and solve it in a
different way to see if we get
01:16:10.020 --> 01:16:10.940
the same answer.
01:16:10.940 --> 01:16:12.190
So we'll do that.
01:16:20.430 --> 01:16:26.350
If J is equal to 1, that means
that on the first flip of the
01:16:26.350 --> 01:16:31.780
coin you've got heads, you
became 1 up and you stopped.
01:16:31.780 --> 01:16:36.440
So the experiment was over
at time 1 if the
01:16:36.440 --> 01:16:39.070
first toss was a head.
01:16:39.070 --> 01:16:43.350
So if the first toss was a tail,
on the other hand, what
01:16:43.350 --> 01:16:48.030
has to happen in order
for you to win?
01:16:48.030 --> 01:16:51.920
Well, you're sitting there at
minus 1 and you're going to
01:16:51.920 --> 01:16:55.900
stop when you get to plus 1.
01:16:55.900 --> 01:16:59.670
Every time you move up one or
down one, which means if
01:16:59.670 --> 01:17:03.320
you're ever going to get from
minus 1 up to plus 1, you've
01:17:03.320 --> 01:17:07.410
got to go through 0 in
order to get there.
01:17:07.410 --> 01:17:11.410
So let's look at the expected
time that it takes to get to 0
01:17:11.410 --> 01:17:14.820
for the first time, and then
we'll look at the expected
01:17:14.820 --> 01:17:20.350
time that it takes to get from
0 to 1 for the first time.
01:17:20.350 --> 01:17:24.960
Well, the expected time that it
takes to get from minus 1
01:17:24.960 --> 01:17:29.560
to 0 is exactly the same as
the expected time that it
01:17:29.560 --> 01:17:31.980
takes to get from 0 to 1.
01:17:31.980 --> 01:17:39.700
So, the equation that you then
have is the expected value of
01:17:39.700 --> 01:17:42.690
J is going to be 1.
01:17:42.690 --> 01:17:46.930
This is the first step you have
to take anyway, and with
01:17:46.930 --> 01:17:52.260
probability 1 minus p you went
to minus 1, and if you went to
01:17:52.260 --> 01:17:57.965
minus 1 you still have J-bar
left to go to get to 0 again,
01:17:57.965 --> 01:18:06.960
and another J-bar to get to 1,
so J-bar is equal to 1 plus
01:18:06.960 --> 01:18:10.940
failing the first time it
takes 2J bar to get
01:18:10.940 --> 01:18:12.720
back up to plus 1.
01:18:12.720 --> 01:18:19.870
If you analyze this equation,
J-bar is equal to 1 plus
01:18:19.870 --> 01:18:26.490
2J-bar minus 2Jp-bar, you work
that out and it's just 1 over
01:18:26.490 --> 01:18:27.980
2p minus 1.
01:18:27.980 --> 01:18:31.850
So fortunately we get the same
answer as we got with the
01:18:31.850 --> 01:18:33.100
Wald's equality.
01:18:36.580 --> 01:18:39.010
Let's go one again.
01:18:39.010 --> 01:18:43.880
Let's look at what happens if
you go to the casino and you
01:18:43.880 --> 01:18:50.130
play a game where you
win $1 or lose $1.
01:18:50.130 --> 01:18:55.350
But the probability in casinos
is somehow always tilted to be
01:18:55.350 --> 01:18:56.600
a little less.
01:19:00.790 --> 01:19:06.080
If you play for red or black in
roulette, what happens is
01:19:06.080 --> 01:19:11.440
that red or black are equally
likely, but there's the 0 and
01:19:11.440 --> 01:19:15.090
double 0 where the house
always wins.
01:19:15.090 --> 01:19:18.470
So your probability of winning
is always just a little less
01:19:18.470 --> 01:19:23.990
than 1/2, so that's the
situation we have here.
01:19:23.990 --> 01:19:27.450
It's still possible
to win and stop.
01:19:27.450 --> 01:19:32.610
Like if you win on the
first toss, then
01:19:32.610 --> 01:19:34.880
you're going to stop.
01:19:34.880 --> 01:19:38.340
So J equals 1 with
probability p.
01:19:38.340 --> 01:19:42.780
J equals 3 if you lose,
and then you win,
01:19:42.780 --> 01:19:43.930
and then you win.
01:19:43.930 --> 01:19:48.660
So J equals 3 with probability
p squared times 1 minus b.
01:19:48.660 --> 01:19:52.680
And there are lots of other
situations you can look at J
01:19:52.680 --> 01:19:57.150
equals 5 and see when you win
then, and you can count all
01:19:57.150 --> 01:19:58.990
these things up.
01:19:58.990 --> 01:20:04.730
Let's try to find a simple
way of counting them up.
01:20:04.730 --> 01:20:08.880
Let's let theta be the
probability that J is less
01:20:08.880 --> 01:20:10.960
than infinity.
01:20:10.960 --> 01:20:13.480
In other words, theta is the
probability that you're ever
01:20:13.480 --> 01:20:15.010
going to stop.
01:20:15.010 --> 01:20:18.610
Sometimes in this game you won't
ever stop because you're
01:20:18.610 --> 01:20:21.820
going to lose for while, and
as soon as you lose a fair
01:20:21.820 --> 01:20:26.090
amount you're very unlikely to
ever recover because you're
01:20:26.090 --> 01:20:27.640
drifting South all the time.
01:20:40.970 --> 01:20:43.580
Since you're drifting South,
theta is going to
01:20:43.580 --> 01:20:46.750
be less than 1.
01:20:46.750 --> 01:20:49.050
So how do we analyze this?
01:20:49.050 --> 01:20:53.980
Same way as before, given that
J is greater than 1, in other
01:20:53.980 --> 01:20:57.970
words, given that you lose on
the first trial, the event J
01:20:57.970 --> 01:21:04.310
less than infinity requires that
at some time you go from
01:21:04.310 --> 01:21:06.580
minus 1 back up to 1.
01:21:06.580 --> 01:21:12.170
In other words, there's some
time m at which s sub m minus
01:21:12.170 --> 01:21:14.330
s sub 1 is equal to 1.
01:21:14.330 --> 01:21:17.760
s sub m is the time at which
you first get from minus 1
01:21:17.760 --> 01:21:21.860
back up to 0, and then there's
some time that it takes to get
01:21:21.860 --> 01:21:25.530
from 0 up to 1, and
there's some
01:21:25.530 --> 01:21:27.820
probability that ever happened.
01:21:27.820 --> 01:21:33.400
The probability that you ever
reach 0 is equal to theta, the
01:21:33.400 --> 01:21:38.720
probability you ever get from 0
up to plus 1 is theta again.
01:21:38.720 --> 01:21:42.740
So theta is equal to p, which
is the probability that you
01:21:42.740 --> 01:21:46.640
win right away, plus probability
you lose on the
01:21:46.640 --> 01:21:53.990
first toss, and you ever get
from minus 1 to 0 and you ever
01:21:53.990 --> 01:21:55.650
get from 0 to 1.
01:21:55.650 --> 01:21:59.500
So theta equals p plus 1 minus
p times theta squared.
01:21:59.500 --> 01:22:01.010
There are two solutions
to this.
01:22:04.420 --> 01:22:09.120
One is that theta it is p over
1 minus p and the other is
01:22:09.120 --> 01:22:13.230
theta equals 1, which we
know is impossible.
01:22:13.230 --> 01:22:16.610
And thus J is defective,
and Wald's equation is
01:22:16.610 --> 01:22:19.240
inapplicable.
01:22:19.240 --> 01:22:23.600
But the solution that we have,
which is useful to know, is
01:22:23.600 --> 01:22:28.970
that your probability of ever
winning is p over 1 minus p.
01:22:28.970 --> 01:22:30.220
Nice to know.
01:22:33.130 --> 01:22:37.280
That's what happens when
you go to a casino.
01:22:37.280 --> 01:22:41.570
Finally, let's consider the
interesting case which is p
01:22:41.570 --> 01:22:43.470
equals 1/2.
01:22:43.470 --> 01:22:47.850
The limit as p approaches 1/2
from below, probability that J
01:22:47.850 --> 01:23:02.840
less than infinity goes from
p over 1 minus p goes to 1.
01:23:02.840 --> 01:23:07.830
In other words, in a fair game,
and this is not obvious,
01:23:07.830 --> 01:23:11.350
there's probability 1 that you
will eventually get to the
01:23:11.350 --> 01:23:16.270
point where s sub
n is equal to 1.
01:23:16.270 --> 01:23:23.550
You can sort of see this if you
view this in terms of the
01:23:23.550 --> 01:23:24.840
central limit theorem.
01:23:27.850 --> 01:23:30.910
And we looked pretty hard at the
central limit theorem in
01:23:30.910 --> 01:23:33.910
the case of these binary
experiments.
01:23:33.910 --> 01:23:42.170
As n gets bigger and bigger, the
standard deviation of this
01:23:42.170 --> 01:23:45.720
grid grows as the square
root of n.
01:23:45.720 --> 01:23:48.900
Now the standard deviation is
growing as the square root of
01:23:48.900 --> 01:23:53.675
n and over a period of n
you're wobbling around.
01:23:56.560 --> 01:24:00.050
You can see that it doesn't
make any sense to have the
01:24:00.050 --> 01:24:03.630
standard deviation growing with
the square root of n if
01:24:03.630 --> 01:24:06.820
you don't at some point go from
the positive half down to
01:24:06.820 --> 01:24:09.150
the negative half.
01:24:09.150 --> 01:24:12.350
If you always stay in the
positive half after a certain
01:24:12.350 --> 01:24:16.920
point, then that much time later
you're always going to
01:24:16.920 --> 01:24:22.070
stay in the same half again, and
that doesn't work in terms
01:24:22.070 --> 01:24:24.600
of the square root of n.
01:24:24.600 --> 01:24:28.850
So you can convince yourself on
the basis of a very tedious
01:24:28.850 --> 01:24:31.670
argument what this very simple
argument shows you.
01:24:35.010 --> 01:24:38.430
And as soon as we start studying
Markov chains with
01:24:38.430 --> 01:24:41.710
uncountably infinite number of
states, we're going to analyze
01:24:41.710 --> 01:24:43.470
this case again.
01:24:43.470 --> 01:24:46.720
In other words, this is a very
interesting situation because
01:24:46.720 --> 01:24:48.750
something very interesting
happens here and we're going
01:24:48.750 --> 01:24:51.880
to analyze it at least
three different ways.
01:24:51.880 --> 01:24:54.560
So if you don't believe
this way, you will
01:24:54.560 --> 01:24:57.780
believe the other way.
01:24:57.780 --> 01:25:04.351
And what happens as p approaches
1/2 from above you
01:25:04.351 --> 01:25:08.040
know the expected value of
J approaches infinity.
01:25:08.040 --> 01:25:12.020
So what's going to happen at p
equals 1/2 as the expected
01:25:12.020 --> 01:25:17.500
time for you to win goes to
infinity, the probability of
01:25:17.500 --> 01:25:23.460
your winning goes to 1, and
Wald's equality does not hold
01:25:23.460 --> 01:25:28.800
because the expected value of x
is equal to 0, and you look
01:25:28.800 --> 01:25:33.050
at 0 times infinity and it
doesn't tell you anything.
01:25:33.050 --> 01:25:39.520
The expected value of s sub J s
sub n at every value of n is
01:25:39.520 --> 01:25:44.610
0, but s sub n when the
experiment ends is equal to 1.
01:25:44.610 --> 01:25:49.760
So there's something very funny
going on here and if you
01:25:49.760 --> 01:25:52.800
think in practical terms though,
it takes an infinite
01:25:52.800 --> 01:25:58.250
time to make your $1.00 and more
important, it requires
01:25:58.250 --> 01:26:00.610
access to an infinite capital.
01:26:00.610 --> 01:26:04.570
We will see that if you limit
your capital, if you limit how
01:26:04.570 --> 01:26:09.050
far South you can go, then in
fact you don't win with
01:26:09.050 --> 01:26:13.720
probability 1, because you
can go there also.
01:26:13.720 --> 01:26:16.500
Next time we will apply this
to renewal processes.