The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare continue to
offer high quality educational resources for free.
To make a donation or view additional materials from
hundreds of MIT courses, visit MIT OpenCourseWare at
ocw.mit.edu.
PROFESSOR: OK, I guess we're all set for getting close to
the end, coming now to a race about whether we could say
anything meaningful about Martingales or not.
But I think we can.
I want to spend a little time reviewing the Wald identity
today and also sequential tests.
It turns out that last time on the slides--
I didn't get the thresholds confused--
I got hypothesis 0 and hypothesis 1 interchanged from
the way we usually do them.
And it doesn't make any difference.
There's no difference between hypothesis 0 and hypothesis 1.
And you can do it either way you want to.
But it gets very confusing when you switch from one to
the other when you're halfway through an argument.
So I'm going to go through part of that again today.
And we will get revised slides on the web, so that if you
want to see them with the hypotheses done in a
consistent way, you will see it there.
That should be on there by this afternoon I hope.
OK, so let's go on and review what Mr. Wald said.
He was talking about a random walk.
Random walk consists of a bunch of a sequence of IID
random variables.
The random walk consists of the sequence of partial sums
of those random variables.
And the question is if this random walk is taking place
and you have two thresholds, one at alpha and one of beta--
and beta is below 0 and alpha is above 0--
and you start at 0, of course, and the question is when do
you cross one of these two thresholds and which
threshold do cross?
What's the probability of crossing each other, and
everything else you can say about this problem.
And it turns out this is a very major problem as far as
stochastic processes are concerned, because it comes up
almost everywhere.
It certainly comes up as far as
hypothesis testing is concerned.
It's probably the major problem there when you get
into sequential analysis.
It's the major problem there.
So it's a very important problem.
And what Wald said was if you let the random variable J be
the stopping time of this random walk, namely, the time
in which the walk first crosses either alpha or
crosses the beta, and then he said no matter what r you
choose and the range of points where the moment generating
function g sub X of r exists.
Y You can pick any r in that range, and then what you get
is this strange looking equality here.
And I pointed out last time it just wasn't all that strange,
because if instead of using the stopping time of when you
cross a threshold, if instead you used as a stopping time
just some particular end.
You go for some number of steps, and then you stop.
And at that point, you have the expected
value of E to the rsn.
The expected value of E to the rsn by definition is the
moment generating function at r of S sub n, which is exactly
equal to the minus J times gamma n times gamma of r.
So all we're doing here, all that Wald did--
as it turns out, it was quite a bit--
was to say that when you replace a fixed end with a
stopping time, you still get the same result.
We're stating it here just for the case of two thresholds.
Wald stated it in much general terms.
We'll use it a more general terms, when we say more about
martingales.
X, now remember is the underlying random variable.
S sub n is the sum of the X's.
If X bar is less than 0, and if gamma r star equals 0, r
star is the r at which gamma of r equals 0.
It's the second root of gamma of r.
Gamma of r, if you remember, looks like this.
This is r star here.
This is the expected value of X as the slope here.
And we're assuming that X bar is less than 0 for this.
I don't know how that greater than 0 got in.
And then what it says is the probability that SJ is greater
than or equal to alpha is less than or equal to the minus
alpha r star.
And last time, remember, we went through a long, messy
bunch of equations for that.
I looked at it again, and this is just the simple old Markov
inequality again.
All you do to get this is you say, OK, think of the random
variable, E to the r SJ.
SJ is a very complicated random variable, but it's a
random variable, nonetheless.
So either the rSJ is a random variable and the expected
value of that random variable is at r star, the expected
value of it is just one.
I'll write down.
It'll be easier.
The expected value e to the r star S sub J is equal to 1.
And therefore, the probability that E to the r star SJ is a
greater than or equal to E to the r star alpha is just less
than or equal to 1 over E to the r star alpha, OK?
And that's what the inequality says.
So that's all there is to it.
OK, what?
AUDIENCE: I don't really see why these two [INAUDIBLE]?
They don't [INAUDIBLE].
PROFESSOR: You need x1 negative so that you get
another root so that r star exists.
If r star is positive, if the expected value of x is
positive, then r star is down here at negative r.
I mean, you're talking about the other
threshold in a sense.
OK, this is valid for all lower thresholds.
And it's also valid for no threshold.
OK, in other words, this equation here does not have
beta in it at all.
So this equation is an upper bound on the probability, that
you're going to cross that threshold of alpha.
And that upper bound is valid, no matter where you put the
lower bound at all.
So you can go to the limit as the lower
bound goes to infinity.
And this inequality should still be valid.
You have a homework problem where you actually prove that.
Sometimes when things go to infinity, funny things happen.
And that proves that nothing funny happens then.
So what happens then is the probability that you ever
cross a threshold at plus alpha, when you have a random
variable, which has a negative mean, is this exponent here.
And we also sort of showed by looking at the turn off bound
that this bound is pretty tight.
So in other words, what this is saying is when you're
looking at threshold crossing problems--
this quantity here, this quantity where the second root
of gamma of r is--
that's sort of the crucial parameter
that you want to know.
Usually the first thing you want to know about a random
variable is its mean, its variance, all sorts
of things like that.
This is saying if you're interested in thresholds,
forget about all those things, look at r star.
If r star is positive that means it means is negative, so
there's no problem there.
But this one quantity here is sort of the most important
parameter of all of these problems.
OK, so let's go back to look at a hypothesis testing again,
where we're looking at the likelihood ratio of being the
ratio of the density for hypothesis 0 divided by
hypothesis 1.
What you get then is you observe this sequence Y sub n.
These are the observations that you're taking.
In other words, nature at the beginning of this whole
experiment chooses either H equals 0 or H equals 1.
At that point, you start to make measurements.
Now whether nature chooses H equals 0 before or after or
when doesn't make any difference.
The point is the experiment consists of nature choosing
one of these two hypotheses.
You know all the probabilities that exist in the world in
this model.
You go making these measurements.
All you of observe is these measurements.
You don't observe what the hypothesis is, so you define
this likelihood ratio of the ratio of the densities of the
vector Y for H equals 0 and the vector Y with H equals 1.
These quantities exist no matter what the a priori
probabilities of the thresholds
are or anything else.
Even without all of that, so long as you have a model which
tells you what the densities of these observations are,
conditional on each hypothesis,
you can define this.
This doesn't depend on a priori probabilities at all.
OK, so now you look at the probability that H is equal to
0, given all these observations divided by the
probability it's equal to 1.
What you get here now, you have the a priori
probabilities, p0 over p1.
Here is the likelihood ratio here.
So what you have this p0 over p1 times the likelihood ratio
of this vector of however many
observations you have observed.
It's just a nice way of breaking up the problem into
the likelihood ratio and the a priori probabilities.
Incidentally, we haven't talked about this at all, but
there's an important idea and all of this hypothesis testing
of a sufficient statistic, and what do you think a sufficient
statistic is.
It's anything from which you can calculate
the likelihood ratio.
In other words, what we're saying here, the point we're
making, is that any intelligent choice of
hypothesis is it based on a threshold test on the
likelihood ratio.
And therefore, the only thing you can really be interested
in in all your observations is just what is
the likelihood ratio?
If they make all these 1,000 observations complicated sort
of thing, you calculate one number.
And that's the only thing you're interested in.
And anything from which that number could be calculated is
a sufficient statistic.
And anything from which it can't be calculated you've
thrown away some of the information that you have.
If you study communication and you study detection, you study
how to receive data that's being sent, what you find is
that right at the beginning, even before you do any
detection, even before you do any filtering, there's some
idea of a sufficient statistic there.
That's what you need in order to calculate everything else.
And you want to make sure that you have that.
So that's an important idea there.
OK, but anyway, the MAP rule, which comes right from this,
says if you have these a priori probabilities, and
you're trying to maximize the probability of choosing
correctly, what do you do?
Well, your probability of H equals 0 was the correct
hypothesis, given all the observations you
made, is in fact this.
The probability that H equals 1 is the correct
hypothesis is this.
What do you do if you want to maximize the probability of
being correct?
You choose the one which is biggest.
In other words, what you do is you look at this number.
And if this number is bigger than 1, you choose 0.
If it's less than 1, you choose hypothesis 1.
And what it turns out to is threshold of rule.
You take this likelihood ratio.
You compare it with p1 over p0.
And in this case, you select h equals 0.
In this case you select H equals 1.
And the last time I just a 1 and 0 reversed, which is fine,
but if you reverse them one place, you want a reverse them
every place.
And every other threshold test does something like this,
except you replace p1 over p0 with some arbitrary threshold.
You say whatever reason you want to find for that
threshold, that's the only intelligent kind of
test you can make.
OK, then we define the log-likelihood ratio of the
logarithm of the likelihood ratio.
And that was nice because it was a sum of this quantity
related to the individual observations.
For each observation you really want to know what f of
Y given H of Y given 0, divided by Y given 1.
You want to divide those two.
You want to take the logarithm of it.
And then you have those numbers, and the sufficient
statistic that you're interested in is just a sum of
those numbers.
So you're looking at a sum of IID, random variable.
IID, why IID?
Well, under the hypothesis that H is equal to 1, those
Yi's are IID.
And therefore under the hypothesis that
H is equal to 1.
Well, little Zi is just a sample value.
If you look at the random variable which has these
sampled values, Z sub i, under the probability measure,
corresponding to H equals 1, those Z sub i's are IID.
So what that says is when you look at these sums of random
variables, the sum of Zi from 1 to n, under hypothesis H
equals 1, what do you get?
You get a random walk.
You get a sum of IID random variables.
If you take more observations, S sub n just changes.
With n changing, then you have a larger number of
observations.
So the random walk goes a little further out, and you
might get closer to a threshold or whatever.
And that's what we're trying to do here.
OK, so the Z sub i's under the hypothesis H equals 1, or IID,
and the moment generating function of the Z sub i's
given H equals 1, is this.
Let's be careful about this.
The sampled values of the Z sub i do not depend on the
hypotheses at all.
Namely, you make an observation.
You make an observation of Y sub i.
You calculate Z sub i from Y sub i.
That has nothing to do with whether H equals
0 or H equals 1.
You try to calculate this moment
generating function, however.
And you want to know what the probability
density of the Y's are.
And you get a different probability density for H
equals 1, then you get on the other hypothesis.
If the observations behaved the same way under both
hypotheses, it wouldn't make much sense to do the
observation.
Unless you have a government grant, and you're trying to
get money out of the government instead of trying
to do anything worthwhile.
Under those circumstances, you keep on making observations.
You now perfectly well that nothing is going
to come from them.
But otherwise, it's a little silly.
So this moment generating function under the hypothesis
H equals 1 is given by this quantity here.
And this density here is the same as this density here.
So you get this density to the 1 minus r power, and you get
this density to the r power.
So you get the product of these two densities.
You integrate it over Y, and that's what gamma 1 of r is.
Now I said that the really important thing in all of
these threshold problems is what is our star?
And for this problem, r star is trivial.
It's always the same.
r star is always equal to 1.
And the reason is when you set r equal to 1 here, this
quantity becomes 1.
This quantity becomes the density of Y
conditional on H equals 0.
When you integrate that, you get 1.
So for all of these hypothesis testing problems, r star is
equal to 1.
Gamma 1 of 1 is equal to 0.
And this is what this curve says.
OK, this is gamma 1 of r here.
This curve starts out here, negative slope.
It comes up here.
r star is equal to 1 in this case.
And that's sort of the end of the story for that.
Now if you are doing a test with a fixed value of n, you
say I'm going to make n observations, it's
all I have time for.
The week is over.
I'm going on vacation next week.
I've got to stop this test.
I've got to write my paper.
Take the end test.
You write your paper.
And what do you do?
You go through the optimal tests the best you can.
And what you find is given H equals 1, an error is going to
occur if the sum of random variables, namely the
log-likelihood likelihood ratio, exceeds the logarithm
of your threshold.
OK, this is whatever threshold you decide to establish.
And we showed before that the probability that S sub n is
greater than or equal to log of the threshold is evaluated
as E to the n times this quantity right here.
The probability of error given H equals 1 is
this quantity here.
Probability of error given H equals 1 is the probability
that the data looks like H equals 0 was a right
hypothesis.
In other words, that you crossed the threshold at plus
alpha, instead of crossing the threshold at beta.
Excuse me.
We have too many cases here we're looking
at, so it gets confusing.
What I'm looking at here is the probability that the
log-likelihood ratio exceeds this threshold data, whatever
we set beta to be.
Eta is set, depending on the cost of making errors of both
types on our a priori beta, if we have any and
all of those things.
And the probability of error given H equals 1 is this
quantity here, which has the threshold in it over there.
We've looked at that a number of times in a lecture.
We looked at it in chapter one.
And then those, we looked at it in chapter seven.
And you calculate it by taking this moment generating
function, drawing attention to it at the point where slope
natural log of eta divided by n.
And then you take where it comes in to this
vertical axis here.
And that's the exponent of the error of probability when
hypothesis 1 is correct.
Now, if the hypothesis is H equals 0 instead, at that
point with H equals 0, the expected value of this
log-likelihood ratio is going to be positive.
The situation is going to be a curve that comes over here,
comes back at some point here.
And what we've showed is that this curve is just a
translation of this curve by 1.
OK, namely if you calculate the moment generating function
for H equals 0, you get the same thing that we got before.
I'm not going to go through all the details of this.
Now you have 0 here instead of 1.
Over here you're going to have just minus r.
And over here you're going to have 1 plus r.
So this whole thing is translated by 1.
The action happens here.
But if you translate it by 1 over in this direction, what
happens is the error of probability is
determined by this.
It's this.
The exponent is this point right here, gamma 1 of r0 plus
1 minus r0, log of eta over n.
And the r0 was again determined by this point at
which the slope is equal to log eta over n.
We did that before.
Then I want to make clear you understood it, because to
really understand it, you have to go through the arithmetic
yourselves at least once.
And you can do that easily by following the notes, because
it does it in almost excruciating detail.
So that's the argument you get.
We had this idea before, of the Neyman-Pearson principle,
which says you don't assume a priori probabilities.
You look at the probability of making an error as being a
trade off between the error you make when H is equal to 1
and the error you make when H is equal to 0.
In terms of the Chernoff bound, this
trade off is very clear.
As you change the exponent that you want to get under H
equals 1, this point moves.
The tangent then moves.
And the exponent over here moves.
So you have this inverted seesaw.
And the exponent for one kind of error is over here.
And the exponent for the other kind of error is over there.
Then the next thing we said was this is really stupid,
unless you're going on vacation this Friday.
If you're not going on vacation this Friday, if
you're really serious about making the right decision,
then what you're going to do is keep on making observations
until you're pretty sure you're right.
Now somebody at the end of the lecture last time pointed out
something, which says that when you do experiments and
you keep on making observations until you get the
data that you want, there's something very
unethical about that.
Is this that kind of unethical behavior?
Or is this really valid?
Well, I claim this is valid, because what we're doing when
we're doing sequential testing is we're deciding what we're
going to do ahead of time.
Namely, we've decided what we're going to do is we're
going to continue testing until we cross a threshold and
threshold gives us a suitable probability of error.
So we're not cooking the books at all.
What we're doing is we're following this preset
procedure we've set up.
And the only question is can we get a very small error
probability by using a smaller number of observations on the
average than what we need otherwise?
Put it in terms of a communication system.
One kind of communication system, you have to send some
data from one point to another.
You're not going to get any feedback on it.
You've got to get the data through the first time.
It's got to be right.
What are you going to do?
You're going to send this data a very large number of times
or use a very powerful coding technique on it.
And by time it gets through, you're going to be very sure
you're right.
Now a much better procedure, and the thing which is used in
almost all communication systems, and the thing which
we use as human beings all the time, and the thing which
control people use all the time, the thing which almost
everybody uses, because most of us have common sense if we
spend some time trying to do these things, is instead of
trying to get it right the first time, we try little bit
to get it right the first time.
And we make sure that if we don't get it right the first
time, we have some way of finding out about it and
getting it right the second time.
And in the scientific way of looking at it, what we do is
we decide ahead of time exactly what our procedure is
going to be for making repetitions--
something called ARQ in communication systems, which
means automatic repeat request.
It's automatic, which means you don't try to make your
decision depending on whether you'd like to receive this 0
or like to receive a 1.
You make the decision ahead of time that if you have a clean
enough answer, you're going to accept it.
If it looks doubtful, you're going to send it over again.
That's exactly the same sort of thing we're doing here.
OK, when we do that, given H equals 1, we again have this S
sub n as a function of n as a random walk.
It's a sum of IID random variables and
conditional on H equals 1.
You have a random walk.
Conditional on H equals 1, you have a negative slope on this
random walk.
The random walk starts out and on the average is going to go
down, and it's going to continue going down forever.
And if you're looking for across some positive
threshold, if it doesn't cross it pretty soon, it's not going
to cross it.
But anyway, we have a test which said we have some
positive threshold.
We have some negative threshold.
If we ever cross the positive threshold, we say
H is equal to 0.
If we ever cross the negative threshold, we say
H is equal to 1.
And then we're done with it.
OK, now, let me give you another argument why that
makes sense.
I gave you one argument last time.
I'll give you another argument this time.
If S sub J is greater than or equal to 0, we're going to
decide that 0 is the correct hypothesis.
If H equals 1 is the correct hypothesis, then we're going
make an error when S sub J is greater
than or equal to alpha.
If S sub J is less than or equal to the beta, we're going
to decide H equals 1.
And conditional on H equals 1, an error is made if SJ is
greater than or equal to alpha.
Conditional on H equals 0, an error is made if SJ is less
than or equal to beta.
OK, so the probability of the error conditional on H equals
1 is the probability that S sub J is greater than or equal
to alpha, given H equals 1, which is less than or equal to
E to the minus alpha or star.
This is the thing that we said before.
r star is the root of gamma of r.
And gamma of r is equal to this.
OK, so, let's just make life a little easier for ourselves
assume that our a priori probabilities are each 1/2.
This is also called maximum likelihood decision.
You take this likelihood ratio, and you just decide on
the basis of the likelihood ratio.
OK, then at the end of trial end, the probability of H
equals 0 given Sn divided by the probability of H equals 1
given Sn, the a prioris cancel out.
It is just E to the S sub n.
That's what it is.
It's the likelihood ratio.
S sub n is the log-likelihood ratio.
So this is what it is.
If you now take probability of H equals 0 on probability that
H equals 1 given S of in, this equation
becomes this equation.
And then the probability of H equals 1 given S sub n is just
E to the minus Sn over 1 plus E to the minus Sn.
Now if Sn is a large number, E to the minus Sn is going to be
totally trivial.
And the probability that H equals 1 given Sn is
essentially E to the minus Sn.
It means when you can choose different values of n, this
very directly gives you a control on what the
probability of error is.
The probability of error is essentially E to the minus Sn.
So if you choose a threshold alpha, what you're doing is
you're guaranteeing that the probability of error cannot be
less than E to the minus alpha.
OK, so this is more than just talking about averages.
This is saying if you use a threshold rule, then what
you're doing is guaranteeing that the probability of error
is never going to be less than this quantity
of specified here.
OK, we saw last time the cost of choosing alpha to be large
is that you have to make a very large number of trials,
at least given H equals 0.
Why don't I worry about the number of trials
for H equals 1?
I mean, it's nothing to be thought through here.
If my thresholds are large, my probability of
error is very small.
The expected values of things for very large log-likelihood
ratios are determined almost entirely by H equals 0.
H equals 1 sometimes.
You sometimes make a mistake, because it's something very,
very unusual.
But that has very little influence on the expected
number of tests you're making.
So what happens then is the expected number of tests you
make under the hypothesis that H is equal to 0--
now we're using Wald's equality
rather than Wald's identity--
it's equal to the expected value of S sub J given H
equals 0, divided by the expected value of Z,
given H equals 0.
Z is the log-likelihood ratio of one trial.
This is just Wald's equality with this
condition thrown into it.
Now what's the expected value of SJ given H equals 0?
It's essentially alpha, and if you want to be more careful,
it's alpha plus the expected overshoot given H equals 0.
And that's divided by the expected value of Z,
given H equals 0.
This is the answer we got last time.
So the number of tests you have to make, if you set a
positive threshold alpha, is essentially the number of
tests you have to make when the hypothesis is equal 0.
So the funny thing which is happening here is that as you
change alpha, you're changing the probability of error for
hypothesis H equals 1.
And you're changing the number of tests you're going to have
to do when H is equal to 0.
When you change beta, it's just the opposite.
So that when you change beta, if you make beta a very large
negative, you have to make an enormous number of tests under
the circumstance that H is equal to 1.
But you might make an error when H is equal to 0.
So the trade off is between number of trials under one
hypothesis, error of probability under the other
hypothesis.
That's almost all we wanted to say about Wald's identity.
There's one other huge thing that we want to talk about.
If you take the first two derivatives of Wald's identity
at r equals 0, you get some interesting things coming out.
I mean, Wald's identity, you can use it any value
of r you want to.
And when you use it for a large value of r, you that an
interesting result about large deviations.
When you use it at a small value of r, you get something
more about typical cases.
So looking at it at r equals 0, what you want to do is you
want to take the derivative with respect
to r of Wald's identity.
This expected value in here we know is equal to 1.
It's equal to 1 whatever value of r we choose.
And therefore, when we take the derivative of this, we
have to get 0.
But we also want to take the derivative of it
to see what we get.
So when you take the derivative of this quantity
here and you don't worry about what exists and
what doesn't exist--
you have to take the derivative here--
so you get an S sub J there.
You take the derivative here, you get a
gamma prime of r there.
If you get SJ minus J times gamma prime of r, and this E
to the what have you just sits there.
You take the derivative of E to something, you never get
rid of the E to something.
You just get piled up stuff in front of it.
OK, so when we evaluate that at r equals 0, what happens?
Well, what's the value of the gamma prime of 0?
It's the expected value of X, yes.
And this quantity here is all equal to 1, so we can forget
about that.
When r is equal to 0, this is equal to 0.
When r is equal to 0, gamma of r is equal to 0.
So this whole thing in here is 0.
So E to the 0 is 1.
So we've got a 1 there.
We got expected value of S sub J minus J times X
bar is equal to 0.
What is that?
That's Wald's equality.
So Wald's equality falls out of the Wald's identity as what
happens as the derivative of Wald's identity
that r equals 0.
Well, since we're so successful with that, let's go
on and take another derivative.
Yes?
AUDIENCE: I guess you want the final equal to 0 [INAUDIBLE].
PROFESSOR: Oh, the final equal to 0 comes from the fact that
this quantity here that you're starting with is equal to 1
for all values of r.
Therefore, I want to take the derivative with
respect to r, I get 0.
So that's one equation.
The other thing is I just go through the mechanics of
taking the derivative.
OK, so let's try to take the second derivative.
Take the second derivative by taking the derivative of the
first derivative.
And what happens is then is this quantity in here I get an
extra term of that sitting over there.
And along with that, I get the derivative of this with
respect to r.
I should probably have written that down there but since I
didn't, let me see if I can do it.
I get the expected value of SJ minus J gamma prime of r.
And this quantity is squared now,
because I have this there.
I'm taking the derivative of this term with respect to r.
And also, I have to take the derivative of this with
respect to r.
So that gives me minus J times gamma double prime of r.
And all of this times E to the r SJ minus J gamma of r.
Now I want to evaluate this at r equals 0.
Evaluating this at r equals 0, this term goes away.
So I wind up with the expected value of SJ minus J gamma
prime of r where minus J gamma double prime of
r is equal to 0.
Well, this doesn't look bad.
But if you try to use it if you expand this term here, you
get a term the expected value of S of J times J. And you can
struggle with that.
And it's ugly.
That's very ugly.
But if you now say, if we have a mean we
can use Wald's equality.
It tells us what we want to know.
If we don't have a mean, then Wald's equality doesn't tell
us anything.
But this is going to tell us something.
So we're going to make the assumption here that r is
equal to 0 and X bar is equal to 0.
And if X bar is equal to 0, gamma prime of
0 is equal to 0.
And gamma double prime of 0 is equal to sigma squared of X.
So you do all of that.
What you get is the expected value of S sub J squared minus
sigma X squared of J is equal to 0.
This is the same kind of thing that we
got from Wald's equality.
From Wald's equality, it didn't tell us anything.
It just gave us a relationship between the expected value of
S sub J and expected value of J. This is
doing the same thing.
It's giving us a relationship between the expected value of
S sub J squared and the expected value of J. So we get
the same kind of quantity.
It's doing the same thing for us.
Now you look at this for a 0 means simple random walk.
Now you would have thought before you started to take
this class that a simple random walk with mean 0 was
the simplest thing in the world.
And we've seen by looking at that it really isn't all that
simple, that you come play these silly games like you can
gamble forever, where with probability 1/2 you lose $1,
with probability 1/2 you win $1--
perfectly fair game.
And with probability one, you to make $1 out of that, and
quit and go home.
And since you to make $1 out of it, and quit and go home,
you can then quickly come back again and it again.
You can make $2.
You can make $10.
You can make $1,000.
You can make $1 million with probability 1.
So the simple random walk is no longer simple.
It becomes puzzling.
But Wald's identity is dealing with two thresholds, one of
alpha and one at beta.
When we apply this and you observe it as a simple random
walk, where you either go up by 1 or go down by 1, each
with probability 1/2, the mean of X is 0 and the
variance of X is 1.
So this quantity here is 1.
You can then play games with what the probability is that
you hit the upper threshold and the probability that you
hit the lower threshold.
I mean, it's done in the text.
You don't have to take my word for it.
And when you do that, what you find is the expected value of
J is equal to minus beta times alpha.
Theta is a negative number, remember, so this is expected
value of J is the magnitude of theta times the
magnitude of alpha.
Now that's a little bizarre, but then you think about it a
little bit.
You think what happens.
And this is really exact.
I mean, this isn't an approximation or anything.
If alpha is very large, and beta is very large and
negative, and you play this random walk game, you're going
to fluctuate a long time.
You're going to disperse slowly.
You're going to disperse according to the square root
of n, or the number of tests you take.
So the amount of time it takes you until you get way out to
these thresholds should be--
to the namely the value that n has to have--
roughly the square of alpha when beta and alpha
are both the same.
This is something more general than that.
It says that if Sn, the stop-when-you're-ahead game,
we make alpha equals 1, the expected value of J depends on
what the lower threshold is.
And that suddenly makes sense, because what that's saying is
if we have a lower threshold at 10, an upper threshold at
one, then most of the time you win.
But when you lose, you lose $10.
When you win, you win, $1.
When you set a lower threshold at 100, when you
lose, you lose $100.
When you win, you win $1.
And suddenly, that stop-when-you're-ahead game
does not look quite as attractive as it did before.
What you're doing is taking a chance where you're probably
going to win of winning $1, and you're risking your life's
assets for it, which doesn't make too much sense anymore.
OK this, I think, it gives you a better idea of what's going
on on the simple random walk than anything else I've seen.
So it's time to start talking about martingales.
A martingale, like most of the other things we've been
talking about in the course, is a
sequence of random variables.
This is a more general kind of sequence than most of them.
Almost all of the processes we've talked about so far have
been the kinds of things you can sort of get your hands on.
And this is defined very abstractly in terms of a
peculiar property that it has.
And then the peculiar property it has is the expected value
as the nth term, in this thing called a martingale,
conditional on knowing the values of all the previous
values, expected value of Z sub n given the value of Z and
minus 1, Z and minus 2, all the way down to Z1 is equal to
Z sub n minus 1.
Namely, the expected value here is what you had there.
The word martingale comes from gambling, where gamblers used
to spend a great deal of time trying to find gambling
strategies when to stop, when to start betting bigger, when
to start betting smaller, when to do all sorts of things, all
sorts of strategies for how to lose less money.
Let me put it that way, because you rarely find that
opportunity where you can play a fair game.
But if you play a fair game, martingales are what sort of
rules on that.
And what that says is if you play this game for a long
time, your capital is Z sub n minus 1.
This says, figure expected capital after you
play one more time.
No matter what strategy you use, your expected capital is
going to be the same as was as the actual
capital the time before.
If this is too abstract for you, and it's too abstract for
me half the time, because I look at this, and I say, gee,
that's not much of a restriction, is it?
What we're talking about is expected values here.
But it's more than that, because it's saying for every
choice of sample value for all of these things, none of them
make any difference, except the last one.
And that's what happens in gambling.
It doesn't make any difference how your capital has gotten to
the point where it is at time n minus 1.
You make a bet in a fair bet, and what you win is solely a
function of what you've bet, if the game is fair.
And that's what this is saying.
So when you write it out this way, the expected value of Zn,
given that 1, the random variable Zn minus 1 has a
particular value Zn minus 1.
You started out with a particular value.
It says that expected value is equal to what you
said at the last time.
And this is true for all sample values Zn minus 1 down
to Z1, which is why it's a much stronger statement than
it appears to be.
now there's a lemma.
I want to talk about that a little bit, because it's a
good time to get you used to what these
expected values mean.
For martingale, the expected value of Zn given Zi, Zi minus
1, all the way down to Z1.
This expected value is equal to Z sub i.
In other words, it's not only that your expected capital,
given all of the past, is equal to what you had on the
last time instant.
If you're not given anything for 100 years back, and all
you know is what your capital was 100 years ago, and if we
think we're playing a fair game all of this time, which
is of course always a question, the expected value
of what we have now, conditional on everything from
100 years back through recorded history, is
just that last term.
In other words, it's the same kind of isolation of the past
from the future as we had with Markov change.
With Markov change, remember, it's only what happens at one
instant, given what happens a one instant, it makes the past
independent of the future.
Here it's not quite that way, because the past and the
future are separated only in terms of the details of the
past, and the expected value the future.
It says expected value of Z sub n given all of the details
of the past, no matter what the details of the past are,
the effective value of Zn is equal to the actual value at
times Z sub i.
So I want to improve this for you, and I warn you you're not
going to follow this proof.
And that's part of the reason for me to do it, because I
want you to go back to Chapter 1 and think that through.
Because in dealing with martingales, you have to think
this through.
Because if you don't think it through, you're stuck with
this notation all the way through.
And if you try to use this notation with martingales,
this is nice notation when you get confused, but you don't
want to use it all the time.
So you have to be able to go through arguments like this.
What I want to show is that if E to the Z3, given Z1 and Z2
is equal to Z2, then one special case of this lemma is
that expected value of Z3 given Z1 is equal to Z1.
And how do we show that?
Well, what we do is we use this law complete expectation.
Well, first we remember the expected value of an arbitrary
random variable X is the expected value of the expected
value of X given Y. Now what does that mean?
The expected value of the random variable X, given Y, is
a random variable.
It's a random variable which depends on Y. That's a
function of the sample value of Y. Namely, if you look at
this quantity up here, expected value of X
given Y equals 1.
Expected value of X given Y equals 2.
Expected value of X given Y equals 3.
We have all of these values here.
We have a probability measure on it.
This is a random variable, which is a function of Y.
You've averaged that over X, but you're left move why
because of the conditioning here.
So this quantity in here is now a function of Y. So when
we take this equation and we add the conditioning on Z1,
namely, this is being used for Z3 and Z2.
Expected value of Z3 is equal to the expected value over Z2
of the expected value of Z3 given Z2, whole thing
dependent on Z1.
OK, so what it says is this expected value is the expected
value of the expected value of Z3 condition on Z2 and Z1.
This quantity here as a function of what?
That's a random variable.
It's a function of what random variables?
AUDIENCE: Z2, Z1.
PROFESSOR: Z1 and Z2, yes.
So this is a function of Z1 and Z2.
What value is it as a function of Z1 and Z2?
It's just equal to Z2.
So this quantity in here is Z2.
so we're asking what's expected value of Z2 given Z1.
And by definition of martingale, it is equal to Z1.
Now I imagine about half you could follow that, and half of
you couldn't, and half of you sort of followed it.
This is a kind of argument we'll be using all the way
through on this stuff.
So make sure you understand it.
I mean, once you get it, it's easy.
And you can apply it in all sorts of places.
So it's worth doing it.
In the same way, you can follow the same kind of
argument through the expected value Z sub i plus 2, using
this total expectation based on Zi plus 1.
And you go through the whole thing.
When you go down to i equals 1, it says the expected value
of z is equal to the expected value of Z1.
If you want to become wealthy, have a wealthy parent, who
leads to a lot of money 20 years ago.
That's the easiest way to make a million dollars is to start
out with 2 million dollars is the way that
some people put it.
OK, let's have some simple examples a martingales.
One of them is a zero-mean random walk.
Mainly, what I'm trying to do here is to show you this
martingales are really pretty general things.
And since there are many very general theorems that hold for
all martingales, you can then apply them to all of these
special cases, which is kind of neat.
To have a zero-mean random walk, let Z sub n be the sum
of X1 plus Xn and the X sub i's are IID and zero mean.
The fact that they're IID makes it a random walk.
The fact that there's zero mean makes it a special zero
mean random walk, and the expected value of Z sub n,
given Zn minus 1.
All the way back, Zn now is Xn plus Zn minus 1.
OK, Zn is the sum of all these random variables.
So you get up n minus 1 of them, and then you add the
last one in.
So it's Zn minus 1 plus Xn, so its expected value of Xn plus
Zn minus 1, given all the stuff before that.
The expected value of Xn, given all this stuff, is what?
Xn is independent of all the other X's, therefore it's
independent of all the earlier Z's.
And therefore, that's just expected value of Xn.
So we have the expected value of Zn minus 1, given Zn minus
1 back to Z1.
What's expected value of Zn minus 1, given Zn minus 1?
Well, it's Zn minus 1.
That's no problem there.
So this is 0.
So this is equal to Zn minus 1, as it's supposed to be.
All of these things you ought to go back and think them
through yourself.
Because the first time you look at martingales, all of
this stuff, it's all pretty easy, but it all looks a
little strange at first.
The next one is sums of arbitrary
dependent random variables.
They're not quite arbitrary.
Suppose you have a sequence of random variables, X sub i, i
greater or equal to 1.
And they satisfy the expected value of Xi, given all the
earlier X of i's is equal to 0.
It's similar to what a martingale is.
But here, we're just saying the Xi's all have expected
value of 0.
And Zn, the sum of these, has to be a martingale.
And I'm not going to write all a proof of that.
I mean, this proof is really the same as this proof.
This is really a pretty important thing, because given
any martingale, you can always look at the partial sums
between the terms of the martingale--
namely, given Z1, Z2, up to Z sub n.
You can always look at Z2 minus Z1.
You can look at Z3 minus Z2.
You could at Z4 minus Z3, and so forth.
And each of the Z's is just the sum of those
other random variables.
So given any martingale in the world, you can always define
the set of arbitrary depending random variables would satisfy
this rule here.
So Zn, in this case, is a martingale.
And if Zn is a martingale, you can always define a set of
random variables, which satisfy this property.
I think it's almost easier to see what a random variable
really has to do with gambling, which is where it
started, by looking at this.
This is not your capital at time n.
This is how much you win or lose at time i.
And what it's saying is your winnings or losings at time i
has zero mean independent of everything in the past.
In other words, in a fair game, you can bet whatever you
want to and depending on what you bet, that's the expected
amount you get on that trial.
And that's what this says.
This says essentially, you're applying a fair game.
So martingales really have to do is fair games.
If you can find fair games, why, that's great.
But we always look for games where we have an edge.
But what you want to avoid is games where Las
Vegas has an edge.
OK so, that's a general one.
Here's an interesting one, because I think this is an
example which you can use.
I mean, in any field you study, there are always
generic examples, which can be used to generate counter
examples to any simple thing you might want to think of.
And this is to me the most interesting one of those for
martingales.
Suppose that Xi is the product of two random variables--
one is either plus 1 or minus 1, each with probability 1/2.
And the other one, Y sub i is anything it wants to be.
I don't care what Y sub i is.
Y sub i is non-negative, might as well make it non-negative.
I don't care about it.
I don't care how it's related to all the other Y sub i's.
All I want is the that the U sub i's are all independent of
all the Y sub i's.
And what happens then?
I take the expected value of X sub i, give it anything in the
past, and what do I get?
U sub i is independent of Y sub i.
And therefore, the expected value of U sub i times Y sub i
is expected value of U sub i, which is what--
plus 1 or minus 1 of probability 1/2 each.
The expected value of U sub i is equal to 0.
That makes expected value of X sub i of 0
whatever the past is.
So you automatically have the--
I don't know what to call in them, the terms between the
terms of a martingale--
the interarrival terms, so to speak.
I mean, it's like those for a renewal process.
Those terms always have mean 0.
And therefore, the sums of these turn out to be this
simple kind of martingale.
So that's a nice martingale to use as counter examples for
almost anything.
The next one is product for martingales.
Product for martingales are things we use quite a bit too,
because now when we're using generating functions, we're in
the habit of multiplying things together.
And that's a useful thing to do.
So the expected value of Z to the n, given Z to the n minus
1, now to Z1, where Zn is this product of terms.
OK, Z sub n then is equal to Xn times Z sub n minus 1,
which is what we're doing here.
Expected value of Zn conditional on the past is the
expected value of Xn times Zn minus 1, conditional on the
past, Xn and Zn minus 1.
Oh, the expected value of Xn for any given value of Zn
minus 1, all the way back, is just the expected
value of X sub n.
So we have expected value of X sub n times the expected value
of Z sub n minus 1, given Zn n minus 1 down to Z1.
So that's just Zn minus 1.
Ah, the missing quantity, fortunately I wrote it here.
The X sub i's are unit means random variables.
And they're IIDs.
They're independent of each other.
And since the X sub i's are independent of each other, X
sub n is independent of Xn minus 1, all
the way back to X1.
Zn minus 1 back to Z1 is a function of Xn
minus 1, down to X1.
So Xn is independent of all those previous Z's also.
That's why I could split this apart in this way.
And suddenly I wind up with Zn n minus 1 again.
So product form martingales work.
Special form of product form martingales--
this again is favored counter example for when you can and
can't get around with going to limits and
interchanging limits.
And it's a simple one.
Suppose that X sub i's are IID, as in
the previous example.
And they're [INAUDIBLE] probably 2 or 0.
I mean, this is a game you often play--
double or nothing.
You start out with dollar.
You play the game.
If you win, you get $2.
If you lose, you're broke.
If you win, you play your $2.
If you win again, you have $4.
If you lose, you're broke.
You play again.
If you win, you have $8.
If you lose again, you're broke.
So the probability that Z sub n, which is your capital after
n trials, is equals to 2 to the n, namely you've won all n
times, is 2 to the minus n.
And every other instance you've lost.
So the probability that Zn is equal to 0 is 1 minus 2
to the minus n.
So for each n, if you calculate the expected value
of Z sub n, it's equal to 1.
Namely, with probability 2 to the minus n, your capital is 2
to the n, so that's 1.
With all the other probability, you have nothing.
So your expected value of Z sub n is always equal to 1.
That's what this product form martingale says.
And this is a product form martingale.
However, the limit as n goes to infinity of Zn is equal to
0 with probability 1.
If you play double or nothing, eventually you lose.
And then you're wiped out.
In other words, there's no real purpose to playing the
game, because eventually you lose.
If you're playing with somebody else, and they're
playing double or nothing, then of course you get their
money eventually.
Or you go broke and the bank that you bank at fails, and
all that stuff.
We won't worry about that.
OK, but the point of this is that the limit as n goes to
infinity of Zn is equal to 0.
And the limit of the expected value of Z sub
n is equal to 1.
And therefore, the limit of Zn and the expected value of the
limit of Zn.
The expected value of the limit of Zn is 0.
The limit as expected value of Zn is equal to one.
So this is a case where you can't interchange limit and
expectation.
It's an easy one to keep in mind, because we all know
about playing double or nothing.
Might as well define submartingales and
supermartingales.
Because the first thing to know about them is they're
like martingales, except they're defined with
inequalities.
And for a submartingale, the expected value of Zn, given
all the previous terms, is greater than or
equal to Zn minus 1.
So submartingales go up.
Supermartingales are the opposite.
Supermartingales goes down.
What else could you expect from a mathematical theory?
Things that should go up, go down.
Things that should go down, go up.
Only thing you have to remember about submartingales
and supermartingales is you figure out what terminology
should have been used, and you remember the terminology they
use was the opposite of what they should use.
I don't know whether I've ever seen stupider
terminology than this.
And someone once explained the reasoning for it, and the
reasoning was stupid too.
So there's no excuse for that one.
We're only going to refer to submartingales in what we're
doing, partly because that's where most of the
neat results are.
And the other thing is if you have to deal with a
supermartingale, what you might as well do is instead of
dealing with a sequence--
Z1, Z2--
deal with the sequence minus Z1, minus Z2, and so forth.
And if you change the sign on all the terms, then you change
supermartingales into submartingales and vice versa.
You don't really have to deal with both of them.
Let me talk briefly about an inequality that I'm sure most
of you heard of.
How many people have heard of Jensen's Inequality?
Maybe half of you, so not everyone.
Well, it's one of the main work horses
of probability theory.
Even though we haven't seen it yet this term, you will see it
many times.
So what a convex function is.
A convex function in simple minded terms is something, a
convex function from r into r.
A real value convex function is a function which has a
positive second derivative everywhere.
So it curves down and comes back up again.
Since you also want to talk about functions which don't
have second derivatives, you want something more
general than that.
So you go from derivatives.
You go back to your high school ideas,
and you draw a picture.
And function is convex.
If all the tangents to the curve lie not strictly below,
but all the tangents can curve lie beneath the curve, so
wherever you draw a tangent, you get something which
doesn't cross the curve.
Magnitude of X is a convex function of X. Magnitude X as
a function of X looks like this.
You go down or up.
And all tangents to this, this goes off to infinity and this
goes off to infinity.
So there's no way to get something like
that in this tangent.
So you have one tangent here.
You have a bunch of tangents along here.
And you have one tangent there.
And they all lie below the curve.
So X bar is a convex function too.
And Jensen's Inequality says if H is convex, and if Z is a
random variable, it has finite expectation, then H of the
expected value of Z is less than or equal to the expected
value of H of Z. You can interchange expected value and
function with inequality like this, if
the function is convex.
Now, why is this true?
You can see why it's true automatically, if you're
dealing with a random variable that has only two values.
If you have two values for the random variable--
Z is a random variable here.
You have one variable here, one value here, one sample
value here.
Look at what the expected value of H of Z is.
The expected value of H of Z is the expected value of this
and this with the appropriate probability put in on it, so
at some point that lies on the straight line
between here and there.
When you look at the H of the expect the value, then you
find the expected value along here.
You can think of finding it along the straight line here.
And then it's that point there.
So since the curve is convex, the H of the expected value of
Z is there's always a bunch of points.
Average them, which lie on a straight
line beneath the curve.
And the expected value of H of Z is taking the average
directly along the curve.
So you get this boosting up everywhere.
It's like saying that the absolute value of expected
value of Z is less than or equal to the expected value of
the absolute value of Z.
And I think I will stop there instead of going on, because
we had a lot of new things today.
And somehow sequential detection always wears one's
mind out in a short period of time.
That should be enough.