The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare continue to
offer high quality educational resources for free.
To make a donation or view additional materials from
hundreds of MIT courses, visit MIT OpenCourseWare at
ocw.mit.edu.
PROFESSOR: OK, I guess it's time to get started.
Next lecture, I'm going to try to summarize what we've done
so that I want to try to finish what we're going to
finish for the term today.
And that means we have a lot of topics that all get
slightly squeezed in together, including the martingale
convergence theorem and the strengthening of the strong
law of large numbers and the Kolmogorov's submartingale
inequalities and stopped martingales.
So all of these are fairly major topics.
One thing that it means is that we certainly aren't going
to do much with the martingale convergence theorem.
It's a major theorem and advanced work.
What we're trying to do here is just give you
some flavor of it.
The other things, as we move up the chain there, we want to
know more and more about it.
And the things on top certainly explain what's going
on in the things below.
OK, let's review what a martingale is.
A sequence of random variables is a martingale if it
satisfies this funny looking condition.
When I write it this way, it's not completely obvious what
it's saying.
But I think we know now the expected value of one thing,
of one random variable, given a set of random variables, is
really a random variable in its own right.
And that's a random variable, which is a function of those
conditioning random variables.
So expected value of Zn, given Zn minus 1 to Z1.
It's a random variable.
It maps each sample point to the conditional expectation of
Zn, conditional in Z1 to Zn minus 1.
For martingale, this expectation has to be this
random variable, Z sub n minus one.
It has to be the most recent random variable you've seen.
And then last time, we proved this lemma, a pretty major
lemma, which says, for a martingale, expected value of
Zn is equal to of Zn, given not all of the past, but just
a certain number of elements of the past Zn conditional and
Zi, back to Z1.
That's equal to Zi.
Expected value of Zn is equal to the expected value of Zi.
We didn't talk about this one last time.
But this is obvious in terms of this.
If you take the expected value of this over all of the
conditioning random variables, then what you get is just the
expected value of Z sub i.
So that's what this says.
OK, question now.
Why is it that if you condition on more random
variables than just Zn in the paths, if you condition on
things into the future, including Z sub n its self,
why is the expected value of Zn given Zm down
to Zn down to Z1?
Why is that equal to Zn and not equal to Zn minus 1?
If you understand this, you understand what conditional
expectations are.
If you don't understand it, you've got to spend some time
really thinking about what conditional expectations are.
OK, how many people can see why this, in fact, has to be
Zn and not something else?
AUDIENCE: We know Zn already.
PROFESSOR: What?
AUDIENCE: Isn't it-- don't you know Zn already?
PROFESSOR: That's right.
Since you know the n already, then you know it.
That's exactly what it says.
And we'll talk more about this later because we actually use
this a bunch of times when we're going on.
But what it says is that for any given value of each of
these random variables, in other words, for a given value
of Z sub n, the expected value of Z sub n--
I mean, if I just wrote it, it's what's the expected value
of Z sub n, given Z sub n?
What's the expected value of Z sub n--
I'll make it even easier.
And you can see it more clearly.
What's the expected value of Z sub n, given that Z sub n is
equal to a particular value z sub n.
OK, now I hope you can see why it is that this is
equal to Z sub n.
That's the only thing that the random
variable Z sub n can be.
For this sample point and the conditioning, we're assuming
the sample value for which Z sub n is equal to little z sub
n, and therefore that's what this random
variable has to be.
That's what you said in one sentence.
And I'm saying it in five sentences.
It's hard enough that you want to say it in five sentences
because this is not an obvious thing.
It's not an obvious thing until you really have a sense
of what these conditional expectations mean.
And that's part of our function for the last couple
of weeks, to sort out what those things mean because
that's part of understanding what martingales are doing.
OK, when we go one step further on this, we've said
that the expected value of Z sub n, given Z sub i back to Z
sub n is Z sub i, so the expected value of Z sub n,
only given Z sub 1, is equal to Z sub 1.
That's what this says, an expected value of Z sub n then
is equal to the expected value of Z sub 1.
It says these marginal expectations are all the same.
Yes?
AUDIENCE: Why don't you just say that the expectation of
the Zs are constant.
I mean, it seems like this is kind of a roundabout way of
saying that.
PROFESSOR: Yeah, in a sense.
But if you want to--
almost all the examples you can think of, it's hard to
figure out what these Z sub n's are because the Z sub n's
are given in terms of all of the previous random variables.
If you want to sort out what a martingale is, and you can't
even understand what Z sub 1 is, then you're in trouble.
I mean, it makes the theorem more abstract to say the
expected value of Z sub n is a constant random variable
rather than saying which random variable it is.
And it's obvious what random variable it is.
I mean, Z1 is one example of it.
So you're right.
You could say that way.
OK, so we talked about a number of simple examples of
martingales last time.
All of these examples assume that the expected value of the
magnitude of Z sub n is less than infinity for all n.
Remember, this does not mean that the expected value of Z
sub n is bounded over all n.
I mean, you can have expected value of Z to the n
can be 2 to the n.
It can be shooting off to infinity very, very fast.
But it still is finite for every value of n.
That's what this assumption is.
Later on, when we talk about the martingale convergence
theorem, we'll assume that these expectations are
bounded, which is a much, much stronger constraint.
OK, we talked about these examples last time.
All of them are pretty important because any time
you're trying to do a problem with martingales, trying to
prove something for martingales, what I like to do
first is to look at all the simple examples I know and try
to get some insight from them as to whether the result is
true or whether it's not true.
And if you can't see from the examples what's going on, then
you're sort of stuck for the most part, unless you're lucky
enough to construct the kind of proof you would
find in a math book.
Now math book theorem proofs are beautiful
because they're the--
I mean, mathematicians work to get the shortest possible
proof they can, which has no holes in it at all.
And that makes it very elegant.
But when you're trying to understand it, what is often
done is somebody starts out understanding a theorem, and
they write a proof which runs for three pages.
And then they think about it for two weeks.
They cut it down to one page.
They think about it for another two weeks.
They cut it down to half a page.
And then they publish it.
And all they do is publish the half page.
Everybody is stuck.
Nobody knows where this came from.
So what I'm trying to do here is, at least in some cases, to
give you a little more than the half page to give you some
idea of where these things come from, the extent I can.
OK so the zero mean random walk--
if Z sub n is equal to some of Xi, where Xi are IID and zero
mean , then this zero mean random walk is, in fact, a
martingale, just satisfies the conditions.
This one is probably the most important of all of the simple
examples because it says if Z sub n is a sum of random
variables--
don't know what the random variables so are-- the
condition on the random variables is the expected
value of X sub i, given all the previous ones is 0.
This is a general example because every martingale in
the world you can look at the increments of that martingale,
namely you can define X sub n to be Z sub n minus
Z sub n minus 1.
And as soon as you do that, X sub n satisfies
this condition here.
And what you've got for sure is a martingale, so that this
condition here, that the X sub is are satisfying, is really
the same as a martingale condition.
It's just that when people are talking about martingales,
they're talking about the sums of random variables.
Here we're just talking about the random variables
themselves.
It's like when we talk about sum of IID random variables,
we prove the laws of large numbers and everything.
What we're really talking about there
is IID random variables.
What we're really talking about here is these random
variables, which has the property that no matter what's
happened in the past, the expected value of this new
random variable is equal to 0.
People call these fair games.
And they call martingales examples of fair games.
And martingale, the expected value of Z sub n, you can
think of it as you're expecting that
worth of time n.
And with these underlying random variables, the X sub i,
the X sub i is, in a sense, your profit at time n.
And what this says is your profit at time n is
independent of everything in the past, independent of every
sample value of everything in the past.
And this is why people call it fair game.
It's really a very strong definition of a fair game.
I mean, it's saying an awful lot.
I mean, everybody says life is not fair.
Surely by this definition, life is not even close to fair
because when you look at all your past--
I mean, you try to learn from your past.
This is saying in these kinds of gambling games, you can't
learn from the past.
You can't do anything with it so long as you're interested
only in the expectation.
The expectation of X sub i is equal to 0, no matter what all
the earlier random variables are.
This, I think, gives you a sense of what a martingale is,
probably better than the original definition.
OK, another one we talked about last time,
this is very specific.
Suppose that X sub i is equal to the product of two random
variables, U sub i times Y sub i.
The U sub i here are IID equiprobable, plus or minus 1.
And the Y sub is are independent of the U sub is.
They can be anything at all.
And when you take these Y sub is, which are anything at all,
but these quantities here, which are IID 1 and minus 1,
when you look at this product here, any positive number this
can be is equiprobable with the
corresponding negative number.
And that means when you take the expectation of X sub i,
given any old thing in the past, this U sub i is enough
to make the expectation equal to 0.
So this is a fairly strong kind of example also, which
gives you a sense of what these things are.
Product form martingales--
you use product form martingales primarily to find
counter examples of theorems.
If you stated a theorem and it isn't true--
almost all the examples I know of of reasonable martingale
theorems which are not true, you look at a product
martingale, and very often you look at this product
martingale down here, and you find out either that the
theorem is not true, which lets you stop looking at it.
Or it says, well, it still isn't clear from that.
OK, so the product form martingale, there's a sequence
of IID unit-mean random variables.
And Zn, which is the product, is then a martingale, if you
assume this condition up here, of course.
And this condition here, the probability that the n-th
order product, namely the product of n of these sample
values, if you get one 0, the product is 0.
So you're done.
So the only question is, do you get all 1s?
Or do you get something other than all 1s?
If you get all 1s, then the product of these random
variables is 2 to the n.
You get 2 to the n with probability 2 to the minus n.
And you get zero with all the rest of the probability.
The limit as n goes to infinity if Z sub n is equal
to zero with probability 1, namely eventually
you go down to 0.
And you stay there forever after.
And with this very small probability, you get to some
humongous number.
And you keep going up until eventually you lose, and you
go down to 0.
So the limit for the expected value of Z sub n--
for every n the expected value of Z sub n is equal to 1.
That's what makes this example interesting.
The limit of the expected value of the Z sub
ns is equal to 1.
But the Z sub ns themselves go to 0 with probability 1.
And the reason for that is that you had this enormous
growth here with very small probability.
But it's enough to keep the expectation equal to 1.
OK, then we started to talk about sub and super
martingales.
And I told you if you can't remember what the definition
of a submartingale is in terms of is it less than or equal or
greater than or equal, just remember that it's not
what it should be.
It's the opposite of what it should be.
So the expected value of Z sub n given the past is greater
than or equal to Z sub n minus 1.
That means it's a submartingale.
In other words, submartingales grow in time.
Supermartingales shrink in time.
And that's strange.
But that's the way it is.
If this quantity is a submartingale, then minus Zn
is a supermartingale and vice versa.
So I'm going to only talk about submartingales from now
on because supermartingales just do everything the same,
but just look at minus signs instead of plus signs.
So why bother yourself with one thing
extra to think about?
So for submartingales, the expected value of Z sub n
given the past, given part of the past from i down to 1, is
greater than or equal to Z sub i.
That's essentially the same as that theorem we've stated
before, which said that for martingales, the expected
values of Zn given Zi down to Z1 was equal to Z sub i.
You remember we proved that in detail last time because that
was a crucially important theorem.
You take that proof and you put this inequality in it
instead of a quality, and it immediately gives you this.
You just follow that proof step by step, putting an
equality in in place of a quality.
And same thing here, the expected value of Z sub n is
greater than or equal to the expected value of Z sub i.
That's true for all i.
So the expected value of Zn is also greater than or equal to
the expected value Z1.
In other words, the expected values of these random
variables, in fact, always grow.
Or if they don't grow, they at least stay the same.
They can't shrink.
OK, we started to talk about convex functions last time.
I want to remind you what they are.
A function which carries the real numbers into the real
numbers is convex, if each tangent of the curve lies on
or below the curve.
Here's a picture of it.
Here's a function h of x, a one-dimensional function.
So you can draw x on the line.
And h of x is something which goes up and down.
And here's another example.
H of x is equal to the magnitude of x.
You're usually used to thinking of convex functions
in terms of functions that have a positive second
derivative.
Taking the geometric view, you get something considerably
more general because it includes all of these cases,
as well as all of these cases.
And this idea of tangents lying on or below the line
gives you the linkage here.
You have something which comes down, goes to 0, and
then goes up again.
Think of drawing tangents to this curve.
Tangents have to have a slope starting here and
going around to here.
And all of those tangents hit at that point there.
So this is a very pathological thing.
But all the tangents indeed do lie below the curve.
So you've satisfied the condition.
The lemma is Jensen's inequality.
It says, if h is convex and Z is a random variable with
finite expectation, then h of the expected value of Z is
less than or equal to the expected value of h of Z. This
seems sort of obvious, perhaps.
It's one of those things which either seems obvious or it
doesn't seem obvious.
And if it doesn't seem obvious, it doesn't become
obvious terribly easily.
But what I want to do here is to convince you of why it's
true by looking at a little triangle here.
I can think of the random variable Z as having three
possible values--
one over here where we get this point comes into the
curve, one here, and one here.
Now if I take those three possible values of x and I
think of assigning all possible probability
assignments to those three possible values, what happens?
If I assign all the probability over here, I get
that point.
If I assign all the probability
here, I get that point.
If I assign all the probability
here, I get that point.
And for everything else, it lies inside that triangle.
And you can convince yourselves
of that pretty easily.
So for all of those probability measures where the
expected value lies on this line, what you get is
something between this and this as the expected value of
h of Z. So you get something above the line for the
expected value for h.
Of expected values of Z, you just get this point right
there, which is clearly smaller than anything you can
generate out of that triangle or quadrilateral or any kind
of straight line figure that you draw, which is the set of
expected values you can get from probabilities using the
points on that--
well, it's what I said it was.
OK, Jensen's inequality leads to the following theorem.
And I'm not going to prove it here in class.
It's one of those theorems which is sort of
obvious and not quite.
So if you want to see the proof, you can look at it.
Or if you want to, you can just believe it.
If Z sub n is a martingale or it's a submartingale and if h
is convex, then the expected value of the magnitude of h of
Zn is less than infinity for all n.
Then h of Zn is a submartingale.
In other words, when you have that convex function, you go
from something which is a martingale, which will be what
you would get on the line to something which is bigger than
that, so that what you get is the fact that expected value
of h of Z is, in fact, growing with time, faster than the
martingale itself is growing.
OK, so one example of this is if z of n is a martingale,
then the absolute value of Z sub n is a submartingale.
Submartingales are martingales.
Martingales are submartingales also.
So I don't have to keep saying that if it's a martingale or a
submartingale.
I can just say if it's a submartingale.
This theorem is usually stated as, if Z of n is a martingale,
then h of Zn is a submartingale, which is true.
But just as obviously, if Zn is a submartingale, which is
more general, h of Zn is also a submartingale.
So you don't get out of the realm of submartingales by
taking convex functions.
And you also get that Z squared is a martingale.
And E to the rZn is a martingale because all of
those are convex functions.
So when you want to look at any of those, you just go from
talking about a martingale to talking about a submartingale.
And life is easy again.
OK, major topic-- stopped martingales.
We've talked about stopping rules before.
And a stopping rule, we were interested in stopping rules
when we were mostly talking about renewal processes.
But stopping rules can be applied to any sequence of
random variables at all.
What a stopping rule is, you remember, is you have a
sequence of random variables, any kind of random variable.
And a stopping rule is a rule where the time that you stop
is determined by the things that you've seen up until the
time that you stop.
You can't peak at future values.
You have to look at these sample values one by one as
they arrive.
And a stopping rule is something which, when you get
to the point you want to stop, you know that you want to stop
there from the sample values you've already observed.
So when you're playing poker with somebody, which I think
we talked about before, and you make a bet and you lose,
you cannot withdraw your bet.
You cannot say, I stopped!
I stopped before!
The time that you stop depends on what you've already seen up
until the time that you stop.
We talked about possibly defective
random variables before.
I realized I never defined a possibly
defective random variable.
In fact, somebody asked me afterwards if it could be just
any old thing at all.
And I said, no.
And here's what it is.
It's a mapping from the sample space to a set of real values,
to the extended real values.
And it has the property that for a defective random
variable, the mapping can give you plus infinity.
Or it can give you minus infinity.
And it can give you either one of those with positive
probability rather than just 0 probability.
So it applies to these cases where you have a threshold, a
single threshold, for a random walk.
And you might cross the threshold, or you might never
cross the threshold.
So it applies to conditions where sometimes you stop and
sometimes you just keep on going forever.
So it's nice for that kind of situation.
The other provisos a random variable back when we defined
random variables still hold for a possibly defective
random variable.
So you have a distribution function.
It's just the distribution function doesn't necessarily
go to 1, and it doesn't necessarily start at 0.
It could be somewhere in between.
OK, so a stop process for a possibly defective stopping
time satisfies Z sub n star, which is Z sub n star is the
stopping time.
Let me start on that.
We have now defined stopping rules.
We now want to define a stop process.
A stop process is a process which runs along until you
decide you're going to stop.
But before when we stopped, the game was over and nothing
else happened.
Here, the idea is to game continues forever.
But you stop playing, OK?
So the sequence of random variables continues forever.
But the random variable of interest to you is this
quantity Z sub n star, which at the point you stopped, then
all subsequent Z sub n's are just equal to
that stopped value.
So you're talking about some kind of gambling game now,
perhaps, where you play for a while.
And you're playing some game where the
game continues forever.
And you make your bets according to
some strange algorithm.
And when you've made $10, you say, that's
all I want to make.
I'm happy with that.
And I'm not going to become the kind of gambler who can
never stop.
So I'm going to stop at that point.
So your capital remains $10 forever after, although the
game keeps on going.
And if you start out with a capital of $10 and you lose it
all and you can't borrow anything, then you stop also
when your capital becomes minus $10.
So you can see that this is a useful thing when you're
talking about threshold crossings because when you
have a random walk and you have a threshold crossing, you
can stop at that point.
And then, you just stay there forever after.
But if you cross the other threshold, you stay there
forever after.
And that makes it very convenient because you can
look at what has happens out at infinity as a way saying
what the value of a game was at the time you stopped.
So you don't have to worry about what was the value of
the game at the stopping point.
You can keep on going forever.
And you can talk about what the stopped process is.
And my guess--
or if you've read ahead a little bit,
which I hope you have--
you will know that the main theorem here is that if you
start out with a Martingale and you stop it someplace, you
still have a Martingale.
In other words, if you add stopping as one of your
options in gambling and it's a fair game, if you can find a
fair game any place, and you stop, then it's
still a fair game.
The stop process is still a fair game.
And that's as it should be because if it's a fair game,
you should be able to stop.
So for example, a given gambling strategy is Zn is the
net worth at time n.
You can modify that to stop when Zn
reaches some given value.
So the stopped process remains at that value forever.
And Zn follows the original strategy.
Here's the main theorem here.
If j is a possibly defective stopping rule for a Martingale
or a sub-Martingale and Zn greater than or equal to 1,
then the stop process, Zn star, is a Martingale if the
original processes is a Martingale and it's a
sub-Martingale if the original process is a Martingale.
And the proof is the following.
You can almost say this looks obvious.
If it looks obvious to you, you
should admire your intuition.
If it doesn't look obvious to you, you should admire your
mathematical insight.
And either way, the kind of intuition is that before
stopping occurs, Z sub n star is equal to Z sub n.
And after you stop, Z sub n star is constant.
So it satisfies a Martingale condition because it's not
going up and it's not going down.
But in fact, when you try to think through the whole thing,
it's not quite enough.
It's the kind of thing where after you look at it for a
while, you say, yes, it has to be true.
But why?
And you can't explain why it's true.
I'm going to go through the proof here.
And mostly the reason is that the proof I have in the notes,
I can't understand it anymore.
Well, I can sort of understand it when I correct a
few errors in it.
But I think this proof gives you a much better
idea why it's true.
And I think you can follow it in real time.
Whereas that proof, I couldn't follow it in real time, or
fairly extended time.
OK, so this stop process, I can express it in the
following way.
And let me try to explain why this is.
If your stopping rule tells you that you stop at time m
for a particular sample sequence, this indicator
function, i of sub j equals n, this function here is 1 for
all sample sequences for which you stop at time m.
And it's 0 for all other sequences.
So the value of this stop process at time n is going to
be the value at which it stopped, which is Zm, when you
have this indicator function, which is j equals m.
And if it hasn't stopped yet, it's going to be Z sub n,
which is what it really is.
And it hasn't stopped.
So the stop process hasn't yet stopped.
So Z sub n star is equal to Z sub n.
OK, so as far as the magnitude is concerned, the magnitude of
Z sub n is going to be less than or equal to the sum of
those magnitudes.
And the sum of those magnitudes, you can just
ignore the indicator functions because they're either 0 or 1.
So we can upper bound them by 1.
So Z sub n star is less than or equal to the sum over m
less than n of Z sub n plus Z sub n.
And this means that the expected value of Z sub n star
has to be less than infinity because what it is in this
bound here is a sum of finite numbers.
When you take a finite sum-- this is a finite sum.
There are only n plus 1 terms in it.
When you take a finite sum of finite numbers, you get
something finite.
So expected value of Z sub n is--.
Excuse me.
I was a little bit quick about that.
The expected value of Z sub n star is now less than or equal
to the expected value of each of the Z sub n's plus the
expected value of Z sub n.
Since the Z process is a Martingale, you know that all
of those expected values are finite.
And since all of those expected values are finite,
the expected value of Z sub n star is finite as we said.
OK, so let's try to trace out what happens if we look at the
expected value of Z sub n star conditional on the past
history up until time n minus 1.
We'll rewrite Z sub n star in terms of this expression here.
So it's the sum over m less than n of the expected value
of the stopping point if the stopping point was equal to m
plus the expected value of Z sub n if the stopping point
was greater than n.
So we just want to analyze all of those terms.
So we look at them.
There's nothing complicated about it.
The expected value of this term here, expected value of
Zm times i of j equals m given Zn minus 1.
And now, we're going to be child-like about it.
And we're going to assume a particular sample sequence,
which is equal to little z n minus 1.
What is this expected value here?
It has to be Z sub n if j is equal to m.
Why is that?
That's the argument I was just going through before.
What's the expected value of a random variable conditional on
the random variable, that same random variable, having a
particular value?
The fact that you're given a large number of these
quantities doesn't make any difference.
The main thing that you're given here is the value of Z
sub n being little z sub m, which says this quantity is
equal to little z sub m if j is equal to m.
That's equal to 0 if j is unequal to m, which in fact is
just equal to Zm times the indicator function of j.
OK, so you--
and the same thing happens for the indicator
function of j equals n.
This should be j greater than or equal to n.
This is Zn minus 1.
And we add these things up.
And what we get, finally, is this sum here.
And now, you look at this last term here, which is a
combination of here and here.
And this is just the indicator function for j greater than or
equal to n minus 1.
And if you look back at the definition of Z sub n star,
this is just Z star of n minus 1.
I see a lot of blank faces.
But this is the kind of thing you almost
have to look at twice.
So we'll let it go with that.
So this shows that the expected value of Z sub n star
given the past of the original process is equal to Z
star n minus 1.
That's not quite what you want.
You want the expected value of Zn star given Z star of n
minus 1 to be equal to Z star n minus 1.
In other words, you want to be able to replace this quantity
in here with Z star n minus 1.
And the question is, how do you do that?
And that's what bothered me about the proof in the notes
because it didn't even talk about that.
So the argument is Z star n minus 1 in the past is really
a function of the past of the original process.
If I give you the sample values of the original
process, you can tell me where the process stops.
And you can say what the stop process is both before and
after that point.
So the stop values are a function of
the unstopped values.
So now what I can do is for every sample point of the
original process leading to a given sequence of the stop
process, we're going to have expected value Zn star given
these values here as equal to Z star n minus 1.
And since that's true for all of the values for which this
leads to that, this is true also.
So that proves it.
I'm doing this primarily because I think you ought to
be tortured with at least one proof in every lecture.
And the other thing is the proof in the notes was not
quite sufficient.
So I wanted to add to it here.
So now, you have a proof of it.
Consequences of the theorem, this is for sub-Martingales,
the marginal expected values of the stopped process lies in
between the expected value of Z1 and the expected
value of Z sub n.
In other words, when you take the stop process, it in some
sense is intermediate between what happens at time 1 and
what happens for the original process at time n.
It can't grow any faster than the original process.
This is also almost intuitively obvious.
And it's proven in section 7.8.
So you can find it there.
It's quite a bit easier than the proof I just went through.
The proof I just went through was a fairly difficult and
fairly tricky proof.
Partly I went through that proof because everything we do
from now on--
we're not going to do an awful lot of things.
But the Martingale convergence theorem, the strong law of
large numbers, and all of the other results we're talking
about all depend critically on that theorem that
we just went through.
In other words, it's a really major theorem.
It's not trivial little thing.
OK, this one is fairly major, too.
But it follows very easily from the other one.
Do I want to talk about this, this generating function
product of the Martingale?
No.
Let's let that go.
Let's not--
not that important.
So this is that, too.
No, I guess I better go back to that.
We need it.
OK, let's look at the generating function product
Martingale that we had for a random walk.
So X sub n is a sequence of IID random variables.
The partial sums form the variables of a random walk.
Sn is a random walk where Sn is a sum.
For any r such that gamma of r exists, we then define Z sub n
to be a Martingale, this Martingale here.
That's called a generating function Martingale.
Zn is a Martingale.
The expected value of Zn is equal to 1.
You can see immediately from this that the expected value
of Zn is equal to 1.
You don't need any of the theory we've gone through
because the expected value of the e to the r Sn is what?
It's a moment generating function to the n-th power.
That's just this term here.
So this has to be equal to 1 for all n.
The fact that this is a Martingale comes from that
example of product form Martingale
that we went through.
So there's nothing very sophisticated here.
OK, so if we assume that gamma of r exists and we let Zn be
this Martingale, well, this is just what we said before.
So you see it here.
Let j be the non-defective stopping time that stops when
either alpha greater than 0 or beta less than 0.
Since this is a stopping time, the expected value of e to the
Zn star is equal to 1 for all n greater than or equal to 1.
And the limit as n goes to infinity of Z sub n is then
going to be equal to the process at the
time where you stopped.
After you stop, you stay the same.
So you never move.
And the expected value of Z sub j is just
this quantity here.
What does that look like?
That's the Wald identity coming up again.
That's the Wald identity coming up for a random walk
with two thresholds.
The nice thing about doing it this way is you can see that
the proof applies to many other situations.
You can have almost any stopping rule you want.
And you still get the Wald identity.
So it has a much more general form than we had before.
This business here, the limit of Z sub n star going to Z sub
j is a little fishy.
The proof in the notes is fine.
This limit does in fact equal this limit.
What's bizarre is that the expected value of this limit
is not necessarily equal to the expected value of Z sub j.
So make a note to yourselves that if you ever want to use
Wald's identity in this more general case, think carefully
about what's going on as you go to the limit.
Because it can be a little bit tricky there.
It's not always what it looks like.
OK, so we're on to Mr. Kolmogorov again.
Kolmogorov was the guy who did so many things in the subject.
Most important, he said for a firm foundation to start with,
he was the one that said you really need a model.
And you really need some axioms.
And then, he went on with all these other neat things that
we've talked about from time to time.
His sub-Martingale inequality is a fairly simple result.
But this follows after that stopping theorem that we've
just talked about.
And everything else depends on this.
So there's a sort of chain that runs through this whole
development.
And if you go further in Martingales, you find that
this is just an absolutely major theorem which comes up
all the time.
And it's wonderful because it's so simple.
OK, so let's let Z sub n be a non-negative sub-Martingale.
And then, for any positive integer m and any number a
bigger than 0, the probability that the maximum of the first
n terms of this Martingale is greater than or equal to the
quantity a is the expected value of Z sub n divided by a.
This looks like the Markov inequality.
If instead of taking the maximum from 1 to m we just
look at Z sub n, the probability that Z sub n is
greater than or equal to a, then we get less than or equal
to Z of m divided by a.
So what this is saying is it's really strengthening that
Markov inequality and saying, you don't have to restrict
yourself to Z sub m.
You can instead look at all of the terms up until m.
And this bound here you get in the Markov inequality really
covers the maximum of all of those terms to start out with,
which says that for any m, you can look at the maximum over
an enormous sum of terms if you want to.
And it does this nice thing.
OK, I'm going to prove this also.
But this proof is simple.
So you can follow it in real time, I think.
So we want to start out with letting j be the stopping
time, which is essentially the smallest term where you've
crossed the threshold at a.
And if you haven't crossed a threshold at a, then it's
equal to the last term.
So here's the specific stopping rule that
we're going to use.
If Zn is greater than or equal to a for any n, then j is the
smallest n for which Zn is greater than a.
It's the first time at which we've crossed that threshold.
If Zn is less than a for all n up until m, then we make j
equal to n.
So we're insisting on stopping at some point.
This is not a defective stopping rule.
It's a real stopping rule because you've set the limit
on how far you want to look.
OK, so the process has to stop by time m.
The value of the process at the time you stop--
remember this thing we've called Z sub j, which is the
value at the stopping times.
Z sub j is greater than or equal to a if you stopped
before time m.
And Z sub n is--
well, we're saying that Z sub j is greater than or equal to
a if and only if Zn is greater than or equal to a for some n
less than or equal to m.
If you haven't crossed a threshold by time m, then Z
sub m is equal to Z sub n.
But it's not above a.
So the stopping time is this largest possible value that
we've got until the process stops by time n.
Zj greater than or equal to a if and only if we've crossed a
threshold for some n less than or equal to m.
So the probability that we've crossed the threshold from 1
to n to n is equal to the probability that Z sub j is
greater than or equal or a, which is less than or equal to
the expected value of Z sub j divided by a.
Since the process must be stopped by time m, we have Z
sub j is equal to Z sub m star.
And the stop process, f time m and Z sub n, is
less than or equal--
expected value of the stop process is less than or equal
to the expected value of the original process.
Why is that?
That's that theorum we just proved somewhere.
Yeah, this one here, OK?
That's submartingale consequence.
OK.
So that completes the proof.
And it's 10:30 now.
So the Kolmogorove submartingale inequality is
really a strengthening of the Markov inequality.
So you get this extra soup to nuts form of it.
Chebyshev inequality can be strengthened in the same way.
That's called a Kolmogorove inequality also.
Kolmogorove just got in here before anybody else and he
took those axioms that he made up--
and he was a smart guy--
and he developed this whole school of
probability in Russia.
And along with that, since he had these original results, he
just almost wiped up the field before anybody else knew what
was going on.
Partly a consequence of the fact that mathematicians in
most other parts of the world didn't believe that there was
any good probability theory going on in Russia.
So they weren't really conscious of this until he
cleaned up the whole field.
So if you want to be famous mathematician, you should move
away from the US and go to Upper Turkestan or something.
And you then clean up the whole field the same way that
Kolmogorove did.
OK.
So the strengthening of the Kolmogorove inequality.
What the result says is let Zn be a submartingale with the
expected value of Zn squared less than infinity.
Then the probability that the maximum of these terms, up to
n, is greater than or equal to b, is less than or equal to
the expected value of Zm squared divided by b squared.
You'll notice that that's almost the same thing as the
submartingale inequality.
This one says the probability that the maximum of Z sub i is
greater than or equal to i.
And this one says THE probability that the maximum Z
n is greater than or equal to b is less than or equal to the
expected value of Zn squared over b squared.
If you can't prove this, go back and look at the proof of
the Chebyshev inequality.
The proof of this given the submartingale inequality is
exactly the same as the proof of the Chebyshev inequality
given the Markov inequality.
You just go through the same steps and it's fairly simple.
OK.
So that is a nice result.
What happens if you apply this to a random walk?
If you apply it to a random walk, what you do is replace
this is Z sub n with the sum of random variables, and the
random walk, minus the mean of those random variables.
We have seen that a zero-mean random walk is a martingale.
So what we're going to do next is to use that zero-mean
random walk of the martingale and then what this says is the
probability that the maximum, from 1, less
than or equal to n--
less than or equal to m, of Sn minus nX bar.
That's Zn because we're subtracting off the main.
The probability of that is greater than or equal to, and
we just give b another name--
m times epsilon--
and it's less than or equal to the expected
value of Z sub m squared.
What's the expected value of this quantity squared?
It's n times sigma squared.
Because S sub m is just the sum of m IID random variables,
which have variance sigma squared.
So you take the expected value of this quantity squared, and
this n times the variance of X. So this then becomes sigma
squared times m divided by m squared times epsilon squared,
and the m cancels out.
This gives you the Chebyshev inequality with the extra
feature, but it deals with the whole sum from 1 up to m.
Now, you look at this and you say, gee.
Wouldn't it be absolutely wonderful if instead of going
from 1 to m, this went from m to infinity?
Because then you'd be saying the maximum of these terms--
the maximum is less than or equal to something.
And you'd have the strong law of large numbers all sitting
there for you.
And life was not that good, but almost as good, because we
can now do the strong law of large numbers assuming only
IID random variables with the variance.
So we're going to use that expression that we just did--
we're going to plug it into what we need for the strong
law of large numbers.
Again, I'm going to give you the idea of the proof of that.
I wasn't going to do that, but I looked at the proof in the
notes, and I had trouble understanding that too.
You understand, I have a problem here.
I write things two years ago, I look at them now.
I have a bad memory, so I have trouble understanding them.
So I recreate a new proof, which looks obvious to me now
because I've done it right now, and in two years, it
might look just as difficult.
So if you look at this half proof here and you can't
understand it, let me know, and I'll go back to the
drawing board and work on something else.
OK.
So the theorem says let X sub i be a sequence of IID random
variables with mean x bar and standard deviation sigma less
than infinity.
So I'm trying to do the strong law of large numbers before we
assume the fourth moment.
Here, I'm only assuming a second moment.
If you work really hard, you can do it with the first
absolute moment.
OK.
Let the S sub n be the sum of n random variables, then for
any epsilon--
oh, I don't need an epsilon there.
I don't know where that epsilon came from.
Just cross that out.
It doesn't belong.
The probability that the limit, as n goes to infinity,
of S n over n is equal to X bar.
The probability of that event--
event happens for a whole bunch of sample sequences.
It doesn't happen for others.
And this says that the probability of the class of
infinite length sequences for which that
happens is equal to 1.
That's the statement of the strong law of large numbers
that we had before.
It says that the probability of the set of sequences for
which the sample average approaches the main
probability of that set of sequences is equal to 1.
OK.
So the idea of the proof is going to be
the following thing.
And what I'm going to use is this Chebyshev inequality
we've already done.
But since Chebyshev inequality, in this new form--
namely the Kolmogorove inequality only goes up to n,
what I'm going to do is look at successively
larger values of n.
So I'm going to try to crawl my way up on infinity by
taking first a short length, then a longer length, then a
longer length, and a the longer length.
So I'm going to take this quantity here, which was the
quantity in the Kolmogorove submartingale inequality.
I'm going to ask what's the probability that the union of
all of these things, from m equals sum k, which I'm going
to let go to infinity later, what's the
probability of this union?
And the terms in the union, instead of going from 1 to n
to n, I want to replace n by 2 to the m.
The maximum of this.
The probability that this is greater than or equal to 2 to
the m times epsilon--
that's the biggest term times epsilon.
I want to see that this is less than or
equal to this quantity.
Now, why is this less than or equal to that?
AUDIENCE: Union bound.
PROFESSOR: What?
AUDIENCE: Union bound.
PROFESSOR: Union bound, yes.
That's all it is.
I've just applied the union bound to this.
This is less than or equal to the probability of this for m
equals k, plus the probability of this for m equals k plus 1,
and so forth.
Each of these terms is sigma squared over 2 to the m times
epsilon squared.
That's what we had on the last page, I hope.
Yes.
Sigma squared over m times epsilon squared.
Remember, we replaced m by 2 to the m, so this has changed
in that way.
And now we can sum this.
And when we sum it, we just get 2 sigma squared over 2 to
the k times epsilon squared.
What are we doing here?
What's the whole of this?
The Kolmogorove submartingale inequality, lets us, instead
of looking at just one value of n, let's us look at a whole
bunch of values altogether and maximize over them.
So what I'm going do is use the Kolmogorove submartingale
inequality over one big bunch of things and then over
another much bigger bunch of things, then over another
much, much bigger set of things.
And because I'm hopping over these much larger sequences, I
can now sum this quantity here, which I couldn't do if I
only had an m here.
If I replaced this 2 to the m by m and I tried to sum this,
what would happen?
It's a harmonic series and it diverges.
So what I've been able to do--
or, really, what Kolmogorove was able to do--
was instead of summing over all m, he was summing over
bunches of things and using this maximum here.
So this probability is less than or equal
to something finite.
If I now let k go to infinity, this term goes to 0, which
says that the tail end of this whole big thing goes to 0 as k
gets larger, for any epsilon at all.
But now, this doesn't quite look like what I want it to
look like, so what I'm going to do is find something which
is smaller than this that looks like what I would like
it to look like.
So I'm going to lower bound this quantity here by this
quantity here.
I still have the same union here.
Instead of finding the max over 1 to n to 2 to the n, I
will get a probability which is smaller because I'll only
maximize over part of those terms.
I'll only go from 2 to the m minus 1, less than or equal to
n, less than or equal to 2 to the n.
So I'm maximizing over a smaller set of terms, which
makes the probability of this smaller.
And then I'm replacing the 2 the m here by 2 times n,
because now this bound is between 2 to the n minus 1 and
2 to the m.
So I can replace it that way.
And now, it's exactly the sum that I want.
Yeah?
AUDIENCE: What does it mean to maximize the [INAUDIBLE]?
Now n is [INAUDIBLE].
So it looks like you're maximizing over inequalities.
Is it something like that?
PROFESSOR: Well, Yeah no.
I probably want to take this quantity, subtract off--
no.
What I really have to do to make this make any sense is
write it as the maximum over the same set of things S n
over n minus x bar, greater than or equal to 2 epsilon.
OK?
Thank you.
AUDIENCE: OK.
Another thing--
you're more likely to be bigger
than a smaller quantity.
Your n is smaller than 2 to the m, you're more likely to
be bigger than a smaller quantity.
So you're not bounding correctly, it would seem.
Oh, no.
You're doing 2 times m.
PROFESSOR: Well, there's no problem here.
I think the question is here.
Can I reduce this maximum down to a smaller sum and get a
smaller probability?
Oh, yes.
There's a smaller probability that this smaller max will
exceed a limit than if this will exceed a limit.
So I should do it in two steps.
In fact, you pointed it out here.
I should do it in three steps.
So the first step replaces this
maximum with this maximum.
Then the second step is going to go through that step.
And the third one is going to replace the m here with the n.
Anyway.
I think it's OK, with a few minor twiddles.
And before the term is over, I will get a new set of notes
out on the web, and you can check them to see if you're
actually satisfied with it, OK?
OK.
So finally, the martingale convergence theorem.
I'm not even going to try to prove this at all, but you
might have some imagination for how this follows from
dealing with stop processes, also.
What it says is Z sub n is a martingale again.
We're going to assume that there's something finite m, so
the expected value of Zn is less than or equal to n.
So what I'm saying is the expected value of Z sub n is
now bounded--
it's not finite, it's more than finite--
it's bounded.
It can never exceed this quantity.
I can have an expected value Z sub n, which is equal to 2 to
the n, and that's fine for every n.
But it's not bounded.
This quantity, I'm assuming it's bounded.
And then, according to this super theorem, there's a
random variable, Z. And don't ask what Z is.
Z is usually a very complicated random variable.
All this is doing is saying it exists.
You don't know what it is.
Such that the limit, as n goes to infinity, is Z sub n, is
equal to this random variable.
In other words, the limit of Z sub n minus Z is equal to 0 as
n goes to infinity.
And the texts proves the theorem with the additional
constraint that the expected value of Z sub
n squared is bounded.
Either one of those bounds is a very big constraint on these
martingales.
So the way you use these theorems is you take an
original problem that you're dealing with and you twist it
around, then you massage it and you do all sorts of things
to it all in order to get another martingale, which
satisfies this bound in the constraint.
Then you apply this theorem, and then you go back to where
you started.
So that's the sort of general way of dealing with this.
And you see this theorem being used in all sorts of strange
places where you would never expect it to be used.
For those of you in the communication field, about a
couple of years ago, there was a very famous paper dealing
with something called polar codes, which is
a new kind of coding--
very careful.
And they guy, in order to prove that these codes
worked, used that.
I don't know how--
I haven't checked it out yet-- but that was
crucial in his proofs.
So he had to turn these things into martingales somehow and
then use that proof.
OK.
We talked about branching processes, about the
remarkable things about them.
This theorem applies directly to these branching processes.
A branching process, you remember--
the number of elements or organisms or whatever, at time
n is the number of offspring of the set of elements at
time n minus 1.
Each element at each time has a random number of offspring,
which is independent of time that's independent of all the
other elements.
And it's a random variable Y. And the expected value of X
sub n is going to be X sub n minus 1 times the expected
value of Y, because X sub n minus 1 is the number of
elements in the n minus first generation.
Y bar is the expected number of offspring
of each one of them.
So you look at it and you say, ah.
That theorem doesn't work.
No good.
You walk away.
Then somebody else who is really interested in branching
processes says, oh, this process is growing as--
I mean it's growing by Y bar every unit of time.
So I should be able to deal with that somehow.
So I say, OK.
Let's look at the number of elements in the n-th
generation and divide it by Y to the n.
When you do this, what happens?
The expected value of X n divided by Y bar to the n is
going to be equal to X sub n minus 1 over Y bar sub n minus
1 times
AUDIENCE: You need to [INAUDIBLE].
PROFESSOR: Yes.
That would help, wouldn't it.
Thank you.
Given X n minus 1 divided by Y bar n times n minus 1.
And Xn minus 2 over Y bar n minus 2.
And so if we're given these things, we don't have to worry
about this quantity if we're just given a number in each
generation.
If we're given a number in generation n minus 1, the
expected value of X sub n over Y n is just X n minus 1.
Expected value, we pick up another value of Y divided by
Y bar to the n, which is X n minus 1 over Y
bar the n minus 1.
OK.
So the theorem applies here because that expected value is
just 1, then.
And this is a martingale.
So the theorem says that this quantity
approaches a random variable.
And what does that mean?
Well, if you observe this process for a long time, it
might die out.
If it dies out and it stays died out-- it
never comes back again.
And if it doesn't die out, it's going to start to grow.
And if it starts to grow, it's going to start to
grow in this why.
After a very long time, if it's growing, X sub n minus 1
is humongous, and the law of large numbers says that the
next generation should have very close to X sub n minus 1
times the Y bar elements in it.
So it says that after you get started, this thing wobbles
around trying to decide whether it's going to go to
zero or decide to get large.
But if one starts to get large, then it becomes very
stable from that time on.
And it's going to increase by Y bar with each unit of time.
If it decides it's going to die out, it goes to 0 and it
stays there.
So what the theorem is saying is just what I just said--
namely X to the n over Y bar n in fact either grows in this
very regular Y or it goes to 0 and it stays there.
So this random variable is 0 with a probability that the
process dies out, and we evaluated that before.
The other values of it are very hard to evaluate.
The other values depend on how long this thing takes.
If it's not going to go to 0, how long does it take before
it really takes off?
And sometimes it takes a long time before it really takes
off, sometimes it takes a short time, and that's what
the random variable Z is.
But the random variable Z says that after a very long time,
the value of this process is X sub n is going to be
approximately Z times this quantity here, which is
growing exponentially.
OK.
That gives us the martingale convergence theorem.
Next time, I will try to review at least the whole
course from Markov chains on.