WEBVTT
00:00:00.500 --> 00:00:04.570
We have observed in the simple
example from the previous clip
00:00:04.570 --> 00:00:08.900
that when the Markov chain
initially starts in state one,
00:00:08.900 --> 00:00:11.970
the probability that it
finds itself in state one
00:00:11.970 --> 00:00:17.050
after a long period of time
converges to a constant value,
00:00:17.050 --> 00:00:21.300
in our case, 2/7.
00:00:21.300 --> 00:00:25.880
In addition, if the Markov chain
initially starts in state two,
00:00:25.880 --> 00:00:28.740
the probability that it
finds itself in state one
00:00:28.740 --> 00:00:31.760
after a long period
of time also converges
00:00:31.760 --> 00:00:33.260
to the same constant value, 2/7.
00:00:37.500 --> 00:00:41.650
Are these two properties
of long term convergence
00:00:41.650 --> 00:00:45.140
and of vanishing effect
of the initial state
00:00:45.140 --> 00:00:50.240
over the long term
convergence always true?
00:00:50.240 --> 00:01:07.530
Mathematically, we are asking
the question, is rij of n pi j
00:01:07.530 --> 00:01:12.150
when n goes to infinity?
00:01:12.150 --> 00:01:16.450
The answer is that for
nice Markov chains,
00:01:16.450 --> 00:01:22.880
this will be true, but
this is not always true.
00:01:22.880 --> 00:01:25.630
Consider the first question.
00:01:25.630 --> 00:01:29.910
Does rij(n) always
converge to something
00:01:29.910 --> 00:01:32.539
as n goes to infinity?
00:01:32.539 --> 00:01:36.116
Look at the following
simple Markov chain.
00:01:36.116 --> 00:01:42.140
When in state two, you
will never be in state two
00:01:42.140 --> 00:01:44.740
at the next transition.
00:01:44.740 --> 00:01:51.280
You will end up next in either
state one or state three.
00:01:54.140 --> 00:01:57.870
However, no matter
where you end up,
00:01:57.870 --> 00:02:01.810
you're sure that the next
transition will bring you back
00:02:01.810 --> 00:02:06.080
to state two, either
here or from here.
00:02:06.080 --> 00:02:14.340
In other words, for n odd,
r22 of n will always be 0,
00:02:14.340 --> 00:02:19.485
and for n even, r22
of n will always be 1.
00:02:22.079 --> 00:02:26.040
And so r22 of n
will never converge.
00:02:26.040 --> 00:02:30.350
It will always alternate
between 1 or 0.
00:02:30.350 --> 00:02:32.670
Convergence has failed.
00:02:32.670 --> 00:02:36.240
That chain has a
periodic structure,
00:02:36.240 --> 00:02:38.390
and we will see in
the next lecture
00:02:38.390 --> 00:02:43.070
that if periodicity is
absent from a chain,
00:02:43.070 --> 00:02:47.570
then we don't have a
problem with convergence.
00:02:47.570 --> 00:02:51.240
Consider now the second question
dealing with a vanishing
00:02:51.240 --> 00:02:56.200
importance of initial states
when convergence occurs.
00:02:56.200 --> 00:02:59.690
For this, consider the
following Markov chain.
00:03:04.370 --> 00:03:10.320
If you start in state one,
there is no way you can escape.
00:03:10.320 --> 00:03:12.310
You are certain to
stay there forever.
00:03:15.430 --> 00:03:21.750
So r11 of n will always be 1.
00:03:21.750 --> 00:03:26.560
On the other hand, if
you start in state three,
00:03:26.560 --> 00:03:30.400
there is no way you will
ever reach state one.
00:03:30.400 --> 00:03:34.650
So r31 of n will be 0.
00:03:37.500 --> 00:03:40.410
The initial state
of where you started
00:03:40.410 --> 00:03:45.020
does matter in this example,
and its influence never
00:03:45.020 --> 00:03:46.900
vanishes in the long run.
00:03:46.900 --> 00:03:50.610
The second nice property
has failed here.
00:03:50.610 --> 00:03:53.860
And here, this has to do
with the Markov structure
00:03:53.860 --> 00:03:57.150
where some states are not
accessible from some other
00:03:57.150 --> 00:04:00.880
states, and we will address
this in the final portion
00:04:00.880 --> 00:04:03.410
of this lecture.
00:04:03.410 --> 00:04:10.790
Finally, let us calculate
r21 of n for large n.
00:04:10.790 --> 00:04:16.329
So you start in state
two, and you ask yourself,
00:04:16.329 --> 00:04:21.519
what is the probability that
I will end up in state one
00:04:21.519 --> 00:04:25.950
after n steps for n large?
00:04:25.950 --> 00:04:30.810
Well, when you start in two,
you may stay in two for a while
00:04:30.810 --> 00:04:33.850
by doing this kind of
transition and this transition
00:04:33.850 --> 00:04:35.830
and this transition.
00:04:35.830 --> 00:04:41.270
But eventually, with probability
one, you will escape.
00:04:41.270 --> 00:04:45.200
Either you will go
to state one, or you
00:04:45.200 --> 00:04:46.760
will escape to state three.
00:04:49.810 --> 00:04:54.632
And in that case, you
will never go back to two.
00:04:54.632 --> 00:04:58.090
If you are in one, you will
never go back here to two,
00:04:58.090 --> 00:05:02.150
and from three, you will
never go back to two.
00:05:02.150 --> 00:05:05.440
Because of the symmetry
between these probabilities
00:05:05.440 --> 00:05:12.060
here-- 0.3 on this side
and 0.3 on this side--
00:05:12.060 --> 00:05:15.160
when you do escape
state two, you
00:05:15.160 --> 00:05:21.840
are equally likely to escape
toward one or toward three.
00:05:21.840 --> 00:05:28.975
So what you have is that
r21 of n will be 1/2.