WEBVTT

00:00:00.800 --> 00:00:03.220
The other thing that we
discussed in the last lecture

00:00:03.220 --> 00:00:06.410
was a classification of the
different states of the Markov

00:00:06.410 --> 00:00:09.270
chain into different types.

00:00:09.270 --> 00:00:11.440
A Markov chain in
general has states

00:00:11.440 --> 00:00:15.740
that are recurrent, which means
that from that recurrent state,

00:00:15.740 --> 00:00:19.380
you can go somewhere else and
then from that somewhere else

00:00:19.380 --> 00:00:22.090
you can always come back to it.

00:00:22.090 --> 00:00:25.080
So if you have a Markov
chain of this form

00:00:25.080 --> 00:00:27.910
and you start in state
nine, the options for you

00:00:27.910 --> 00:00:32.000
is to either go to state
three or to state five.

00:00:32.000 --> 00:00:34.630
But no matter what,
if you go to three,

00:00:34.630 --> 00:00:37.036
you can come back always,
and if you go to five,

00:00:37.036 --> 00:00:38.410
you can always
come back as well.

00:00:38.410 --> 00:00:42.830
So clearly nine here would be
a recurrent state, and three

00:00:42.830 --> 00:00:45.890
for the same reason,
and five as well.

00:00:45.890 --> 00:00:49.120
Now, if you look at
the state six or seven,

00:00:49.120 --> 00:00:50.180
it is the same thing.

00:00:50.180 --> 00:00:53.600
Starting from six, the
only way that you can go to

00:00:53.600 --> 00:00:57.400
is to either stay at six or go
to seven, and then in that case

00:00:57.400 --> 00:00:58.360
you always come back.

00:00:58.360 --> 00:01:01.620
And same thing from seven,
you can either go to six

00:01:01.620 --> 00:01:03.540
and that's it's,
actually, and come back.

00:01:03.540 --> 00:01:07.520
So both of these are
recurrent as well.

00:01:07.520 --> 00:01:09.930
So in a case the state
is not recurrent,

00:01:09.930 --> 00:01:11.030
we will call it transient.

00:01:11.030 --> 00:01:14.220
So let's look at for
example state one.

00:01:14.220 --> 00:01:18.260
From state one, if you go from
one to two and then go to six,

00:01:18.260 --> 00:01:20.080
there is no way to
come back to one.

00:01:20.080 --> 00:01:23.810
So the state 1 is transient,
and for the same reason

00:01:23.810 --> 00:01:25.520
the state four
will be transient,

00:01:25.520 --> 00:01:27.620
and the state 2
will be transient.

00:01:27.620 --> 00:01:28.810
What about eight?

00:01:28.810 --> 00:01:32.900
Well, the same reason, the
state is transient as well.

00:01:32.900 --> 00:01:36.440
So what we have seen also is
the notion of a recurrent class.

00:01:36.440 --> 00:01:39.630
A recurrent class is,
again, a collection

00:01:39.630 --> 00:01:43.320
of recurrent states that can
communicate between each other.

00:01:43.320 --> 00:01:47.550
So here, for this specific
example, we have two classes.

00:01:47.550 --> 00:01:52.970
This is one class, right,
so it's a class one.

00:01:52.970 --> 00:01:56.140
Let's call it
recurrent class one.

00:01:56.140 --> 00:02:02.550
And this is a recurrent
class, recurrent class two.

00:02:02.550 --> 00:02:05.610
So here again we have two
classes instead of one,

00:02:05.610 --> 00:02:10.788
because if you are in
one of these classes,

00:02:10.788 --> 00:02:13.970
there is no way that you can
find a path to go to one state

00:02:13.970 --> 00:02:15.710
here and vice versa.

00:02:15.710 --> 00:02:18.480
If you are in one of
these states here,

00:02:18.480 --> 00:02:22.670
there is no path that would lead
you to that recurrent class.

00:02:22.670 --> 00:02:25.950
In the case where you have two
recurrent classes, like here

00:02:25.950 --> 00:02:29.980
or more, it is pretty
clear that in the long run,

00:02:29.980 --> 00:02:32.200
the steady state behavior
of the Markov chain

00:02:32.200 --> 00:02:34.980
will really depend
on where you started.

00:02:34.980 --> 00:02:37.960
So for example, if
your Markov chain

00:02:37.960 --> 00:02:42.720
started in that recurrent
class, there is no probability

00:02:42.720 --> 00:02:46.500
that in the long run it will be
in that class, and vice versa.

00:02:46.500 --> 00:02:48.600
If it started here,
the probability

00:02:48.600 --> 00:02:52.800
of being in that recurrent
class in the long run is zero.

00:02:52.800 --> 00:02:56.390
So the long run behavior
of the Markov chain

00:02:56.390 --> 00:03:00.260
will depend on the
initial condition.

00:03:00.260 --> 00:03:03.685
In the case where you have
only one recurrent class,

00:03:03.685 --> 00:03:06.420
let's forget about that
portion, for example,

00:03:06.420 --> 00:03:09.330
and you have only
that portion here.

00:03:09.330 --> 00:03:12.470
Then maybe the initial
condition will not

00:03:12.470 --> 00:03:14.970
matter in the long
run, but in fact it's

00:03:14.970 --> 00:03:17.470
not going to be always
the case, depending

00:03:17.470 --> 00:03:19.920
on the recurrent
class being periodic

00:03:19.920 --> 00:03:23.380
or not as we will
see in the next clip.