WEBVTT
00:00:00.101 --> 00:00:00.600
All right.
00:00:00.600 --> 00:00:03.550
So let us revisit the
example from last lecture.
00:00:03.550 --> 00:00:06.880
So we have a Markov chain
with two states, one and two,
00:00:06.880 --> 00:00:11.502
and this Markov chain has
a single recurrent class.
00:00:11.502 --> 00:00:12.950
All right.
00:00:12.950 --> 00:00:15.970
And then also it's
not periodic right,
00:00:15.970 --> 00:00:19.060
because we have self
transition of this type.
00:00:19.060 --> 00:00:23.810
So as a result, this is well
defined and these steady state
00:00:23.810 --> 00:00:32.208
probabilities from 1 to m, in
that case for us, m = 2, right?
00:00:32.208 --> 00:00:34.970
So let us write the
system and solve
00:00:34.970 --> 00:00:38.940
the system of linear equation
for this example here.
00:00:38.940 --> 00:00:41.176
So what we have is pi
1 equals pi 1 times 0.5
00:00:41.176 --> 00:00:51.880
plus pi 2 times 0.2.
00:00:51.880 --> 00:00:55.500
So that's the first equation
that corresponds to j equals 1.
00:00:55.500 --> 00:00:58.024
Now, for j equals 2, pi
2 equals pi 1 times 0.5
00:00:58.024 --> 00:01:07.410
plus pi 2 times 0.8.
00:01:07.410 --> 00:01:09.580
So we have a system
of two equations
00:01:09.580 --> 00:01:13.390
with two unknowns,
pi 1 and pi 2.
00:01:13.390 --> 00:01:18.060
Let us rewrite them, I
pass this one on this side
00:01:18.060 --> 00:01:19.432
and this one on this side.
00:01:19.432 --> 00:01:21.182
So we get pi 1 times
1 minus 0.5 minus 0.5
00:01:21.182 --> 00:01:30.084
equals pi 2 times 0.2.
00:01:30.084 --> 00:01:31.834
And this one pi 2 times
1 minus 0.8 is 0.2
00:01:31.834 --> 00:01:41.340
equals pi 1 times 0.5.
00:01:41.340 --> 00:01:45.700
We realize that these two
happen to be the same,
00:01:45.700 --> 00:01:50.350
so they are not enough to
define a unique solution,
00:01:50.350 --> 00:01:52.240
so we have to add
another equation,
00:01:52.240 --> 00:01:54.580
and we know that these
are probabilities.
00:01:54.580 --> 00:01:59.100
So pi 1 plus pi 2 has to
be one, and so now we're
00:01:59.100 --> 00:02:01.930
going to keep one of these
two, let's say this one,
00:02:01.930 --> 00:02:03.141
I'm going to write it here.
00:02:03.141 --> 00:02:05.266
And we can rewrite it by
saying that pi 1 times 1/2
00:02:05.266 --> 00:02:06.182
equals pi 2 times 1/5.
00:02:14.730 --> 00:02:18.770
So now, we're going
to take that, replace
00:02:18.770 --> 00:02:26.010
pi 1 equals 2/5 of pi 2
is the result of that.
00:02:26.010 --> 00:02:30.690
And we're going to use that
pi 1 and replace it here.
00:02:30.690 --> 00:02:38.400
So we end it by 2 times
2/5 plus 1 equals 1,
00:02:38.400 --> 00:02:46.730
which means that from here, we
get that pi 2 equals 5 plus 2/7
00:02:46.730 --> 00:02:52.540
so 5/7, and then we use
that and place it here
00:02:52.540 --> 00:03:00.280
and we end up having pi 1
equals 2/5 times 5/7 equals 2/7,
00:03:00.280 --> 00:03:03.510
and we check 5
plus 2 equals 7, so
00:03:03.510 --> 00:03:05.350
these are real probabilities.
00:03:05.350 --> 00:03:07.370
So the probabilitiy
that you find yourself
00:03:07.370 --> 00:03:12.627
at state one at time 1 trillion
would be approximately 2/7.
00:03:12.627 --> 00:03:14.210
The probability that
you find yourself
00:03:14.210 --> 00:03:19.760
at state one at time 2 trillions
is again approximately 2/7.
00:03:19.760 --> 00:03:22.860
So essentially what we have
here is the probability
00:03:22.860 --> 00:03:27.370
of being in that state one
settles in a steady value.
00:03:27.370 --> 00:03:30.140
That's what the steady
state convergence means.
00:03:30.140 --> 00:03:32.960
It's convergence of
probabilities, not convergence
00:03:32.960 --> 00:03:34.130
of the process itself.
00:03:34.130 --> 00:03:37.350
Again, the process will
keep jumping back and forth,
00:03:37.350 --> 00:03:40.240
but the steady state probability
will settle for a given value
00:03:40.240 --> 00:03:44.300
here in one, that will be
2/7, and the steady state
00:03:44.300 --> 00:03:49.720
probability in being in
two will settle to 5/7.
00:03:49.720 --> 00:03:52.355
And finally in this
example, and more
00:03:52.355 --> 00:03:55.420
generally when we have a single
class and no periodicity,
00:03:55.420 --> 00:03:58.520
the initial state
does not matter.