WEBVTT
00:00:01.780 --> 00:00:05.600
OK, so we have just seen that
if we have a single recurrent
00:00:05.600 --> 00:00:08.740
class, which is not periodic,
then the Markov chain reaches
00:00:08.740 --> 00:00:13.560
steady state, and the steady
state probabilities satisfy
00:00:13.560 --> 00:00:16.730
the following
system of equations.
00:00:16.730 --> 00:00:19.760
These equations are essential
in the study of Markov chains,
00:00:19.760 --> 00:00:21.150
and they have a name.
00:00:21.150 --> 00:00:23.780
They are called the
balance equations.
00:00:23.780 --> 00:00:26.140
In fact, it's worth
looking at them
00:00:26.140 --> 00:00:29.990
in a somewhat different way than
how we introduced them so far.
00:00:29.990 --> 00:00:32.475
Using a frequency
interpretation.
00:00:32.475 --> 00:00:34.880
Along the way, it
will shed some light
00:00:34.880 --> 00:00:39.640
on the how or why they are
called balance equations.
00:00:39.640 --> 00:00:41.660
Intuitively, one
can sometimes think
00:00:41.660 --> 00:00:43.700
of probabilities as frequencies.
00:00:43.700 --> 00:00:46.420
For example, if we keep
tossing a fair coin, which
00:00:46.420 --> 00:00:49.700
has a probability half of Heads,
and then in the long run, half
00:00:49.700 --> 00:00:51.960
of the time we are
going to see Heads.
00:00:51.960 --> 00:00:54.900
So let us try an interpretation
of this kind for pi
00:00:54.900 --> 00:00:58.940
of j, the steady state
probability of state j.
00:00:58.940 --> 00:01:01.270
Imagine the evolution
of a Markov chain
00:01:01.270 --> 00:01:04.440
as a particle jumping
from state to state.
00:01:04.440 --> 00:01:07.015
And imagine an observer
at a given state.
00:01:07.015 --> 00:01:10.160
So imagine that you
have an observer here,
00:01:10.160 --> 00:01:11.039
in a given state j.
00:01:11.039 --> 00:01:13.240
And imagine that
this observer will
00:01:13.240 --> 00:01:17.890
keep counting every time the
particle visits the state j.
00:01:17.890 --> 00:01:19.840
So you have this
observer keeping
00:01:19.840 --> 00:01:23.780
track every time the particle is
in state j and keep recording.
00:01:23.780 --> 00:01:26.050
So, for example, one
record at time two,
00:01:26.050 --> 00:01:31.080
and saw it at time
four, eight, maybe n.
00:01:31.080 --> 00:01:34.490
And you can look at the
total number of time
00:01:34.490 --> 00:01:37.530
this observer saw the
particle being in j,
00:01:37.530 --> 00:01:38.789
and you can define it.
00:01:38.789 --> 00:01:42.700
Let's call it y of j of n.
00:01:42.700 --> 00:01:47.400
So yj of n represents
the total number of ones
00:01:47.400 --> 00:01:49.690
that you have up to time n.
00:01:49.690 --> 00:01:52.289
So it's the total
number, so we divide by n
00:01:52.289 --> 00:01:53.780
to have the frequency.
00:01:53.780 --> 00:01:56.940
So here that would be
the frequency of time
00:01:56.940 --> 00:02:02.040
this observer saw the particle
in state j up to time n.
00:02:02.040 --> 00:02:07.150
Well, when n is very,
very large, so n large,
00:02:07.150 --> 00:02:12.690
that frequency
approaches pi of j.
00:02:12.690 --> 00:02:14.820
In fact, we can make
it more rigorous
00:02:14.820 --> 00:02:18.800
by saying that that
converges to pi of j
00:02:18.800 --> 00:02:21.700
when n goes to infinity
in a rigorous fashion
00:02:21.700 --> 00:02:24.190
that we will not discuss here.
00:02:24.190 --> 00:02:29.210
So we have now a frequency
interpretation of pi of j.
00:02:29.210 --> 00:02:32.470
Now, let us think
about a frequency
00:02:32.470 --> 00:02:36.510
interpretation of
transitions from 1 to j.
00:02:36.510 --> 00:02:39.870
So again, you have
a new observer,
00:02:39.870 --> 00:02:42.200
and this observer
look at it here,
00:02:42.200 --> 00:02:45.720
and every time the particle
pass here, he put a one.
00:02:45.720 --> 00:02:48.810
So, for example, maybe
one here and here.
00:02:48.810 --> 00:02:53.730
So if you think about it,
you're looking at from 1
00:02:53.730 --> 00:02:57.060
to j, and of n, that would
be the total number of ones
00:02:57.060 --> 00:03:00.340
that you observe
here up to time n.
00:03:00.340 --> 00:03:03.630
And if you divide by 1,
that's the frequency,
00:03:03.630 --> 00:03:05.750
and so what is this frequency?
00:03:05.750 --> 00:03:08.360
Well, let's look at it this way.
00:03:08.360 --> 00:03:10.890
So how often do we
have such a transition?
00:03:10.890 --> 00:03:15.020
Well, a fraction pi1 of
the time, the particle
00:03:15.020 --> 00:03:19.050
is in state 1, and
whenever at state 1,
00:03:19.050 --> 00:03:24.140
there is going to be a
probability p1j of going there.
00:03:24.140 --> 00:03:27.620
There might be other
ways to go, but out
00:03:27.620 --> 00:03:30.990
of all the time the
particle is in state one,
00:03:30.990 --> 00:03:36.750
the frequency of time it will
transition to j will be pi 1j.
00:03:36.750 --> 00:03:40.600
So out of all possible
transitions that can happen,
00:03:40.600 --> 00:03:42.900
the fraction of
these transitions
00:03:42.900 --> 00:03:49.000
that will happen from 1 to
j will be pi 1 times p1j.
00:03:52.079 --> 00:03:54.820
Again, this is when
n is large, and this
00:03:54.820 --> 00:03:58.040
can be made more rigorous.
00:03:58.040 --> 00:04:03.400
Now, what's the total frequency
of transitions into state j?
00:04:03.400 --> 00:04:06.190
So these are
transitions leaving.
00:04:09.020 --> 00:04:12.290
These are the transitions
of interest here.
00:04:12.290 --> 00:04:19.750
So think about a third observer
looking here and recording
00:04:19.750 --> 00:04:23.270
every time the particle
goes through here, here,
00:04:23.270 --> 00:04:26.610
here, or here.
00:04:26.610 --> 00:04:29.150
So what is the frequency
of transition here?
00:04:29.150 --> 00:04:33.630
Well, it will be the sum of
all the possible transitions
00:04:33.630 --> 00:04:35.170
that we have observed there.
00:04:35.170 --> 00:04:40.034
And so this is going to be this
and that corresponds to this.
00:04:42.880 --> 00:04:44.980
Now, the last step
of the argument
00:04:44.980 --> 00:04:50.880
is to see that the
particle is in state j, if
00:04:50.880 --> 00:04:55.680
and only if the last
transition was into state j.
00:04:55.680 --> 00:04:58.940
And this explains that this
part, which we have calculated
00:04:58.940 --> 00:05:03.980
here, will be the same as this
one and that explain that.
00:05:03.980 --> 00:05:07.220
So this equation expresses
exactly the statement
00:05:07.220 --> 00:05:08.030
that we made.
00:05:08.030 --> 00:05:11.100
That's useful intuition
to have, and we
00:05:11.100 --> 00:05:15.020
are going to see an example
later on how it gives us
00:05:15.020 --> 00:05:18.425
shortcuts into
analyzing Markov chains.