WEBVTT
00:00:00.990 --> 00:00:04.270
So we have just seen
that a clever trick based
00:00:04.270 --> 00:00:06.890
on the frequency interpretation
of the transitions
00:00:06.890 --> 00:00:10.350
between successive
states, like here,
00:00:10.350 --> 00:00:13.530
allows us to write a simple
set of equations which
00:00:13.530 --> 00:00:19.800
can be solved recursively,
given here, giving pi i plus 1
00:00:19.800 --> 00:00:22.330
as a function of pi of i.
00:00:22.330 --> 00:00:25.720
More specifically,
we have pi i plus 1
00:00:25.720 --> 00:00:30.570
equals pi of i times p of i.
00:00:30.570 --> 00:00:33.330
Divide by q of i plus 1.
00:00:33.330 --> 00:00:38.040
And this is true for
i equal 0 up to m.
00:00:38.040 --> 00:00:42.620
And to start the recursion,
we need to find pi of 0.
00:00:42.620 --> 00:00:49.170
And this can be done using this
normalization condition-- which
00:00:49.170 --> 00:01:00.110
leads to pi of 0 times 1 plus
p0 over q1 plus et cetera
00:01:00.110 --> 00:01:02.650
equals 1.
00:01:02.650 --> 00:01:04.970
Let's illustrate the
details of this procedure
00:01:04.970 --> 00:01:06.710
on a special case.
00:01:06.710 --> 00:01:09.750
Let's assume that all
the p's are the same
00:01:09.750 --> 00:01:12.450
and all the q's are the same.
00:01:12.450 --> 00:01:16.390
So this is a special case
in which we are interested.
00:01:16.390 --> 00:01:21.030
So at each point in time, if
we are somewhere in the middle,
00:01:21.030 --> 00:01:24.110
you have probability
p of moving up,
00:01:24.110 --> 00:01:27.770
and probability
q of moving down.
00:01:27.770 --> 00:01:32.500
Define rho to be the
ratio of p over q.
00:01:32.500 --> 00:01:35.930
Rho can be interpreted as
the frequency of going up
00:01:35.930 --> 00:01:38.420
versus the frequency
of going down.
00:01:38.420 --> 00:01:41.190
If it's a service system,
you can think of it
00:01:41.190 --> 00:01:44.470
as a measure of how
loaded the system is.
00:01:44.470 --> 00:01:48.340
If p equals q, that means that
if you are at this state--
00:01:48.340 --> 00:01:52.500
you are equally likely
to move left or right.
00:01:52.500 --> 00:01:54.720
So the chain does
not have a tendency
00:01:54.720 --> 00:01:59.070
to move in that direction
or in that direction.
00:01:59.070 --> 00:02:04.880
If rho is bigger than 1,
so that p is bigger than q,
00:02:04.880 --> 00:02:08.430
it means that whenever we are
at some state in the middle,
00:02:08.430 --> 00:02:12.050
we are more likely to
move right, as opposed
00:02:12.050 --> 00:02:14.800
to moving left.
00:02:14.800 --> 00:02:17.230
Which means that the
chain has a tendency
00:02:17.230 --> 00:02:19.440
to move in that direction.
00:02:19.440 --> 00:02:23.240
And if you think of this as a
number of customers in queue,
00:02:23.240 --> 00:02:26.450
it means your system has the
tendency to become loaded
00:02:26.450 --> 00:02:28.160
and to build up a queue.
00:02:28.160 --> 00:02:31.480
So rho being bigger
than 1 corresponds
00:02:31.480 --> 00:02:34.390
to a heavy load,
where queues build up.
00:02:34.390 --> 00:02:37.079
Rho less than one
corresponds to the system
00:02:37.079 --> 00:02:39.370
where queues have the
tendency to drain down.
00:02:39.370 --> 00:02:42.690
The system is going to
move in that direction.
00:02:42.690 --> 00:02:46.210
Now let us write
down these equations
00:02:46.210 --> 00:02:48.350
for that special case.
00:02:48.350 --> 00:02:52.090
We end up with that,
which is pi i times rho,
00:02:52.090 --> 00:02:54.100
by definition of rho.
00:02:54.100 --> 00:02:58.760
Once you look at this equation,
you realize that pi of 1
00:02:58.760 --> 00:03:01.860
is pi of 0 times rho.
00:03:01.860 --> 00:03:07.280
And pi of 2 is pi of
1 times rho equals
00:03:07.280 --> 00:03:10.740
pi of 0 times rho square.
00:03:10.740 --> 00:03:12.810
And so on and so forth.
00:03:12.810 --> 00:03:16.200
And you find that you
can express pi of i
00:03:16.200 --> 00:03:18.750
as pi of 0 times
rho at the power
00:03:18.750 --> 00:03:24.600
i for any possible
i between 0 and m.
00:03:24.600 --> 00:03:31.780
And now if we use the
normalization condition,
00:03:31.780 --> 00:03:36.370
we get that pi of 0
times 1 plus rho plus rho
00:03:36.370 --> 00:03:44.050
squared plus rho at the
power m is equal to 1.
00:03:44.050 --> 00:03:45.970
Let's now complete
the calculations
00:03:45.970 --> 00:03:47.920
for two special cases.
00:03:47.920 --> 00:03:51.610
If rho is equal to 1,
that means p equals q.
00:03:51.610 --> 00:03:56.860
Then pi i equals
pi of 0 for all i.
00:03:56.860 --> 00:04:00.500
It means that all the steady
state probabilities are equal.
00:04:00.500 --> 00:04:04.690
This special case is called
a symmetric random walk.
00:04:04.690 --> 00:04:07.640
So you start at the
state at a point in time.
00:04:07.640 --> 00:04:11.260
Either you stay in place, or
you have an equal probability
00:04:11.260 --> 00:04:14.220
of going left or right.
00:04:14.220 --> 00:04:16.589
There is no bias in
either direction.
00:04:16.589 --> 00:04:18.510
You might think that
in such a process,
00:04:18.510 --> 00:04:20.428
you will tend to
get stuck either
00:04:20.428 --> 00:04:23.390
near one end or the other end.
00:04:23.390 --> 00:04:25.740
It turns out that
no, in the long run,
00:04:25.740 --> 00:04:28.250
the symmetric random
walk is equally likely
00:04:28.250 --> 00:04:30.660
to be at any of those states.
00:04:30.660 --> 00:04:34.659
And for the special case--
this equation here-- is simply
00:04:34.659 --> 00:04:40.159
that pi of 0 times
1 plus m equals one.
00:04:40.159 --> 00:04:47.520
That means that pi of 0
equals 1 over 1 plus m.
00:04:47.520 --> 00:04:49.350
Which is consistent
with the fact
00:04:49.350 --> 00:04:51.440
that all steady-state
probabilities are the same.
00:04:51.440 --> 00:04:52.890
They are all equally likely.
00:04:52.890 --> 00:04:54.370
They are end states.
00:04:54.370 --> 00:04:59.850
And so each one of them, pi i
is pi of 0, which is 1 over 1
00:04:59.850 --> 00:05:00.690
plus m.
00:05:00.690 --> 00:05:02.840
The Markov chain
is equally likely
00:05:02.840 --> 00:05:07.590
to be in any of these m plus
1 states in the long run.
00:05:07.590 --> 00:05:12.630
Suppose now instead of p equals
q, that m is very, very large,
00:05:12.630 --> 00:05:13.970
a very large number.
00:05:13.970 --> 00:05:16.410
Let's take m going to infinity.
00:05:16.410 --> 00:05:20.050
And suppose that the system
is on the stable side.
00:05:20.050 --> 00:05:23.320
That means that
p is less than q,
00:05:23.320 --> 00:05:25.960
which means that there's
a tendency for customers
00:05:25.960 --> 00:05:28.280
to be served faster
than they arrive.
00:05:28.280 --> 00:05:33.840
In other words, the chain is
drifting toward that direction.
00:05:33.840 --> 00:05:38.010
So that means that
rho is less than 1
00:05:38.010 --> 00:05:41.560
and what it means is that
this infinite series, when
00:05:41.560 --> 00:05:45.200
m goes to infinity, is
the geometric series.
00:05:45.200 --> 00:05:49.760
And this series is going
to be 1 over 1 minus rho.
00:05:49.760 --> 00:05:57.980
That is, this infinite
series is 1 over 1 minus rho.
00:05:57.980 --> 00:06:02.110
And since pi of 0 is 1
over this infinite series,
00:06:02.110 --> 00:06:06.800
we end up having pi
0 equals 1 minus rho.
00:06:06.800 --> 00:06:13.940
And since we have pi of i equals
pi 0 times rho at the power i,
00:06:13.940 --> 00:06:19.225
we end up having that pi
of i equals pi of 0, which
00:06:19.225 --> 00:06:27.662
is 1 minus rho times
rho at the power i,
00:06:27.662 --> 00:06:31.470
for i equal-- this
pi i can be seen
00:06:31.470 --> 00:06:34.080
as coming from the
probability distribution.
00:06:34.080 --> 00:06:37.640
They tell us that if we observe
that chain at time-- let's
00:06:37.640 --> 00:06:40.710
say one billion--
and ask-- where
00:06:40.710 --> 00:06:43.580
is the state of
the Markov chain?
00:06:43.580 --> 00:06:47.490
The answer will be the
chain is in state zero, that
00:06:47.490 --> 00:06:51.659
is, the system is empty with
a probability 1 minus rho,
00:06:51.659 --> 00:06:54.040
or there is one
customer in the system.
00:06:54.040 --> 00:06:57.240
And that happens with
probability 1 minus rho times
00:06:57.240 --> 00:06:58.640
rho.
00:06:58.640 --> 00:07:00.420
And so on.
00:07:00.420 --> 00:07:04.330
So the distribution
can be drawn like that.
00:07:04.330 --> 00:07:11.012
You have here i corresponding to
a state and if you put pi of i
00:07:11.012 --> 00:07:19.240
here, 0 here, then 1, 2, 3--
then pi of 0 is 1 minus rho
00:07:19.240 --> 00:07:19.740
here.
00:07:23.090 --> 00:07:29.590
pi of 1 will be rho times
1 minus rho and pi of 2
00:07:29.590 --> 00:07:30.270
and so forth.
00:07:33.550 --> 00:07:35.500
So if you look at this
distribution here,
00:07:35.500 --> 00:07:37.530
it's pretty much a
geometric distribution,
00:07:37.530 --> 00:07:41.640
except that it has shifted so
that it starts at 0 instead
00:07:41.640 --> 00:07:43.180
of starting at 1.
00:07:43.180 --> 00:07:45.220
So it's a shifted geometric.
00:07:48.830 --> 00:07:51.320
This model is the first
and simplest model
00:07:51.320 --> 00:07:54.960
that one encounters when
studying queuing theory.
00:07:54.960 --> 00:07:57.740
So a final note--
the PMF that we
00:07:57.740 --> 00:08:00.750
have here has an expected value.
00:08:00.750 --> 00:08:06.090
And the expectation is
given here-- e of x of m
00:08:06.090 --> 00:08:12.950
is-- let me rewrite it here--
it's rho over 1 minus rho.
00:08:12.950 --> 00:08:16.540
And this formula-- which
is interesting to anyone
00:08:16.540 --> 00:08:18.820
who tries to analyze a
system of this kind--
00:08:18.820 --> 00:08:25.160
tells you the following-- that
as long as rho is less than 1,
00:08:25.160 --> 00:08:28.340
then the expected number
of customers in the system
00:08:28.340 --> 00:08:29.890
is finite.
00:08:29.890 --> 00:08:34.169
But if rho, this little rho,
becomes very close to 1,
00:08:34.169 --> 00:08:36.760
then you're going to have
1 over something that
00:08:36.760 --> 00:08:38.400
is very close to 0.
00:08:38.400 --> 00:08:41.299
And that number will
be very, very big.
00:08:41.299 --> 00:08:44.460
So when rho becomes
very close to 1,
00:08:44.460 --> 00:08:47.850
that means the load factor
is something like-- let's say
00:08:47.850 --> 00:08:52.100
0.99-- you expect to have a
very large number of customers
00:08:52.100 --> 00:08:55.300
in the system at any given time.