1 00:00:00,500 --> 00:00:02,270 In this final lecture, we will first 2 00:00:02,270 --> 00:00:05,700 review the various properties of a nice Markov chain, which 3 00:00:05,700 --> 00:00:09,550 ensures steady state behavior, and go over some of the notions 4 00:00:09,550 --> 00:00:12,090 in more detail with some examples. 5 00:00:12,090 --> 00:00:15,540 Providing some insights on how good an approximation 6 00:00:15,540 --> 00:00:18,750 we have when we use steady state probabilities 7 00:00:18,750 --> 00:00:21,880 to characterize the behavior of a Markov chain, which 8 00:00:21,880 --> 00:00:26,520 has run for a long time, but not an infinite amount of time. 9 00:00:26,520 --> 00:00:29,170 We will then consider a classical application 10 00:00:29,170 --> 00:00:32,420 of Markov chains, which has to do with the design of a phone 11 00:00:32,420 --> 00:00:33,710 system. 12 00:00:33,710 --> 00:00:38,130 This is a famous problem, which was posed, analyzed, and solved 13 00:00:38,130 --> 00:00:41,290 by a Danish engineer by the name Erlang. 14 00:00:41,290 --> 00:00:44,090 It was more than 100 years ago when phones just 15 00:00:44,090 --> 00:00:46,300 started to exist, but we will see 16 00:00:46,300 --> 00:00:49,580 that this methodology remains relevant to design 17 00:00:49,580 --> 00:00:52,220 similar systems in today's world. 18 00:00:52,220 --> 00:00:54,150 We will then make use of all what 19 00:00:54,150 --> 00:00:56,620 we have learned so far in order to calculate 20 00:00:56,620 --> 00:00:59,140 some interesting short term behaviors of Markov 21 00:00:59,140 --> 00:01:03,130 chains having more than one recurrent classes. 22 00:01:03,130 --> 00:01:06,430 We will introduce the notion of absorbing states, 23 00:01:06,430 --> 00:01:07,990 and we will show how to calculate 24 00:01:07,990 --> 00:01:10,950 the probability of ending up in such a state, 25 00:01:10,950 --> 00:01:14,690 as well as related quantities such as the expected time it 26 00:01:14,690 --> 00:01:16,600 takes to do so. 27 00:01:16,600 --> 00:01:19,010 As a classical example, we will look 28 00:01:19,010 --> 00:01:21,310 at the gambler continuously playing 29 00:01:21,310 --> 00:01:23,855 a simple game of chance, say a lottery, 30 00:01:23,855 --> 00:01:27,730 until he either accumulates a given amount of money 31 00:01:27,730 --> 00:01:29,870 or loses all his money. 32 00:01:29,870 --> 00:01:32,280 Both of these states are absorbing. 33 00:01:32,280 --> 00:01:35,120 What are their corresponding probabilities? 34 00:01:35,120 --> 00:01:39,090 After this lecture, you will be able to answer such questions.