WEBVTT

00:00:01.090 --> 00:00:04.510
In this lecture, we introduce
Markov chains, a general class

00:00:04.510 --> 00:00:06.910
of random processes
with many applications

00:00:06.910 --> 00:00:09.890
dealing with the evolution
of dynamical systems.

00:00:09.890 --> 00:00:12.730
They have been used in
physics, chemistry, information

00:00:12.730 --> 00:00:15.900
sciences, queuing theory,
internet applications,

00:00:15.900 --> 00:00:20.910
statistics, finance, games,
music, genetics, baseball,

00:00:20.910 --> 00:00:22.730
history, you name it.

00:00:22.730 --> 00:00:26.920
So what make these processes
so powerful and practical?

00:00:26.920 --> 00:00:29.660
Well, as opposed to the
Bernoulli and Poisson

00:00:29.660 --> 00:00:32.530
processes, which are
memoryless in the sense

00:00:32.530 --> 00:00:36.040
that the future does
not depend on the past,

00:00:36.040 --> 00:00:39.600
Markov chains are
more elaborate as they

00:00:39.600 --> 00:00:42.380
allow the representation
of situations where

00:00:42.380 --> 00:00:48.340
the future depends on the
past and, to some extent,

00:00:48.340 --> 00:00:53.370
could be predicted
from the past.

00:00:53.370 --> 00:00:56.470
More precisely, we are
going to consider models

00:00:56.470 --> 00:01:00.610
where the influence of
the past on the future

00:01:00.610 --> 00:01:06.110
is summarized by the notion
of a state, which evolves

00:01:06.110 --> 00:01:09.680
over time according to some
probability distribution.

00:01:09.680 --> 00:01:14.020
That's the link between
the past and the future.

00:01:14.020 --> 00:01:17.410
We will restrict ourselves
to discrete time Markov

00:01:17.410 --> 00:01:20.270
chains in which
the state changes

00:01:20.270 --> 00:01:23.050
at certain discrete time steps.

00:01:23.050 --> 00:01:26.660
The state at time t
plus 1, which is here,

00:01:26.660 --> 00:01:29.850
is a function of
the state at time t,

00:01:29.850 --> 00:01:32.900
and there is some
noise, or randomness.

00:01:32.900 --> 00:01:37.060
As another view, this is what
we will cover in this lecture.

00:01:37.060 --> 00:01:39.970
We will first introduce
the basic concepts

00:01:39.970 --> 00:01:44.310
using the example of a checkout
counter at the supermarket.

00:01:44.310 --> 00:01:47.450
We will then abstract
from the example

00:01:47.450 --> 00:01:50.530
and give some
general definitions.

00:01:50.530 --> 00:01:53.430
Afterwards, we will look
at various questions,

00:01:53.430 --> 00:01:56.900
such as predicting what could
happen in the future given

00:01:56.900 --> 00:01:59.650
the current state
of our systems.

00:01:59.650 --> 00:02:02.800
We will end this lecture
by giving some key

00:02:02.800 --> 00:02:06.470
structural properties
of Markov processes.

00:02:06.470 --> 00:02:08.389
So let us start.