The following content is provided by a
Creative Commons license.
Your support will help MIT OpenCourseWare continue to
offer high quality educational resources for free.
To make a donation or to view additional materials from
hundreds of MIT courses, visit MIT OpenCourseWare at
ocw.mit.edu.
PROFESSOR: I just want to be explicit about it because with
all of the abstraction of QAM looking at it in terms of
Hilbert filters and all of that, you tend to forget that
what's really going on in terms of implementation is
something very, very simple.
We said that what we were doing with QAM is, we're
trying to essentially build two PAM systems in parallel.
So what we do is, we have two different data
streams coming in.
Each one there's a new signal, each t seconds. t is the time
separation between times when we transmit something.
So the sequence of symbols come in.
What we form is then -- well, actually this
isn't a real operation.
Because everybody does this in whatever way suits them.
And the only thing that's important is getting from here
over to there.
And what we're doing in doing that is taking each of these
signals, multiplying them by a pulse, to give this waveform
the pulses delayed according to what signal it is.
And if you like to think of that in terms of taking the
signal and viewing it as an impulse, so you have a train
of impulses weighted by these values here.
And then filtering that, fine.
That's a good way to think about it.
But, at any rate you wind up with this.
With the other data stream which is going through here --
yes?
AUDIENCE: [INAUDIBLE]
PROFESSOR: You couldn't argue that was band limited also.
AUDIENCE: [INAUDIBLE]
PROFESSOR: But it's band limited in the same way.
I mean, visualize --
I mean, the Fourier transform of p of t minus k t is simply
the Fourier transform of p of t, which is multiplied by a
sinusoid in f at this point.
So it doesn't change the bandwidth of that.
In other words, when you take a function and you move it in
time, the Fourier transform is going to be rotating in a
different way.
But its frequency components will not change at all.
All of them just change in phase because
of that time change.
AUDIENCE: [INAUDIBLE]
PROFESSOR: When you look at it in frequency, well no, because
we're adding up all these things in
frequency at that point.
But each one of these pulses is still constrained within
the same bandwidth.
AUDIENCE: [INAUDIBLE]
PROFESSOR: The transform of a pulse train does not look like
a pulse train in frequency, no.
When you take a pulse train in time, and you take the
transform of each one of these pulses, what you get is a
pulse which, where in frequency there is a rotation.
In other words, the phase is changing.
But in fact, what you wind up with is a pulse within the
same frequency band.
I mean, simply take the Fourier transform and look at
it, and see which frequencies it has in it.
In they don't change.
And when you weight it with coefficients, the Fourier
transform will be scaled, will be multiplied by a scalar.
But that won't change which frequencies are there, either.
So in fact, when you do this --
I should make that clearer in the notes.
It's a little confusing.
AUDIENCE: [INAUDIBLE]
PROFESSOR: It'll be band limited in the same way,
though, yes.
AUDIENCE: [INAUDIBLE]
PROFESSOR: It won't look the same.
But the Nyquist criterion theorem remains the same.
I mean, if you look at the proof of it, it's -- well I'll
go back and look at it.
I think if you look at it, you see that it actually --
I think you see that this time variation here in fact does
not make any difference.
I mean, I know it doesn't make any difference.
But the question is, how to explain that it doesn't make a
difference.
And I'll go through this.
And I will think about that.
And I'll change the notes if need be.
But anyway, it's not something you should worry about.
Anyway.
What happens here is, you get this pulse train, which is
just a sum of these pulses.
And they all interact with each other.
So at this point we have intersymbol interference.
We multiply this by a cosine wave.
We multiply this by a sine wave.
And what we get then is some output, x of t.
Which is in the frequency band.
But if you take the low pass frequency here, which goes
from 0 up to sum w, up at passband, we go from f c minus
w to f c plus w.
Same thing here.
So we have this function, which is then constrained.
But it's a passband constraint to twice the frequency which
we have here.
When we take x sub t, and we try to go through a
demodulator.
The thing we're going to do is to multiply,
again, by the cosine.
Multiply again by the sine.
Incidentally, this is minus sine and minus sine here.
When you implement it, you can use plus sine or minus sine,
whichever you want to.
So long as you use the same thing here and there.
It doesn't make any difference.
It's just that when you look at things in terms of complex
exponentials, this turns into a minus sine.
Oh, and this turns into a minus sine.
We then filter by q of t.
We filter this by q of t.
The same filter.
And we wind up then going through the T-spaced sampler.
The T-spaced sampler and the output use of k prime.
And the reason for this is that if you look at the system
broken right here, this is simply a PAM waveform.
And when we take this PAM waveform, the only thing we're
doing is we're multiplying it by a cosine.
Then we're multiplying it by a cosine again.
When we get a cosine squared term, half of it comes back
down to baseband.
Half of it goes up to twice the carrier frequency.
And this filter, in principle, is going to filter out that
part at 2 times the carrier frequency.
And it's also going to do the right thing to this filter.
To turn it into something where, when we sample, we
actually get these components back again.
So the only thing that's going on here is simply, this is a
PAM system where, when we multiply by cosine twice, the
thing we're getting is a baseband component again,
which is exactly the same as this quantity here.
Except it's --
why, I when I multiply this by this, don't I have
to divide by --
don't I have to multiply by 2?
I don't know.
But we don't.
And when we get all through the thing we come out with u
sub k prime, and we come out with u sub k double prime.
It is easier to think about this in terms of complex
frequencies.
When we think about it in terms of cosines and sines,
we're doing it that way because we have to implement
it this way.
Because if you implement a Hilbert filter, you're taking
a complex function, filtering it by complex waveform, which
means you've got to build four separate filters.
One to take the real part with the real part of the filter.
The real part with the complex part of the filter.
Also the imaginary part of the waveform with the real part of
the filter, and so forth.
So you don't implement it that way, but that's the easy way
to look at what's happening.
It's the easy way to analyze it.
This just does the same thing.
When we use this double sideband quadrature carrier
implementation of QAM, the filters p and q
really have to be real.
Why do they have to be real?
Well, because otherwise, we're mixing what's going on here
with what's going on here.
In other words, you can't take -- this will not be a real
waveform, so you can't just multiply it by a cosine wave.
In the other implementation we had, you have this, which is a
complex waveform, plus this, which is
another complex waveform.
You multiply it by e to the 2 pi i f c t.
And then you take the real part of it.
And everything comes out in the wash.
Here we need this to be real and this to be real in order
to be able to implement this just by multiplying by cosine
and multiplying by sine.
So the standard QAM signal set makes it just
parallel PAM systems.
If, in fact, you don't use a standard QAM set; namely, if
these two things are mixed up in some kind of circular
signal set or something, then what comes out here has to be
demodulated in the same way.
So you don't have the total separation.
But the real part of the system is the
same in both cases.
I think the notes for a little confusing about that.
Finally, as a practical detail here, we have talked several
times about the fact that every time people do
complicated filtering, and complicated filtering involves
very sharp filters.
Every time you make a very sharp filter at baseband, the
way you do it is, you do it digitally.
Namely, you first take this waveform.
You sample it very, very rapidly at a rapid pace.
And then you take those digital signals and you pass
those through a digital filter.
And that gives you the waveform that you're
interested in.
When you do that, you're faced with having a waveform coming
into the receiver which has a band down at baseband.
Which it has width w.
You also have a band up at twice the carrier frequency.
And if you sample that whole thing, according to all of the
things we've done with aliasing and everything, those
samples mix what's up at twice the carrier frequency with
what's down here at baseband.
And you really have to do some filtering to get rid of once
or twice the carrier frequency.
This really isn't much of a practical issue, because you
can hardly avoid doing that much filtering just in doing
the sampling.
I mean, you can't build the sampler that doesn't filter
the waveform a little bit too.
But you have to be conscious of the fact that you have to
get rid of those double frequency terms before you go
into the digital domain.
So, you have to think of it as a little bit of filtering
followed by a rapid sampler, followed
by a careful filtering.
Which does this function q of t here.
OK.
Want to talk a little bit about the number of degrees of
freedom we have when we do this.
If we look at quadrature amplitude modulation, anytime
you get confused about bandwidths, it's probably
better to look at the sample time, t, which is something
that never changes.
And that always remains the same.
When you look at the sample spacing, the baseband
bandwidth, as we've seen many times, the nominal baseband
bandwidth is 1 over 2t.
Namely, that's what's called the Nyquist bandwidth.
It's what you get if you use sinc functions, which strictly
puts everything in this baseband and
nothing outside of it.
And we've seen that by using the Nyquist criterion and
building sharp filters, we can make the actual bandwidth as
close to this as we want to make it.
When we modulate this up to passband, we're going to have
a passband bandwidth which is 1 over t.
Because, in fact, this baseband function is going
from minus 1 over 2t to plus 1 over 2t.
When we modulate that up, we get something that goes from f
c minus 1 over 2t to f c plus 1 over 2t.
Which is really an overall bandwidth of 1 over t.
If we look at this over a large time interval, t0, we
have two degrees of freedom.
Namely, the real part and the imaginary part.
Namely, we have two different real number signals coming
into this transmitter.
Each t seconds.
So that over a period of t0, some much larger period, we
have t0 over capital T different signals that we're
transmitting.
Each of those signals has two degrees of freedom in it.
So we wind up with 2 t0 times w real degrees of freedom.
Namely, 2 t0 over T. There's 2 t0 times w, where w is the
passband bandwidth.
What I want to do now, to show you that actually nothing is
being lost and nothing is being gained, is to look at a
baseband system where you use pulse amplitude modulation.
But, in fact, you use a humongously wide
bandwidth to do this.
So it becomes something like these ultra-wideband systems
that people are talking about now.
You use this very, very wide bandwidth, and you see how
many signals you can pack into this very wide
bandwidth using PAM.
And what's the answer?
There are 2 t0 w 0 real degrees of freedom.
And a baseband bandwidth w 0.
This is what we determine when we were dealing with PAM.
It simply corresponded to the fact that the baseband
bandwidth was 1 over 2 t 0.
Therefore we got two signals coming in, each 1 over w 0.
So this was the number of real degrees of freedom we have.
Now, the thing we want to do is to take
this humongous bandwidth.
And to say, what happens if instead of using this wide
bandwidth system, we use a lot of little narrow bands.
So we make narrow bands send QAM in the narrow bands, where
in fact we're packing signals into the real part and the
imaginary part, which is what we decided we ought to do.
And how do these two systems compare.
Well, if we take a very wide bandwidth w0, with lots of
little bandwidths, w, up at various passbands, all stacked
on top of each other.
Like here's zero.
Here's w0 which is up at five gigahertz or whatever you want
to make it.
We put lots of little passbands in here,
each of width w.
And the question is how many different frequency
intervals do we get?
How many different signals can we multiplex together here by
using different frequency bands?
Which is exactly the way people who use QAM normally
transmit data.
Well, the answer is, you get w0 over w
different bands here.
So how many degrees of freedom do you have?
Well, we said we had 2 t 0 w real degrees of freedom with
each QAM system.
We have w0 over w different QAM systems all stacked on top
of each other.
So, when you take w equals w0 over m you wind up with 2 t0
w, 2 t0 w 0 real degrees of freedom, overall using QAM.
Exactly the same thing if you use PAM.
This corresponds to these orthonormal series we were
thinking about, where we were thinking in terms of taking
segmenting time, and then using a Fourier series within
each segment of time.
And looking at the number of different
coefficients that we got.
And we then looked at these sinc weighted expansions of
the same type.
We got the same number of degrees of freedom.
This is simply saying that PAM and QAM use every one of those
degrees of freedom.
And there aren't any more degrees of freedom available.
If you're stuck with using some very large time interval
t0 and some very large frequency interval, w0, you're
stuck with that many real degrees of
freedom, and real waveform.
There's nothing else available, and PAM and QAM are
perfectly efficient as far as their
user spectrum is concerned.
They both do the same thing.
How about single sideband?
That does the same thing too.
I mean, with single sideband, you're sending real signals
but you cut off half the bandwidth at passband.
So, again, you get the same number of signals for unit
time and per unit bandwidth.
And most systems, except double sideband quadrature
carrier, which is just obviously wasting bandwidth,
all do the same thing.
So, it's not hard to use all of these
available degrees of freedom.
And the only thing that makes these degrees of freedom
arguments complicated is, there's no way you can find
the pulse, which is perfectly limited to 1 over w in time
and 1 over w in bandwidth.
If you make it limited in time, it
spills out in bandwidth.
If you make it limited in bandwidth, it
spills out in time.
So we always have to fudge a little bit as far as seeing
how these things go together.
Let's talk about one sort of added practical detail that
you always need in any QAM system.
In fact, you always need it in a PAM system, also.
And you need to do it somehow or other.
There's this question of, how do you make sure that the
receiver is working on the same clock as the transmitter
is working on?
How do you make sure that when we talk about a cosine wave at
the receiver, the phase of that cosine wave is, quote,
the same as the phase at the transmit?
So, if the transmitted waveform at the upper passband
-- at positive frequencies is u of t times e to
the 2 pi i f c t.
This is this is one of the reasons why you want to think
in terms of complex notation.
If you try to analyze QAM phase lock using
cosines and sines --
I mean, you're welcome to do it, but it's
just a terrible mess.
If you look at it in terms of complex signals, it's a very
easy problem.
But it's most easy when you look at it in terms of just
what's going on at positive frequencies.
So the thing we're doing here is, when we modulate, we're
taking u of t.
We multiply by e to the 2 pi i f c t.
What happens to any arbitrary phase that we have in the
transmitter?
Well, nothing happens to it.
We just define time in such a way that it's not there.
At the receiver, we're going to be multiplying again by e
to the minus 2 pi i f c t.
There's going to be some phase in there because the
receiver's clock and the transmitter's
clock are not the same.
Now, the thing which is complicated about this is that
at the receiver there's some propagation delay from what
there is at the transmitter.
So what we're really saying here is that when the waveform
u of t gets to the receiver --
in other words, after we demodulate, what do we get?
That's the practical question to ask.
I mean you have an arbitrary sinusoid, complex sinusoid at
the transmitter.
You have an arbitrary complex sinusoid at the receiver.
When you multiply by e to the 2 pi i f c t, and then at the
receiver you demodulate by multiplying e to the minus 2
pi i f c t, there's some added phase in there just because
you're multiplying by some sinusoid in
an arbitrary phase.
What do you wind up with when you get done with that?
You get the received waveform, u of t, times e to the i phi.
Now, suppose you're sending analog data.
What happens here?
Well, with analog data, you're absolutely up a
creek without an oar.
There's nothing you can do.
But with digital data there is something you can do.
And I think this diagram makes it clear, if you can see the
little arrows on it.
What you're transmitting is QAM signals out of a QAM
constellation.
So the signals -- so you're either transmitting this, or
this, or this, or this, and so forth.
Now, at the receiver we haven't analyzed noise yet.
We're going to start doing that later today.
But if you visualize noise being added up at passband and
then coming down to baseband again, we can visualize what's
happening as the noise just adds something to each one of
these points.
I mean, assuming that we know exactly what the timing is for
the time being.
So that we sample at exactly the
right time at the receiver.
And that's a separate issue.
But let's assume that we do know what the timing is.
We do sample at the correct time.
And the only question we're trying to find out now is, how
do we choose this phase right on this sinusoid that we're
using at the demodulator.
Because at the demodulator, there's no way to know whether
phi is zero or whether phi is something else.
But after we got all done doing this, if we sample at
the right time, what we're going to receive, if there's
some small error in phase, is that this diagram is all
rotated around a little bit.
Any time you send this point, what you receive is going to
be a point up here.
If you send this point it's going to be a point here.
If phi is negative, everything will go the other wa.y How do
I know whether it's positive phi or negative phi?
I don't.
I just do the same thing either place.
Everybody I know who worries about phase locked loops
ignores the sign and simply makes sure that they're using
the same sign at the transmitter and the receiver.
If they don't, when they build it, the thing goes out of lock
immediately.
And they try switching the sign, and if it works then,
they say uh-huh, I now have the right sign.
I mean, it's like these factors of two.
You always put them in after you're done.
Anyway, I think I've done it right here.
So the received waveform is now going to be u of t times e
to the i phi.
It will have a little bit of noise added to it.
So what we're getting, then, if you look at a scope, or
whatever people use as scope these days.
And people have done this for so many years that they call
this an eye pattern.
And what they see, after looking at many, many received
signals, is little blobs around each of these points
due to noise.
And they also see the set of points rotated
around a little bit.
And if the noise a small, and if the phase error is small,
you can actually see this diagram rotated around sitting
at some orientation.
And the question is, if you know that's what's happening,
what do you do about it?
Well, if you have this diagram rotated around a little bit,
you want to change the phase of your demodulating carrier.
So the thing you do is, you measure the errors that you're
making when you go from these received signals.
And you make decisions on them.
You then look at the phase of these errors.
If all the phases are positive, then you know that
you have to change the phase of this carrier one way.
And if all the phases are negative, you change
it the other way.
Now, why does this work?
It works because these clocks are very stable clocks.
They wander in phase slowly.
I mean, you can't lock them perfectly.
And since they wander in phase slowly.
And since you're receiving data quickly, you can average
this carrier recovery over many, many data symbols.
Which is what people do.
So, you average it over many symbol.
And you gradually change the phase as the
phase is getting off.
How much trouble does that cause in trying to actually
receive the data?
Hardly any.
Because if the phase changes are slow enough, then you
average over a long enough interval, the actual phase
errors are going to be extraordinarily small.
So it's the noise errors that look big and the phase errors
that look very small.
When you average over a long period of time, the noise is
sort of independent from one period to another.
So the noise averages out.
And the thing you see, then, is the phase errors.
So, so long as the phase is slow and the noise is quick,
you first decode the data.
After you decode the data, you average over what the phase
errors are.
And then you go and you change the clock.
And people spend their lives designing how to make these
phase lock loops.
And there's a lot of interesting technology there.
But the point is, the idea is almost trivially simple.
And this is what it is.
Since the phase is changing slowly, you can change the
phase slowly.
You can keep the phase error very, very small.
And therefore, the data recovery due to the noise
works almost as well as if you had no phase error at all.
How do you recover from frequency errors?
Well, if I can recover from phase errors, I'm recovering
from frequency errors also.
Because I had this clock which is running around.
And I'm slaving the clock at the receiver to the clock at
the transmitter.
So, so long as I have very little frequency error to
start with, simply by keeping this phase right, I can also
keep the carrier frequency right.
Because the phase is just going around at the speed it's
supposed to be going around.
So this will also serve as a frequency lock as well as a
phase lock.
When you're trying to do both together, you build the phase
lock loop a little different, obviously.
What's the problem with this?
You look this diagram, and you say, OK, suppose there's some
huge error where for a long period of time, things get
very noisy.
What's going to happen?
Well, if you can't decode the data for a long period of
time, suddenly these phase errors build up.
The channel stops being noisy.
And what could have happened is that your phase could be
off by pi over 2.
Which means what you see, what you should be seeing is this.
What you're actually seeing is this.
You can't tell the difference.
In other words, you decode every symbol wrong.
Because you're making an error of pi over 2 in
every one of them.
You decode this as this, you decode this as this, this as
this, and so forth.
So every point is wrong.
What will you do to recover from this?
I mean, suppose you were designing a system where in
fact you had no feedback to the transmitter at all.
You have feedback from the transmitter, you just,
obviously every once in a while you stop and you
resynchronize.
You don't have any feedback, what do you do?
Yeah?
AUDIENCE: [INAUDIBLE]
PROFESSOR: Differential phase shift key, yes.
I mean, this isn't quite phase shift key, but
it's the same idea.
In other words, the thing that you do is, you don't --
usually when the data is coming in, you don't map it
directly into a signal point.
The thing that you do is map the data into a phase change
from one period of time to the next.
I mean, you map it into an amplitude.
And then you map it into a change of phase
in this set of points.
Or change of phase within this set of points.
Or a change of phase within the outer set of points.
And then if, due to some disaster, you get off by pi
over 2, it won't hurt you.
Because the changes are still the same, except for the
errors that you make when this process starts.
So in fact, a lot of these things that people have been
doing for years, you would think of almost immediately.
The only thing you have to be careful about, when you invent
new things to build new systems, is you have to have
somebody to check whether somebody else
has a patent on it.
So, you invent something and then you check whether
somebody else patented it.
So that's what this slide says.
Bingo.
It's time to go on to talk about additive noise and
random processes.
How many of you have seen random processes before?
How many of you have seen it in a graduate context as
opposed to an undergraduate context?
A smaller number.
Many of you, if you got your degrees outside of the US, if
you've studied this as an undergraduate, you probably
know what students here have learned in their various
graduate courses.
So you might know a lot more about it.
What we're going to do here is I'll try to teach you just
enough about random processes that you can figure out what's
going on with the kinds of noise that we're
looking at this term.
Which involves two kinds of noise.
One, something called additive noise, which is what we're
going to be dealing with now.
And the other is much later, when we start studying
wireless systems, we have to deal with a
different kind of noise.
People sometimes call it multiplicative noise, but
really what it is, is these channels fade in various ways.
And the fading is a random phenomena.
And therefore we have to learn how to deal with that in some
kind of probabilistic way also.
So the thing we're going to do is we're going to visualize
what gets received as what gets transmitted plus noise.
Now, at this point, I've said OK, we know how to deal with
compensating for phase errors.
So we'll assume that away, it's our
usual layered approach.
What kind of other assumptions are we making here?
When I say the received signal is the transmitted
signal plus the noise.
Well, I mean, I could look at this and say, this doesn't
tell me anything.
This is simply defining the noise.
There's the difference between what I
receive and what I transmit.
So when I say this is additive noise, I'm not saying anything
other than the definition of the noise, is this difference.
But when we look at this, we're really going to be
interested in trying to analyze what this process is.
And we're going to be interested in analyzing what
this is as a process also.
Here, I'm just looking at sample functions.
And what I'd like to be able to do is say this sample
function, namely, what's going into the channel, is one of a
large set of possibilities.
If it weren't one of a large set of possibilities, there
would be no sense at all to transmitting it, because the
receiver would know what it was going to be.
And you wouldn't have to transmit it, it would just
print it out at the output.
So, this is some kind of random process which is
determined by the binary data which is coming into the
encoder and the signals it gets mapped
into, and so forth.
This reviewing of some kind of noise waveform.
What we're going to assume is that in fact these two random
phenomena are independent of each other.
And when I write this, you would believe that if nobody
told you to think about it.
Because it just looks like that's what it's saying.
Here's some noise phenomena that's added to the signal
phenomenon.
If in fact I told you, well, this noise here happens to be
twice the input then you'd say, foul.
What I'm really doing is calling something noise which
is really just a scaling of the input.
Which is really a more trivial problem.
So we don't want to do that.
Another thing I could try to do, and people tried to do for
a long time in the communication field, was to
say, well, studying random processes is too complicated.
Let me just say there's a range of possibilities,
possible things for what I transmit.
There's a range of possible waveforms for the noise.
And I want to make a system that works for both these
ranges of different things.
And it just turns out that that doesn't work.
It works pretty well for the transmitter.
It doesn't work at all for the noise.
Because sometimes the noise is what it should be.
And sometimes the noise is abnormally large.
And when you're trying to decide what kind of coding
devices you want to use, what kind of modulation you want to
use, and all these other questions, you really need to
know something about the probabilistic structure here.
So we're going to view this as a sample
function of a random process.
And that means we're going to have to say what a random
process is.
And so far, the only thing we're going to say is, when we
talk about it as a random process, we'll call it capital
N of t instead of little n of t.
And the idea there is that for every time, for every instant
of time, when you talking about random processes, you
usually call instants of time epochs.
And you call them epochs because you're using time for
too many different things.
And calling it an epoch helps you keep straight what you're
talking about.
So for each epoch t, n of t is a random variable.
And this little n of t is just some sample value of that
random variable.
So we're going to assume that we have some probabilistic
description of this very large
collection of random variables.
And the thing which makes this a little bit tricky,
mathematically is that we have an uncountably infinite number
of random of variables to worry about here.
x sub t is known at the transmitter but
unknown at the receiver.
And what we're eventually concerned about is, how does
the receiver figure out what was transmitted.
So we'd better view x of t as a sample
function of a random process.
x of t, from the receiver's viewpoint.
Namely, the transmitter knows what it is.
This is this continual problem we run into with probability.
When you define a probability model for something, it always
depends on whose viewpoint you're taking.
I mean you could view my voice waveform as a random process.
And in a sense it is.
As far as I'm concerned it's almost a random process,
because I don't always know what I'm saying.
But as far as you're concerned, it's much more of a
random process than it is from my viewpoint.
If you understand the English language, which all of you do,
fortunately, it's one kind of random process.
If you don't understand that, it's just some noise waveform
where you need a much more complicated model
to deal with it.
So everything we have to do here is based on these models
of these random processes, which are based on reasonable
assumptions of what we're trying to do with the model.
When we do this in terms of random processes, we have the
input random process, and we have the noise random process.
And if we add up two random processes, it doesn't take
much imagination to figure out that what we get is another
random process.
It's random both because of this and because of this.
Now, we implicitly are assuming a whole bunch of
things when we write things out this way.
We're not explicitly assuming them, but what we want to do
is explain the fact that we're implicitly assuming them and
then make it explicit.
One of those things that we're doing here is, we're avoiding
all attenuation in what's been transmitted.
We've been doing that all along.
We've been talking about received waveforms as being
equal to the transmitted waveform.
Which means that we're assuming that the receiver
knows what the attenuation is.
And if it knows what the attenuation is, there's no
point in putting in this scale factor everywhere.
So we might as well look at what's being transmitted as
being the same as what's being received, subject to this
attenuation, which is just some known factor which we're
not going to consider.
There's also this question of delay.
And we're assuming that the delay is known perfectly.
So that assumption is built into here.
y of t is x of t without delay, plus z of t.
And now, after studying these modulation things, we can add
a few more things which are implicitly put into here.
One of them is that we know what the received phase is.
So that in fact we don't have any phase
error on all of this.
So we're saying, let's get rid of that also.
So we have no timing error, no phase error.
All of these errors are disappearing because what we
want to focus on is just the errors that come from this
additive noise.
Whatever the additive noise is.
But we want to assume that this additive noise is
independent of this process.
So we're saying, when we know all of these things about --
or when the receiver knows all of these things about what's
being transmitted, except for the actual noise waveform, how
do we deal with the problem?
We implicitly mean that z of t is independent of x of t,
because otherwise it would be just very misleading to define
n z of t as the difference between what you receive and
what you transmit.
So we're going to make that assumption explicit here.
These are standing assumptions until we start to study
wireless systems.
And another standing assumption which we'll make is
the kind of assumption we've been making in
this course all along.
Which some people like, usually theoreticians.
And some people often don't like, namely practitioners.
But which if both the theoreticians and
practitioners understood better what the game was, I
think they would agree to doing things this way.
And what we're going to do here is, we're not going to
think of what happens when people make measurements of
this noise and then try to create a stochastic model of
it from all of these measurements.
What we're going to do is more in the light of what Shannon
did then when he invented information theory.
Which is to say, let's try to understand the very simplest
situations first.
If we can understand those very simplest situations
namely, we will invent whatever
noise we want to invent.
To start to understand what noise does to communication.
And we will look at the simplest processes we can.
Which seem to make any sense at all of us.
And then after we understand a few of those, then we will
say, OK, what happens if there's some particular
phenomenon going on in this channel.
And when you look at what practitioners do, what in fact
they do is, everything they do is based on
various rules of thumb.
Where do these rules of thumb come from?
They come from the theoretical papers that were written 50
years before, if they're very out of date practitioners.
25 years before if they're relatively up to date
practitioners.
Or one year before if they're really reading all the
literature as it comes out.
But rules of thumb usually aren't based on what's
happening as far as papers are concerned now.
It's usually based on some composite of what people have
understood over a large number of years.
So in fact, the way you understand the subject is to
create these toy models.
And the toy models are what we're going
to be creating now.
Understand the toy models, and then move from the toy models
to trying to understand other things.
This is the process we'll be going through when we look at
this additive noise channel, which doesn't make much sense
for wireless channels, as it turns out.
But it's part of what you deal with on a wireless channel.
And then, on a wireless channel, we'll say, well, are
these other effects concerned also?
And we'll try to deal with these other effects as an
addition to the effects that we know how to deal width.
We've already done that as far as phase noise is concerned.
Namely, we've already said that since phase effects occur
slowly, we can in fact get rid of them almost entirely.
And therefore we don't have to worry about them here.
So, in fact we have done what we've said that we believe
ought to be done.
Namely, since we now know how to get rid of the phase noise,
we're now looking at additive noise and saying, how can we
get rid of that.
Actually, if you look at the argument we went through with
phase noise, we were really assuming that we knew how to
deal with channel noise as part of that.
And in fact when you get done knowing how to deal with
channel noise, you can go back to that
phase locked loop argument.
And you'll see a little more clearly, or a little more
precisely, what's going on there.
When we started talking about waveforms, if you remember
back to when we were talking about compression, we really
faked this issue of random processes.
If you remember the thing we did, everything was random
while we were talking about taking discrete sources and
coding for them.
When we then looked at how do you do quantization on
sequences of real numbers or sequences of complex numbers,
we again took a probabilistic model.
And we determined how to do quantization in terms of that
probabilistic model.
And in terms of that probabilistic model, we
figured out what ought to be done.
And then, if you remember, when we went to the problem of
going from waveforms to sequences, and our pattern
was, take a waveform.
Turn it into a sequence.
Quantize the sequence, then encode the digital sequence.
We cheated.
Because we didn't talk at all about random processes there.
And the thing we did instead is, we simply converted source
waveforms to sequences and said that only the
probabilistic description of the sequence is relevant.
And, you know, that's actually true.
Because so long as you decide that what you're going to do
is go from waveforms to sequences, and in fact you
have decided on which particular orthonormal
expansion you're going to use, then everything from there on
is determined just by what you know about the probabilistic
description of that sequence.
And everything else is irrelevant.
I mean, each waveform maps into a sequence.
When you're going back, sequences map into waveforms,
at least in this L2 sense, which is what determines
energy difference.
And energy differences usually are criterion for whether
we've done a good job or not.
So, in fact, we could avoid the problem that way simply by
looking at the sequences.
And it makes some sense to do that because we're looking at
mean square error.
And that was why we looked at mean square error.
But now we can't do that any more.
Because now, in fact, the noise is occurring on the
waveform itself.
So we can't just say, let's turn this into a sequence and
see what the noise on the waveform does to the sequence.
Because, in fact, that won't work at this point.
So random process z of t is a collection of random
variables, one for each t and r.
So, in other words, this is really a collection of an
uncountably infinite number of random variables.
And for each epoch t, the random
variable z of t is a function.
Capital Z of t and omega.
Omega is the sample point we have in this sample space.
Where does the sample space come from?
We're imagining it.
We don't know any way to construct a
sample space very well.
But we know if we study probability theory that we
need a sample space to talk about probabilities.
So we're going to imagine different sample spaces and
see what the consequences of those are.
I mean, that's the point of this general point of view
that we have.
That we start out looking at simple models and go from
there to complex models.
Part of the model is the sample space that we have.
So for each sample point that we're dealing with, the
collection --
I should have written this differently.
Let me change this a little bit to
make it a little clearer.
z of t omega for t and r is a sample function
z of t; t and r.
So if I look at this, as a function of two variables.
One is time and one is this random collection of
things we can't see.
If I hold omega fixed, this thing becomes
a function of time.
And in fact it's a sample function, it's the sample
function corresponding to that particular element in the
sample space.
And don't try to ask what the sample space is.
The sample space --
I mean, the sample point here --
really specifies what this sample function is.
It specifies what the input sample function is.
It specifies what everything else you're interested in is.
So it's just an abstract entity to say, if we're
constructing probabilities we need something to
construct them on.
So that's the sample space that we can construct these
probabilities on.
The random processes --
Oh.
OK.
We want to look at two different things here.
Here we're looking at a fixed sample point, which means that
this process for a fixed sample point corresponds to a
sample function.
If instead I look a fixed time, a fixed epoch, t and r,
and I say what does this correspond to as I look at z
of t and omega over all omega in this big
messy thing, fix t.
What do I have there?
I have a random variable.
If I look at a fixed t, and I'm looking over the whole
sample space, that's what you mean by a random variable.
If I look at -- if I hold omega fixed and look over the
time axis, what I have is a sample function.
I hope that makes it a little clearer what the game we're
playing is.
Because this random thing, the random process that we're
dealing with, we can best visualize it as a function of
two things, both time and sample point.
The sample point in this big sample space is what controls
everything.
It tells you what every random variable in the world is.
And what the relationship of every random variable
in the world is.
So, a random process is defined, then, by some rule
which establishes a joint density for all finite values
of k, all particular sets of time, and all particular
arguments here.
What am I trying to do here?
I'm trying to admit right from the outset that if I try to
define this process for an uncountably infinite number of
times, I'm going to be dead in the water.
So I had better be happy with the idea of, at least in
principle I would like to be able to find out what the
joint probability density is for any finite set
of points in time.
And if I can find that, I will say I have
defined this random process.
And I'll see what I can do with it.
So we need rules that let us find this kind
of probability density.
That's the whole objective here.
Because I don't know how to generate a random process that
does more than that, that talks about
everything all at once.
You can read mathematics texts to try to do this, but I don't
know any way of getting from that kind of general
mathematics down to anything that we could make sense about
in terms of models of reality.
The favorite way we're going to generate this kind of rule
is to think of the random process as being a sum of
random variables multiplied by orthonormal functions.
In other words, we will have sample values in time, z of t,
which are equal to z sub i phi sub i of t.
And for each sample function in time, we will then have,
for that particular time, as I vary over capital Omega, I'm
going to be varying over the values of z sub i.
These are just fixed functions.
And what that's going to give me is a set of random
variables here.
So in a way we're doing very much the same thing as we did
with functions.
With functions we said, when we're dealing with these the
equivalence classes, things get very, very complicated.
The way we're going to visualize a function is in
terms of an orthonormal expansion.
And with orthonormal expansions, if two functions
are equivalent, they both have the
same orthonormal expension.
So now we're saying the same thing when we're dealing with
this more general thing, which is stochastic processes.
This is really what the signal space viewpoint of
communication is.
It's the viewpoint that you view these random processes in
terms of orthonormal expansions where the
coefficients are random variables.
And if I deal with a finite number of random variables
here, everything is very simple.
Because you know how to add random variables, right?
I mean, at least in principle, you all know how
to add random variables.
You've been doing that any time you take
a probability class.
A quarter of the exercises you deal with are adding random
variables together and finding various things about the sum
of those random variables.
So that's the same thing we're going to be doing here.
The only thing is, we can do this for any real value of t,
just by adding up this fixed
collection of random variables.
And you add it up at a different arguments here
because as t changes, phi sub i of t changes.
And this is sort of what we did when we were talking about
source waveforms.
It's very much the same idea that we were looking at.
Because with source waveforms, when we turned the waveform
into a sequence we were turning the waveform into a
sequence by in fact using an orthonormal expansion.
And then what was happening is that the coefficients in the
orthonormal expansion, we were using to actually represent
the random process.
So we're doing the same thing here.
The same argument.
So, somehow we have to figure out what these
joint densities are.
So, as I said before, when we don't know what to do, we make
a simple assumption.
And the simple assumption we're going to make is that
these random variables at different points in time, of
noise, we're going to assume that they're Gaussian.
And a normal Gaussian random variable; this, I hope, is
familiar to you, has the density 1 over the square root
of 2 pi, e to the minus n squared over 2.
So it's just density that has this nice bell-shaped curve.
It's an extraordinarily smooth thing.
It has a mean of 0.
It has a variance of 1.
How many of you could prove that the
variance of this is 1?
Well, in a while you'll all be able to prove it.
Because there are easy ways of doing it.
Other than looking at a table of integrals, which we'll do.
An arbitrary Gaussian random variable is just where you
take this normal variable.
These normalized Gaussian random variables are also
traditionally called normal variables.
I mean, they have such a central position in all of
probability.
And when somebody talks about a normal random variable --
I mean, there's sort of the sense that random variable
should be normal.
That's the typical thing.
It's what should happen.
And any time you don't have normal random variables,
you're dealing with something bizarre and strange.
And that's carrying it a little too far, but in fact,
when we're looking at noise, that's not a bad idea.
If you shift this random variable by a mean z bar, and
you scale it by -- excuse me, scale it by sigma.
Then the density of the result becomes 1 over the square root
of 2 pi sigma squared, e to the minus z minus z bar over 2
sigma squared.
In other words, the shifting just shifts the
whole thing by z bar.
So this density peaks up around z bar, rather than
peaking up around 0, which makes sense.
And if you scale it by sigma, then you need a
sigma in here, too.
When you have this probability density, you're going to be
scaling it this way.
Scaling it that way is what we mean by scaling it.
If you scale it this way to make sigma squared larger, the
variance larger, this thing still has to integrate to 1.
So you have to pop it down a little bit also.
Which is why this sigma squared appears here in the
denominator.
So anytime we scale a random variable, it's got to be
scaled this way.
And it's also got to be scaled down.
So that's why these Gaussian random
variables look this way.
We're going to be dealing with these Gaussian random
variables so much that we'll refer to this random variable
as being normal with mean z bar and sigma
variance sigma squared.
And if we don't say anything other than it's normal, we
mean it's normal with mean 0 and variance 1.
Which is the nicest case of all.
It's this nice bell-shaped curve which everything should
correspond to.
They tend to be good models of things
for a number of reasons.
One of them is the central limit theorem.
And the central limit theorem, I'm not going to state it
precisely here.
I'm certainly not going to prove it.
One of the nastiest things to prove in probability theory is
the central limit theorem.
It's an absolute bear.
I mean, it seems so simple.
And you try to prove it, and it's ugly.
But, anyway, the central limit theorem says that if you add
up a large number of Gaussian random variables, of
independent Gaussian random variables, and you scale the
sum down to make it variance one, you
have to scale it down.
Then when you get all done adding them all up, what you
wind up with is a Gaussian random variable.
This is saying something a little more precise than the
law of large numbers.
The law of large numbers says you add up a bunch of
independent or almost independent random variables.
And you scale them down.
And you get a mean, which is something.
This is saying something more.
It's saying you add them all up, and you scale them down in
the right way.
And you get not only the mean, now, but you also get the
shape of the density around the mean.
Which is this Gaussian shape, if you add a lot of them.
It has a bunch of extremal properties.
In a sense, it's the most random
of all random variables.
Not going to talk about that, but every once in a while, one
of these things will come up.
When you start doing an optimization in probability
theory, over all possible random variables of a given
mean and variance, you often find that the answer that you
come up with either when you maximize or when you minimize
is Gaussian.
If you get this answer when you maximize, then what you
get when you minimize is usually a random variable,
which is zero or something.
And vice versa.
So that either the minimum or the maximum of a
problem makes sense.
And the one that makes sense, the
answer is usually Gaussian.
At least, it's Gaussian for the problems
for which it's Gaussian.
But it's Gaussian for an awful lot of problems.
It's very easy to manipulate analytically.
Now, you look at that density, and you say, my God, that
can't be easy to manipulate analytically.
But in fact, after you get used to two or three very
small tricks, in fact you'll find out that it's very easy
to manipulate analytically.
It's one of the easiest random variables to work with.
Far easier than a binary random variable.
If we add up a bunch of binary random variables,
and what do you get?
You get a binomial random variable.
And binomial random variables, when you add up enough of
them, they get uglier and uglier.
Except they start to look Gaussian.
These Gaussian random variables, if you add up a
bunch of them, and what do you get?
You get a Gaussian random variable.
Yes?
AUDIENCE: Central limit theorem applies to random
variables or Gaussian random variables?
PROFESSOR: Central limit theorem
applies to random variables.
AUDIENCE: [INAUDIBLE]
PROFESSOR: Oh, I'm sorry.
I shouldn't have.
No.
I said, when you add a bunch of Gaussian random variables,
you get exactly a Gaussian random --
when you add up a bunch of jointly Gaussian random
variables, and we'll find out what that means in a while,
you get exactly a Gaussian random variable.
When you add up a bunch of random variables which are
independent, or which are close enough to independent,
and which have several restrictions on them and you
scale them the right way.
What you get as you add more and more of them, tends toward
a Gaussian distribution.
And I don't want to state all of those conditions.
I mean, we don't need all those conditions here.
Because the only thing we're relying on is when we look at
noise, we're looking at a collection of a very large
number of phenomena.
And because we're looking at a very large number of
phenomena, we can sort of model it as being an addition
of a lot of little random variables.
And because of that, in an enormous number of situations,
when you look at these things, what you get is fairly close
to a Gaussian shape.
And it's a model that everybody uses for noise.
If you read papers in the communication field, or in the
control field, or in a bunch of other fields, and people
start talking about what happens when noise enters the
picture, sometimes they will talk about their model for the
noise process.
Sometimes they won't.
But if they don't talk about what noise model they're
using, it's a very safe bet that what they're using is a
Gaussian process model.
In other words, this is the kind of model people use
because it's a model that makes a
certain amount of sense.
Now, there's one other thing you ought to be aware of here.
It's something that we've talked about a little bit in
other contexts.
When we're looking at a noise waveform that you have to deal
with in a modem or something like that, this modem only
experiences one noise waveform in its life.
So, what do we mean by saying that it's random?
Well, it's an act of faith, in a sense.
But it's not so much.
Because if you're designing communication equipment, and
you're cookie cuttering cellphones or something like
that, or something else, what you're doing is designing a
piece of equipment which is supposed to work in an
enormous number of different places.
So that each one you're manufacturing is experiencing
a different waveform.
So that as far as the designing the device is
concerned, what you're interested in is not that one
waveform that one of these things experiences, but in
fact the ensemble of waveforms that all of them experience.
Now, when you look at the ensemble of waveforms that all
of these things experience, over all of this large variety
of different contexts; noise here, noise there, noise
there, suddenly the model of a large collection of very small
things makes even better sense.
Because in each context, in each place, you have some
forms of noise.
When you look at this over all the different contexts that
you're looking at, then suddenly you have many, many
more things that you, in a sense, are just adding up if
you want to make a model here.
So we're going to use this model for a while.
We will find out that it leads us to lots of nice things.
And so forth.
So, we said that we'd be happy if we could deal with at least
studying a random process over some finite set of times.
So it makes sense to look at a finite
set of random variables.
So we're going to look at a k-tuple of random variables.
And we're going to call that -- what do you think we're
going to call a k-tuple of random variables?
We're going to call it a random vector.
In fact, when you look at random vectors, you can
describe them as being elements of a vector space.
And we're not going to do that because most of you are
already sufficiently confused about doing functions as
vectors, and adding on the idea of a random process as a
vector would be very unkind.
And we're not going to do it because we
don't need to do it.
We're simply calling these things vectors because we want
to use the notation of vector space, which will be
convenient here.
So, if we have a collection of random variables n 1 to n k,
and they're all IID, independent identically
distributed.
They're all normal with mean zero and variance one, the
joint density of them is what?
Well, you know what the joint density is.
If they're independent and they're all the same, you just
multiply their densities together.
So it's 1 over 2 pi with a k over 2.
e to the minus n 1 squared minus n 2
squared, and so forth.
Divided by 2.
That's why this density form is a
particularly convenient thing.
You have something in the exponent.
When you have independent random variables,
you multiply densities.
Which means you add what's in the exponent.
Which means everything is very nice.
So you wind up adding up all these sample values for each
of the random variables.
With our notation looking at norm squared, this is just the
norm squared for this sequence of sample values.
And look at the sequence of samples values is a actual
vector, n 1 up to n sub k.
So we wind up width the norm of the vector n squared
divided by 2.
If you draw a picture of that, you have spherical symmetry.
Every set of sample vectors which have the same energy in
them has the same probability density.
If you draw it in two dimensions, you get --
it looks like a target where the biggest probability
density is in here, and the probability density goes down
as you move out.
If you look at it in three dimensions, if that same
thing's looked at in a sphere.
And if you look at it in four dimensions, well, it's the
same thing, but I can't draw it, I can't imagine it.
But it's still a spherical symmetry.
In the one minute that I have remaining, I'm going to --
I might derive a Gaussian density.
I'm going to call a bunch of random variables zero-mean
jointly Gaussian if they're all derived by linear
combinations on some set of normal IID
Gaussian in random variables.
In other words, if each of them is a linear combination
of these IID random variables, I will call the collection z 1
up to z k a jointly Gaussian random variable.
In other words, jointly Gaussian does not mean that z
1 is Gaussian, z 2 is Gaussian, up
to z k being Gaussian.
It means much more than that.
It means that all of them are linear combinations of some
underlying set of independent random variables.
In the homework that was passed out this time, you will
look at two different ways of generating a pair of random
variables which are each Gaussian, but which are not
jointly Gaussian.
Which you can't view in this way.
So this is important.
Jointly Gaussian makes sense physically because when we
look at random variables, we say it makes sense to look at
Gaussian random variables because of a collection of a
lot of noise variables.
If you look at two different random variables, each of them
is going to be a collection of
different small noise variables.
And they're each going to be some weighted sum over a
different set of noise variables.
So it's reasonable to expect each of them to be expressible
in this way.
I guess I'm not going to derive
what I wanted to derive.
I'll do it next time.
It's easier to think of this in matrix form.
In other words, I'll take the vector of sample values, z 1
up to z sub k.
Each one of them is a linear combination of these normal
random variables, n 1 up to n sub n.
So I can do it as z equal to matrix a times n.
This is a vector of real numbers.
This is a vector of real numbers, this is a matrix.
And if I make this a square matrix, than what we're doing
is mapping each of the unit vectors, each single noise
vector, into the i'th column of a.
In other words, if I have a noise vector where n 1 is
equal to 0, n 2 is equal to 0, n sub i is equal to 1 and
everything else is equal to 0, what that generates in terms
of this random vector z is just the vector a sub 1.
So.
What I'm going to do when I come back to this next time
is, if I take a little cube in the n space, and I map it into
what goes on in the z space, this unit vector here is going
to map into a vector here.
The unit vector here is going to map into a vector here.
So cubes map into parallelograms.
Next time, the thing we will say is, if you look at the
determinant of a matrix, the determinant of a matrix, the
most fundamental definition of it, is that it's the volume of
a parallelepiped.
In fact, it's the volume of this parallelepiped.
So we'll do that next time.