1
00:00:07 --> 00:00:07.93
OK.
2
00:00:07.93 --> 00:00:13
So, coming nearer the end of
the course, this lecture will be
3
00:00:13 --> 00:00:18
a mixture of the linear algebra
that comes with a change of
4
00:00:18 --> 00:00:19
basis.
5
00:00:19 --> 00:00:24.37
And a change of basis from one
basis to another basis is
6
00:00:24.37 --> 00:00:27
something you really do in
applications.
7
00:00:27 --> 00:00:33
And, I would like to talk about
those applications.
8
00:00:33 --> 00:00:37
I got a little bit involved
with compression.
9
00:00:37 --> 00:00:40
Compressing a signal,
compressing an image.
10
00:00:40 --> 00:00:42
And that's exactly
change-of-basis.
11
00:00:42 --> 00:00:46
And then, the main theme in
this chapter is th- the
12
00:00:46 --> 00:00:49
connection between a linear
transformation,
13
00:00:49 --> 00:00:54
which doesn't have to have
coordinates, and the matrix that
14
00:00:54 --> 00:00:59.7
tells us that transformation
with respect to coordinates.
15
00:00:59.7 --> 00:01:05
So the matrix is the
coordinate-based description of
16
00:01:05 --> 00:01:09
the linear transformation.
17
00:01:09 --> 00:01:16
Let me start out with the nice
part, which is just to tell you
18
00:01:16 --> 00:01:21
something about image
compression.
19
00:01:21 --> 00:01:24
Those of you -- well,
everybody's going to meet
20
00:01:24 --> 00:01:29
compression, because you know
that the amount of data that
21
00:01:29 --> 00:01:33
we're getting -- well,
these lectures are compressed.
22
00:01:33 --> 00:01:37
So that, actually,
probably you see my motion as
23
00:01:37 --> 00:01:37
jerky?
24
00:01:37 --> 00:01:40
Shall I use that word?
25
00:01:40 --> 00:01:41
Have you looked on the web?
26
00:01:41 --> 00:01:43
I should like to find a better
word.
27
00:01:43 --> 00:01:45
Compressed, let's say.
28
00:01:45 --> 00:01:48
So the complete signal is,
of course, in those video
29
00:01:48 --> 00:01:51
cameras, and in the videotape,
but that goes to the bottom of
30
00:01:51 --> 00:01:54.24
building nine,
and out of that comes a jumpy
31
00:01:54.24 --> 00:01:57
motion because it uses a
standard system for compressing
32
00:01:57 --> 00:01:58
images.
33
00:01:58 --> 00:02:08
And, you'll notice that the
stuff that sits on the board
34
00:02:08 --> 00:02:18
comes very clearly,
but it's my motion that needs a
35
00:02:18 --> 00:02:23
whole lot of bits,
right?
36
00:02:23 --> 00:02:29
So, and if I were to run up and
back up there and back,
37
00:02:29 --> 00:02:35.89
that would need too many bits,
and I'd be compressed even
38
00:02:35.89 --> 00:02:36
more.
39
00:02:36 --> 00:02:39
So, what does compression mean?
40
00:02:39 --> 00:02:44
Let me just think of a still
image.
41
00:02:44 --> 00:02:47
And of course,
satellites, and computations of
42
00:02:47 --> 00:02:51
the climate, computations of
combustion, the computers and
43
00:02:51 --> 00:02:54
sensors of all kinds are just
giving us overwhelming amounts
44
00:02:54 --> 00:02:55
of data.
45
00:02:55 --> 00:02:56
The Web is, too.
46
00:02:56 --> 00:02:59
Now, some compression can be
done with no loss.
47
00:02:59 --> 00:03:02
Lossless compression is
possible just using,
48
00:03:02 --> 00:03:05
sort of, the fact that there
are redundancies.
49
00:03:05 --> 00:03:11
But I'm talking here about
lossy compression.
50
00:03:11 --> 00:03:15
So I'm talking about -- here's
an image.
51
00:03:15 --> 00:03:19
And what does an image consist
of?
52
00:03:19 --> 00:03:25
It consists of a lot of little
pixels, right?
53
00:03:25 --> 00:03:30
Maybe five hundred and twelve
by five hundred and twelve.
54
00:03:30 --> 00:03:34
Two to the ninth by two to the
ninth pixels,
55
00:03:34 --> 00:03:39
and so this is pixel number
one, one, so that's a pixel.
56
00:03:39 --> 00:03:43.89
And if we're in black and
white, the typical pixel would
57
00:03:43.89 --> 00:03:49
tell us a gray-scale,
from zero to two fifty five.
58
00:03:49 --> 00:03:53
So a pixel is usually a value
of one of the xi,
59
00:03:53 --> 00:03:59
so this would be the i-th
pixel, is -- it's usually a real
60
00:03:59 --> 00:04:03
number on a scale from zero to
two fifty five.
61
00:04:03 --> 00:04:06
In other words,
two to the eighth
62
00:04:06 --> 00:04:08
possibilities.
63
00:04:08 --> 00:04:12
So usually, that's the
standard, so that's eight --
64
00:04:12 --> 00:04:15
eight bits.
65
00:04:15 --> 00:04:23
But then we have that for every
pixel, so we have five hundred
66
00:04:23 --> 00:04:30.61
and twelve squared pixels,
we're really operating x is a
67
00:04:30.61 --> 00:04:34
vector in R^n,
but what is n?
68
00:04:34 --> 00:04:40
n is five hundred and twelve
squared.
69
00:04:40 --> 00:04:44
That's our problem,
right there.
70
00:04:44 --> 00:04:51
A pixel is a vector that gives
us the information about the
71
00:04:51 --> 00:04:52
image.
72
00:04:52 --> 00:04:53
I'm sorry.
73
00:04:53 --> 00:05:02
The image that comes through is
a vector of that length that --
74
00:05:02 --> 00:05:06.47
that's the information that we
have about the image,
75
00:05:06.47 --> 00:05:10
if it's a color image,
we would have three times that
76
00:05:10 --> 00:05:15
length, because we'd need three
coordinates to get color.
77
00:05:15 --> 00:05:20
So it would be three times five
hundred and twelve squared.
78
00:05:20 --> 00:05:25
It's an enormous amount of
information, and we couldn't
79
00:05:25 --> 00:05:30
send out the image for these
lectures without compressing it.
80
00:05:30 --> 00:05:33
It would overload the system.
81
00:05:33 --> 00:05:35
So it has to be compressed.
82
00:05:35 --> 00:05:40
The standard compression,
and still used with lectures
83
00:05:40 --> 00:05:42
is, called JPEG.
84
00:05:42 --> 00:05:46
I think that stands for Joint
Photographic Experts Group.
85
00:05:46 --> 00:05:49
They established a system of
compression.
86
00:05:49 --> 00:05:52
And I just want to tell you
what it's about.
87
00:05:52 --> 00:05:54
It's a change-of-basis.
88
00:05:54 --> 00:05:55
What basis do we have?
89
00:05:55 --> 00:05:58
The current basis we have is,
you could say,
90
00:05:58 --> 00:06:00
the standard basis is,
every pixel,
91
00:06:00 --> 00:06:02
give a value.
92
00:06:02 --> 00:06:11
So that's like we have a vector
x which is five hundred and
93
00:06:11 --> 00:06:17.45
twelve squared long and,
in the i-th position,
94
00:06:17.45 --> 00:06:25
we get a number like one twenty
one or something.
95
00:06:25 --> 00:06:28
The pixel next to it might be
one twenty four,
96
00:06:28 --> 00:06:31
maybe where my tie begins to
enter, so if it was mostly blue
97
00:06:31 --> 00:06:35
shirt, this would be a slight
difference in shading,
98
00:06:35 --> 00:06:37
but pretty close,
then the tie would be a
99
00:06:37 --> 00:06:40
different color,
so we might have quite a few
100
00:06:40 --> 00:06:44.27
pixels for the blue shirt,
and a whole lot more for the
101
00:06:44.27 --> 00:06:47
blackboard, that are very close.
102
00:06:47 --> 00:06:49
And that's what are very
correlated.
103
00:06:49 --> 00:06:53
And that's what gives us the
possibility of compression.
104
00:06:53 --> 00:06:56
For example,
before the lecture starts,
105
00:06:56 --> 00:07:00
if we had a blank blackboard,
then there's an image,
106
00:07:00 --> 00:07:04
but it would make no sense to
take that image and tell you
107
00:07:04 --> 00:07:07
what it is pixel by pixel.
108
00:07:07 --> 00:07:14
I mean, there's a case in which
all pixel values,
109
00:07:14 --> 00:07:22
all gray levels are the same --
or practically the same,
110
00:07:22 --> 00:07:31
depending on the erasing of the
board, but extremely close --
111
00:07:31 --> 00:07:35
and, so that's an image where
the standard basis is lousy.
112
00:07:35 --> 00:07:38
That's the basic fact,
that the standard basis which
113
00:07:38 --> 00:07:42
gives the value of every pixel
makes no use of the fact that
114
00:07:42 --> 00:07:47
we're getting a whole lot of
pixels whose gray levels -- the
115
00:07:47 --> 00:07:51
neighboring pixels tend to have
the same gray level as their
116
00:07:51 --> 00:07:52
neighbors.
117
00:07:52 --> 00:07:58
So how do we take advantage of
that fact?
118
00:07:58 --> 00:08:05
Well, one basis vector that
would be extremely nice to
119
00:08:05 --> 00:08:13
include in the basis would be a
vector of all ones.
120
00:08:13 --> 00:08:19
That's not in our standard
basis, so let me just write
121
00:08:19 --> 00:08:27
again, the standard basis is our
one, and all the rest zeroes,
122
00:08:27 --> 00:08:33
zero, one, and all the rest,
zeroes, everybody knows what
123
00:08:33 --> 00:08:37
these standard basis is.
124
00:08:37 --> 00:08:42.65
Now, any other basis for R --
so this is -- for this very
125
00:08:42.65 --> 00:08:47
high-dimensional space -- now
I'm going to speak about a
126
00:08:47 --> 00:08:48
better basis.
127
00:08:48 --> 00:08:54
Better basis -- and let me just
emphasize, one vector that would
128
00:08:54 --> 00:08:59
be extremely nice to have in
that basis is the vector of all
129
00:08:59 --> 00:09:00
ones.
130
00:09:00 --> 00:09:02
Why is that?
131
00:09:02 --> 00:09:07
Let me just say again,
because that vector of all
132
00:09:07 --> 00:09:12
ones, by itself,
one vector is able to
133
00:09:12 --> 00:09:17
completely give the information
on a solid image.
134
00:09:17 --> 00:09:23
Of course, our image won't be
solid, it will have a mix of
135
00:09:23 --> 00:09:26
solid and signal.
136
00:09:26 --> 00:09:31
So having that one vector in
the basis is going to save us a
137
00:09:31 --> 00:09:32
whole lot.
138
00:09:32 --> 00:09:36
Now, the question is,
what other vectors should be in
139
00:09:36 --> 00:09:37
the basis?
140
00:09:37 --> 00:09:42
The extreme vector in the basis
might be a vector of one minus
141
00:09:42 --> 00:09:45
one, one minus one,
one minus one.
142
00:09:45 --> 00:09:49
That would be a vector that
shows --
143
00:09:49 --> 00:09:53
I mean, that's like a
checkerboard vector,
144
00:09:53 --> 00:09:54
right?
145
00:09:54 --> 00:09:59
That's a vector that would,
if the image was like a huge
146
00:09:59 --> 00:10:03
checkerboard of plus,
minus, plus,
147
00:10:03 --> 00:10:07
minus, plus,
minus, that vector would carry
148
00:10:07 --> 00:10:10
the whole signal.
149
00:10:10 --> 00:10:15
But much more common would be
maybe to have half the image,
150
00:10:15 --> 00:10:18
darker and the other half
lighter.
151
00:10:18 --> 00:10:23
So another vector that might be
quite useful in here would be
152
00:10:23 --> 00:10:25
half ones and half minus ones.
153
00:10:25 --> 00:10:30
I'm just trying to get across
the idea of that a basis could
154
00:10:30 --> 00:10:35
be where, that first of all,
we've got the bases at our
155
00:10:35 --> 00:10:37
disposal.
156
00:10:37 --> 00:10:39
Like, we're free to choose
that.
157
00:10:39 --> 00:10:42
And it's a billion-dollar
decision what we choose.
158
00:10:42 --> 00:10:46
So, and TV people would rather
pre- would prefer one basis
159
00:10:46 --> 00:10:50
based on the way the signal is
scanned, and movie people would
160
00:10:50 --> 00:10:53
prefer another,
I mean, there's giant politics
161
00:10:53 --> 00:10:56
in this question that really
reduces to a linear algebra
162
00:10:56 --> 00:10:59
problem, what basis to choose.
163
00:10:59 --> 00:11:08
I'll just mention the best
known basis, which JPEG uses,
164
00:11:08 --> 00:11:15
-- let me put that here -- is
the Fourier basis.
165
00:11:15 --> 00:11:24
So when you use the Fourier
basis, that includes --
166
00:11:24 --> 00:11:29.23
this is the constant vector,
the D C vector if we're
167
00:11:29.23 --> 00:11:33
electrical engineers,
the l- vector of all ones,
168
00:11:33 --> 00:11:37.52
so it would include one,
one, one, one.
169
00:11:37.52 --> 00:11:41
Often eight by eight is a good
choice.
170
00:11:41 --> 00:11:45
Eight by eight is a good
choice.
171
00:11:45 --> 00:11:49
So, what do I mean by this
eight by eight?
172
00:11:49 --> 00:11:55
I mean that the big signal,
which is five twelve by five
173
00:11:55 --> 00:12:00
twelve, gets broken down,
and JPEG does this,
174
00:12:00 --> 00:12:03.17
into eight by eight blocks.
175
00:12:03.17 --> 00:12:08
And we -- sort of,
this is too much to deal with
176
00:12:08 --> 00:12:10
at once.
177
00:12:10 --> 00:12:14
So what JPEG does is take this
eight by eight block,
178
00:12:14 --> 00:12:17.2
which is sixty four
coefficients,
179
00:12:17.2 --> 00:12:21
sixty four, pixels,
and changes the basis on that
180
00:12:21 --> 00:12:21
piece.
181
00:12:21 --> 00:12:25
And then, now,
let's see, I was going to write
182
00:12:25 --> 00:12:29
down Fourier,
so you remember Fourier as this
183
00:12:29 --> 00:12:33
vector of all ones,
and then, the vector --
184
00:12:33 --> 00:12:38
oh, well, actually,
I gave a lecture earlier about
185
00:12:38 --> 00:12:43
the Fourier matrix,
this matrix whose columns are
186
00:12:43 --> 00:12:46
powers of a complex number w.
187
00:12:46 --> 00:12:52
I won't repeat that,
because I don't want to go into
188
00:12:52 --> 00:12:57
the details of the Fourier
basis, just to tell you how
189
00:12:57 --> 00:13:00
compression works.
190
00:13:00 --> 00:13:02
So what happens in JPEG?
191
00:13:02 --> 00:13:05.58
What happens to the video,
to each image,
192
00:13:05.58 --> 00:13:07
of these lectures?
193
00:13:07 --> 00:13:10
It gets broken into eight by
eight blocks.
194
00:13:10 --> 00:13:10
OK.
195
00:13:10 --> 00:13:13
Within each block,
we have sixty four
196
00:13:13 --> 00:13:16
coefficients,
sixty four basis vectors,
197
00:13:16 --> 00:13:20
sixty four pixels,
and we change basis in sixty
198
00:13:20 --> 00:13:25
four dimensional space using
these Fourier vectors.
199
00:13:25 --> 00:13:33
Just note, that was a lossless
step.
200
00:13:33 --> 00:13:36
Let me emphasize.
201
00:13:36 --> 00:13:41
In comes the signal x.
202
00:13:41 --> 00:13:44
We change basis.
203
00:13:44 --> 00:13:50
This is the basis change.
204
00:13:50 --> 00:13:54
Change basis.
205
00:13:54 --> 00:13:56
Choose a better basis.
206
00:13:56 --> 00:14:00
So it produces,
the coefficients c.
207
00:14:00 --> 00:14:06
So sixty four pixels come in,
sixty four coefficients come
208
00:14:06 --> 00:14:07
out.
209
00:14:07 --> 00:14:10
Now comes the compression.
210
00:14:10 --> 00:14:13
Now come -- this was lossless.
211
00:14:13 --> 00:14:19
It's just -- we know that R --
R sixty four has plenty of
212
00:14:19 --> 00:14:23
bases, and we've chosen one.
213
00:14:23 --> 00:14:28
Now, in that basis,
we write the signal in that
214
00:14:28 --> 00:14:33
basis, and that's what my
lecture -- that's the math part
215
00:14:33 --> 00:14:35
of my lecture.
216
00:14:35 --> 00:14:38
Now here's the application
part.
217
00:14:38 --> 00:14:43
The next part is going to be
the compression step.
218
00:14:43 --> 00:14:44
And that's lossy.
219
00:14:44 --> 00:14:48
We're going to lose
information.
220
00:14:48 --> 00:14:54
And what will actually happen
at that step?
221
00:14:54 --> 00:15:02
Well, one thing we could do is
just throw away the small
222
00:15:02 --> 00:15:04
coefficients.
223
00:15:04 --> 00:15:13
So that's called thresholding,
we set some threshold.
224
00:15:13 --> 00:15:16
Every coefficient,
every basis vector that's not
225
00:15:16 --> 00:15:20
in there more than the threshold
value, and we set them threshold
226
00:15:20 --> 00:15:23
so that our eye can't see the
difference, or can hardly see
227
00:15:23 --> 00:15:26
the difference,
whether we throw away that
228
00:15:26 --> 00:15:28.86
little bit of that basis vector
or keep it.
229
00:15:28.86 --> 00:15:32.07
So this compression step
produces a compressed set of
230
00:15:32.07 --> 00:15:32
coefficients.
231
00:15:32 --> 00:15:35
I'll just keep going here.
232
00:15:35 --> 00:15:39
So it keeps going,
this compression step produces
233
00:15:39 --> 00:15:41
some coefficient c hat.
234
00:15:41 --> 00:15:43
And with many zeroes.
235
00:15:43 --> 00:15:47.27
So that's where the compression
came.
236
00:15:47.27 --> 00:15:52
Probably, there is enough of
this vector of all ones -- we
237
00:15:52 --> 00:15:55
very seldom throw that away.
238
00:15:55 --> 00:15:59
Usually, its coefficient will
be large.
239
00:15:59 --> 00:16:03
But the coefficient of
something like this,
240
00:16:03 --> 00:16:08
that quickly alternative
vector, there's probably very
241
00:16:08 --> 00:16:12
little of that in any smooth
signal.
242
00:16:12 --> 00:16:16
That's high-frequency -- this
is low-frequency,
243
00:16:16 --> 00:16:18
zero frequency.
244
00:16:18 --> 00:16:23
This stuff is the highest
frequency we could have,
245
00:16:23 --> 00:16:27
and if the noise,
the jitter is producing that
246
00:16:27 --> 00:16:31.14
sort of output,
but a smooth lecture like this
247
00:16:31.14 --> 00:16:35
one is, has very little of that
highest frequency,
248
00:16:35 --> 00:16:39
very little noise in this
lecture.
249
00:16:39 --> 00:16:44
OK, so we throw away whatever
there is, and we're left with
250
00:16:44 --> 00:16:48
just a few coefficients,
and then we reconstruct a
251
00:16:48 --> 00:16:51.16
signal using those coefficients.
252
00:16:51.16 --> 00:16:55
We take those coefficients,
times their basis vectors,
253
00:16:55 --> 00:16:59
but this sum doesn't have sixty
four terms any more.
254
00:16:59 --> 00:17:04
Probably, it has about two or
three terms.
255
00:17:04 --> 00:17:06.76
So that would -- say it has
three terms.
256
00:17:06.76 --> 00:17:10
From sixty four down to three,
that's compression of twenty
257
00:17:10 --> 00:17:11
one to one.
258
00:17:11 --> 00:17:14
That's the kind of compression
you're looking for.
259
00:17:14 --> 00:17:17
And everybody is looking for
that sort of compression.
260
00:17:17 --> 00:17:20
Let's see, I guess I met the
problem with the FBI and
261
00:17:20 --> 00:17:22
fingerprints.
262
00:17:22 --> 00:17:27
So there's a whole lot of still
images.
263
00:17:27 --> 00:17:34
You know, with your thumb,
you make these inky marks which
264
00:17:34 --> 00:17:40
go somewhere.
it used to go to Washington and
265
00:17:40 --> 00:17:44.76
get stored in a big file.
266
00:17:44.76 --> 00:17:48
So Washington had a file of
thirty million murderers,
267
00:17:48 --> 00:17:50.91
cheaters on quizzes,
other stuff,
268
00:17:50.91 --> 00:17:53
and actually,
there was no way to retrieve
269
00:17:53 --> 00:17:54
them in time.
270
00:17:54 --> 00:17:58.45
So suppose you're at the police
station, they say,
271
00:17:58.45 --> 00:18:02
OK, this person may have done
this, check with Washington,
272
00:18:02 --> 00:18:07
have they got -- are his or her
fingerprints on file?
273
00:18:07 --> 00:18:11
Well, Washington won't know the
answer within a week if it's got
274
00:18:11 --> 00:18:14
filing cabinets full of
fingerprints.
275
00:18:14 --> 00:18:17
So of course,
the natural step is digitizing.
276
00:18:17 --> 00:18:22
So all fingerprints are now
digitized, so now it's at least
277
00:18:22 --> 00:18:26
electronic, but still there's
too much information in each
278
00:18:26 --> 00:18:27
one.
279
00:18:27 --> 00:18:32
I mean, you can't search
through that many,
280
00:18:32 --> 00:18:39.5
fingerprints if the digital
image is five twelve squared by
281
00:18:39.5 --> 00:18:45
five twelve squared,
if it's that many pixels.
282
00:18:45 --> 00:18:48
So you get compressed.
283
00:18:48 --> 00:18:54
So the FBI had to decide what
basis to choose for compression
284
00:18:54 --> 00:18:56
of fingerprints.
285
00:18:56 --> 00:19:02
And then they built a big new
facility in West Virginia,
286
00:19:02 --> 00:19:07
and that's where fingerprints
now are sent.
287
00:19:07 --> 00:19:11
So I think, if you get your
fingerprints done now at the
288
00:19:11 --> 00:19:13.82
police station,
if it's an up-to-date police
289
00:19:13.82 --> 00:19:17
station, it happens digitally,
and the signal is sent
290
00:19:17 --> 00:19:20
digitally, and then in West
Virginia, it's compressed and
291
00:19:20 --> 00:19:21
indexed.
292
00:19:21 --> 00:19:24
And then, if they want to find
you, they can do it within
293
00:19:24 --> 00:19:27
minutes instead of within a
week.
294
00:19:27 --> 00:19:28
OK.
295
00:19:28 --> 00:19:33
So this compression comes up
for signals, for images,
296
00:19:33 --> 00:19:38
for video -- which is,
like these lectures -- there's
297
00:19:38 --> 00:19:39
another aspect.
298
00:19:39 --> 00:19:45
You could treat the video as
one still image after another
299
00:19:45 --> 00:19:50
one, and compress each one,
and then run them and make a
300
00:19:50 --> 00:19:51
video.
301
00:19:51 --> 00:19:58
But that misses --
well, you can see why that's
302
00:19:58 --> 00:19:59.78
not optimal.
303
00:19:59.78 --> 00:20:06
In a video thing,
you have a sequence of images,
304
00:20:06 --> 00:20:13
so video is really a sequence
of images but what about one
305
00:20:13 --> 00:20:16
image to the next image?
306
00:20:16 --> 00:20:21
They're extremely correlated.
307
00:20:21 --> 00:20:25
I mean that I'm getting an
image every split-second,
308
00:20:25 --> 00:20:27
and also, I'm moving slightly.
309
00:20:27 --> 00:20:31.93
That's what's producing the,
jumpy motion on the video.
310
00:20:31.93 --> 00:20:35
But I'm not,
like, you know -- each image in
311
00:20:35 --> 00:20:39
the sequence is pretty close to
the one before.
312
00:20:39 --> 00:20:42
So you have to use,
like, prediction and
313
00:20:42 --> 00:20:43
correction.
314
00:20:43 --> 00:20:48
I mean, the image of me one
instant -- one time-step later,
315
00:20:48 --> 00:20:52.56
you would assume would be the
same, and then plus a small
316
00:20:52.56 --> 00:20:53.45
correction.
317
00:20:53.45 --> 00:20:57
And you would only code and
digitize the correction,
318
00:20:57 --> 00:21:00
and compress the correction.
319
00:21:00 --> 00:21:06
So a sequence of images that's
highly correlated and the
320
00:21:06 --> 00:21:12
problem in compression is always
to use this correlation,
321
00:21:12 --> 00:21:15.75
this fact that,
in time, or in space,
322
00:21:15.75 --> 00:21:21
things don't change instantly,
they're very often smooth
323
00:21:21 --> 00:21:25
changes, and,
you can predict one value from
324
00:21:25 --> 00:21:28
the previous value.
325
00:21:28 --> 00:21:29
OK.
326
00:21:29 --> 00:21:36
So those are applications which
are pure linear algebra.
327
00:21:36 --> 00:21:41
I could, well,
maybe you'll allow me to tell
328
00:21:41 --> 00:21:48
you, and the book describes,
the new basis that's the
329
00:21:48 --> 00:21:52
competition for Fourier.
330
00:21:52 --> 00:21:55
So the competition for Fourier
is called wavelets,
331
00:21:55 --> 00:21:58
and I can describe what that
basis is like,
332
00:21:58 --> 00:22:01
say, in the eight by eight
case.
333
00:22:01 --> 00:22:05
So the eight by eight wavelet
basis is the vector of all ones,
334
00:22:05 --> 00:22:09
eight ones, then the vector of
four ones and four minus ones,
335
00:22:09 --> 00:22:13
then the vector of two ones,
and two minus ones,
336
00:22:13 --> 00:22:15.4
and four zeroes.
337
00:22:15.4 --> 00:22:24
And also the vector of four
zeroes and two ones and two
338
00:22:24 --> 00:22:26
minus ones.
339
00:22:26 --> 00:22:34
So now I'm up to four,
and I need four more,
340
00:22:34 --> 00:22:35
right?
341
00:22:35 --> 00:22:37
For R^8?
342
00:22:37 --> 00:22:43
The next basis vector will be
one minus one and six zeroes,
343
00:22:43 --> 00:22:48
and then three more like that,
with the one minus one there,
344
00:22:48 --> 00:22:50
and there, and there.
345
00:22:50 --> 00:22:55
So those are eight vectors in
eight-dimensional space,
346
00:22:55 --> 00:23:00
those are called wavelets,
and it's a very simple
347
00:23:00 --> 00:23:04
wavelet choice,
it's a more sophisticated
348
00:23:04 --> 00:23:04
choice.
349
00:23:04 --> 00:23:08
This is a little jumpy,
to jump between one and minus
350
00:23:08 --> 00:23:09
one.
351
00:23:09 --> 00:23:11
And, actually,
you can see,
352
00:23:11 --> 00:23:15
now, suppose you compare the
wavelet basis with the Fourier
353
00:23:15 --> 00:23:16
basis above.
354
00:23:16 --> 00:23:21.38
How could I write this guy,
which is in the Fourier basis,
355
00:23:21.38 --> 00:23:25.46
it's an eight --
it's a vector in R^8.
356
00:23:25.46 --> 00:23:29
How would I write that as a
combination of the wavelet
357
00:23:29 --> 00:23:30
basis?
358
00:23:30 --> 00:23:34
Have I told you enough about
the wavelet basis that you can
359
00:23:34 --> 00:23:39
see, how does this very fast guy
-- what combination of the
360
00:23:39 --> 00:23:42.16
wavelet basis is that very fast
guy?
361
00:23:42.16 --> 00:23:46
It would be this one -- it
would be the sum of these four,
362
00:23:46 --> 00:23:48
right?
363
00:23:48 --> 00:23:51
That very fast guy will be that
one minus one,
364
00:23:51 --> 00:23:53
and the next one,
and the next one,
365
00:23:53 --> 00:23:54
and the next one.
366
00:23:54 --> 00:23:57
So this is the sum of those
last four wavelets.
367
00:23:57 --> 00:23:59
This one, we've kept,
and so on.
368
00:23:59 --> 00:24:02
So, each -- well,
every -- well,
369
00:24:02 --> 00:24:03
that's what a basis does.
370
00:24:03 --> 00:24:07
Every vector in R^8 is some
combination of those,
371
00:24:07 --> 00:24:14
and for the linear algebra --
so the linear algebra is this
372
00:24:14 --> 00:24:18
step, find the coefficient.
373
00:24:18 --> 00:24:23
That's the step we want to
take.
374
00:24:23 --> 00:24:31
What if I give you the basis,
like this wavelet basis,
375
00:24:31 --> 00:24:38
and I give you the pixel --
so here are the pixel values,
376
00:24:38 --> 00:24:41
P1, P2, down to P8 -- what's
the job?
377
00:24:41 --> 00:24:43
What's the linear algebra here?
378
00:24:43 --> 00:24:46
So these are the values,
this is in the standard basis,
379
00:24:46 --> 00:24:47
right?
380
00:24:47 --> 00:24:50.59
Those are just the values at
eight successive points.
381
00:24:50.59 --> 00:24:53
I guess I'm dropping down to
one dimension,
382
00:24:53 --> 00:24:57
instead of eight by eight,
I'm just going to take eight
383
00:24:57 --> 00:25:00
pixel values along that first
top row.
384
00:25:00 --> 00:25:02
So what do I want to do?
385
00:25:02 --> 00:25:07
In standard basis,
here are the pixel values.
386
00:25:07 --> 00:25:12
I want to write that as a
combination of c1 times this
387
00:25:12 --> 00:25:17
guy, plus c2 times this guy,
plus c3, these are the
388
00:25:17 --> 00:25:20
coefficients,
plus c4 times this one -- do
389
00:25:20 --> 00:25:24
you see what I'm doing?
390
00:25:24 --> 00:25:32
I want to write this vector P
as a combination of c1 times the
391
00:25:32 --> 00:25:39
first wavelet plus c8 times the
eighth wavelet.
392
00:25:39 --> 00:25:42
That's the transform step.
393
00:25:42 --> 00:25:47
That's the lossless step.
394
00:25:47 --> 00:25:51
That's the step from P -- oh,
I'm calling it P here,
395
00:25:51 --> 00:25:56
and I called it x there,
so let me -- at the risk of
396
00:25:56 --> 00:26:01
moving, and therefore making
this jumpy -- suppose the signal
397
00:26:01 --> 00:26:04.92
I'm now calling P,
that a pixel values,
398
00:26:04.92 --> 00:26:08
and I'm looking for the
coefficients.
399
00:26:08 --> 00:26:11
OK, tell me how to do it.
400
00:26:11 --> 00:26:14
If I give you eight basis
vectors, and I give you the
401
00:26:14 --> 00:26:17
input signal,
and I ask for the coefficients,
402
00:26:17 --> 00:26:18
what do I do?
403
00:26:18 --> 00:26:19
What's the step?
404
00:26:19 --> 00:26:22
I'm trying to solve this,
I want to know the eight
405
00:26:22 --> 00:26:24
coefficients,
so I'm changing from the
406
00:26:24 --> 00:26:27.39
standard basis,
which is just the eight
407
00:26:27.39 --> 00:26:31
gray-scale values to the wavelet
basis, where the same vector is
408
00:26:31 --> 00:26:34
represented by eight numbers.
409
00:26:34 --> 00:26:43
It's got to take eight numbers
to tell you a vector in R^8,
410
00:26:43 --> 00:26:51
and those eight numbers are the
coefficients of the basis.
411
00:26:51 --> 00:26:58
Look, we've done this thing
before.
412
00:26:58 --> 00:27:06
There is the equation in vector
notation, we want to see it as a
413
00:27:06 --> 00:27:07
matrix.
414
00:27:07 --> 00:27:14
This is a combination of
columns of the wavelet matrix,
415
00:27:14 --> 00:27:14
right?
416
00:27:14 --> 00:27:19
This is P equals c1,
c2, down to c8,
417
00:27:19 --> 00:27:24
and these guys are the columns.
418
00:27:24 --> 00:27:28
I mean, this is the step that
we're constantly taking in this
419
00:27:28 --> 00:27:32
course, the first basis vector
goes in the first column,
420
00:27:32 --> 00:27:36
the second basis vector goes in
the second column,
421
00:27:36 --> 00:27:40
and so on, the eight columns of
this wavelet matrix are the
422
00:27:40 --> 00:27:41
eight basis vectors.
423
00:27:41 --> 00:27:43.93
This is a wavelet matrix W.
424
00:27:43.93 --> 00:27:50.15
So, the step to change basis --
so now I'm finally coming to
425
00:27:50.15 --> 00:27:55.73
this change-of-basis,
so the change of basis that,
426
00:27:55.73 --> 00:28:00
let me stay with this board,
but -- well,
427
00:28:00 --> 00:28:03
let me just go above it,
here.
428
00:28:03 --> 00:28:09
So the standard basis,
we know, the wavelet basis we
429
00:28:09 --> 00:28:16
have here, and the transform is
simply, solve the equations,
430
00:28:16 --> 00:28:16
P=W C.
431
00:28:16 --> 00:28:22
So the coefficients
are W inverse P.
432
00:28:22 --> 00:28:23
Right.
433
00:28:23 --> 00:28:26
This shows a critical point.
434
00:28:26 --> 00:28:31
A good basis has a nice,
fast, inverse.
435
00:28:31 --> 00:28:34.21
So good basis means what?
436
00:28:34.21 --> 00:28:34
Eh?
437
00:28:34 --> 00:28:40.18
So this is like the
billion-dollar competition,
438
00:28:40.18 --> 00:28:42.86
and it's not over yet.
439
00:28:42.86 --> 00:28:51
People are going to come up
with better bases than these.
440
00:28:51 --> 00:28:55
So a good basis will be,
first good thing would be fast.
441
00:28:55 --> 00:29:00
I have to be able to multiply
by W fast, and multiply by W --
442
00:29:00 --> 00:29:01
by its inverse fast.
443
00:29:01 --> 00:29:06
That's -- if a basis doesn't
allow you to do that fast,
444
00:29:06 --> 00:29:11
then it's going to take so much
time that you can't afford it.
445
00:29:11 --> 00:29:15.98
So these bases -- the Fourier
basis, everybody said,
446
00:29:15.98 --> 00:29:20
OK, I know how to deal quickly
with the Fourier basis,
447
00:29:20 --> 00:29:24
because we have something
called the Fast Fourier
448
00:29:24 --> 00:29:25.26
Transform.
449
00:29:25.26 --> 00:29:29
So there's a FFT that came in
my earlier lecture,
450
00:29:29 --> 00:29:33
and comes in the last chapter
of the
451
00:29:33 --> 00:29:37.89
book, so change-of-basis is
done -- if, for the Fourier
452
00:29:37.89 --> 00:29:43
basis, it's done fast by the FFT
and there's a fast wavelet
453
00:29:43 --> 00:29:43
transform.
454
00:29:43 --> 00:29:47
I can change,
for this wavelet example,
455
00:29:47 --> 00:29:49
this matrix is easy to invert.
456
00:29:49 --> 00:29:55
It's just somebody had a smart
idea in choosing that wavelet
457
00:29:55 --> 00:30:00
basis and inverting it,
it has a nice inverse.
458
00:30:00 --> 00:30:02
Actually, you can see why it
has a nice inverse.
459
00:30:02 --> 00:30:05
Do you see any property of
these eight basis vectors?
460
00:30:05 --> 00:30:09.08
Well, I've only written five of
them, but if you see that
461
00:30:09.08 --> 00:30:12
property for those five,
you'll see it for the three
462
00:30:12 --> 00:30:12
remaining.
463
00:30:12 --> 00:30:15
Well, if I give you those eight
vectors and ask,
464
00:30:15 --> 00:30:17
what's a nice property?
465
00:30:17 --> 00:30:22.58
Well, you would say,
first, they're all ones and
466
00:30:22.58 --> 00:30:25
minus ones and zeroes.
467
00:30:25 --> 00:30:31
So every multiplication is very
fast using -- just in binary.
468
00:30:31 --> 00:30:37
But what's the other great
property of those vectors?
469
00:30:37 --> 00:30:40
Anybody see it?
470
00:30:40 --> 00:30:42
So, of course,
when I think about a basis,
471
00:30:42 --> 00:30:45
one nice property -- I don't
have to have it,
472
00:30:45 --> 00:30:48
but I'm happy if it's there --
is that they're orthogonal.
473
00:30:48 --> 00:30:51
If the basis vectors are
orthogonal, then I'm in good
474
00:30:51 --> 00:30:52
shape.
475
00:30:52 --> 00:30:53
And these are...
do you see?
476
00:30:53 --> 00:30:57
Take the dot product of that
with that, you get four plus
477
00:30:57 --> 00:31:00
ones and four minus ones,
you get zero.
478
00:31:00 --> 00:31:03
Take the dot product of that
with that.
479
00:31:03 --> 00:31:07
You get two plus ones and two
minus ones.
480
00:31:07 --> 00:31:10
Or the dot product of that with
that.
481
00:31:10 --> 00:31:13.34
Two plus ones and two minus
ones.
482
00:31:13.34 --> 00:31:17.96
You can easily check that
that's an orthogonal basis.
483
00:31:17.96 --> 00:31:20
It's not orthonormal.
484
00:31:20 --> 00:31:23
To fix it up,
I should divide by the length,
485
00:31:23 --> 00:31:25
to make them unit vectors.
486
00:31:25 --> 00:31:26
Let's suppose I do that.
487
00:31:26 --> 00:31:29.49
So somewhere in here,
I've got to account for the
488
00:31:29.49 --> 00:31:32
fact that this has length square
root of eight,
489
00:31:32 --> 00:31:35
that has length square root of
four, that has length square
490
00:31:35 --> 00:31:36
root of two.
491
00:31:36 --> 00:31:40
But that's just a constant
factor that's easy to --
492
00:31:40 --> 00:31:44
so suppose we've done that.
493
00:31:44 --> 00:31:48
Then, tell me what's W inverse?
494
00:31:48 --> 00:31:54
That's what chapter four,
section four point four was
495
00:31:54 --> 00:31:55
about.
496
00:31:55 --> 00:32:03
If we have orthonormal columns
then the inverse is the same as
497
00:32:03 --> 00:32:06
the transpose.
498
00:32:06 --> 00:32:09
So if we have a fast way to
multiply by W,
499
00:32:09 --> 00:32:12
which we do,
the inverse is going to look
500
00:32:12 --> 00:32:16
just the same,
and we'll have a fast way to do
501
00:32:16 --> 00:32:16
W inverse.
502
00:32:16 --> 00:32:20
So that's the wavelet basis
passes this requirement for
503
00:32:20 --> 00:32:21
fast.
504
00:32:21 --> 00:32:22
We can use it fast.
505
00:32:22 --> 00:32:27
But there's a second
requirement, is it any good?
506
00:32:27 --> 00:32:31
Because the the very fastest
thing we could do is not to
507
00:32:31 --> 00:32:32
change basis at all.
508
00:32:32 --> 00:32:33
Right?
509
00:32:33 --> 00:32:36
The fastest thing would be,
OK, stay with the standard
510
00:32:36 --> 00:32:39
basis, stay with eight pixel
values.
511
00:32:39 --> 00:32:42
But that was poor from
compression point of view,
512
00:32:42 --> 00:32:42
right?
513
00:32:42 --> 00:32:46
Those eight pixel values,
if I just took those eight
514
00:32:46 --> 00:32:50.03
numbers, I can't throw some of
those away.
515
00:32:50.03 --> 00:32:54
If I throw away ninety percent
-- if I compress ten to one,
516
00:32:54 --> 00:32:57
and throw away ninety percent
of my pixel values,
517
00:32:57 --> 00:33:00.29
well, my picture's just gone
dark.
518
00:33:00.29 --> 00:33:04
Whereas, the basis that was
good, the wavelet basis or the
519
00:33:04 --> 00:33:06
Fourier basis,
if I throw away c5,
520
00:33:06 --> 00:33:10
c6, c7, and c8,
all I'm throwing away is little
521
00:33:10 --> 00:33:15
blips that are probably there in
very small amounts.
522
00:33:15 --> 00:33:23
So the second property that we
need is good compression.
523
00:33:23 --> 00:33:30
So first, it has to be fast,
and secondly,
524
00:33:30 --> 00:33:38.5
a few basis vectors should come
close to the signal.
525
00:33:38.5 --> 00:33:42
So a few is enough.
526
00:33:42 --> 00:33:44.5
Can I write it that way?
527
00:33:44.5 --> 00:33:48
A few basis vectors are enough
to reproduce the image just
528
00:33:48 --> 00:33:52
exactly as on a video of these
18.06 lectures.
529
00:33:52 --> 00:33:55.93
Uh, I don't know what the
compression rate is,
530
00:33:55.93 --> 00:33:59
I'll ask, David,
who does the compression --
531
00:33:59 --> 00:34:02
and, by the way,
I'll try to get the lectures,
532
00:34:02 --> 00:34:07
that are relevant for the quiz
up onto the Web in time.
533
00:34:07 --> 00:34:11
So I'll send them a message
today.
534
00:34:11 --> 00:34:18
So, he's using the Fourier
basis because the JPEG -- so
535
00:34:18 --> 00:34:24
JPEG two thousand,
which will be the next standard
536
00:34:24 --> 00:34:30
for image compression,
will include wavelets.
537
00:34:30 --> 00:34:34
So, I mean, you're actually
getting a kind of up-to-date,
538
00:34:34 --> 00:34:40
picture of where this big world
of signal and image processing
539
00:34:40 --> 00:34:40
is.
540
00:34:40 --> 00:34:44
That Fourier is what everybody
knew, and what people
541
00:34:44 --> 00:34:48
automatically used,
and the new one is wavelets,
542
00:34:48 --> 00:34:53
where this is the simplest set
of wavelets.
543
00:34:53 --> 00:34:58
And this isn't the one that the
FBI uses, by the way,
544
00:34:58 --> 00:35:04
the FBI uses a smoother
wavelet, instead of jumping from
545
00:35:04 --> 00:35:07
one to minus one,
it's a smooth,
546
00:35:07 --> 00:35:11
Cutoff.
and, that's what we'll be in in
547
00:35:11 --> 00:35:13
JPEG two thousand.
548
00:35:13 --> 00:35:17
OK, so that's that application.
549
00:35:17 --> 00:35:23
Now, let me come to the math,
the linear algebra part of the
550
00:35:23 --> 00:35:24
lecture.
551
00:35:24 --> 00:35:29
Well, we've actually seen a
change-of-basis.
552
00:35:29 --> 00:35:33
So let -- let me just review
that
553
00:35:33 --> 00:35:39
eh-eh change-of-basis idea,
and then the i- and then the
554
00:35:39 --> 00:35:42
transformation to a matrix.
555
00:35:42 --> 00:35:42
OK.
556
00:35:42 --> 00:35:48
So this, I hope you see that
these applications are really
557
00:35:48 --> 00:35:49
big.
558
00:35:49 --> 00:35:54
Now, I have to talk a little
about change-of-basis,
559
00:35:54 --> 00:35:56
and a little about that.
560
00:35:56 --> 00:35:58
The matrix.
561
00:35:58 --> 00:35:58
OK.
562
00:35:58 --> 00:36:00
OK.
563
00:36:00 --> 00:36:00
OK.
564
00:36:00 --> 00:36:02
So change-of-basis.
565
00:36:02 --> 00:36:10
Basically, forgive that put,
OK, I have, I have my vector in
566
00:36:10 --> 00:36:16
one basis, and I want to change
to a different one.
567
00:36:16 --> 00:36:21
Actually, you saw it for the
wavelet case.
568
00:36:21 --> 00:36:29
So I need the -- let the matrix
W, and the columns of W be the
569
00:36:29 --> 00:36:33
new basis vectors.
570
00:36:33 --> 00:36:41
Then the change-of-basis
involves, just as it did there,
571
00:36:41 --> 00:36:43
W inverse.
572
00:36:43 --> 00:36:50
So we have the vector,
say, x, in the old basis,
573
00:36:50 --> 00:36:56
and that converts to a vector,
let's say, c,
574
00:36:56 --> 00:37:03
in the new basis,
and the relation is exactly
575
00:37:03 --> 00:37:10.05
what we had there,
that x is W c.
576
00:37:10.05 --> 00:37:16
That's the step we have to
take.
577
00:37:16 --> 00:37:25
There's a matrix W that gives
us a change-of-basis.
578
00:37:25 --> 00:37:26
OK.
579
00:37:26 --> 00:37:35
What I want to do is think
about transformations on
580
00:37:35 --> 00:37:39
matrices.
581
00:37:39 --> 00:37:45
So here's the question to
complete this lecture.
582
00:37:45 --> 00:37:51
Suppose I have a linear
transformation T.
583
00:37:51 --> 00:37:59
So we would think of it as an
eight -- as a n by n matrix.
584
00:37:59 --> 00:38:05
And it's computed with respect
to a certain basis.
585
00:38:05 --> 00:38:07
So T -- no, I'm sorry.
586
00:38:07 --> 00:38:12
I've got the transformation T,
period.
587
00:38:12 --> 00:38:18
That's taking eight-dimensional
space to eight-dimensional
588
00:38:18 --> 00:38:19
space.
589
00:38:19 --> 00:38:23
Now, let's get matrices in
there.
590
00:38:23 --> 00:38:24
OK.
591
00:38:24 --> 00:38:33
So, with respect to a first
basis, say v1 up to v8,
592
00:38:33 --> 00:38:36
it has a matrix A.
593
00:38:36 --> 00:38:42
I'm just setting up letters
here.
594
00:38:42 --> 00:38:52
With respect to a second basis,
say, I'll make it u1 up to --
595
00:38:52 --> 00:38:59
or w1, since I've used (w)s,
w1 up to w8,
596
00:38:59 --> 00:39:04
it has a matrix B.
597
00:39:04 --> 00:39:09.34
And my question is,
what's the connection between A
598
00:39:09.34 --> 00:39:09
and B?
599
00:39:09 --> 00:39:15
How is the matrix -- the
transformation T is settled.
600
00:39:15 --> 00:39:17
We could say,
it's a rotation,
601
00:39:17 --> 00:39:19
for example.
602
00:39:19 --> 00:39:22
So that would be one
transformation of
603
00:39:22 --> 00:39:27
eight-dimensional space,
just spin it a little.
604
00:39:27 --> 00:39:29.94
Or project it.
605
00:39:29.94 --> 00:39:33
Or whatever linear
transformation we've got.
606
00:39:33 --> 00:39:38
Now, we have to remember -- my
first step is to remind you how
607
00:39:38 --> 00:39:40
you create that matrix A.
608
00:39:40 --> 00:39:44
Then my second step is,
we would use the same method to
609
00:39:44 --> 00:39:49
create B, but because it came
from the same transformation,
610
00:39:49 --> 00:39:53
there's got to be a relation
between A and B.
611
00:39:53 --> 00:39:57
What's the relation between A
and B?
612
00:39:57 --> 00:40:01
And let me jump to the answer
on that one.
613
00:40:01 --> 00:40:05
That if I have the same
transformation,
614
00:40:05 --> 00:40:09.35
and I'm compute on its matrix
in one basis,
615
00:40:09.35 --> 00:40:13
and then I computer it in
another basis,
616
00:40:13 --> 00:40:17
those two matrices are similar.
617
00:40:17 --> 00:40:22
So these two matrices are
similar.
618
00:40:22 --> 00:40:28
Now, do you remember what
similar matrices meant?
619
00:40:28 --> 00:40:30
Similar.
620
00:40:30 --> 00:40:36
A is similar to -- the two
matrices are similar.
621
00:40:36 --> 00:40:37
Similar.
622
00:40:37 --> 00:40:42
And what do I mean by that?
623
00:40:42 --> 00:40:48
I mean that I take the matrix
B, and I can compute it from the
624
00:40:48 --> 00:40:53
matrix A using some similarity,
some matrix M on one side,
625
00:40:53 --> 00:40:56
and M inverse on the other.
626
00:40:56 --> 00:41:00
And this M will be the
change-of-basis matrix.
627
00:41:00 --> 00:41:06.03
This part of the lecture is,
admittedly, compressed.
628
00:41:06.03 --> 00:41:11
What I wanted you to -- it's
really the conclusion that I
629
00:41:11 --> 00:41:13
want you to spot.
630
00:41:13 --> 00:41:19
Now, I have to go back and say,
what does it mean for A to be
631
00:41:19 --> 00:41:24
the matrix of this
transformation T.
632
00:41:24 --> 00:41:29
So I have to remind you what
that meant, that was in the last
633
00:41:29 --> 00:41:30
lecture.
634
00:41:30 --> 00:41:34
Then this is the conclusion
that if I change to a different
635
00:41:34 --> 00:41:39.36
basis, we now know -- see,
if I change to a different
636
00:41:39.36 --> 00:41:41
basis, two things happen.
637
00:41:41 --> 00:41:44
Every vector has new
coordinates.
638
00:41:44 --> 00:41:49.08
There, the rule is this one,
between the old coordinates and
639
00:41:49.08 --> 00:41:49.98
the new ones.
640
00:41:49.98 --> 00:41:53
Every matrix changes,
every transformation has a new
641
00:41:53 --> 00:41:53
matrix.
642
00:41:53 --> 00:41:57
And the new matrix is related
this way, the M could be the
643
00:41:57 --> 00:41:58
same as the W.
644
00:41:58 --> 00:42:01
The M there would be the W
here.
645
00:42:01 --> 00:42:01
OK.
646
00:42:01 --> 00:42:05.35
So, can I, in the remaining
minutes, recapture my lecture --
647
00:42:05.35 --> 00:42:09
the end of my lecture that was
just before Thanksgiving,
648
00:42:09 --> 00:42:11
about the matrix?
649
00:42:11 --> 00:42:12
OK.
650
00:42:12 --> 00:42:15
What's the matrix?
651
00:42:15 --> 00:42:20
And I'll just take one basis.
652
00:42:20 --> 00:42:30.68
So now this part is going to go
onto this board here.
653
00:42:30.68 --> 00:42:34.25
What is the matrix?
654
00:42:34.25 --> 00:42:36
What is A?
655
00:42:36 --> 00:42:36
OK.
656
00:42:36 --> 00:42:43
Using a basis v1 up to v8.
657
00:42:43 --> 00:42:43
Mm.
658
00:42:43 --> 00:42:43
OK.
659
00:42:43 --> 00:42:46
What's the point?
660
00:42:46 --> 00:42:49
The point is,
if I know what the
661
00:42:49 --> 00:42:55
transformation does to those
eight basis vectors,
662
00:42:55 --> 00:42:57
I know it completely.
663
00:42:57 --> 00:43:04
I know T, I know everything
about T, I know T completely
664
00:43:04 --> 00:43:11
from knowing T of V -- what T
does to v1, what T does to v2,
665
00:43:11 --> 00:43:15
what T does to v8.
666
00:43:15 --> 00:43:16
Why is that?
667
00:43:16 --> 00:43:18
It's because T is a linear
transformation.
668
00:43:18 --> 00:43:23
So that if I know what these
outputs are -- so these are the
669
00:43:23 --> 00:43:26
inputs v1 up to v8,
these are the outputs from the
670
00:43:26 --> 00:43:29
transformation,
like everyone rotated,
671
00:43:29 --> 00:43:32
everyone projected,
whatever transformation I've
672
00:43:32 --> 00:43:36
done, then why is it that I know
everything?
673
00:43:36 --> 00:43:41
How does linearity work?
674
00:43:41 --> 00:43:42
Why?
675
00:43:42 --> 00:43:53
This is because every x is some
combination of these basis
676
00:43:53 --> 00:44:00
vectors, right?
c1v1, c2v2, c8v8,
677
00:44:00 --> 00:44:05
they were a basis.
678
00:44:05 --> 00:44:10
That's the whole point of a
basis, that every vector is a
679
00:44:10 --> 00:44:16
combination of the basis vectors
in exactly one way.
680
00:44:16 --> 00:44:18.66
And then, what is T of x?
681
00:44:18.66 --> 00:44:22
The point is,
I claim that we know T of x
682
00:44:22 --> 00:44:27
completely for every x,
because every x is a
683
00:44:27 --> 00:44:32.89
combination of those --
and now we use the linear
684
00:44:32.89 --> 00:44:38
transformation part to say that
the output from x has to be c1
685
00:44:38 --> 00:44:44
times the output from v1 plus v2
times the output from v2,
686
00:44:44 --> 00:44:45
and so on.
687
00:44:45 --> 00:44:49
Up through c8 times the output
from v8.
688
00:44:49 --> 00:44:52
So this is like just saying,
OK.
689
00:44:52 --> 00:44:57
We know everything when we know
what T does to each basis
690
00:44:57 --> 00:44:59
vector.
691
00:44:59 --> 00:44:59
OK.
692
00:44:59 --> 00:45:05.49
So those are the eight things
we need.
693
00:45:05.49 --> 00:45:12
Now -- but we need these
answers in this basis.
694
00:45:12 --> 00:45:21
So this first output is some
combination of the eight basis
695
00:45:21 --> 00:45:22
vectors.
696
00:45:22 --> 00:45:29
So write T acting on the first
input --
697
00:45:29 --> 00:45:35
in other words,
write the first output as a
698
00:45:35 --> 00:45:42
combination of the basis
vectors, say a11 v1 + a21 v2 and
699
00:45:42 --> 00:45:44.71
so on a81 v8.
700
00:45:44.71 --> 00:45:50
Write T of v2 as some
combination a12 of v1,
701
00:45:50 --> 00:45:53.38
a22 of v2 and so on.
702
00:45:53.38 --> 00:46:00
I'm creating the matrix A,
column by column.
703
00:46:00 --> 00:46:06
Those numbers go in the first
column, these numbers go in the
704
00:46:06 --> 00:46:10
second column,
the matrix A that thi- this --
705
00:46:10 --> 00:46:15
this is our matrix that
represents T in this basis is
706
00:46:15 --> 00:46:18
these numbers.
a11 down to a18,
707
00:46:18 --> 00:46:21
a21 down to a28,
and so on.
708
00:46:21 --> 00:46:21
OK.
709
00:46:21 --> 00:46:23
That's the recipe.
710
00:46:23 --> 00:46:27
In other words,
if I give you a transformation,
711
00:46:27 --> 00:46:30
and a basis.
712
00:46:30 --> 00:46:33
So that's what I have to give
you.
713
00:46:33 --> 00:46:40
The inputs are the basis and to
tell you what the transformation
714
00:46:40 --> 00:46:40
is.
715
00:46:40 --> 00:46:46
And then, you tell me -- you
compute T for each basis,
716
00:46:46 --> 00:46:51
expand that result in the
basis, and that gives you the
717
00:46:51 --> 00:46:57
sixty four numbers that go into
the matrix A.
718
00:46:57 --> 00:47:06
Let me suppose -- let's close
with the best example of all.
719
00:47:06 --> 00:47:11
Suppose v1 to v8,
this basis, is the
720
00:47:11 --> 00:47:13
eigenvectors.
721
00:47:13 --> 00:47:22
Suppose we have an eigenvector
basis so that T(vi) is in the
722
00:47:22 --> 00:47:25
same direction of vi.
723
00:47:25 --> 00:47:29
Now, my question is,
what is A?
724
00:47:29 --> 00:47:36
Can you carry through the
steps?
725
00:47:36 --> 00:47:40
Let's do them together,
because we can do it in one
726
00:47:40 --> 00:47:41
minute.
727
00:47:41 --> 00:47:44
So, we've chosen this perfect
basis.
728
00:47:44 --> 00:47:47
And, actually,
with signal image processing,
729
00:47:47 --> 00:47:50
they might look for the
eigenvectors.
730
00:47:50 --> 00:47:55
But that would take more
calculation time that just
731
00:47:55 --> 00:47:59
saying, OK, we'll use the
wavelet basis.
732
00:47:59 --> 00:48:03
Or, OK, we'll use the Fourier
basis.
733
00:48:03 --> 00:48:08
But the very best basis is the
eigenvector basis.
734
00:48:08 --> 00:48:10
OK, what's the matrix?
735
00:48:10 --> 00:48:15.19
So, what's the first column of
the matrix?
736
00:48:15.19 --> 00:48:18
How do I get the first column?
737
00:48:18 --> 00:48:22
I take the first basis vector
v1.
738
00:48:22 --> 00:48:28
I opt -- I look to see,
what does the transformation do
739
00:48:28 --> 00:48:29
to it?
740
00:48:29 --> 00:48:32
The output is lambda one v1.
741
00:48:32 --> 00:48:38
I express that output as a
combination so the first input
742
00:48:38 --> 00:48:38
is v1.
743
00:48:38 --> 00:48:41
Its output is lambda one v1.
744
00:48:41 --> 00:48:47
Now write lambda one v1 as a
combination of the basis
745
00:48:47 --> 00:48:51
vectors, well,
it's already done.
746
00:48:51 --> 00:48:59
It's just lambda one times the
first basis vector and zero
747
00:48:59 --> 00:49:01
times the others.
748
00:49:01 --> 00:49:08
So this first column will have
lambda one and zeroes.
749
00:49:08 --> 00:49:08
OK.
750
00:49:08 --> 00:49:10
Second input is v2.
751
00:49:10 --> 00:49:13
Output is lambda two v2.
752
00:49:13 --> 00:49:20
OK, write that output as a
combination of the (v)s.
753
00:49:20 --> 00:49:23
It's already done.
754
00:49:23 --> 00:49:27
It's just lambda two times the
second v.
755
00:49:27 --> 00:49:32
So we need, in the second
column, we have lambda two times
756
00:49:32 --> 00:49:33
the second v.
757
00:49:33 --> 00:49:37
Well, you see what's coming,
that in that basis,
758
00:49:37 --> 00:49:42
in the eigenvector basis,
the matrix is diagonal.
759
00:49:42 --> 00:49:47
So that's the perfect basis,
that's the basis we'd love to
760
00:49:47 --> 00:49:52
have for image processing,
but to find the eigenvectors of
761
00:49:52 --> 00:49:56
our pixel matrix would be too
expensive.
762
00:49:56 --> 00:50:01
So we do something cheaper and
close, which is to choose a good
763
00:50:01 --> 00:50:03
basis like wavelets.
764
00:50:03 --> 00:50:04
OK, thanks.
765
00:50:04 --> 00:50:08
So I'll -- quiz review on
Wednesday, all day.
766
00:50:08 --> 00:50:11
Thanks.