1
00:00:07 --> 00:00:07.93
OK.

2
00:00:07.93 --> 00:00:13
So, coming nearer the end of
the course, this lecture will be

3
00:00:13 --> 00:00:18
a mixture of the linear algebra
that comes with a change of

4
00:00:18 --> 00:00:19
basis.

5
00:00:19 --> 00:00:24.37
And a change of basis from one
basis to another basis is

6
00:00:24.37 --> 00:00:27
something you really do in
applications.

7
00:00:27 --> 00:00:33
And, I would like to talk about
those applications.

8
00:00:33 --> 00:00:37
I got a little bit involved
with compression.

9
00:00:37 --> 00:00:40
Compressing a signal,
compressing an image.

10
00:00:40 --> 00:00:42
And that's exactly
change-of-basis.

11
00:00:42 --> 00:00:46
And then, the main theme in
this chapter is th- the

12
00:00:46 --> 00:00:49
connection between a linear
transformation,

13
00:00:49 --> 00:00:54
which doesn't have to have
coordinates, and the matrix that

14
00:00:54 --> 00:00:59.7
tells us that transformation
with respect to coordinates.

15
00:00:59.7 --> 00:01:05
So the matrix is the
coordinate-based description of

16
00:01:05 --> 00:01:09
the linear transformation.

17
00:01:09 --> 00:01:16
Let me start out with the nice
part, which is just to tell you

18
00:01:16 --> 00:01:21
something about image
compression.

19
00:01:21 --> 00:01:24
Those of you -- well,
everybody's going to meet

20
00:01:24 --> 00:01:29
compression, because you know
that the amount of data that

21
00:01:29 --> 00:01:33
we're getting -- well,
these lectures are compressed.

22
00:01:33 --> 00:01:37
So that, actually,
probably you see my motion as

23
00:01:37 --> 00:01:37
jerky?

24
00:01:37 --> 00:01:40
Shall I use that word?

25
00:01:40 --> 00:01:41
Have you looked on the web?

26
00:01:41 --> 00:01:43
I should like to find a better
word.

27
00:01:43 --> 00:01:45
Compressed, let's say.

28
00:01:45 --> 00:01:48
So the complete signal is,
of course, in those video

29
00:01:48 --> 00:01:51
cameras, and in the videotape,
but that goes to the bottom of

30
00:01:51 --> 00:01:54.24
building nine,
and out of that comes a jumpy

31
00:01:54.24 --> 00:01:57
motion because it uses a
standard system for compressing

32
00:01:57 --> 00:01:58
images.

33
00:01:58 --> 00:02:08
And, you'll notice that the
stuff that sits on the board

34
00:02:08 --> 00:02:18
comes very clearly,
but it's my motion that needs a

35
00:02:18 --> 00:02:23
whole lot of bits,
right?

36
00:02:23 --> 00:02:29
So, and if I were to run up and
back up there and back,

37
00:02:29 --> 00:02:35.89
that would need too many bits,
and I'd be compressed even

38
00:02:35.89 --> 00:02:36
more.

39
00:02:36 --> 00:02:39
So, what does compression mean?

40
00:02:39 --> 00:02:44
Let me just think of a still
image.

41
00:02:44 --> 00:02:47
And of course,
satellites, and computations of

42
00:02:47 --> 00:02:51
the climate, computations of
combustion, the computers and

43
00:02:51 --> 00:02:54
sensors of all kinds are just
giving us overwhelming amounts

44
00:02:54 --> 00:02:55
of data.

45
00:02:55 --> 00:02:56
The Web is, too.

46
00:02:56 --> 00:02:59
Now, some compression can be
done with no loss.

47
00:02:59 --> 00:03:02
Lossless compression is
possible just using,

48
00:03:02 --> 00:03:05
sort of, the fact that there
are redundancies.

49
00:03:05 --> 00:03:11
But I'm talking here about
lossy compression.

50
00:03:11 --> 00:03:15
So I'm talking about -- here's
an image.

51
00:03:15 --> 00:03:19
And what does an image consist
of?

52
00:03:19 --> 00:03:25
It consists of a lot of little
pixels, right?

53
00:03:25 --> 00:03:30
Maybe five hundred and twelve
by five hundred and twelve.

54
00:03:30 --> 00:03:34
Two to the ninth by two to the
ninth pixels,

55
00:03:34 --> 00:03:39
and so this is pixel number
one, one, so that's a pixel.

56
00:03:39 --> 00:03:43.89
And if we're in black and
white, the typical pixel would

57
00:03:43.89 --> 00:03:49
tell us a gray-scale,
from zero to two fifty five.

58
00:03:49 --> 00:03:53
So a pixel is usually a value
of one of the xi,

59
00:03:53 --> 00:03:59
so this would be the i-th
pixel, is -- it's usually a real

60
00:03:59 --> 00:04:03
number on a scale from zero to
two fifty five.

61
00:04:03 --> 00:04:06
In other words,
two to the eighth

62
00:04:06 --> 00:04:08
possibilities.

63
00:04:08 --> 00:04:12
So usually, that's the
standard, so that's eight --

64
00:04:12 --> 00:04:15
eight bits.

65
00:04:15 --> 00:04:23
But then we have that for every
pixel, so we have five hundred

66
00:04:23 --> 00:04:30.61
and twelve squared pixels,
we're really operating x is a

67
00:04:30.61 --> 00:04:34
vector in R^n,
but what is n?

68
00:04:34 --> 00:04:40
n is five hundred and twelve
squared.

69
00:04:40 --> 00:04:44
That's our problem,
right there.

70
00:04:44 --> 00:04:51
A pixel is a vector that gives
us the information about the

71
00:04:51 --> 00:04:52
image.

72
00:04:52 --> 00:04:53
I'm sorry.

73
00:04:53 --> 00:05:02
The image that comes through is
a vector of that length that --

74
00:05:02 --> 00:05:06.47
that's the information that we
have about the image,

75
00:05:06.47 --> 00:05:10
if it's a color image,
we would have three times that

76
00:05:10 --> 00:05:15
length, because we'd need three
coordinates to get color.

77
00:05:15 --> 00:05:20
So it would be three times five
hundred and twelve squared.

78
00:05:20 --> 00:05:25
It's an enormous amount of
information, and we couldn't

79
00:05:25 --> 00:05:30
send out the image for these
lectures without compressing it.

80
00:05:30 --> 00:05:33
It would overload the system.

81
00:05:33 --> 00:05:35
So it has to be compressed.

82
00:05:35 --> 00:05:40
The standard compression,
and still used with lectures

83
00:05:40 --> 00:05:42
is, called JPEG.

84
00:05:42 --> 00:05:46
I think that stands for Joint
Photographic Experts Group.

85
00:05:46 --> 00:05:49
They established a system of
compression.

86
00:05:49 --> 00:05:52
And I just want to tell you
what it's about.

87
00:05:52 --> 00:05:54
It's a change-of-basis.

88
00:05:54 --> 00:05:55
What basis do we have?

89
00:05:55 --> 00:05:58
The current basis we have is,
you could say,

90
00:05:58 --> 00:06:00
the standard basis is,
every pixel,

91
00:06:00 --> 00:06:02
give a value.

92
00:06:02 --> 00:06:11
So that's like we have a vector
x which is five hundred and

93
00:06:11 --> 00:06:17.45
twelve squared long and,
in the i-th position,

94
00:06:17.45 --> 00:06:25
we get a number like one twenty
one or something.

95
00:06:25 --> 00:06:28
The pixel next to it might be
one twenty four,

96
00:06:28 --> 00:06:31
maybe where my tie begins to
enter, so if it was mostly blue

97
00:06:31 --> 00:06:35
shirt, this would be a slight
difference in shading,

98
00:06:35 --> 00:06:37
but pretty close,
then the tie would be a

99
00:06:37 --> 00:06:40
different color,
so we might have quite a few

100
00:06:40 --> 00:06:44.27
pixels for the blue shirt,
and a whole lot more for the

101
00:06:44.27 --> 00:06:47
blackboard, that are very close.

102
00:06:47 --> 00:06:49
And that's what are very
correlated.

103
00:06:49 --> 00:06:53
And that's what gives us the
possibility of compression.

104
00:06:53 --> 00:06:56
For example,
before the lecture starts,

105
00:06:56 --> 00:07:00
if we had a blank blackboard,
then there's an image,

106
00:07:00 --> 00:07:04
but it would make no sense to
take that image and tell you

107
00:07:04 --> 00:07:07
what it is pixel by pixel.

108
00:07:07 --> 00:07:14
I mean, there's a case in which
all pixel values,

109
00:07:14 --> 00:07:22
all gray levels are the same --
or practically the same,

110
00:07:22 --> 00:07:31
depending on the erasing of the
board, but extremely close --

111
00:07:31 --> 00:07:35
and, so that's an image where
the standard basis is lousy.

112
00:07:35 --> 00:07:38
That's the basic fact,
that the standard basis which

113
00:07:38 --> 00:07:42
gives the value of every pixel
makes no use of the fact that

114
00:07:42 --> 00:07:47
we're getting a whole lot of
pixels whose gray levels -- the

115
00:07:47 --> 00:07:51
neighboring pixels tend to have
the same gray level as their

116
00:07:51 --> 00:07:52
neighbors.

117
00:07:52 --> 00:07:58
So how do we take advantage of
that fact?

118
00:07:58 --> 00:08:05
Well, one basis vector that
would be extremely nice to

119
00:08:05 --> 00:08:13
include in the basis would be a
vector of all ones.

120
00:08:13 --> 00:08:19
That's not in our standard
basis, so let me just write

121
00:08:19 --> 00:08:27
again, the standard basis is our
one, and all the rest zeroes,

122
00:08:27 --> 00:08:33
zero, one, and all the rest,
zeroes, everybody knows what

123
00:08:33 --> 00:08:37
these standard basis is.

124
00:08:37 --> 00:08:42.65
Now, any other basis for R --
so this is -- for this very

125
00:08:42.65 --> 00:08:47
high-dimensional space -- now
I'm going to speak about a

126
00:08:47 --> 00:08:48
better basis.

127
00:08:48 --> 00:08:54
Better basis -- and let me just
emphasize, one vector that would

128
00:08:54 --> 00:08:59
be extremely nice to have in
that basis is the vector of all

129
00:08:59 --> 00:09:00
ones.

130
00:09:00 --> 00:09:02
Why is that?

131
00:09:02 --> 00:09:07
Let me just say again,
because that vector of all

132
00:09:07 --> 00:09:12
ones, by itself,
one vector is able to

133
00:09:12 --> 00:09:17
completely give the information
on a solid image.

134
00:09:17 --> 00:09:23
Of course, our image won't be
solid, it will have a mix of

135
00:09:23 --> 00:09:26
solid and signal.

136
00:09:26 --> 00:09:31
So having that one vector in
the basis is going to save us a

137
00:09:31 --> 00:09:32
whole lot.

138
00:09:32 --> 00:09:36
Now, the question is,
what other vectors should be in

139
00:09:36 --> 00:09:37
the basis?

140
00:09:37 --> 00:09:42
The extreme vector in the basis
might be a vector of one minus

141
00:09:42 --> 00:09:45
one, one minus one,
one minus one.

142
00:09:45 --> 00:09:49
That would be a vector that
shows --

143
00:09:49 --> 00:09:53
I mean, that's like a
checkerboard vector,

144
00:09:53 --> 00:09:54
right?

145
00:09:54 --> 00:09:59
That's a vector that would,
if the image was like a huge

146
00:09:59 --> 00:10:03
checkerboard of plus,
minus, plus,

147
00:10:03 --> 00:10:07
minus, plus,
minus, that vector would carry

148
00:10:07 --> 00:10:10
the whole signal.

149
00:10:10 --> 00:10:15
But much more common would be
maybe to have half the image,

150
00:10:15 --> 00:10:18
darker and the other half
lighter.

151
00:10:18 --> 00:10:23
So another vector that might be
quite useful in here would be

152
00:10:23 --> 00:10:25
half ones and half minus ones.

153
00:10:25 --> 00:10:30
I'm just trying to get across
the idea of that a basis could

154
00:10:30 --> 00:10:35
be where, that first of all,
we've got the bases at our

155
00:10:35 --> 00:10:37
disposal.

156
00:10:37 --> 00:10:39
Like, we're free to choose
that.

157
00:10:39 --> 00:10:42
And it's a billion-dollar
decision what we choose.

158
00:10:42 --> 00:10:46
So, and TV people would rather
pre- would prefer one basis

159
00:10:46 --> 00:10:50
based on the way the signal is
scanned, and movie people would

160
00:10:50 --> 00:10:53
prefer another,
I mean, there's giant politics

161
00:10:53 --> 00:10:56
in this question that really
reduces to a linear algebra

162
00:10:56 --> 00:10:59
problem, what basis to choose.

163
00:10:59 --> 00:11:08
I'll just mention the best
known basis, which JPEG uses,

164
00:11:08 --> 00:11:15
-- let me put that here -- is
the Fourier basis.

165
00:11:15 --> 00:11:24
So when you use the Fourier
basis, that includes --

166
00:11:24 --> 00:11:29.23
this is the constant vector,
the D C vector if we're

167
00:11:29.23 --> 00:11:33
electrical engineers,
the l- vector of all ones,

168
00:11:33 --> 00:11:37.52
so it would include one,
one, one, one.

169
00:11:37.52 --> 00:11:41
Often eight by eight is a good
choice.

170
00:11:41 --> 00:11:45
Eight by eight is a good
choice.

171
00:11:45 --> 00:11:49
So, what do I mean by this
eight by eight?

172
00:11:49 --> 00:11:55
I mean that the big signal,
which is five twelve by five

173
00:11:55 --> 00:12:00
twelve, gets broken down,
and JPEG does this,

174
00:12:00 --> 00:12:03.17
into eight by eight blocks.

175
00:12:03.17 --> 00:12:08
And we -- sort of,
this is too much to deal with

176
00:12:08 --> 00:12:10
at once.

177
00:12:10 --> 00:12:14
So what JPEG does is take this
eight by eight block,

178
00:12:14 --> 00:12:17.2
which is sixty four
coefficients,

179
00:12:17.2 --> 00:12:21
sixty four, pixels,
and changes the basis on that

180
00:12:21 --> 00:12:21
piece.

181
00:12:21 --> 00:12:25
And then, now,
let's see, I was going to write

182
00:12:25 --> 00:12:29
down Fourier,
so you remember Fourier as this

183
00:12:29 --> 00:12:33
vector of all ones,
and then, the vector --

184
00:12:33 --> 00:12:38
oh, well, actually,
I gave a lecture earlier about

185
00:12:38 --> 00:12:43
the Fourier matrix,
this matrix whose columns are

186
00:12:43 --> 00:12:46
powers of a complex number w.

187
00:12:46 --> 00:12:52
I won't repeat that,
because I don't want to go into

188
00:12:52 --> 00:12:57
the details of the Fourier
basis, just to tell you how

189
00:12:57 --> 00:13:00
compression works.

190
00:13:00 --> 00:13:02
So what happens in JPEG?

191
00:13:02 --> 00:13:05.58
What happens to the video,
to each image,

192
00:13:05.58 --> 00:13:07
of these lectures?

193
00:13:07 --> 00:13:10
It gets broken into eight by
eight blocks.

194
00:13:10 --> 00:13:10
OK.

195
00:13:10 --> 00:13:13
Within each block,
we have sixty four

196
00:13:13 --> 00:13:16
coefficients,
sixty four basis vectors,

197
00:13:16 --> 00:13:20
sixty four pixels,
and we change basis in sixty

198
00:13:20 --> 00:13:25
four dimensional space using
these Fourier vectors.

199
00:13:25 --> 00:13:33
Just note, that was a lossless
step.

200
00:13:33 --> 00:13:36
Let me emphasize.

201
00:13:36 --> 00:13:41
In comes the signal x.

202
00:13:41 --> 00:13:44
We change basis.

203
00:13:44 --> 00:13:50
This is the basis change.

204
00:13:50 --> 00:13:54
Change basis.

205
00:13:54 --> 00:13:56
Choose a better basis.

206
00:13:56 --> 00:14:00
So it produces,
the coefficients c.

207
00:14:00 --> 00:14:06
So sixty four pixels come in,
sixty four coefficients come

208
00:14:06 --> 00:14:07
out.

209
00:14:07 --> 00:14:10
Now comes the compression.

210
00:14:10 --> 00:14:13
Now come -- this was lossless.

211
00:14:13 --> 00:14:19
It's just -- we know that R --
R sixty four has plenty of

212
00:14:19 --> 00:14:23
bases, and we've chosen one.

213
00:14:23 --> 00:14:28
Now, in that basis,
we write the signal in that

214
00:14:28 --> 00:14:33
basis, and that's what my
lecture -- that's the math part

215
00:14:33 --> 00:14:35
of my lecture.

216
00:14:35 --> 00:14:38
Now here's the application
part.

217
00:14:38 --> 00:14:43
The next part is going to be
the compression step.

218
00:14:43 --> 00:14:44
And that's lossy.

219
00:14:44 --> 00:14:48
We're going to lose
information.

220
00:14:48 --> 00:14:54
And what will actually happen
at that step?

221
00:14:54 --> 00:15:02
Well, one thing we could do is
just throw away the small

222
00:15:02 --> 00:15:04
coefficients.

223
00:15:04 --> 00:15:13
So that's called thresholding,
we set some threshold.

224
00:15:13 --> 00:15:16
Every coefficient,
every basis vector that's not

225
00:15:16 --> 00:15:20
in there more than the threshold
value, and we set them threshold

226
00:15:20 --> 00:15:23
so that our eye can't see the
difference, or can hardly see

227
00:15:23 --> 00:15:26
the difference,
whether we throw away that

228
00:15:26 --> 00:15:28.86
little bit of that basis vector
or keep it.

229
00:15:28.86 --> 00:15:32.07
So this compression step
produces a compressed set of

230
00:15:32.07 --> 00:15:32
coefficients.

231
00:15:32 --> 00:15:35
I'll just keep going here.

232
00:15:35 --> 00:15:39
So it keeps going,
this compression step produces

233
00:15:39 --> 00:15:41
some coefficient c hat.

234
00:15:41 --> 00:15:43
And with many zeroes.

235
00:15:43 --> 00:15:47.27
So that's where the compression
came.

236
00:15:47.27 --> 00:15:52
Probably, there is enough of
this vector of all ones -- we

237
00:15:52 --> 00:15:55
very seldom throw that away.

238
00:15:55 --> 00:15:59
Usually, its coefficient will
be large.

239
00:15:59 --> 00:16:03
But the coefficient of
something like this,

240
00:16:03 --> 00:16:08
that quickly alternative
vector, there's probably very

241
00:16:08 --> 00:16:12
little of that in any smooth
signal.

242
00:16:12 --> 00:16:16
That's high-frequency -- this
is low-frequency,

243
00:16:16 --> 00:16:18
zero frequency.

244
00:16:18 --> 00:16:23
This stuff is the highest
frequency we could have,

245
00:16:23 --> 00:16:27
and if the noise,
the jitter is producing that

246
00:16:27 --> 00:16:31.14
sort of output,
but a smooth lecture like this

247
00:16:31.14 --> 00:16:35
one is, has very little of that
highest frequency,

248
00:16:35 --> 00:16:39
very little noise in this
lecture.

249
00:16:39 --> 00:16:44
OK, so we throw away whatever
there is, and we're left with

250
00:16:44 --> 00:16:48
just a few coefficients,
and then we reconstruct a

251
00:16:48 --> 00:16:51.16
signal using those coefficients.

252
00:16:51.16 --> 00:16:55
We take those coefficients,
times their basis vectors,

253
00:16:55 --> 00:16:59
but this sum doesn't have sixty
four terms any more.

254
00:16:59 --> 00:17:04
Probably, it has about two or
three terms.

255
00:17:04 --> 00:17:06.76
So that would -- say it has
three terms.

256
00:17:06.76 --> 00:17:10
From sixty four down to three,
that's compression of twenty

257
00:17:10 --> 00:17:11
one to one.

258
00:17:11 --> 00:17:14
That's the kind of compression
you're looking for.

259
00:17:14 --> 00:17:17
And everybody is looking for
that sort of compression.

260
00:17:17 --> 00:17:20
Let's see, I guess I met the
problem with the FBI and

261
00:17:20 --> 00:17:22
fingerprints.

262
00:17:22 --> 00:17:27
So there's a whole lot of still
images.

263
00:17:27 --> 00:17:34
You know, with your thumb,
you make these inky marks which

264
00:17:34 --> 00:17:40
go somewhere.
it used to go to Washington and

265
00:17:40 --> 00:17:44.76
get stored in a big file.

266
00:17:44.76 --> 00:17:48
So Washington had a file of
thirty million murderers,

267
00:17:48 --> 00:17:50.91
cheaters on quizzes,
other stuff,

268
00:17:50.91 --> 00:17:53
and actually,
there was no way to retrieve

269
00:17:53 --> 00:17:54
them in time.

270
00:17:54 --> 00:17:58.45
So suppose you're at the police
station, they say,

271
00:17:58.45 --> 00:18:02
OK, this person may have done
this, check with Washington,

272
00:18:02 --> 00:18:07
have they got -- are his or her
fingerprints on file?

273
00:18:07 --> 00:18:11
Well, Washington won't know the
answer within a week if it's got

274
00:18:11 --> 00:18:14
filing cabinets full of
fingerprints.

275
00:18:14 --> 00:18:17
So of course,
the natural step is digitizing.

276
00:18:17 --> 00:18:22
So all fingerprints are now
digitized, so now it's at least

277
00:18:22 --> 00:18:26
electronic, but still there's
too much information in each

278
00:18:26 --> 00:18:27
one.

279
00:18:27 --> 00:18:32
I mean, you can't search
through that many,

280
00:18:32 --> 00:18:39.5
fingerprints if the digital
image is five twelve squared by

281
00:18:39.5 --> 00:18:45
five twelve squared,
if it's that many pixels.

282
00:18:45 --> 00:18:48
So you get compressed.

283
00:18:48 --> 00:18:54
So the FBI had to decide what
basis to choose for compression

284
00:18:54 --> 00:18:56
of fingerprints.

285
00:18:56 --> 00:19:02
And then they built a big new
facility in West Virginia,

286
00:19:02 --> 00:19:07
and that's where fingerprints
now are sent.

287
00:19:07 --> 00:19:11
So I think, if you get your
fingerprints done now at the

288
00:19:11 --> 00:19:13.82
police station,
if it's an up-to-date police

289
00:19:13.82 --> 00:19:17
station, it happens digitally,
and the signal is sent

290
00:19:17 --> 00:19:20
digitally, and then in West
Virginia, it's compressed and

291
00:19:20 --> 00:19:21
indexed.

292
00:19:21 --> 00:19:24
And then, if they want to find
you, they can do it within

293
00:19:24 --> 00:19:27
minutes instead of within a
week.

294
00:19:27 --> 00:19:28
OK.

295
00:19:28 --> 00:19:33
So this compression comes up
for signals, for images,

296
00:19:33 --> 00:19:38
for video -- which is,
like these lectures -- there's

297
00:19:38 --> 00:19:39
another aspect.

298
00:19:39 --> 00:19:45
You could treat the video as
one still image after another

299
00:19:45 --> 00:19:50
one, and compress each one,
and then run them and make a

300
00:19:50 --> 00:19:51
video.

301
00:19:51 --> 00:19:58
But that misses --
well, you can see why that's

302
00:19:58 --> 00:19:59.78
not optimal.

303
00:19:59.78 --> 00:20:06
In a video thing,
you have a sequence of images,

304
00:20:06 --> 00:20:13
so video is really a sequence
of images but what about one

305
00:20:13 --> 00:20:16
image to the next image?

306
00:20:16 --> 00:20:21
They're extremely correlated.

307
00:20:21 --> 00:20:25
I mean that I'm getting an
image every split-second,

308
00:20:25 --> 00:20:27
and also, I'm moving slightly.

309
00:20:27 --> 00:20:31.93
That's what's producing the,
jumpy motion on the video.

310
00:20:31.93 --> 00:20:35
But I'm not,
like, you know -- each image in

311
00:20:35 --> 00:20:39
the sequence is pretty close to
the one before.

312
00:20:39 --> 00:20:42
So you have to use,
like, prediction and

313
00:20:42 --> 00:20:43
correction.

314
00:20:43 --> 00:20:48
I mean, the image of me one
instant -- one time-step later,

315
00:20:48 --> 00:20:52.56
you would assume would be the
same, and then plus a small

316
00:20:52.56 --> 00:20:53.45
correction.

317
00:20:53.45 --> 00:20:57
And you would only code and
digitize the correction,

318
00:20:57 --> 00:21:00
and compress the correction.

319
00:21:00 --> 00:21:06
So a sequence of images that's
highly correlated and the

320
00:21:06 --> 00:21:12
problem in compression is always
to use this correlation,

321
00:21:12 --> 00:21:15.75
this fact that,
in time, or in space,

322
00:21:15.75 --> 00:21:21
things don't change instantly,
they're very often smooth

323
00:21:21 --> 00:21:25
changes, and,
you can predict one value from

324
00:21:25 --> 00:21:28
the previous value.

325
00:21:28 --> 00:21:29
OK.

326
00:21:29 --> 00:21:36
So those are applications which
are pure linear algebra.

327
00:21:36 --> 00:21:41
I could, well,
maybe you'll allow me to tell

328
00:21:41 --> 00:21:48
you, and the book describes,
the new basis that's the

329
00:21:48 --> 00:21:52
competition for Fourier.

330
00:21:52 --> 00:21:55
So the competition for Fourier
is called wavelets,

331
00:21:55 --> 00:21:58
and I can describe what that
basis is like,

332
00:21:58 --> 00:22:01
say, in the eight by eight
case.

333
00:22:01 --> 00:22:05
So the eight by eight wavelet
basis is the vector of all ones,

334
00:22:05 --> 00:22:09
eight ones, then the vector of
four ones and four minus ones,

335
00:22:09 --> 00:22:13
then the vector of two ones,
and two minus ones,

336
00:22:13 --> 00:22:15.4
and four zeroes.

337
00:22:15.4 --> 00:22:24
And also the vector of four
zeroes and two ones and two

338
00:22:24 --> 00:22:26
minus ones.

339
00:22:26 --> 00:22:34
So now I'm up to four,
and I need four more,

340
00:22:34 --> 00:22:35
right?

341
00:22:35 --> 00:22:37
For R^8?

342
00:22:37 --> 00:22:43
The next basis vector will be
one minus one and six zeroes,

343
00:22:43 --> 00:22:48
and then three more like that,
with the one minus one there,

344
00:22:48 --> 00:22:50
and there, and there.

345
00:22:50 --> 00:22:55
So those are eight vectors in
eight-dimensional space,

346
00:22:55 --> 00:23:00
those are called wavelets,
and it's a very simple

347
00:23:00 --> 00:23:04
wavelet choice,
it's a more sophisticated

348
00:23:04 --> 00:23:04
choice.

349
00:23:04 --> 00:23:08
This is a little jumpy,
to jump between one and minus

350
00:23:08 --> 00:23:09
one.

351
00:23:09 --> 00:23:11
And, actually,
you can see,

352
00:23:11 --> 00:23:15
now, suppose you compare the
wavelet basis with the Fourier

353
00:23:15 --> 00:23:16
basis above.

354
00:23:16 --> 00:23:21.38
How could I write this guy,
which is in the Fourier basis,

355
00:23:21.38 --> 00:23:25.46
it's an eight --
it's a vector in R^8.

356
00:23:25.46 --> 00:23:29
How would I write that as a
combination of the wavelet

357
00:23:29 --> 00:23:30
basis?

358
00:23:30 --> 00:23:34
Have I told you enough about
the wavelet basis that you can

359
00:23:34 --> 00:23:39
see, how does this very fast guy
-- what combination of the

360
00:23:39 --> 00:23:42.16
wavelet basis is that very fast
guy?

361
00:23:42.16 --> 00:23:46
It would be this one -- it
would be the sum of these four,

362
00:23:46 --> 00:23:48
right?

363
00:23:48 --> 00:23:51
That very fast guy will be that
one minus one,

364
00:23:51 --> 00:23:53
and the next one,
and the next one,

365
00:23:53 --> 00:23:54
and the next one.

366
00:23:54 --> 00:23:57
So this is the sum of those
last four wavelets.

367
00:23:57 --> 00:23:59
This one, we've kept,
and so on.

368
00:23:59 --> 00:24:02
So, each -- well,
every -- well,

369
00:24:02 --> 00:24:03
that's what a basis does.

370
00:24:03 --> 00:24:07
Every vector in R^8 is some
combination of those,

371
00:24:07 --> 00:24:14
and for the linear algebra --
so the linear algebra is this

372
00:24:14 --> 00:24:18
step, find the coefficient.

373
00:24:18 --> 00:24:23
That's the step we want to
take.

374
00:24:23 --> 00:24:31
What if I give you the basis,
like this wavelet basis,

375
00:24:31 --> 00:24:38
and I give you the pixel --
so here are the pixel values,

376
00:24:38 --> 00:24:41
P1, P2, down to P8 -- what's
the job?

377
00:24:41 --> 00:24:43
What's the linear algebra here?

378
00:24:43 --> 00:24:46
So these are the values,
this is in the standard basis,

379
00:24:46 --> 00:24:47
right?

380
00:24:47 --> 00:24:50.59
Those are just the values at
eight successive points.

381
00:24:50.59 --> 00:24:53
I guess I'm dropping down to
one dimension,

382
00:24:53 --> 00:24:57
instead of eight by eight,
I'm just going to take eight

383
00:24:57 --> 00:25:00
pixel values along that first
top row.

384
00:25:00 --> 00:25:02
So what do I want to do?

385
00:25:02 --> 00:25:07
In standard basis,
here are the pixel values.

386
00:25:07 --> 00:25:12
I want to write that as a
combination of c1 times this

387
00:25:12 --> 00:25:17
guy, plus c2 times this guy,
plus c3, these are the

388
00:25:17 --> 00:25:20
coefficients,
plus c4 times this one -- do

389
00:25:20 --> 00:25:24
you see what I'm doing?

390
00:25:24 --> 00:25:32
I want to write this vector P
as a combination of c1 times the

391
00:25:32 --> 00:25:39
first wavelet plus c8 times the
eighth wavelet.

392
00:25:39 --> 00:25:42
That's the transform step.

393
00:25:42 --> 00:25:47
That's the lossless step.

394
00:25:47 --> 00:25:51
That's the step from P -- oh,
I'm calling it P here,

395
00:25:51 --> 00:25:56
and I called it x there,
so let me -- at the risk of

396
00:25:56 --> 00:26:01
moving, and therefore making
this jumpy -- suppose the signal

397
00:26:01 --> 00:26:04.92
I'm now calling P,
that a pixel values,

398
00:26:04.92 --> 00:26:08
and I'm looking for the
coefficients.

399
00:26:08 --> 00:26:11
OK, tell me how to do it.

400
00:26:11 --> 00:26:14
If I give you eight basis
vectors, and I give you the

401
00:26:14 --> 00:26:17
input signal,
and I ask for the coefficients,

402
00:26:17 --> 00:26:18
what do I do?

403
00:26:18 --> 00:26:19
What's the step?

404
00:26:19 --> 00:26:22
I'm trying to solve this,
I want to know the eight

405
00:26:22 --> 00:26:24
coefficients,
so I'm changing from the

406
00:26:24 --> 00:26:27.39
standard basis,
which is just the eight

407
00:26:27.39 --> 00:26:31
gray-scale values to the wavelet
basis, where the same vector is

408
00:26:31 --> 00:26:34
represented by eight numbers.

409
00:26:34 --> 00:26:43
It's got to take eight numbers
to tell you a vector in R^8,

410
00:26:43 --> 00:26:51
and those eight numbers are the
coefficients of the basis.

411
00:26:51 --> 00:26:58
Look, we've done this thing
before.

412
00:26:58 --> 00:27:06
There is the equation in vector
notation, we want to see it as a

413
00:27:06 --> 00:27:07
matrix.

414
00:27:07 --> 00:27:14
This is a combination of
columns of the wavelet matrix,

415
00:27:14 --> 00:27:14
right?

416
00:27:14 --> 00:27:19
This is P equals c1,
c2, down to c8,

417
00:27:19 --> 00:27:24
and these guys are the columns.

418
00:27:24 --> 00:27:28
I mean, this is the step that
we're constantly taking in this

419
00:27:28 --> 00:27:32
course, the first basis vector
goes in the first column,

420
00:27:32 --> 00:27:36
the second basis vector goes in
the second column,

421
00:27:36 --> 00:27:40
and so on, the eight columns of
this wavelet matrix are the

422
00:27:40 --> 00:27:41
eight basis vectors.

423
00:27:41 --> 00:27:43.93
This is a wavelet matrix W.

424
00:27:43.93 --> 00:27:50.15
So, the step to change basis --
so now I'm finally coming to

425
00:27:50.15 --> 00:27:55.73
this change-of-basis,
so the change of basis that,

426
00:27:55.73 --> 00:28:00
let me stay with this board,
but -- well,

427
00:28:00 --> 00:28:03
let me just go above it,
here.

428
00:28:03 --> 00:28:09
So the standard basis,
we know, the wavelet basis we

429
00:28:09 --> 00:28:16
have here, and the transform is
simply, solve the equations,

430
00:28:16 --> 00:28:16
P=W C.

431
00:28:16 --> 00:28:22
So the coefficients
are W inverse P.

432
00:28:22 --> 00:28:23
Right.

433
00:28:23 --> 00:28:26
This shows a critical point.

434
00:28:26 --> 00:28:31
A good basis has a nice,
fast, inverse.

435
00:28:31 --> 00:28:34.21
So good basis means what?

436
00:28:34.21 --> 00:28:34
Eh?

437
00:28:34 --> 00:28:40.18
So this is like the
billion-dollar competition,

438
00:28:40.18 --> 00:28:42.86
and it's not over yet.

439
00:28:42.86 --> 00:28:51
People are going to come up
with better bases than these.

440
00:28:51 --> 00:28:55
So a good basis will be,
first good thing would be fast.

441
00:28:55 --> 00:29:00
I have to be able to multiply
by W fast, and multiply by W --

442
00:29:00 --> 00:29:01
by its inverse fast.

443
00:29:01 --> 00:29:06
That's -- if a basis doesn't
allow you to do that fast,

444
00:29:06 --> 00:29:11
then it's going to take so much
time that you can't afford it.

445
00:29:11 --> 00:29:15.98
So these bases -- the Fourier
basis, everybody said,

446
00:29:15.98 --> 00:29:20
OK, I know how to deal quickly
with the Fourier basis,

447
00:29:20 --> 00:29:24
because we have something
called the Fast Fourier

448
00:29:24 --> 00:29:25.26
Transform.

449
00:29:25.26 --> 00:29:29
So there's a FFT that came in
my earlier lecture,

450
00:29:29 --> 00:29:33
and comes in the last chapter
of the

451
00:29:33 --> 00:29:37.89
book, so change-of-basis is
done -- if, for the Fourier

452
00:29:37.89 --> 00:29:43
basis, it's done fast by the FFT
and there's a fast wavelet

453
00:29:43 --> 00:29:43
transform.

454
00:29:43 --> 00:29:47
I can change,
for this wavelet example,

455
00:29:47 --> 00:29:49
this matrix is easy to invert.

456
00:29:49 --> 00:29:55
It's just somebody had a smart
idea in choosing that wavelet

457
00:29:55 --> 00:30:00
basis and inverting it,
it has a nice inverse.

458
00:30:00 --> 00:30:02
Actually, you can see why it
has a nice inverse.

459
00:30:02 --> 00:30:05
Do you see any property of
these eight basis vectors?

460
00:30:05 --> 00:30:09.08
Well, I've only written five of
them, but if you see that

461
00:30:09.08 --> 00:30:12
property for those five,
you'll see it for the three

462
00:30:12 --> 00:30:12
remaining.

463
00:30:12 --> 00:30:15
Well, if I give you those eight
vectors and ask,

464
00:30:15 --> 00:30:17
what's a nice property?

465
00:30:17 --> 00:30:22.58
Well, you would say,
first, they're all ones and

466
00:30:22.58 --> 00:30:25
minus ones and zeroes.

467
00:30:25 --> 00:30:31
So every multiplication is very
fast using -- just in binary.

468
00:30:31 --> 00:30:37
But what's the other great
property of those vectors?

469
00:30:37 --> 00:30:40
Anybody see it?

470
00:30:40 --> 00:30:42
So, of course,
when I think about a basis,

471
00:30:42 --> 00:30:45
one nice property -- I don't
have to have it,

472
00:30:45 --> 00:30:48
but I'm happy if it's there --
is that they're orthogonal.

473
00:30:48 --> 00:30:51
If the basis vectors are
orthogonal, then I'm in good

474
00:30:51 --> 00:30:52
shape.

475
00:30:52 --> 00:30:53
And these are...
do you see?

476
00:30:53 --> 00:30:57
Take the dot product of that
with that, you get four plus

477
00:30:57 --> 00:31:00
ones and four minus ones,
you get zero.

478
00:31:00 --> 00:31:03
Take the dot product of that
with that.

479
00:31:03 --> 00:31:07
You get two plus ones and two
minus ones.

480
00:31:07 --> 00:31:10
Or the dot product of that with
that.

481
00:31:10 --> 00:31:13.34
Two plus ones and two minus
ones.

482
00:31:13.34 --> 00:31:17.96
You can easily check that
that's an orthogonal basis.

483
00:31:17.96 --> 00:31:20
It's not orthonormal.

484
00:31:20 --> 00:31:23
To fix it up,
I should divide by the length,

485
00:31:23 --> 00:31:25
to make them unit vectors.

486
00:31:25 --> 00:31:26
Let's suppose I do that.

487
00:31:26 --> 00:31:29.49
So somewhere in here,
I've got to account for the

488
00:31:29.49 --> 00:31:32
fact that this has length square
root of eight,

489
00:31:32 --> 00:31:35
that has length square root of
four, that has length square

490
00:31:35 --> 00:31:36
root of two.

491
00:31:36 --> 00:31:40
But that's just a constant
factor that's easy to --

492
00:31:40 --> 00:31:44
so suppose we've done that.

493
00:31:44 --> 00:31:48
Then, tell me what's W inverse?

494
00:31:48 --> 00:31:54
That's what chapter four,
section four point four was

495
00:31:54 --> 00:31:55
about.

496
00:31:55 --> 00:32:03
If we have orthonormal columns
then the inverse is the same as

497
00:32:03 --> 00:32:06
the transpose.

498
00:32:06 --> 00:32:09
So if we have a fast way to
multiply by W,

499
00:32:09 --> 00:32:12
which we do,
the inverse is going to look

500
00:32:12 --> 00:32:16
just the same,
and we'll have a fast way to do

501
00:32:16 --> 00:32:16
W inverse.

502
00:32:16 --> 00:32:20
So that's the wavelet basis
passes this requirement for

503
00:32:20 --> 00:32:21
fast.

504
00:32:21 --> 00:32:22
We can use it fast.

505
00:32:22 --> 00:32:27
But there's a second
requirement, is it any good?

506
00:32:27 --> 00:32:31
Because the the very fastest
thing we could do is not to

507
00:32:31 --> 00:32:32
change basis at all.

508
00:32:32 --> 00:32:33
Right?

509
00:32:33 --> 00:32:36
The fastest thing would be,
OK, stay with the standard

510
00:32:36 --> 00:32:39
basis, stay with eight pixel
values.

511
00:32:39 --> 00:32:42
But that was poor from
compression point of view,

512
00:32:42 --> 00:32:42
right?

513
00:32:42 --> 00:32:46
Those eight pixel values,
if I just took those eight

514
00:32:46 --> 00:32:50.03
numbers, I can't throw some of
those away.

515
00:32:50.03 --> 00:32:54
If I throw away ninety percent
-- if I compress ten to one,

516
00:32:54 --> 00:32:57
and throw away ninety percent
of my pixel values,

517
00:32:57 --> 00:33:00.29
well, my picture's just gone
dark.

518
00:33:00.29 --> 00:33:04
Whereas, the basis that was
good, the wavelet basis or the

519
00:33:04 --> 00:33:06
Fourier basis,
if I throw away c5,

520
00:33:06 --> 00:33:10
c6, c7, and c8,
all I'm throwing away is little

521
00:33:10 --> 00:33:15
blips that are probably there in
very small amounts.

522
00:33:15 --> 00:33:23
So the second property that we
need is good compression.

523
00:33:23 --> 00:33:30
So first, it has to be fast,
and secondly,

524
00:33:30 --> 00:33:38.5
a few basis vectors should come
close to the signal.

525
00:33:38.5 --> 00:33:42
So a few is enough.

526
00:33:42 --> 00:33:44.5
Can I write it that way?

527
00:33:44.5 --> 00:33:48
A few basis vectors are enough
to reproduce the image just

528
00:33:48 --> 00:33:52
exactly as on a video of these
18.06 lectures.

529
00:33:52 --> 00:33:55.93
Uh, I don't know what the
compression rate is,

530
00:33:55.93 --> 00:33:59
I'll ask, David,
who does the compression --

531
00:33:59 --> 00:34:02
and, by the way,
I'll try to get the lectures,

532
00:34:02 --> 00:34:07
that are relevant for the quiz
up onto the Web in time.

533
00:34:07 --> 00:34:11
So I'll send them a message
today.

534
00:34:11 --> 00:34:18
So, he's using the Fourier
basis because the JPEG -- so

535
00:34:18 --> 00:34:24
JPEG two thousand,
which will be the next standard

536
00:34:24 --> 00:34:30
for image compression,
will include wavelets.

537
00:34:30 --> 00:34:34
So, I mean, you're actually
getting a kind of up-to-date,

538
00:34:34 --> 00:34:40
picture of where this big world
of signal and image processing

539
00:34:40 --> 00:34:40
is.

540
00:34:40 --> 00:34:44
That Fourier is what everybody
knew, and what people

541
00:34:44 --> 00:34:48
automatically used,
and the new one is wavelets,

542
00:34:48 --> 00:34:53
where this is the simplest set
of wavelets.

543
00:34:53 --> 00:34:58
And this isn't the one that the
FBI uses, by the way,

544
00:34:58 --> 00:35:04
the FBI uses a smoother
wavelet, instead of jumping from

545
00:35:04 --> 00:35:07
one to minus one,
it's a smooth,

546
00:35:07 --> 00:35:11
Cutoff.
and, that's what we'll be in in

547
00:35:11 --> 00:35:13
JPEG two thousand.

548
00:35:13 --> 00:35:17
OK, so that's that application.

549
00:35:17 --> 00:35:23
Now, let me come to the math,
the linear algebra part of the

550
00:35:23 --> 00:35:24
lecture.

551
00:35:24 --> 00:35:29
Well, we've actually seen a
change-of-basis.

552
00:35:29 --> 00:35:33
So let -- let me just review
that

553
00:35:33 --> 00:35:39
eh-eh change-of-basis idea,
and then the i- and then the

554
00:35:39 --> 00:35:42
transformation to a matrix.

555
00:35:42 --> 00:35:42
OK.

556
00:35:42 --> 00:35:48
So this, I hope you see that
these applications are really

557
00:35:48 --> 00:35:49
big.

558
00:35:49 --> 00:35:54
Now, I have to talk a little
about change-of-basis,

559
00:35:54 --> 00:35:56
and a little about that.

560
00:35:56 --> 00:35:58
The matrix.

561
00:35:58 --> 00:35:58
OK.

562
00:35:58 --> 00:36:00
OK.

563
00:36:00 --> 00:36:00
OK.

564
00:36:00 --> 00:36:02
So change-of-basis.

565
00:36:02 --> 00:36:10
Basically, forgive that put,
OK, I have, I have my vector in

566
00:36:10 --> 00:36:16
one basis, and I want to change
to a different one.

567
00:36:16 --> 00:36:21
Actually, you saw it for the
wavelet case.

568
00:36:21 --> 00:36:29
So I need the -- let the matrix
W, and the columns of W be the

569
00:36:29 --> 00:36:33
new basis vectors.

570
00:36:33 --> 00:36:41
Then the change-of-basis
involves, just as it did there,

571
00:36:41 --> 00:36:43
W inverse.

572
00:36:43 --> 00:36:50
So we have the vector,
say, x, in the old basis,

573
00:36:50 --> 00:36:56
and that converts to a vector,
let's say, c,

574
00:36:56 --> 00:37:03
in the new basis,
and the relation is exactly

575
00:37:03 --> 00:37:10.05
what we had there,
that x is W c.

576
00:37:10.05 --> 00:37:16
That's the step we have to
take.

577
00:37:16 --> 00:37:25
There's a matrix W that gives
us a change-of-basis.

578
00:37:25 --> 00:37:26
OK.

579
00:37:26 --> 00:37:35
What I want to do is think
about transformations on

580
00:37:35 --> 00:37:39
matrices.

581
00:37:39 --> 00:37:45
So here's the question to
complete this lecture.

582
00:37:45 --> 00:37:51
Suppose I have a linear
transformation T.

583
00:37:51 --> 00:37:59
So we would think of it as an
eight -- as a n by n matrix.

584
00:37:59 --> 00:38:05
And it's computed with respect
to a certain basis.

585
00:38:05 --> 00:38:07
So T -- no, I'm sorry.

586
00:38:07 --> 00:38:12
I've got the transformation T,
period.

587
00:38:12 --> 00:38:18
That's taking eight-dimensional
space to eight-dimensional

588
00:38:18 --> 00:38:19
space.

589
00:38:19 --> 00:38:23
Now, let's get matrices in
there.

590
00:38:23 --> 00:38:24
OK.

591
00:38:24 --> 00:38:33
So, with respect to a first
basis, say v1 up to v8,

592
00:38:33 --> 00:38:36
it has a matrix A.

593
00:38:36 --> 00:38:42
I'm just setting up letters
here.

594
00:38:42 --> 00:38:52
With respect to a second basis,
say, I'll make it u1 up to --

595
00:38:52 --> 00:38:59
or w1, since I've used (w)s,
w1 up to w8,

596
00:38:59 --> 00:39:04
it has a matrix B.

597
00:39:04 --> 00:39:09.34
And my question is,
what's the connection between A

598
00:39:09.34 --> 00:39:09
and B?

599
00:39:09 --> 00:39:15
How is the matrix -- the
transformation T is settled.

600
00:39:15 --> 00:39:17
We could say,
it's a rotation,

601
00:39:17 --> 00:39:19
for example.

602
00:39:19 --> 00:39:22
So that would be one
transformation of

603
00:39:22 --> 00:39:27
eight-dimensional space,
just spin it a little.

604
00:39:27 --> 00:39:29.94
Or project it.

605
00:39:29.94 --> 00:39:33
Or whatever linear
transformation we've got.

606
00:39:33 --> 00:39:38
Now, we have to remember -- my
first step is to remind you how

607
00:39:38 --> 00:39:40
you create that matrix A.

608
00:39:40 --> 00:39:44
Then my second step is,
we would use the same method to

609
00:39:44 --> 00:39:49
create B, but because it came
from the same transformation,

610
00:39:49 --> 00:39:53
there's got to be a relation
between A and B.

611
00:39:53 --> 00:39:57
What's the relation between A
and B?

612
00:39:57 --> 00:40:01
And let me jump to the answer
on that one.

613
00:40:01 --> 00:40:05
That if I have the same
transformation,

614
00:40:05 --> 00:40:09.35
and I'm compute on its matrix
in one basis,

615
00:40:09.35 --> 00:40:13
and then I computer it in
another basis,

616
00:40:13 --> 00:40:17
those two matrices are similar.

617
00:40:17 --> 00:40:22
So these two matrices are
similar.

618
00:40:22 --> 00:40:28
Now, do you remember what
similar matrices meant?

619
00:40:28 --> 00:40:30
Similar.

620
00:40:30 --> 00:40:36
A is similar to -- the two
matrices are similar.

621
00:40:36 --> 00:40:37
Similar.

622
00:40:37 --> 00:40:42
And what do I mean by that?

623
00:40:42 --> 00:40:48
I mean that I take the matrix
B, and I can compute it from the

624
00:40:48 --> 00:40:53
matrix A using some similarity,
some matrix M on one side,

625
00:40:53 --> 00:40:56
and M inverse on the other.

626
00:40:56 --> 00:41:00
And this M will be the
change-of-basis matrix.

627
00:41:00 --> 00:41:06.03
This part of the lecture is,
admittedly, compressed.

628
00:41:06.03 --> 00:41:11
What I wanted you to -- it's
really the conclusion that I

629
00:41:11 --> 00:41:13
want you to spot.

630
00:41:13 --> 00:41:19
Now, I have to go back and say,
what does it mean for A to be

631
00:41:19 --> 00:41:24
the matrix of this
transformation T.

632
00:41:24 --> 00:41:29
So I have to remind you what
that meant, that was in the last

633
00:41:29 --> 00:41:30
lecture.

634
00:41:30 --> 00:41:34
Then this is the conclusion
that if I change to a different

635
00:41:34 --> 00:41:39.36
basis, we now know -- see,
if I change to a different

636
00:41:39.36 --> 00:41:41
basis, two things happen.

637
00:41:41 --> 00:41:44
Every vector has new
coordinates.

638
00:41:44 --> 00:41:49.08
There, the rule is this one,
between the old coordinates and

639
00:41:49.08 --> 00:41:49.98
the new ones.

640
00:41:49.98 --> 00:41:53
Every matrix changes,
every transformation has a new

641
00:41:53 --> 00:41:53
matrix.

642
00:41:53 --> 00:41:57
And the new matrix is related
this way, the M could be the

643
00:41:57 --> 00:41:58
same as the W.

644
00:41:58 --> 00:42:01
The M there would be the W
here.

645
00:42:01 --> 00:42:01
OK.

646
00:42:01 --> 00:42:05.35
So, can I, in the remaining
minutes, recapture my lecture --

647
00:42:05.35 --> 00:42:09
the end of my lecture that was
just before Thanksgiving,

648
00:42:09 --> 00:42:11
about the matrix?

649
00:42:11 --> 00:42:12
OK.

650
00:42:12 --> 00:42:15
What's the matrix?

651
00:42:15 --> 00:42:20
And I'll just take one basis.

652
00:42:20 --> 00:42:30.68
So now this part is going to go
onto this board here.

653
00:42:30.68 --> 00:42:34.25
What is the matrix?

654
00:42:34.25 --> 00:42:36
What is A?

655
00:42:36 --> 00:42:36
OK.

656
00:42:36 --> 00:42:43
Using a basis v1 up to v8.

657
00:42:43 --> 00:42:43
Mm.

658
00:42:43 --> 00:42:43
OK.

659
00:42:43 --> 00:42:46
What's the point?

660
00:42:46 --> 00:42:49
The point is,
if I know what the

661
00:42:49 --> 00:42:55
transformation does to those
eight basis vectors,

662
00:42:55 --> 00:42:57
I know it completely.

663
00:42:57 --> 00:43:04
I know T, I know everything
about T, I know T completely

664
00:43:04 --> 00:43:11
from knowing T of V -- what T
does to v1, what T does to v2,

665
00:43:11 --> 00:43:15
what T does to v8.

666
00:43:15 --> 00:43:16
Why is that?

667
00:43:16 --> 00:43:18
It's because T is a linear
transformation.

668
00:43:18 --> 00:43:23
So that if I know what these
outputs are -- so these are the

669
00:43:23 --> 00:43:26
inputs v1 up to v8,
these are the outputs from the

670
00:43:26 --> 00:43:29
transformation,
like everyone rotated,

671
00:43:29 --> 00:43:32
everyone projected,
whatever transformation I've

672
00:43:32 --> 00:43:36
done, then why is it that I know
everything?

673
00:43:36 --> 00:43:41
How does linearity work?

674
00:43:41 --> 00:43:42
Why?

675
00:43:42 --> 00:43:53
This is because every x is some
combination of these basis

676
00:43:53 --> 00:44:00
vectors, right?
c1v1, c2v2, c8v8,

677
00:44:00 --> 00:44:05
they were a basis.

678
00:44:05 --> 00:44:10
That's the whole point of a
basis, that every vector is a

679
00:44:10 --> 00:44:16
combination of the basis vectors
in exactly one way.

680
00:44:16 --> 00:44:18.66
And then, what is T of x?

681
00:44:18.66 --> 00:44:22
The point is,
I claim that we know T of x

682
00:44:22 --> 00:44:27
completely for every x,
because every x is a

683
00:44:27 --> 00:44:32.89
combination of those --
and now we use the linear

684
00:44:32.89 --> 00:44:38
transformation part to say that
the output from x has to be c1

685
00:44:38 --> 00:44:44
times the output from v1 plus v2
times the output from v2,

686
00:44:44 --> 00:44:45
and so on.

687
00:44:45 --> 00:44:49
Up through c8 times the output
from v8.

688
00:44:49 --> 00:44:52
So this is like just saying,
OK.

689
00:44:52 --> 00:44:57
We know everything when we know
what T does to each basis

690
00:44:57 --> 00:44:59
vector.

691
00:44:59 --> 00:44:59
OK.

692
00:44:59 --> 00:45:05.49
So those are the eight things
we need.

693
00:45:05.49 --> 00:45:12
Now -- but we need these
answers in this basis.

694
00:45:12 --> 00:45:21
So this first output is some
combination of the eight basis

695
00:45:21 --> 00:45:22
vectors.

696
00:45:22 --> 00:45:29
So write T acting on the first
input --

697
00:45:29 --> 00:45:35
in other words,
write the first output as a

698
00:45:35 --> 00:45:42
combination of the basis
vectors, say a11 v1 + a21 v2 and

699
00:45:42 --> 00:45:44.71
so on a81 v8.

700
00:45:44.71 --> 00:45:50
Write T of v2 as some
combination a12 of v1,

701
00:45:50 --> 00:45:53.38
a22 of v2 and so on.

702
00:45:53.38 --> 00:46:00
I'm creating the matrix A,
column by column.

703
00:46:00 --> 00:46:06
Those numbers go in the first
column, these numbers go in the

704
00:46:06 --> 00:46:10
second column,
the matrix A that thi- this --

705
00:46:10 --> 00:46:15
this is our matrix that
represents T in this basis is

706
00:46:15 --> 00:46:18
these numbers.
a11 down to a18,

707
00:46:18 --> 00:46:21
a21 down to a28,
and so on.

708
00:46:21 --> 00:46:21
OK.

709
00:46:21 --> 00:46:23
That's the recipe.

710
00:46:23 --> 00:46:27
In other words,
if I give you a transformation,

711
00:46:27 --> 00:46:30
and a basis.

712
00:46:30 --> 00:46:33
So that's what I have to give
you.

713
00:46:33 --> 00:46:40
The inputs are the basis and to
tell you what the transformation

714
00:46:40 --> 00:46:40
is.

715
00:46:40 --> 00:46:46
And then, you tell me -- you
compute T for each basis,

716
00:46:46 --> 00:46:51
expand that result in the
basis, and that gives you the

717
00:46:51 --> 00:46:57
sixty four numbers that go into
the matrix A.

718
00:46:57 --> 00:47:06
Let me suppose -- let's close
with the best example of all.

719
00:47:06 --> 00:47:11
Suppose v1 to v8,
this basis, is the

720
00:47:11 --> 00:47:13
eigenvectors.

721
00:47:13 --> 00:47:22
Suppose we have an eigenvector
basis so that T(vi) is in the

722
00:47:22 --> 00:47:25
same direction of vi.

723
00:47:25 --> 00:47:29
Now, my question is,
what is A?

724
00:47:29 --> 00:47:36
Can you carry through the
steps?

725
00:47:36 --> 00:47:40
Let's do them together,
because we can do it in one

726
00:47:40 --> 00:47:41
minute.

727
00:47:41 --> 00:47:44
So, we've chosen this perfect
basis.

728
00:47:44 --> 00:47:47
And, actually,
with signal image processing,

729
00:47:47 --> 00:47:50
they might look for the
eigenvectors.

730
00:47:50 --> 00:47:55
But that would take more
calculation time that just

731
00:47:55 --> 00:47:59
saying, OK, we'll use the
wavelet basis.

732
00:47:59 --> 00:48:03
Or, OK, we'll use the Fourier
basis.

733
00:48:03 --> 00:48:08
But the very best basis is the
eigenvector basis.

734
00:48:08 --> 00:48:10
OK, what's the matrix?

735
00:48:10 --> 00:48:15.19
So, what's the first column of
the matrix?

736
00:48:15.19 --> 00:48:18
How do I get the first column?

737
00:48:18 --> 00:48:22
I take the first basis vector
v1.

738
00:48:22 --> 00:48:28
I opt -- I look to see,
what does the transformation do

739
00:48:28 --> 00:48:29
to it?

740
00:48:29 --> 00:48:32
The output is lambda one v1.

741
00:48:32 --> 00:48:38
I express that output as a
combination so the first input

742
00:48:38 --> 00:48:38
is v1.

743
00:48:38 --> 00:48:41
Its output is lambda one v1.

744
00:48:41 --> 00:48:47
Now write lambda one v1 as a
combination of the basis

745
00:48:47 --> 00:48:51
vectors, well,
it's already done.

746
00:48:51 --> 00:48:59
It's just lambda one times the
first basis vector and zero

747
00:48:59 --> 00:49:01
times the others.

748
00:49:01 --> 00:49:08
So this first column will have
lambda one and zeroes.

749
00:49:08 --> 00:49:08
OK.

750
00:49:08 --> 00:49:10
Second input is v2.

751
00:49:10 --> 00:49:13
Output is lambda two v2.

752
00:49:13 --> 00:49:20
OK, write that output as a
combination of the (v)s.

753
00:49:20 --> 00:49:23
It's already done.

754
00:49:23 --> 00:49:27
It's just lambda two times the
second v.

755
00:49:27 --> 00:49:32
So we need, in the second
column, we have lambda two times

756
00:49:32 --> 00:49:33
the second v.

757
00:49:33 --> 00:49:37
Well, you see what's coming,
that in that basis,

758
00:49:37 --> 00:49:42
in the eigenvector basis,
the matrix is diagonal.

759
00:49:42 --> 00:49:47
So that's the perfect basis,
that's the basis we'd love to

760
00:49:47 --> 00:49:52
have for image processing,
but to find the eigenvectors of

761
00:49:52 --> 00:49:56
our pixel matrix would be too
expensive.

762
00:49:56 --> 00:50:01
So we do something cheaper and
close, which is to choose a good

763
00:50:01 --> 00:50:03
basis like wavelets.

764
00:50:03 --> 00:50:04
OK, thanks.

765
00:50:04 --> 00:50:08
So I'll -- quiz review on
Wednesday, all day.

766
00:50:08 --> 00:50:11
Thanks.