Video Lectures

Lecture 32: ImageNet is a Convolutional Neural Network (CNN), The Convolution Rule

Description

Professor Strang begins the lecture talking about ImageNet, a large visual database used in visual object recognition software research. ImageNet is an example of a convolutional neural network (CNN). The rest of the lecture focuses on convolution.

Summary

Convolution matrices have \(\leq\) \(n\) parameters (not \(n\)2).
Fewer weights to compute in deep learning
Component \(k\) from convolution \(c*d\): Add all \(c(j)d(k-j)\)
Convolution Rule: \(F(c*d) = Fc\) times \(Fd\) (component by component)
\(F\) = Fourier matrix with \(j\), \(k\) entry \(= \exp (2 \pi i j k /n)\)

Related section in textbook: IV.2

Instructor: Prof. Gilbert Strang

Problems for Lecture 32
From textbook Section IV.2

4. Any two circulant matrices of the same size commute: \(CD=DC\). They have the same eigenvectors \(\boldsymbol{q}_k\) (the columns of the Fourier matrix \(F\)). Show that the eigenvalues \(\lambda_k(CD)\) are equal to \(\lambda_k(C)\) times \(\lambda_k(D)\).

Course Info

Departments
As Taught In
Spring 2018
Learning Resource Types
Lecture Videos
Problem Sets
Instructor Insights