Lecture 1: Course Overview, Newton's Method for RootFinding

Square Roots via Newton's Method (PDF)
Square Roots

Problem Set 1 (PDF)
Problem set 1 notebook
Solutions to Problem Set 1 (PDF)
Solutions to Problem Set 1 notebook

Lecture 2: FloatingPoint Arithmetic

FloatingPoint Arithmetic, the IEEE Standard (PDF) (Courtesy of PerOlof Persson. Used with permission.)
FloatingPoint Arithmetic
Some Myths about FloatingPoint Arithmetic (PDF)
Julia & IJulia CheatSheet (PDF)
Introduction to Julia (PDF)
Julia for Numerical Computation in MIT Courses

[No problem set]

Lecture 3: FloatingPoint Summation and Backwards Stability

Notes on the Accuracy of Naive Summation (PDF)
Backwards Stability of Recursive Summation (PDF)

[No problem set]

Lecture 4: Norms on Vector Spaces

Notes on the Equivalence of Norms (PDF)

[No problem set]

Lecture 5: Condition Numbers

[No handout/notebook]

Problem set 2 (PDF)
Solutions to Problem Set 2 (PDF)

Lecture 6: Numerical Methods for Ordinary Differential Equations

Modern Differential Equations Solver Software: Where We Are and Where We're Headed (PDF  2.4MB) (Courtesy of Christopher Rackauckas. Used with permission.)

[No problem set]

Lecture 7: The SVD, its Applications, and Condition Numbers

[No handout/notebook]

[No problem set]

Lecture 8: Linear Regression and the Generalized SVD

Many Viewpoints on Linear Regression

[No problem set]

Lecture 9: Solving the Normal Equations by QR and GramSchmidt

[No handout/notebook]

[No problem set]

Lecture 10: Modified GramSchmidt and Householder QR

Householder Reflectors and Givens Rotations (PDF) (Courtesy of PerOlof Persson. Used with permission.)
Classical vs. Modified GramSchmidt

[No problem set]

Lecture 11: Matrix Operations, Caches, and Blocking

Performance Experiments with Matrix Multiplication (PDF)
IdealCache Terminology (PDF)

Problem set 3 (PDF)
Solutions to Problem Set 3 (PDF)

Lecture 12: CacheOblivious Algorithms and Spatial Locality

Experiments with CacheOblivious Matrix Multiplication (PDF)
Experiments with Memory Access and Matrices

[No problem set]

Lecture 13: LU Factorization and Partial Pivoting

[No handout/notebook]

[No problem set]

Lecture 14: Cholesky Factorization and other Specialized Solvers. Eigenproblems and Schur Factorizations

[No handout/notebook]

[No problem set]

Lecture 15: Eigensolver Algorithms: Companion Matrices, IllConditioning, and Hessenberg Factorization

Hessenberg Factorization (PDF)

[No problem set]

Lecture 16: The Power Method and the QR Algorithm

[No handout/notebook]

[No problem set]

Lecture 17: Shifted QR and Rayleigh Quotients

[No handout/notebook]

[No problem set]

Lecture 18: Krylov Methods and the Arnoldi Algorithm

[No handout/notebook]

[No problem set]

Lecture 19: Arnoldi and Lanczos with Restarting

Why Restarting Arnoldi/Lanczos is not Trivial (PDF)
Experiments with Arnoldi Iterations

[No problem set]

Lecture 20: The GMRES Algorithm and Convergence of GMRES and Arnoldi

[No handout/notebook]

Problem set 4 (PDF)
Solutions to Problem Set 4 (PDF)

Lecture 21: Preconditioning Techniques. The ConjugateGradient Method

LargeScale Linear Algebra: Dense Matrix Methods

[No problem set]

Lecture 22: Convergence of Conjugate Gradient

[No handout/notebook]

[No problem set]

Lecture 23: Biconjugate Gradient Algorithms

[No handout/notebook]

[No problem set]

Lecture 24: SparseDirect Solvers

Sparse Matrix Algorithms (PDF) (Courtesy of PerOlof Persson. Used with permission.)
SparseDirect Solvers in Julia

[No problem set]

Lecture 25: Overview of Optimization Algorithms

A Brief Overview of Optimization Problems (PDF)

Midterm exam (PDF)
Solutions to midterm exam (PDF)
Midterm exams and solutions from previous years

Lecture 26: Adjoint Methods

Adjoint Methods (PDF)

[No problem set]

Lecture 27: Adjoint Methods for Eigenproblems and Recurrence Relations

Adjoint Methods and Sensitivity Analysis for Recurrence Relations (PDF)
Recurrence Relation

[No problem set]

Lecture 28: TrustRegions Methods and the CCSA Algorithm

[No handout/notebook]

[No problem set]

Lecture 29: Lagrange Dual Problems

Lagrangian, Lagrange Dual Function and Dual Problem

[No problem set]

Lecture 30: QuasiNewton Methods and the BFGS Algorithm
Lecture 31: Derivation of the BFGS Update

QuasiNewton Optimization: Origin of the BFGS Update (PDF)

[No problem set]

Lecture 32: DerivativeFree Optimization by Linear and Quadratic Approximations

[No handout/notebook]

[No problem set]

Lecture 33: Numerical Integration and the Convergence of the Trapezoidal Rule

Numerical Integration and the Redemption of the Trapezoidal Rule (PDF)
Fourier Cosine Series Examples (PDF)

[No problem set]

Lecture 34: ClenshawCurtis Quadrature

[No handout/notebook]

[No problem set]

Lecture 35: Chebyshev Approximation

[No handout/notebook]

[No problem set]

Lecture 36: Integration with Weight Functions, and Gaussian Quadrature

[No handout/notebook]

[No problem set]

Lecture 37: Adaptive and Multidimensional Quadrature

[No handout/notebook]

[No problem set]

Lecture 38: The Discrete Fourier Transform (DFT) and FFT Algorithms

Fast Fourier Transform Algorithms (PDF)

[No problem set]

Lecture 39: FFT Algorithms and FFTW

Fast Fourier Transform and Fast Fourier Transform in the West (PDF  2.6MB)

[No problem set]
