18.335J | Spring 2019 | Graduate

Introduction to Numerical Methods

Resource Index

This resource index gives users access to most of the course resources in a single location.

Lecture 1: Course Overview, Newton’s Method for Root-Finding

Square Roots via Newton’s Method (PDF)

Square Roots

Problem Set 1 (PDF)

Problem set 1 notebook

Solutions to Problem Set 1 (PDF)

Solutions to Problem Set 1 notebook

Lecture 2: Floating-Point Arithmetic

Floating-Point Arithmetic, the IEEE Standard (PDF) (Courtesy of Per-Olof Persson. Used with permission.)

Floating-Point Arithmetic

Some Myths about Floating-Point Arithmetic (PDF)

Julia & IJulia Cheat-Sheet (PDF)

Introduction to Julia (PDF)

Julia for Numerical Computation in MIT Courses

[No problem set]
Lecture 3: Floating-Point Summation and Backwards Stability

Notes on the Accuracy of Naive Summation (PDF)

Backwards Stability of Recursive Summation (PDF)

[No problem set]
Lecture 4: Norms on Vector Spaces Notes on the Equivalence of Norms (PDF) [No problem set]
Lecture 5: Condition Numbers [No handout/notebook]

Problem set 2 (PDF)

Solutions to Problem Set 2 (PDF)

Lecture 6: Numerical Methods for Ordinary Differential Equations Modern Differential Equations Solver Software: Where We Are and Where We’re Headed (PDF - 2.4MB) (Courtesy of Christopher Rackauckas. Used with permission.) [No problem set]
Lecture 7: The SVD, its Applications, and Condition Numbers [No handout/notebook] [No problem set]
Lecture 8: Linear Regression and the Generalized SVD Many Viewpoints on Linear Regression [No problem set]
Lecture 9: Solving the Normal Equations by QR and Gram-Schmidt [No handout/notebook] [No problem set]
Lecture 10: Modified Gram-Schmidt and Householder QR

Householder Reflectors and Givens Rotations (PDF) (Courtesy of Per-Olof Persson. Used with permission.)

Classical vs. Modified Gram-Schmidt

[No problem set]
Lecture 11: Matrix Operations, Caches, and Blocking

Performance Experiments with Matrix Multiplication (PDF)

Ideal-Cache Terminology (PDF)

Problem set 3 (PDF)

Solutions to Problem Set 3 (PDF)

Lecture 12: Cache-Oblivious Algorithms and Spatial Locality

Experiments with Cache-Oblivious Matrix Multiplication (PDF)

Experiments with Memory Access and Matrices

[No problem set]
Lecture 13: LU Factorization and Partial Pivoting [No handout/notebook] [No problem set]
Lecture 14: Cholesky Factorization and other Specialized Solvers. Eigenproblems and Schur Factorizations [No handout/notebook] [No problem set]
Lecture 15: Eigensolver Algorithms: Companion Matrices, Ill-Conditioning, and Hessenberg Factorization Hessenberg Factorization (PDF) [No problem set]
Lecture 16: The Power Method and the QR Algorithm [No handout/notebook] [No problem set]
Lecture 17: Shifted QR and Rayleigh Quotients [No handout/notebook] [No problem set]
Lecture 18: Krylov Methods and the Arnoldi Algorithm [No handout/notebook] [No problem set]
Lecture 19: Arnoldi and Lanczos with Restarting

Why Restarting Arnoldi/Lanczos is not Trivial (PDF)

Experiments with Arnoldi Iterations

[No problem set]
Lecture 20: The GMRES Algorithm and Convergence of GMRES and Arnoldi [No handout/notebook]

Problem set 4 (PDF)

Solutions to Problem Set 4 (PDF)

Lecture 21: Preconditioning Techniques. The Conjugate-Gradient Method Large-Scale Linear Algebra: Dense Matrix Methods [No problem set]
Lecture 22: Convergence of Conjugate Gradient [No handout/notebook] [No problem set]
Lecture 23: Biconjugate Gradient Algorithms [No handout/notebook] [No problem set]
Lecture 24: Sparse-Direct Solvers

Sparse Matrix Algorithms (PDF) (Courtesy of Per-Olof Persson. Used with permission.)

Sparse-Direct Solvers in Julia

[No problem set]
Lecture 25: Overview of Optimization Algorithms A Brief Overview of Optimization Problems (PDF)

Midterm exam (PDF)

Solutions to midterm exam (PDF)

Midterm exams and solutions from previous years

Lecture 26: Adjoint Methods Adjoint Methods (PDF) [No problem set]
Lecture 27: Adjoint Methods for Eigenproblems and Recurrence Relations

Adjoint Methods and Sensitivity Analysis for Recurrence Relations (PDF)

Recurrence Relation

[No problem set]
Lecture 28: Trust-Regions Methods and the CCSA Algorithm [No handout/notebook] [No problem set]
Lecture 29: Lagrange Dual Problems Lagrangian, Lagrange Dual Function and Dual Problem [No problem set]

Lecture 30: Quasi-Newton Methods and the BFGS Algorithm

Lecture 31: Derivation of the BFGS Update

Quasi-Newton Optimization: Origin of the BFGS Update (PDF) [No problem set]
Lecture 32: Derivative-Free Optimization by Linear and Quadratic Approximations [No handout/notebook] [No problem set]
Lecture 33: Numerical Integration and the Convergence of the Trapezoidal Rule

Numerical Integration and the Redemption of the Trapezoidal Rule (PDF)

Fourier Cosine Series Examples (PDF)

[No problem set]
Lecture 34: Clenshaw-Curtis Quadrature [No handout/notebook] [No problem set]
Lecture 35: Chebyshev Approximation [No handout/notebook] [No problem set]
Lecture 36: Integration with Weight Functions, and Gaussian Quadrature [No handout/notebook] [No problem set]
Lecture 37: Adaptive and Multidimensional Quadrature [No handout/notebook] [No problem set]
Lecture 38: The Discrete Fourier Transform (DFT) and FFT Algorithms Fast Fourier Transform Algorithms (PDF) [No problem set]
Lecture 39: FFT Algorithms and FFTW Fast Fourier Transform and Fast Fourier Transform in the West (PDF - 2.6MB) [No problem set]

Course Info

As Taught In
Spring 2019
Learning Resource Types
Problem Sets with Solutions
Exams with Solutions
Lecture Notes