MAT 321 - Numerical Analysis and Scientific Computing - Fall 2025

Instructor: Marc Aurèle Gilles

---

Course Description

I like David Bindel's description of his numerical analysis class: "Scientists, engineers, mathematicians, and computer scientists use models to describe everything from the ringing of bells to the evolution of animal populations to the relationships between web pages. We turn to computers to help us analyze all but the simplest such models; but how can an inherently discrete device such as a computer solve continuous problems quickly and reliably? This is the fundamental question we address in (...)" MAT321.

More concretely, we cover the fundamental algorithms of computational mathematics: matrix decompositions, eigenvalue iterations, iterative linear algebra, optimization, interpolation, numerical quadrature, and more. We analyze the stability and accuracy of these algorithms and explore their applications to image processing, data science, and the numerical solution of differential equations. Along the way, we cover the majority of the top 10 algorithms of the 20th century.

In lectures, we cover the derivation and theory of algorithms, often illustrated by numerical experiments. In problem sets, we discuss applications and derive and implement additional algorithms. Exams are proof-based.

Problem Sets

• Stability, rootfinding [hw1.ipynb]
• Finite difference methods, structured matrix-vector products, image alignment, Givens rotations [hw2.ipynb]
• Matrix equations, displacement rank, deblurring images, kernel matrices [hw3.ipynb]
• Eigenvalue problems, Bartels-Stewart, Poisson's equation, spectral embedding [hw4.ipynb]
• The FFT, Chebyshev polynomials, Gauss-Legendre, preconditioning Krylov methods, ADI [hw5.ipynb]

Precepts

• Rootfinding [precept1.ipynb]
• Finite-differences [precept2.ipynb]
• Matrix norms [precept3.ipynb]
• QR decomposition [precept4.ipynb]
• Midterm review
• LU and Sherman-Morrison-Woodbury [precept6.ipynb]
• Eigenvalue iterations [precept7.ipynb], extra questions
• Reduction to Hessenberg [precept8.ipynb], extra questions
• Krylov subspace methods [precept9.ipynb], extra questions
• Interpolation [precept10.ipynb], extra questions
• Numerical quadrature [precept11.ipynb], extra questions
• Initial value problems [precept12.ipynb], extra questions

Lecture Topics

• Lecture 1: Introduction, root-finding, bisection
  Reading: Süli & Mayers, Chapter 1

• Lecture 2: Newton's method, secant method, fixed point iterations
  Reading: Süli & Mayers, Chapter 1

• Lecture 3: Floating point arithmetic
  Reading: Note on floating point arithmetic (files/notes), Trefethen & Bau, Chapter 13

• Lecture 4: Intro to numerical linear algebra: complexity, matrices, vectors, norms
  Reading: Trefethen & Bau, Chapters 1-4

• Lecture 5: The singular value decomposition, the eigenvalue decomposition
  Reading: Trefethen & Bau, Chapters 4-6

• Lecture 6: The QR decomposition, Gram-Schmidt and Householder
  Reading: Trefethen & Bau, Chapters 7, 8, 10

• Lecture 7: Least-squares problem, ill-conditioning, and regularization
  Reading: Trefethen & Bau, Chapter 11; Note on ill-conditioning (files/notes)

• Lecture 8: The LU decomposition, pivoting
  Reading: Trefethen & Bau, Chapters 20, 21

• Lecture 9: The Cholesky decomposition, structured linear solvers
  Reading: Trefethen & Bau, Chapter 23; Note on structured linear systems (files/notes)

• Lecture 10: Rootfinding in high-dimensions, Gradient descent
  Reading: Notes

• Lecture 11: Newton and quasi-Newton in high dimensions
  Reading: Notes

• Lecture 12: Midterm

• Lecture 13: Intro to eigenvalue problems
  Reading: Trefethen & Bau, Chapters 24-26

• Lecture 14: Power iterations
  Reading: Trefethen & Bau, Chapter 27

• Lecture 15: Subspace iterations, the QR algorithm
  Reading: Trefethen & Bau, Chapters 28, 29

• Lecture 16: The Arnoldi and Lanczos iterations
  Reading: Trefethen & Bau, Chapters 32-34, 36

• Lecture 17: Iterative linear algebra, Krylov subspace methods
  Reading: Trefethen & Bau, Chapter 35

• Lecture 18: The conjugate-gradient method
  Reading: Trefethen & Bau, Chapter 38

• Lecture 19: Polynomial interpolation
  Reading: Süli & Mayers, Sections 6.1-6.3

• Lecture 20: Numerical quadrature
  Reading: Süli & Mayers, Chapter 7

• Lecture 21: Approximation theory
  Reading: Süli & Mayers, Chapters 8 and 9

• Lecture 22: Initial value problems, Euler's method
  Reading: Süli & Mayers, Sections 12.1-12.3

• Lecture 23: Runge-Kutta and implicit schemes
  Reading: Süli & Mayers, Sections 12.4-12.5

• Lecture 24: Recap, the top 10 algorithms
  Reading: Notes

Textbooks Used

• Numerical linear algebra by Lloyd N. Trefethen and David Bau
• An introduction to numerical analysis by Endre Süli and David F. Mayers

Other Resources

• A First Course in Numerical Methods by Uri M. Ascher and Chen Grief
• David Bindel's numerical analysis course notes
• Matrix Computations by Gene H. Golub and Charles F. Van Loan
• Numerical Algorithms: Methods for Computer Vision, Machine Learning, and Graphics by Justin Solomon
• Applied Numerical Linear Algebra by James W. Demmel
• Numerical Optimization by Jorge Nocedal and Stephen Wright

---

Note: Instructors interested in using these materials are welcome to contact me for source files and solutions.