Lecture notes for "Applied Numerical Linear Algebra", Spring 2022

Lectures will be on-line (URL provided via bcourses), recorded, and posted in .mp4 (video) format on bcourses.berkeley.edu, through Jan 28, back in person after that. These lectures will be recordings of me speaking, writing on a "virtual whiteboard", and occasionally displaying prepared figures, powerpoint, or doing live Matlab demonstrations. We will try to post typed course notes before each lecture (in .txt and .pdf), and a pdf of the "virtual white board" after each on-line class meeting. One "lecture" covers one topic, which might take more or less than one 80-minute class meeting. You can find all the notes and virtual white boards from the last offering in Fall 20 here.

In addition, a student generously typed up all the class notes from this last offering in latex. These notes, which may turn into a new edition of the textbook, have not been proofread, so are only posted on bcourses. Suggested comments or corrections are welcome, in any of these materials!

If I notice any minor mistakes in the virtual whiteboard after recording, I will correct them before posting, using red ink to make changes easier to see. I will also indicate below the relevant sections of the textbook to read.
  • Jan 18: Lecture 1: Course Outline
  • Typed notes, in .txt and .pdf (updated 1/18, 6:09am)
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, in .pdf (7MB)
  • Textbook: read sections 1.1, 1.2, 1.3
  • Jan 20: Lecture 2: Complete Lecture 1, then Floating Point Arithmetic and Error Analysis
  • Typed notes, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, in .pdf (6MB)
  • Textbook: read sections 1.4, 1.5, 1.6
  • Jan 25: Lecture 3: Complete Floating Point Arithmetic and Error Analysis
  • Typed notes (updated Jan 23), in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, in .pdf (7MB)
  • Textbook: read sections 1.4, 1.5, 1.6
  • Jan 27: Lecture 4: Norms, the SVD, and condition numbers for Ax=b
  • Typed notes, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu. Because of a problem with today's recording, I posted the corresponding recordings from the Fall 2020 semester, broken up into 4 segments of lengths 22, 16, 22 and 18 minutes, with names Ma221_Fall20_Lecture_03_01 through Ma221_Fall20_Lecture_03_04. The last few minutes of the 4th segment include a few topics that will be presented in the Feb 1 lecture.
  • Recorded virtual whiteboard, in .pdf (6MB)
  • Textbook: read sections 1.7, 3.2.3, 2.2 (2.2.1, 2.4.3 and 2.4.4 are optional)
  • Feb 1: Lecture 5: Complete Norms, the SVD, and condition numbers for Ax=b
  • Typed notes, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (5MB)
  • Textbook (as above): read sections 1.7, 3.2.3, 2.2 (2.2.1, 2.4.3 and 2.4.4 are optional)
  • Feb 3: Lecture 6: Real cost of an algorithm, and Matrix Multiplication
  • Typed notes, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (6MB)
  • To see the plot of computer hardware speeds over time, click here. The data being plotted is taken from Fig 1.10 of "Computer Architecture: A Quantitative Approach", Patterson and Hennessy, 2019.
  • Textbook: read sections 2.6.1 and 2.6.2. See also links on the class web page under "References for Communication-Avoiding Algorithms" for more recent results.
  • Feb 8: Lecture 7: Finish Matrix Multiply, start Gaussian Elimination
  • Typed notes, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (5MB)
  • Textbook: read sections 2.3, 2.4.1, 2.4.2, 2.4.4, 2.6.3
  • Feb 10: Lecture 8: Continue Gaussian Elimination
  • Typed notes, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (5MB)
  • Textbook: read sections 2.3, 2.4.1, 2.4.2, 2.4.4, 2.5, 2.6.3
  • Feb 15: Lecture 9: Finish Gaussian Elimination, begin Gaussian Elimination for matrices with special structure. Because of an Apple pencil malfunction, we will post the recorded lectures and virtual whiteboards from Fall 2020.
  • Typed notes, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu. This is broken into 2 segments of lengths 26 and 38 minutes, with names Ma221_Fall20_Lecture_05_05 and Ma221_Fall20_Lecture_06_01, corresponding to the two recorded virtual whiteboards below.
  • Recorded virtual whiteboards, from Fall 20: Part 1: Avoiding Communication in Gaussian Elimation (pdf) (4MB) and Part 2: Intro to Gaussian Elimation for matrices with special structure (pdf) (5MB).
  • Textbook: read section 2.7
  • Feb 17: Lecture 10: Continue Gaussian Elimination for matrices with special structure.
  • Typed notes, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (4MB)
  • Textbook: read section 2.7
  • Feb 22: Lecture 11: Finish Gaussian Elimination for matrices with special structure, start Least Squares
  • Typed notes, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (6MB)
  • Matlab code NestedDissection.m used in lecture
  • Updated survey of sparse direct linear equation solvers used in lecture
  • Textbook: read sections 3.1-3.4
  • Feb 24: Lecture 12: Continue Least Squares
  • Typed notes, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (5MB)
  • Textbook: read sections 3.1-3.4
  • Mar 1: Lecture 13: Finish Least Squares, start Low-Rank Matrices
  • Typed notes, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (6MB)
  • Textbook: read section 3.5. See also the References for Randomized Algorithms on the class web page.
  • Mar 3: Lecture 14: Continue Low-Rank Matrices
  • Typed notes, in .txt and .pdf, Updated Mar 2, 8pm
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (6MB)
  • Textbook: read section 3.5. See also the References for Randomized Algorithms on the class web page.
  • Mar 8: Lecture 15: Finish Low-Rank Matrices
  • Typed notes, in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (6MB)
  • Textbook: read section 3.5. See also the References for Randomized Algorithms on the class web page.
  • Mar 10: Lecture 16: Start Eigenvalue Problems. Because of a recording glitch, we will post the recorded lectures from Fall 2020, and their corresponding recorded whiteboards, as well as today's recorded whiteboard.
  • Typed notes, in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu. This is broken into 3 segments of lengths 22, 17 and 23 minutes, with names Ma221_Fall20_Lecture_09_{01,02,03}, and the corresponding recorded virtual whiteboard here (pdf),
  • Recorded virtual whiteboard from Spr 22, in .pdf (6MB).
  • Textbook: read Chap 4.
  • Mar 15: Lecture 17: Continue Eigenvalue Problems.
  • Typed notes, in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard from Spr 22, in .pdf (6MB).
  • Textbook: read Chap 4.
  • Mar 17: Lecture 18: Finish Eigenvalue Problems.
  • Typed notes, in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard from Spr 22, in .pdf (5MB).
  • Textbook: read Chap 4.
  • Mar 29: Lecture 19: Start Symmetric Eigenvalue Problems and SVD.
  • Typed notes, in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard from Spr 22, in .pdf (5MB).
  • Textbook: read Chap 5.
  • Mar 31: Lecture 20: Continue Symmetric Eigenvalue Problems and SVD.
  • Typed notes, in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard from Spr 22, in .pdf (5.5MB).
  • Textbook: read Chap 5.
  • Apr 5: Lecture 21: Finish Symmetric Eigenvalue Problems and SVD,
    Start Introduction to Iterative Methods for Ax=b and Ax=lambda x
  • Typed notes, in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (5.5MB).
  • Textbook: read sections 6.1 through 6.4.
  • Apr 7: Lecture 22: Finish Introduction to Iterative Methods for Ax=b and Ax=lambda x
  • Typed notes, in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (5.5MB).
  • Textbook: read sections 6.1 through 6.4.
  • Apr 12: Lecture 23: Start Splitting Methods
  • Typed notes, in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (5.5MB).
  • Textbook: read sections 6.5.1 through 6.5.5.
  • Apr 14: Lecture 24: Finish Splitting Methods, start Multigrid
  • Powerpoint slides, in .pdf
  • Recorded virtual whiteboard, in .pdf (5.5MB).
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Textbook: read section 6.9
  • Apr 19: Lecture 25: Finish Multigrid, start Krylov Subspace Methods (KSMs) for A*x=b and Ax=lambda*x: Introduction: Arnoldi and Lanczos
  • Typed notes, in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (3.5MB).
  • Textbook: read section 6.6 through the end of 6.6.1
  • Apr 21: Lecture 26: Krylov Subspace Methods: GMRES and CG for solving Ax=b
  • Typed notes, in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (5MB).
  • Textbook: read sections Chap 6.6.2, 6.6.3 and 6.6.6.
  • Apr 26: Lecture 27: Chebyshev Polynomials: Convergence Analysis of CG, and Accelerating SOR
  • Typed notes, in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (3.5MB).
  • Textbook: read sections Chap 6.5.6, 6.6.4
  • Apr 28: Lecture 28: Preconditioning, and Communication-Avoiding Sparse Iterative Methods
  • Preconditioning: typed notes, in .txt and .pdf,
  • Communication-Avoiding Sparse Iterative Methods: slides, in .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (2.5MB).
  • Textbook: read sections Chap 6.6.5 and 6.10