Lecture notes for "Applied Numerical Linear Algebra", Fall 2024

Lectures will be recorded, and posted in .mp4 (video) format on bcourses.berkeley.edu, These lectures will be recordings of me speaking, writing on a "virtual whiteboard", and occasionally displaying prepared figures, powerpoint, or doing live Matlab demonstrations. We will try to post typed course notes before each lecture (in .txt and .pdf), and a pdf of the "virtual white board" after each on-line class meeting. One "lecture" covers one topic, which might take more than one 80-minute class meeting. You can find all the notes and virtual white boards from the last offering in Fall 23 here.

In addition, a student generously typed up all the class notes from a recent offering in latex. These notes, which may turn into a new edition of the textbook, have not been completely proofread, so are only posted on bcourses, under files as MATH_221_Lecture_Notes.pdf. Suggested comments or corrections are welcome, in any of these materials!

If I notice any minor mistakes in the virtual whiteboard after recording, I will correct them before posting, using red ink to make changes easier to see. I will also indicate below the relevant sections of the textbook to read.
  • Aug 29: Lecture 1: Course Outline
  • Typed notes, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, in .pdf (5.3MB)
  • Textbook: read sections 1.1, 1.2, 1.3
  • Sep 3: Lecture 2: Finish Course Outline, start Floating Point Arithmetic and Error Analysis
  • Finish typed notes above, and start new ones in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, in .pdf (4MB)
  • Textbook: read sections 1.4, 1.5, 1.6
  • Sep 5: Lecture 3: Continue Floating Point Arithmetic and Error Analysis
  • Same typed notes as above, in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, in .pdf (3.3MB)
  • Textbook: read sections 1.4, 1.5, 1.6
  • Sep 10: Lecture 4: Finish Floating Point Arithmetic and Error Analysis; Start Norms, the SVD and condition numbers for Ax=b
  • Finish typed notes above, start new ones in .txt and .pdf,
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, in .pdf (4.5MB) (Note: some mathematical corrections added after lecture, in use of Newton's method, highlighted in red)
  • Textbook: read sections 1.7, 3.2.3, 2.2 (2.2.1, 2.4.3 and 2.4.4 are optional)
  • Sep 12: Lecture 5: Continue Norms, the SVD and condition numbers for Ax=b
  • Same typed notes as above, in .txt and .pdf, (Matlab demo updated Sep 14)
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, in .pdf (4.2MB)
  • Textbook: read sections 1.7, 3.2.3, 2.2 (2.2.1, 2.4.3 and 2.4.4 are optional)
  • Sep 17: Lecture 6: Finish Norms, the SVD and condition numbers for Ax=b; Start Real cost of an algorithm, and Matrix Multiplication
  • Finish typed notes above, start new ones in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu.
  • Recorded virtual whiteboard, in .pdf (4.1MB)
  • To see the plot of computer hardware speeds over time, click here. The data being plotted is taken from Fig 1.10 of "Computer Architecture: A Quantitative Approach", Patterson and Hennessy, 2019.
  • Textbook: read sections 2.6.1 and 2.6.2. See also links on the class web page under "References for Communication-Avoiding Algorithms" for more recent results, and the scribed notes on bcourses.
  • Sep 19: Lecture 7: Finish Real cost of an algorithm, and Matrix Multiplication; Start Gaussian Elimination
  • Finish typed notes above, start new ones in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (4MB)
  • Textbook: read sections 2.3, 2.4.1, 2.4.2, 2.4.4, 2.5, 2.6.3
  • Sep 24: Lecture 8: Continue Gaussian Elimination
  • Same typed notes as above, in .txt and .pdf (Correction to proof of Thm (backward error analysis of LU):When i>j we get L(i,j) by dividing the sum by U(j,j), not A(i,i).)
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (4.6MB)
  • Textbook: read sections 2.3, 2.4.1, 2.4.2, 2.4.4, 2.5, 2.6.3
  • Sep 26: Lecture 9: Finish Gaussian Elimination, Start Gaussian Elimination for Matrices with Special Structure
  • Finish typed notes above, start new ones in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (4MB)
  • Textbook: read sections 2.3, 2.4.1, 2.4.2, 2.4.4, 2.5, 2.6.3
  • Oct 1: Lecture 10: Continue Gaussian Elimination for Matrices with Special Structure
  • Same typed notes as above, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (3MB) (does not include many Matlab "spyplots" of sparse matrices in the video recording, most appear in section 2.7 of the textbook)
  • Textbook: read section 2.7
  • Oct 3: Lecture 11: Finish Gaussian Elimination for Matrices with Special Structure, Start Least Squares
  • Finish typed notes above, start new ones in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (4MB)
  • Textbook: read sections 3.1-3.4
  • Oct 8: Lecture 12: Continue Least Squares
  • Same typed notes as above, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (4MB)
  • Textbook: read sections 3.1-3.4
  • Oct 10: Lecture 13: Finish Least Squares, start Low-Rank Matrices
  • Finish typed notes above, start new ones in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (4.3MB)
  • Textbook: read section 3.5. See also References for Randomized Algorithms on the class web page.
  • Oct 15: Lecture 14: Continue Low-Rank Matrices, including Randomized Linear Algebra
  • Same typed notes as above, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (4.3MB)
  • Textbook: read section 3.5. See also References for Randomized Algorithms on the class web page.
  • Oct 17: Lecture 15: Finish Low-Rank Matrices and Randomized Linear Algebra, including Randomized Linear Algebra, start Eigenvalue Problems
  • Finish typed notes as above, start new one in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (4.2MB)
  • Textbook: read Chap 4.
  • Oct 22: Lecture 16: Continue Eigenvalue Problems
  • Same typed notes as above, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (4.5MB)
  • Textbook: read Chap 4.
  • Oct 24: Lecture 17: Continue Eigenvalue Problems
  • Same typed notes as above, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (3.5MB)
  • Textbook: read Chap 4.
  • Oct 29: Lecture 18: Finish Eigenvalue Problems, start Symmetric Eigenvalue Problems and SVD
  • Finish typed notes as above, start new one in .txt and .pdf
  • Matlab code to explain quadratic convergence of shifted QR iteration
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (3.6MB)
  • Textbook: read Chap 5.
  • Oct 31: Lecture 19: Continue Symmetric Eigenvalue Problems and SVD
  • Same typed notes as above, .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (4.3MB)
  • Textbook: read Chap 5.
  • Nov 5: Lecture 20: Continue Symmetric Eigenvalue Problems and SVD
  • Same typed notes as above, .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (4.1MB)
  • Textbook: read Chap 5.
  • Nov 7: Lecture 21: Start Introduction to Iterative Methods for Ax=b and Ax = lambda x
  • Typed notes in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu (first 17 minutes of sound not recorded, sorry)
  • Recorded virtual whiteboard, .pdf (4.2MB)
  • Textbook: read Sections 6.1 through 6.4
  • Nov 12: Lecture 22: Finish Introduction to Iterative Methods for Ax=b and Ax = lambda x, start Splitting Methods
  • Finish typed notes as above, start new ones in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (4.6MB)
  • Textbook: read Sections 6.5.1 through 6.5.5
  • Nov 14: Lecture 23: Finish Splitting Methods
  • Same typed notes as above, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (4.4MB)
  • Textbook: read Sections 6.5.1 through 6.5.5
  • Nov 19: Lecture 24: Multigrid
  • Powerpoint slides for Multigrid, in .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Textbook: read Section 6.9
  • Nov 21: Lecture 25: Begin Krylov Subspace Methods (KSMs) for A*x=b and A*x=lambda*x, then begin GMRES and CG for solving A*x=b
  • Typed notes for Krylov Subspace Methods, in .txt and .pdf, then for GMRES and CG, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (3.6MB)
  • Textbook: read Sections 6.6.1, 6.6.2
  • Nov 26: Lecture 26: Finish GMRES and CG for solving A*x=b
  • Typed notes as above, in .txt and .pdf
  • Recorded lecture: posted on bcourses.berkeley.edu
  • Recorded virtual whiteboard, .pdf (3.2MB)
  • Textbook: read Section 6.6.3