Lecture notes for "Applied Numerical Linear Algebra", Fall 2020

All the lectures will be prerecorded and posted on bcourses.berkeley.edu, in .mp4 (video) format. These lectures will be recordings of me speaking, writing on a "virtual whiteboard", and occasionally displaying prepared figures, powerpoint, or doing live Matlab demonstrations.

One "lecture" will consist of a single topic, or set of closely related topics. But it will not necessarily take exactly 80 minutes to present. Instead, each lecture will be broken into several shorter recordings, each one covering a subtopic. For example, Lecture 1 is broken into 5 segments, labelled Ma221_Fall20_Lecture_01_01 through Ma221_Fall20_Lecture_01_05. These should be visible in bcourses.berkeley.edu under Media Gallery.

We will also post below two kinds of notes for each lecture: typed notes (in .txt and .pdf formats), and the complete virtual whiteboard of the prerecorded lecture (in .pdf). These notes should cover all the material in all the shorter recordings constituting one lecture. If I notice any minor mistakes in the virtual whiteboard after recording, I will correct them before posting, using red ink to make changes easier to see. I will also indicate below the relevant sections of the textbook to read.
  • Aug 27: Lecture 1: Course Outline
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 5 segments, of lengths 17, 43, 23, 27 and 34 minutes (total=144)
  • Recorded virtual whiteboard, in .pdf (22MB)
  • Textbook: read sections 1.1, 1.2, 1.3
  • Sep 1: Lecture 2: Floating Point Arithmetic and Error Analysis
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 5 segments, of lengths 18, 32, 29, 12, and 37 minutes (total=128). Segment 5 is optional, covering further details of floating point, and its impact on reliability of software and error analysis; segments 1 through 4 total 91 minutes.
  • Recorded virtual whiteboard, in .pdf (15MB)
  • Textbook: read sections 1.4, 1.5, 1.6
  • Sep 3 + Sep 8: Lecture 3: Norms, the SVD, and condition numbers for Ax=b
  • Typed notes, in .txt and .pdf (updated Sep 4, 11:20am)
  • Recorded lecture: broken into 7 segments, of lengths 22, 16, 22, 18, 23, 37 and 10 minutes (total = 148 minutes).
  • The topics in this lecture correspond to what would have been roughly 2 classroom lectures.
  • Recorded virtual whiteboard, in .pdf (18MB) (updated 9/17)
  • Textbook: read sections 1.7, 3.2.3, 2.2 (2.2.1, 2.4.3 and 2.4.4 are optional)
  • Sep 10: Lecture 4: Real cost of an algorithm, and Matrix Multiplication
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 5 segments, of lengths 28, 30, 17, 39 and 17 minutes (total = 131 minutes). Segment 5 is optional, covering Strassen's and related algorithms; segments 1 through 4 total 114 minutes.
  • The topics in this lecture correspond to what would have been roughly 1.5 classroom lectures.
  • Recorded virtual whiteboard, in .pdf (18MB)
  • To see the plot of computer hardware speeds over time, click here. The data being plotted is taken from Fig 1.10 of "Computer Architecture: A Quantitative Approach", Patterson and Hennessy, 2019.
  • Textbook: read sections 2.6.1 and 2.6.2. See also links on the class web page under "References for Communication-Avoiding Algorithms" for more recent results.
  • Sep 15 + Sep 17: Lecture 5: Gaussian Elimination
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 5 segments, of lengths 29, 23, 40, 21 and 27 minutes (total = 140 minutes).
  • The topics in this lecture correspond to what would have been roughly 1.5 classroom lectures.
  • Recorded virtual whiteboard, in .pdf (18MB) (updated 9/18)
  • Textbook: read sections 2.3, 2.4.1, 2.4.2, 2.4.4, 2.5, 2.6.3
  • Sep 22 + Sep 24: Lecture 6: Gaussian Elimination for matrices with special structures
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 5 segments, of lengths 38, 28, 27, 42 and 24 minutes (total = 159 minutes).
  • The topics in this lecture correspond to what would have been roughly 2 classroom lectures.
  • Recorded virtual whiteboard, in .pdf (19MB) (updated 9/21)
  • Matlab code NestedDissection.m used in lecture
  • Updated survey of sparse direct linear equation solvers used in lecture
  • Textbook: read section 2.7
  • Sep 29 + Oct 1: Lecture 7: Least Squares
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 5 segments, of lengths 42, 26, 26, 32 and 23 minutes (total = 129 minutes).
  • The topics in this lecture correspond to what would have been roughly 2 classroom lectures.
  • Recorded virtual whiteboard, in .pdf (19MB)
  • Textbook: read sections 3.1-3.4
  • Oct 6 + Oct 8 + Oct 13: Lecture 8: Low Rank Matrices
  • Typed notes, in .txt and .pdf (updated Oct 6, 9:27am)
  • Recorded lecture: broken into 10 segments, of lengths 12, 24, 21, 21, 12, 12, 22, 13, 15 and 32 minutes (total = 189 minutes). (Note: due to a zoom glitch, Segment 4 is broken into Part A and Part B.)
  • The topics in this lecture correspond to what would have been roughly 3 classroom lectures.
  • Recorded virtual whiteboard, in .pdf (27MB) (updated Oct 6, 9:27am)(updated Oct 8, 6:53am)
  • Textbook: read section 3.5. See also the References for Randomized Algorithms on the class web page.
  • Oct 15 + Oct 20 + Oct 22: Lecture 9: Eigenproblems
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 10 segments of lengths 22, 17, 23, 33, 25, 27, 27, 12, 14, and 32 minutes (total = 232 minutes).
  • The topics in this lecture correspond to what would have been roughly 3 classroom lectures.
  • Recorded virtual whiteboard, in .pdf (31MB)
  • Textbook: read Chap 4
  • Oct 27 + Oct 29 + Nov 3: Lecture 10: Symmetric Eigenproblems and the SVD
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 9 segments of lengths 23, 19, 13, 18, 38, 21, 27, 24, and 12 minutes (total = 195 minutes)
  • The topics in this lecture correspond to what would have been roughly 3 classroom lectures.
  • Recorded virtual whiteboard, in .pdf (27MB) (updated 10/23, 6:40am)
  • Textbook: read Chap 5
  • Nov 5 + Nov 10: Lecture 11: Introduction to Iterative Methods for A*x=b and A*x = lambda*x, and the Poisson Equation (Notes and recorded whiteboard updated Nov 15)
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 5 segments of lengths 35, 25, 20, 10 and 28 minutes (total = 118 minutes)
  • The topics in this lecture correspond to what would have been roughly 2 classroom lectures.
  • Recorded virtual whiteboard, in .pdf (17MB)
  • Textbook: read sections Chap 6.1 through 6.4
  • Nov 12 + Nov 17: Lecture 12: Splitting Methods
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 4 segments of lengths 21, 32, 27 and 27 minutes (total = 107 minutes)
  • The topics in this lecture correspond to what would have been roughly 1.5 classroom lectures.
  • Recorded virtual whiteboard, in .pdf (14MB)
  • Textbook: read sections Chap 6.5.1 through 6.5.5
  • Nov 19: Lecture 13: Multigrid
  • Powerpoint slides, in .pdf
  • Recorded lecture: broken into 3 segments of lengths 28, 20 and 25 minutes (total = 73 minutes)
  • The topics in this lecture correspond to what would have been roughly 1 classroom lecture.
  • Textbook: read section Chap 6.9
  • Nov 24: Lecture 14: Introduction to Krylov Subspace Methods
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 2 segments of lengths 21 and 32 minutes (total = 53 minutes)
  • The topics in this lecture correspond to what would have been roughly half a classroom lecture.
  • Recorded virtual whiteboard, in .pdf (7MB)
  • Textbook: read section Chap 6.6 through the end of 6.6.1
  • Nov 24 and Dec 1: Lecture 15: Krylov Subspace Methods: GMRES and CG
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 2 segments of lengths 36 and 41 minutes (total = 77 minutes)
  • The topics in this lecture correspond to what would have been roughly one classroom lecture.
  • Recorded virtual whiteboard, in .pdf (9MB)
  • Textbook: read sections Chap 6.6.2, 6.6.3 and 6.6.6.
  • Dec 1 and Dec 3: Lecture 16: Chebyshev Polynomials, applied to analyzing CG and accelerating SOR
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 2 segments of lengths 33 and 26 minutes (total = 59 minutes)
  • The topics in this lecture correspond to what would have been roughly one classroom lecture.
  • Recorded virtual whiteboard, in .pdf (7MB)
  • Textbook: read sections Chap 6.5.6 and 6.6.4.
  • Dec 3: Lecture 17: Preconditioning
  • Typed notes, in .txt and .pdf
  • Recorded lecture: broken into 2 segments of lengths 18 and 24 minutes (total = 42 minutes)
  • The topics in this lecture correspond to what would have been roughly half a classroom lecture.
  • Recorded virtual whiteboard, in .pdf (5MB)
  • Textbook: read sections Chap 6.6.5 and 6.10.