Lecture notes for "Applied Numerical Linear Algebra", Fall 2014

These notes are intended to be outlines of the material covered during each lecture, not always comprehensive notes. I will attempt to post them before lecture, but may repost them after lecture with corrections.
  • Aug 29: Lecture 1 - Course outline.
  • Sep 3: Lecture 2 - Floating point arithmetic, roundoff error analysis, and some surprising ways to get high precision despite roundoff.
  • Sep 5: Complete Lecture 2.
  • Sep 8: Lecture 4 (updated 9/10, 8:55am) - Matrix and vectors norms, Singular Value Decomposition, Condition numbers for Ax=b
  • Sep 10: Continue Lecture 4 (updated 9/10, 8:55am)
  • Sep 12: Continue Lecture 4 (updated 9/10, 8:55am)
  • Sep 15: Complete Lecture 4, begin Lecture 7 on Communication-Avoiding Algorithms
  • Sep 17: Continue Lecture 7 (updated 9/17, 10:30am).
  • For a complete proof of the communication lower bound for O(n^3) matrix multiplication (and other linear algebra algorithms), see Minimizing Communication in Numerical Linear Algebra (SIMAX, Sep 2011).
  • For the communication lower bound on Strassen-like matrix multiplication, see Graph Expansion and Communication Costs of Fast Matrix Multiplication (JACM, Dec 2012)
  • Sep 19: Complete Lecture 7
  • Sep 22: Begin Lecture 10 on Gaussian Elimination
  • Sep 24: Continue Lecture 10 (updated 9/24, 10:35am).
  • Sep 26: Complete Lecture 10 (updated 9/26, 10:25am), begin Lecture 12, on Gaussian Elimination for Matrices with Structure
  • Sep 29: Continue Lecture 12
  • Oct 1: Continue Lecture 12
  • Oct 3: Complete Lecture 12 (updated 10/3, 5:00pm)
  • Oct 6: Begin Lecture 16 on Least Squares and QR
  • Oct 8: Continue Lecture 16
  • Oct 10: Complete Lecture 16
  • Oct 13: Begin Lecture 19 on Deterministic and Randomized Algorithms for Low Rank Matrices. The following material provides background on randomized algorithms.
  • "Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions," N. Halko, P. G. Martinsson, J. A. Tropp, SIAM Review 2011, or arxiv.org
  • "An Elementary Proof of a Theorem of Johnson and Lindenstrauss," S. Dasgupta, A. Gupta, Random Structures and Algorithms 2003, or here
  • "Randomized algorithms for matrices and data," M. Mahoney, arxiv.org
  • "Low Rank Approximation and Regression in Input Sparsity Time", K. Clarkson, D. Woodruff, STOC 2013 (co-winner, Best Paper Award), or arxiv.org
  • "Low-distortion Subspace Embeddings in Input-sparsity Time and Applications to Robust Linear Regression," X. Meng, M. Mahoney, arxiv.org
  • Stat 260/CS294 - Randomized Algorithms for Matrices and Data was taught by Michael Mahoney in Fall 2013.
  • Oct 15, Continue Lecture 19
  • Oct 17, Continue Lecture 19 (updated Oct 16, 9:20am)
  • Oct 20, Complete Lecture 19, (updated Oct 20, 1:21pm), begin Lecture 22 on Eigenvalue Problems
  • Oct 22, Continue Lecture 22
  • Oct 24, Continue Lecture 22
  • Oct 27, Continue Lecture 22
  • Oct 29, Continue Lecture 22 (updated Oct 29, 9:00am)
  • Oct 31, Complete Lecture 22, begin Lecture 27 on Symmetric Eigenvalue Problems and the SVD
  • Nov 3, Continue Lecture 27 (update posted Nov 3, 8:17am)
  • Nov 5, Continue Lecture 27 (update posted Nov 5, 5:27am)
  • ``Avoiding Communication in Successive Band Reduction'', G. Ballard, J. Demmel, N. Knight, UCB EECS Tech Report UCB/EECS-2013-131
  • ``Accurate and efficient expression evaluation and linear algebra,'', J. Demmel, I. Dumitriu, O. Holtz, P. Koev, Acta Numerica, v. 17, May 2008
  • ``New Fast and Accurate Jacobi SVD Algorithm, Part I and Part II,'', Z. Drmac, K. Veselic, SIAM J. Mat. Anal. Appl., v 29, 2008 (SIAM Linear Algebra Prize 2009)
  • ``Orthogonal Eigenvectors and Relative Gaps,'' I. Dhillon, B. Parlett, SIAM J. Mat. Anal. Appl. v. 25:3, Mar 2004 (SIAM Linear Algebra Prize 2006)
  • ``Computing the Singular Value Decomposition with High Relative Accuracy,'' J. Demmel, M. Gu, S. Eisenstat, I. Slapnicar, K. Veselic, Z. Drmac, Lin. Alg. Appl., v 299, 1999
  • ``Jacobi's Method is more accurate than QR,'' J. Demmel, K. Veselic, SIAM J. Mat. Anal. Appl., v 27, 1990
  • ``Accurate singular values of bidiagonal matrices,'' J. Demmel, W. Kahan, SIAM J. Sci. Stat. Comp., v 11, 1990 (SIAM Linear Algebra Prize 1991)
  • Nov 7, Continue Lecture 27 (update posted Nov 7, 10:35am)
  • Nov 10, Complete Lecture 27, begin Lecture 31 on Iterative Methods
  • Nov 12, Continue Lecture 31 (update posted Nov 12, 10:36am)
  • Nov 14, Continue Lecture 31
  • Nov 17, Continue Lecture 31 (update posted Nov 17, 10:26am)
  • Nov 19, Continue Lecture 31, start Multigrid in ppt or pdf
  • Nov 21, Continue Multigrid
  • Nov 24, Finish Multigrid (update posted Nov 24, 8:00am), then continue Lecture 31 (another Multigrid update posted Nov 24, 1:25pm)
  • Nov 26, Continue Lecture 31
  • Dec 1, Continue Lecture 31 (update posted Nov 30, 8:40am) (update posted Dec 1, 8:49am)
  • Dec 3, Continue Lecture 31
  • Dec 5, Complete Lecture 31 (update posted Dec 5, 6:00am), present Communication-Avoiding Krylov Subspace Methods (slides 78-100), in pptx or pdf