STAT 30900/CMSC 37810. Mathematical Computation I — Matrix Computation

Department of Statistics
University of Chicago
Fall 2010

This is an introductory course on numerical linear algebra. The course will present a global overview of a number of topics, from classical to modern to state-of-the-art. The fundamental principles and techniques will be covered in depth but towards the end of the course we will also discuss some exciting recent developments.

Numerical linear algebra is quite different from linear algebra. We will be much less interested in algebraic results that follow from the axiomatic definitions of fields and vector spaces but much more interested in analytic results that hold only over the real and complex fields. The main objects of interest are real- or complex-valued matrices, which may come from differential operators, integral transforms, bilinear and quadratic forms, boundary and coboundary maps, Markov chains, graphs, metrics, correlations, hyperlink structures, cell phone signals, DNA microarray measurements, movie ratings by viewers, friendship relations in social networks, etc. Numerical linear algebra provides the mathematical and algorithmic tools for matrix problems that arise in engineering, scientific, and statistical applications.

Announcements

11/25/10: Problem Set 5 posted.

11/19/10: Reference on IEEE floating point standard added. See also Overton's webpage for the book.

11/16/10: Make-up lecture 10:30–11:50AM, Fri, Nov 19, in Eckhart 117.

11/16/10: The original reference for solving total least squares via SVD is Section 6 of: G. Golub, "Some modified matrix eigenvalue problems," SIAM Rev., 15 (1973), no. 2, pp. 318–334.

11/11/10: Problem Set 4 posted.

11/6/10: Make-up lecture 2:30–4PM in Eckhart 110; office hours 1–2:30PM, Mon, Nov 8.

11/1/10: Problem Set 3 posted.

10/25/10: Reminder: Office hours 2–4PM, Wed, Oct 25.

10/19/10: Problem Set 2 posted.

09/27/10: Check this page regularly for announcements.

Lectures

Location: Eckhart Hall, Room 117

Times: 10:30–11:50AM on Tue/Thu.

Course staff

Instructor: Lek-Heng Lim
Office: Eckhart 122
lekheng(at)galton.uchicago.edu
Tel: (773) 702-4263
Office hours: 1:00–2:00 PM Tue/Thu

Course Assistant: Rina Foygel
Office: Ryerson N375
rina(at)galton.uchicago.edu
Office hours: TBA

Syllabus

The last three topics we would only touch upon briefly (no discussion of actual algorithms); they would be treated in greater detail in a second course.

Linear algebra over R or C: How this course differs from your undergraduate linear algebra course.

Three basic matrix decompositions: LU, QR, SVD.

Gaussian elimination revisited: LU and LDU decompositions.

Backward error analysis: Guaranteeing correctness in approximate computations.

Gram-Schmidt orthogonalization revisited: QR and complete orthogonal decompositions.

Solving system of linear equations in the exact and the approximate sense: Linear systems, least squares, data least squares, total least squares.

Low rank matrix approximations and matrix completion.

Iterative methods: Stationary methods and Krylov subspace methods.

Eigenvalue and singular value problems.

Sparse linear algebra: Sparse matrices and sparse solutions.

Problem Sets

Problem set will be assigned biweekly and will be due the following week (except possibly in the weeks when there is a midterm). Collaborations are permitted but you will need to write up your own solutions.

Problem Set 5: Do exercises 15.1, 16.1, 17.1, 17.2, 19.1, 19.2 (posted: Nov 25; due: Dec 2)

Problem Set 4: Do exercises 10.4, 11.1, 12.1, 13.1, 13.2, 14.1, 14.2 and programming exercises: 11.2, 11.3, 12.3, 13.4 (posted: Nov 11; due: Nov 23)

Problem Set 3: Do exercises 6.1, 6.3, 6.5, 7.3, 7.5, 10.1, and programming exercises: 8.2, 9.2, 10.2, 10.3 (posted: Nov 1; due: Nov 9)

Problem Set 2: Do exercises 1.3, 1.4, 2.5, 2.6, 2.7, 3.2, 3.3, 3.6, 4.4, 4.5, 5.3, 5.4 (posted: Oct 19; due: Oct 26)

Problem Set 1: PDF (posted: Sep 28; due: Oct 7)

Bug report on the problem sets or the solutions: lekheng(at)galton.uchicago.edu

Supplementary materials

Course homepage from Fall 2009 (courtesy of Yali Amit). Related course homepages from Fall 2005 and Spring 2006.

Grades

Grade composition: 60% Problem Sets, 40% Final

Textbook

Any one of the following books (listed in order of difficulty) would be acceptable.

L.N. Trefethen, D. Bau, Numerical Linear Algebra, SIAM, 1997.

D. Watkins, Fundamentals of Matrix Computations, 3rd Ed., Wiley, 2010.

G. Golub, C. Van Loan, Matrix Computations, 3rd Ed., John Hopkins, 1996.

J. Demmel, Applied Numerical Linear Algebra, SIAM, 1997.

References

D.S. Bernstein, Matrix Mathematics, 2nd Ed., Princeton, 2009.

G. Golub, G. Meurant, Matrices, Moments and Quadrature with Applications, Princeton, 2010.

N.J. Higham, Accuracy and Stability of Numerical Algorithms, 2nd Ed., SIAM, 2002.

M. Overton, Numerical Computing with IEEE Floating Point Arithmetic, SIAM, 2001.

R. Thisted, Elements of Statistical Computing: Numerical Computation, CRC, 1988.