ENGG 5781 Matrix Analysis and Computations
Term 1, 2023-24

---

Note

• Please read  http://www.cuhk.edu.hk/policy/academichonesty/ for issues related to academic honesty. The Chinese University of Hong Kong places very high importance on honesty in academic work submitted by students, and adopts a policy of zero tolerance on cheating and plagiarism.

---

General Information

Instructor

Prof. Wing-Kin (Ken) Ma

Time and Venue

Lecture: Monday 1:30pm - 3:15pm, Humanities Building 12
                Monday 4:30pm - 5:15pm, Fung King Hey Building Swire Hall 2

Course Description

Matrix analysis and computations are widely used in engineering fields --- such as machine learning, computer vision, systems and control, signal and image processing, optimization, communications and networks, and many more --- and are considered key fundamental tools. This course covers matrix analysis and computations at an advanced or research level. It consists of several parts. The first part focuses on various matrix factorizations, such as eigendecomposition, singular value decomposition, Schur decomposition, QZ decomposition and nonnegative factorization. The second part considers important matrix operations and solutions such as matrix inversion lemmas, linear system of equations, least squares, subspace projections, Kronecker product, Hadamard product and the vectorization operator. Sensitivity and computational aspects are also studied. The third part explores presently frontier or further advanced topics, such as matrix calculus and its various applications, tensor decomposition, and compressive sensing (or managing undetermined systems of equations via sparsity). In every part, relevance to engineering is emphasized and applications are showcased.

Assessment

• Assignments: 60%
• Final Examination: 40%

Learning Resources

Lecture notes in slide form, as well as supplementary materials, are provided; please check this course website regularly.

Recommended readings:

• Gene H. Golub and Charles F. van Loan, Matrix Computations (Fourth Edition), John Hopkins University Press, 2013.
• Roger A. Horn and Charles R. Johnson, Matrix Analysis (Second Edition), Cambridge University Press, 2012.
• Jan R. Magnus and Heinz Neudecker, Matrix Differential Calculus with Applications in Statistics and Econometrics (Third Edition), John Wiley and Sons, New York, 2007.
• Giuseppe Calafiore and Laurent El Ghaoui, Optimization Models, Cambridge University Press, 2014.

---

Lecture Notes

• Lecture 0: Overview
  
• Lecture 1: Basic Concepts, Note (updated, Sep 9, 2023. In particular, the note has an additional section about Gram-Schmidt decomposition)
  
• Lecture 2: Linear Representations and Least Squares, Note
• Lecture 3: Eigenvalues and Eigenvectors, Note
  
• Lecture 4: Positive Semidefinite Matrices and Variational Characterizations of Eigenvalues, Note
  
• Lecture 5: Singular Value Decomposition, Note
  
• Lecture 6: Least Squares Revisited
  
• Lecture 7: Linear Systems, Note
  
• Lecture 8: QR Decomposition
  
• Lecture 9: Kronecker Product
  
• Lecture 10: Advanced Topics---Non-negative Matrix Factorization and Tensor Factorization. Additional Reading: A Slides Related to Nonnegative Matrix Factorization