Course Syllabus
ELEC5470/IEDA6100A - Convex Optimization
Fall 2020-21, HKUST
Description
In the last three decades, a number of fundamental and practical results have been obtained in the area of convex optimization theory. It is a well-developed area, both in the theoretical and practical aspects, and the engineering community has greatly benefited from these recent advances by finding key applications.
This graduate course introduces convex optimization theory and illustrates its use with many applications where convex and nonconvex formulations arise. The emphasis will be on i) the art of unveiling the hidden convexity of problems by appropriate manipulations, ii) a proper characterization of the solution either analytically or algorithmically, and iii) multiple practical ways to approach nonconvex problems.
The course follows a case-study approach by considering recent successful applications of convex optimization published within the last decade in top scientific journals in the areas of signal processing, finance, machine learning, and big data. Problems covered include portfolio optimization in financial markets, filter design, beamforming design in wireless communications, classification in machine learning, circuit design, robust designs under uncertainty, sparse optimization, low-rank optimization, graph learning from data, discrete maximum likelihood decoding, network optimization, distributed algorithms, Internet protocol design, etc.
Textbooks
- Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004. [pdf]
- Daniel P. Palomar and Yonina C. Eldar, Convex Optimization in Signal Processing and Communications, Cambridge University Press, 2009.
Prerequisites
Students are expected to have a solid background in linear algebra. They are also expected to have research experience in their particular area and be capable of reading and dissecting scientific papers.
Grading
| Homework: | 25% | (auditors too) |
| Midterm: | 15% | (auditors too) |
| Class participation: | 10% | (Zoom video on) |
| Final Project: | 50% | (homeworks and midterm are required to be passed) |
Course Schedule
| Date | Week | Lect | Topic |
|---|---|---|---|
| 7-Sep | 1 | 1 | Introduction |
| 2 | Theory: Convex sets and convex functions | ||
| 14-Sep | 2 | 3 | Theory: Convex problems and taxonomy (LP, QP, SOCP, SDP, GP) |
| 4 | Application: Filter design | ||
| 21-Sep | 3 | 5 | Theory: Algorithms primer (Newton, IPM, BCD) |
| 6 | Application: Disciplined convex programming - CVX | ||
| 28-Sep | 4 | 7 | Theory: Lagrange duality and KKT conditions |
| 8 | Application: Waterfilling solutions | ||
| 5-Oct | 5 | 9 | Application: Markowitz portfolio optimization |
| 10 | Theory&Application: Geometric programming (GP) | ||
| 12-Oct | 6 | 11 | Theory&Application: MM and SCA based algorithms |
| 12 | Application: Risk parity portfolio in finance | ||
| 19-Oct | 7 | 13 | Application: Sparsity via l1-norm minimization |
| 14 | Application: Sparse index tracking in finance | ||
| 2-Nov | 8 | - Midterm - | |
| 9-Nov | 9 | 15 | Application: Classification and SVM in machine learning |
| 16 | Application: Low-rank optimization problems (Netflix, video security) | ||
| 16-Nov | 10 | 17 | Application: Robust optimization with applications |
| 18 | Application: Graph learning from data | ||
| 23-Nov | 11 | 19 | Application: ML decoding via SDP relaxation |
| 20 | Application: Rank-constrained SDP and multiuser downlink beamforming | ||
| 30-Nov | 12 | 21 | Theory: Primal/dual decomposition techniques with applications |
| 22 | Application: The Internet as a convex optimization problem |
Lecture Information
Lecture Time: Mon 18:00 – 20:50
Lecture Venue: Online via Zoom and Rm2405 during mixed-mode delivery.
Teaching Team
Instructor: Prof. Daniel P. PALOMAR (https://www.danielppalomar.com)
Email: palomar@ust.hk
Office: 2398 (Lifts 17/18)
Office hours: By email appointment
TAs: Rui ZHOU (rui.zhou@connect.ust.hk) and Zé Vinícius (jvdmc@connect.ust.hk)