Statistical modelling of complex data often requires high-dimensional models to be calibrated to the observed data. Some of the traditional statistical models and methods for training break down in high dimensions, the curse of dimensionality, and particular emphasis on the models and methods is needed. In this course we provide a general introduction to statistical methods and models as well as optimization techniques that are particularly developed to function well in high and infinite dimension.
Course type:
- AI track: elective
- AS track: elective
- Joint curriculum: advanced
Time: Given even years, Autumn
Teachers: Henrik Hult (KTH), Johan Karlsson (KTH)
Examiner: Henrik Hult (KTH)
The participants are assumed to be familiar with concepts in mathematics and applied mathematics at the advanced level. Useful concepts are:
- Basic analysis (metric spaces, Hilbert spaces)
- Probability theory (law of large numbers, central limit theorem, Markov chains)
- Statistics (point estimation, hypothesis testing, linear regression)
- Optimization (linear programming, convex analysis, and gradient descent)
After completed studies, the student is expected to be able to
- provide examples of high-dimensional statistical models and optimization methods and explain the challenges associated with high dimension,
- describe, explain, and compare high-dimensional models and methods in statistics and optimization,
- derive and explain inequalities and concentration of measures in high-dimensional probability theory,
- apply monotone operator theory to derive convergence results for optimization methods,
- apply theoretical concepts and methods in high-dimensional statistics and optimization to solve problems involving high-dimensional data.
Module 1
- Introduction to high-dimensional statistics and optimization
- Background on statistical models, optimization, and iterative methods
Module 2
- Linear regression in high dimension, Lagrange relaxation and the Hahn-Banach theorem
- Concentration of measures and Fenchel duality
Module 3
- Stochastic approximation and Monotone operators
- Compressed sensing / Random projections / Splitting methods
Written lecture notes and recommended literature provided by the teachers.
Written homework assignments (3)
Syllabus (kursplan)
Course page
If you are not a student at KTH you must login via https://canvas.kth.se/login/canvas