Scalable Data Science and Distributed Machine Learning (6hp)

Course type:

• AS track: elective
• AI track: mandatory
• Joint curriculum: advanced

Time: Given even years, Autumn

Teachers: Raazesh Sainudiin

Examiner: Benny Avelin (UU)

The participants are assumed to have a background in mathematics corresponding to the contents of the WASP-course “Introduction to Mathematics and Machine Learning”, including programming experience in at least one high-level language such as Python.

After completing the course the student should be able to:

• describe, explain and analyze the computational complexity of common sequential, parallel and distributed algorithms including sorting, optimizing, performing linear algebraic and graph operations using appropriate abstract machine models.
• describe the time, space and communication complexity of the core distributed algorithms for sorting, joining, optimizing, performing linear algebraic & graph operations.
• design, analyze and implement distributed algorithms for data ‑parallel or model ‑ parallel optimization of ML models from linear regression to DNNs
• analyze the computational complexity and the mathematical assumptions of the distributed algorithms underpinning results of scientific publications in ML/AI
• understand how to design and operate modular feature, training, and inference pipelines for batch, real-time, and LLM-based systems using feature stores as the central data layer to build production ML systems end-to-end.
• understand how to engineer features, manage data across training and inference, implement observability and monitoring, and safely deploy and update ML systems at scale using practical MLOps and data engineering approaches.

The course is given in three modules. In addition to lectures by the organizer there will be invited guest speakers from industry.

Module 1
Practical: Practical: Introduction to Data Science & Machine Learning Systems
Theoretical: Analysis of Sequential, Parallel and Distributed Algorithms

Module 2
Practical: Building Machine Learning Systems
Theoretical: Analysis of Sequential, Parallel and Distributed Algorithms

Module 3
Practical: Diving Deeper
Theoretical: Group Project Work

The course materials (including type-scripted mathematical lecture notes,
recommended books, open MOOCs, etc.) will be made available from the course’s KTH Canvas page.

The required reference book for the practical aspects of the course is freely
available from https://www.hopsworks.ai/lp/full-book-oreilly-building-machine-
learning-systems-with-a-feature-store.

Examination

• Assignments hand-in: students complete weekly comprehension quizzes and
  submit machine-generated receipts from the hands-on labs (run free and locally in Docker on the student’s own laptop) by the specified deadlines, with at least 70% of the points in each assignment; the assignments are autograded on submission, with multiple tries permitted.
• Group project and presentation: students work in groups of 2-4, produce a video presentation and a written report with code, peer-reviewed and generally made publicly available at the end of the course after peer-review akin to academic conference dissemination standards. Graded out of 10 points: method / results quality (5), written report (2), video presentation (1), and a working replay.sh audit chain (2) that is verified against the group’s public git repository.
  Retry: one single retry is allowed for assessment tasks, six months after course
  conclusion. Assessment tasks may be altered for the retry, for example an oral
  examination instead of a group project.

Grades

Fail or Pass