Been going through this MIT course on efficient ML / inference by @songhan_mit and it’s amazing! Covers pruning, quantization, distillation, LLM deployment, diffusion models, etc. Fall 2026 course, so more lectures should keep getting added https://hanlab.mit.edu/courses/2026-fall-65940…

MIT 6.5940 Fall 2026 TinyML and Efficient AI Computing

Source: https://hanlab.mit.edu/courses/2026-fall-65940 This course focuses on efficient machine learning and systems. This is a crucial area as deep neural networks demand extraordinary levels of computation, hindering its deployment on everyday devices and burdening the cloud infrastructure. This course introduces efficient AI computing techniques that enable powerful deep learning applications on resource-constrained devices. Topics include model compression, pruning, quantization, neural architecture search, distributed training, model serving, model parallelism, gradient compression, and on-device fine-tuning. It also introduces application-specific acceleration techniques for large language models and diffusion models. Students will get hands-on experience implementing model compression techniques and deploying large language models on a laptop.

• Time: Tuesday/Thursday 4:00-5:30 PM

• Office Hour: Thursday 5:30-6:30 pm Eastern Time, 54-100

• **Contact:**MIT students who are enrolled in this class can email us atefficientml-staff [at] mit.edu.

• **Prerequisites:**Both 6.191 Computation Structures and 6.390 Intro to Machine Learning. We do not approve prerequisite waiver petitions.‍

• **Cross-registration:**We are unable to accommodate cross-registered students this semester.

• Please do not submit petitions or email the course staff requesting exceptions to these policies.

‍

Instructors

Teaching Assistants

Announcements

Currently no active announcements.

Schedule

Date

Lecture

Logistics

Introduction

Sep 10

Basics of Deep Learning

Sep 15

Lecture

2

:

Basics of Deep Learning

Chapter I: Efficient Inference

Sep 16

Lecture

3

:

Chapter I: Efficient Inference

Pruning and Sparsity (Part I)

Sep 17

Lecture

3

:

Pruning and Sparsity (Part I)

Pruning and Sparsity (Part II)

Sep 22

Lecture

4

:

Pruning and Sparsity (Part II)

Quantization (Part I)

Sep 24

Lecture

5

:

Quantization (Part I)

Lab 0 due

Quantization (Part II)

Sep 29

Lecture

6

:

Quantization (Part II)

Neural Architecture Search (Part I)

Oct 1

Lecture

7

:

Neural Architecture Search (Part I)

Neural Architecture Search (Part II)

Oct 6

Lecture

8

:

Neural Architecture Search (Part II)

Knowledge Distillation

Oct 8

Lecture

9

:

Knowledge Distillation

Student Holiday

Oct 13

Lecture

9

:

Student Holiday

MCUNet: TinyML on Microcontrollers

Oct 15

Lecture

10

:

MCUNet: TinyML on Microcontrollers

Lab 2 due, Lab 3 out

TinyEngine and Parallel Processing

Oct 20

Lecture

11

:

TinyEngine and Parallel Processing

Chapter II: Model Optimization for Generative AI

Oct 21

Lecture

12

:

Chapter II: Model Optimization for Generative AI

Transformer and LLM

Oct 22

Lecture

12

:

Transformer and LLM

LLM Quantization and Deployment

Oct 27

Lecture

13

:

LLM Quantization and Deployment

Lab 3 due, Lab 4 out

LLM Post Training

Oct 29

Lecture

14

:

LLM Post Training

Project ideas out

Long Context LLM

Nov 3

Lecture

15

:

Long Context LLM

Vision Transformer

Nov 5

Lecture

16

:

Vision Transformer

Lab 4 due, Lab 5 out

Diffusion Model (Part I)

Nov 10

Lecture

17

:

Diffusion Model (Part I)

Diffusion Model (Part II)

Nov 12

Lecture

18

:

Diffusion Model (Part II)

Chapter III: Efficient Training

Nov 16

Lecture

19

:

Chapter III: Efficient Training

Distributed Training (Part I)

Nov 17

Lecture

19

:

Distributed Training (Part I)

Lab 5 due

Distributed Training (Part II)

Nov 19

Lecture

20

:

Distributed Training (Part II)

Project proposal due

On-Device Training and Transfer Learning

Nov 24

Lecture

21

:

On-Device Training and Transfer Learning

Thanksgiving

Nov 26

Chapter IV: Advanced Topics

Nov 30

Lecture

22

:

Chapter IV: Advanced Topics

Guest Lecture

Dec 1

Lecture

22

:

Guest Lecture

Final Project Presentation

Dec 3

Lecture

23

:

Final Project Presentation

Final Project Presentation

Dec 8

Lecture

24

:

Final Project Presentation

Final Project Presentation

Dec 10

Lecture

25

:

Final Project Presentation

Dec 13: Project report due

Logistics

The class requirements includefive labs, andone final project. This is a PhD level course, and by the end of this class you shouldhave a good understanding of efficient deep learning techniques, and be able todeploy large language models (LLMs) on your laptop.

Note that this class does not have any tests or exams.

‍

Labs

There will be 5 labs over the course of the semester.

• Lab1: GPU Basics
• Lab2: Pruning
• Lab3: Neural architecture search
• Lab4: Quantization
• Lab5: LLM deployment on laptop

‍

Collaboration Policy

**Labs must be done individually:**each student must hand in their own answers. However, it is acceptable to collaborate when figuring out answers and to help each other solve the problems. We will be assuming that, as participants in a graduate course, you will be taking the responsibility to make sure you personally understand the solution arising from such collaboration. You also must indicate on each homework with whom you have collaborated.

‍

Late Policy

You will be allowed6 total homework late dayswithout penalty for the entire semester. You may be late by up to 6 days on any homework assignment. Once those days are used, you will be penalized according to the following policy:

• Homework is worth full credit at the due time on the due date.
• The allowed late days are counted by day (i.e., each new late day starts at 11:59 pm ET).
• Once the allowed late days are exceeded, the penalty is 50% per late day counted by day.
• The homework is worth zero credit 2 days after exceeding the late day limit.

You must turn in at least 4 of the 5 assignments, even if for zero credit, in order to pass the course.

‍

Final Project

The class project will be carried out in groups, and has three main parts:

• proposal: choose from a list of suggested projects, or propose your own project
• poster presentation
• final report

‍

Previous Courses