Spring 23 - Advanced Topics in Computer Vision (ECE 381V/CS 395T)
- Course title:
ECE 381V/CS 395T: Advanced Topics in Computer Vision
- Term:
Spring 2023
- Meeting times and location:
MW 10:30am -12:00pm (BUR 130)
- After-class platform:
Slack (link sent to registered students)
- Video recording:
Available
Course Description and Prerequisites
This is a research-oriented advanced class that intends to focus on the latest frontier of computer vision. It describes computer vision algorithms that make sense of photographs, video, and other imagery. Applications include robotics, content creation, entertainment, medical image analysis, smart home, security, and HCI, among many others. Through this course, the students will digest and practice their knowledge and skills by many open discussions in classes, and will obtain in-depth experience with a particular research topic through a final project.
Students should have taken the following courses or equivalent: Introduction to Computer Vision (379K), Convex Optimization (381K-18), and Probability & Stochastic Process I (381J).
Previous knowledge of the following courses is helpful, but not necessary: Digital Video (381K-16), Statistical Machine Learning (381V), Data Mining (381L-10), or Cross-Layer Machine Learning HW/SW Design (382V).
Coding experiences with Python are necessary and assumed. Previous knowledge of C/C++, MATLAB or Tensorflow is very helpful, but not necessary.
Instructor Information
- Name:
Dr. Zhangyang (Atlas) Wang
- Telephone number:
512-471-1866
- Email address:
- Office hour time:
Tuesday 10:00am - 11:00am
- Office hour location:
EER 6.886 (instructor office)
TA Information
- TA 1 Name:
- Email address:
- Office hour time:
Thursday 4:00pm - 5:00pm
- Office hour location:
EER 3.854
Textbook and/or Resource Material
This course does not follow any textbook closely. Among many recommended readings are:
-
Pattern Recognition and Machine Learning, Christopher M. Bishop (2006).
-
Computer Vision: Algorithms and Applications, Richard Szeliski (2010).
-
Deep Learning, Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016).
-
Diving into Deep Learning, Aston Zhang, Zack Lipton, Mu Li and Alex Smola (2019).
Grading Policies
Grading will be based on class participation (10%), one mid-term exam (20%), and one final project (70%) (middle-term progress report 15% + presentation 20% + final report 20% + code review 15%). There will be no final exam.
-
One project to receive the Best Project Award, voted by all class members. (+5%)
-
Projects in the novel, interdisciplinary domains (some examples: 5G/6G telecommunication, brain-computer interface, economics & markets, COVID-19, etc.), judged by the instructor. (+2%)
-
For late submission, each additional late day will incur a 10% penalty.
Course Topics
| Date | Topic and materials |
|---|---|
| 1/09 Monday | Topic I: Deep Vision Backbones (1): Building Blocks |
| 1/11 Wednesday | Topic I: Deep Vision Backbones (2): Convolutional Neural Networks |
| 1/16 Monday | - No Class (Martin Luther King, Jr. Day) - |
| 1/18 Wednesday | Topic I: Deep Vision Backbones (3): More Advanced Architectures |
| 1/23 Monday | Topic I: Deep Vision Backbones (4): Vision Transformers Slides |
| 1/25 Wednesday | Topic II: Label-Efficient Learning (1): Semi-Supervised Learning |
| 1/30 Monday | Topic II: Label-Efficient Learning (2): Few-Shot & Active Learning |
| 2/01 Wednesday | Topic II: Label-Efficient Learning (3): Transfer & Self-Supervised Learning Slides |
| 2/06 Monday | Topic III: Resource-Efficient Learning (1): Model Compression |
| 2/08 Wednesday | Topic III: Resource-Efficient Learning (2): Efficient Training and Fine-Tuning |
| 2/13 Monday | Topic III: Resource-Efficient Learning (3): Advanced Sparsity, and Mixture-of-Experts Slides |
| 2/15 Wednesday | Topic IV: Robust Vision (1): Image Enhancement |
| 2/20 Monday | Topic IV: Robust Vision (2): Uncertainty |
| 2/22 Wednesday | Topic IV: Robust Vision (3): Domain Generalization |
| 2/27 Monday | Topic IV: Robust Vision (4): Adversarial Robustnes Slides |
| 3/01 Wednesday | Topic V: AutoML and Meta Learning (1) |
| 3/06 Monday | Topic V: AutoML and Meta Learning (2) |
| 3/08 Wednesday | Topic V: AutoML and Meta Learning (3) Slides |
| 3/13 Monday | - No Class (Spring Break) - |
| 3/15 Wednesday | - No Class (Spring Break) - |
| 3/20 Monday | Topic VI: Generative Adversarial Networks (1) |
| 3/22 Wednesday | Topic VI: Generative Adversarial Networks (2) |
| 3/27 Monday | Topic VI: Generative Adversarial Networks (3) Slides |
| 3/29 Wednesday | Topic VII: Neural Radiance Fields (1): Single Scene Fitting |
| 4/03 Monday | Topic VII: Neural Radiance Fields (2): Cross-Scene Fitting [guest lecture] |
| 4/05 Wednesday | Topic VII: Neural Radiance Fields (3): Beyond NeRFs [guest lecture] Slides |
| 4/10 Monday | Topic VIII: Vision and Language (1): Image to Text (image captioning, VQA) |
| 4/12 Wednesday | Topic VIII: Vision and Language (2): Introduction to Diffusion Models (the game changer!) |
| 4/17 Monday | Topic VIII: Vision and Language (3): Text to Image: Stable Diffusion and the "AIGC" trend Slides |
| 4/19 Wednesday | Class Project Presentation (1) |
| 4/24 Monday | Class Project Presentation (2) |
Acknowledgement
Many materials included in this course are adapted from the existing teaching or tutorial slides, created by colleagues in CMU, Stanford, UIUC, UC Berkeley, GaTech, Microsoft, Google, Meta, DeepMind, NVIDIA, and more. The instructor owes many thanks for their generosity of sharing those materials publicly.