Workshop Description

Deep Learning is a rapidly expanding field with new applications found every day. In this workshop, we will cover the fundamentals of deep learning for the beginner.

We will introduce the math behind training deep learning models: the back-propagation algorithm. Building conceptual understanding of the fundamentals of deep learning will be the focus of the first part of the workshop. We will then cover some of the popular architectures used in deep learning, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), LSTMs, autoencoders and GANs.

There will be a hands-on computing tutorial using Jupyter notebooks to build a basic image classification model via transfer learning. By the end of the workshop, participants will have a firm understanding of the basic terminology and jargon of deep learning and will be prepared to dive into the plethora of online resources and literature available for each specific application area.

About the Instructor

Aashwin Mishra is a Project Scientist at the Machine Learning Initiative at the National Accelerator Laboratory (SLAC). His research focuses on uncertainty quantification, probabilistic modeling, interpretability/explainability, and optimization across physics applications.

About the Teaching Assistant

Rui Yan is a Ph.D. candidate at ICME, Stanford. Her research focuses on developing machine learning algorithms to extract meaningful representations from large-scale, high-dimensional and multi-modal data. She has interned at Meta, Uber AI, and Microsoft Research.

Workshop Materials

Pre-workshop Checklist

The Zoom link to the course can be found here.

The workshop assumes that you have requisite knowledge of:

a) Python programming

b) Numpy

c) basic concepts of Machine Learning (desirable)

For (a), you should have written codes in Python, and at some juncture, defined and used a class.

For (b), you should be comfortable in basic linear algebra operations in Numpy, understand broadcasting, etc.

(c) is desirable, but not required.

Schedule

Session 1 (Monday, August 7 | 8:00 AM - 11:00 AM PDT)

• Review of basic ML concepts
• Overview of deep learning
• Coding a perceptron in numpy
• Understanding fully connected neural networks
• Coding fully connected neural networks in numpy
• Introduction to Torch
• Using the AD facilities in torch to train models
• Using the sub-modules in torch to define and train models

Zoom recording Day 1: Link

Session 2 (Tuesday, August 8 | 8:00 AM - 11:00 AM PDT)

• Understanding Convolutional Neural Networks
• Defining and Training CNNs in torch
• Using GPUs for training
• Data Augmentation

Zoom recording Day 2: Link

Google Colab notebooks

• Linear Regression Exercise in Numpy
• Solutions: Linear Regression Exercise in Numpy
• Shallow Neural Network in Numpy
• Solutions: Shallow Neural Network in Numpy
• Introduction to Torch
• My First Torch Model
• MNIST with an FCNN
• CIFAR10 with CNNs
• CIFAR10 with GPUs
• Data Augmentation for Computer Vision

References

• Model interpretability: SHAP
• Model interpretability: LIME
• Explainable AI tutorial