Dr. Quoc Le from the Google Brain project team (yes, the one that made headlines for creating a cat recognizer) presented a series of lectures at the Machine Learning Summer School (MLSS ’14) in Pittsburgh this week. This is my favorite lecture series from the event till now and I was glad to be able to attend them.

The good news is that the organizers have made available the entire set of video lectures in 4K for you to watch. But since Dr. Le did most of them on the board and did not provide any accompanying slides, I decided to put the contents of the lectures along with the videos here.

In this post I posted Dr. Le’s lecture videos and added content links with short descriptions to help you navigate them better.

## Lecture 1: Neural Networks Review

*[Flash player 8+ and JavaScript enabled needed to view this video.]*

Dr. Le begins his lecture starting from the fundamentals on Neural Networks in case you’d like to brush up your knowledge about them. Otherwise feel free to quickly skim through initial sections but I promise there are interesting things later on. You may use the links below to quickly skip the video to the relevant parts. Let me know in the comments if they don’t work.

#### Contents

- Introduction
- Why Neural Networks: Motivation, Non-linear classification
- Mathematical Expression for NN: Decision function, Minimizing Loss and Gradient Descent (
*Correction in derivative*), Making decision - Backpropagation: Audience questions, Derivation for backpropagation, Backpropagation algorithm

## Lecture 2: NNs in Practice

*[Flash player 8+ and JavaScript enabled needed to view this video.]*

If you have already covered NN in the past then the first lecture may have been a bit dry for you but the real fun begins in this lecture when Dr. Le starts talking about his experiences of using deep learning in practice.

#### Contents

- Stochastic gradient descent
- Clarifications from Lecture 1: Data partitioning is not needed, Derivative of the loss function, Tip – Write unit tests!
- Ideas for practical implementations: Breaking Symmetry, Monitoring Progress on training, Underfitting and overfitting, How to select NN architecture and hyper-parameters, Other tips for improvements
- Deep Neural Networks: Review of why NN, Shallow vs. Deep, Rectified Linear Units, Definitions for deep NN, History of NN
- Deep NN Architectures: Autoencoder, Intuition for using autoencoders for initialization (
*Continued in the next lecture*)

## Lecture 3: Deep NN Architectures

*[Flash player 8+ and JavaScript enabled needed to view this video.]*

In this lecture, Dr. Le finishes his description on NN architectures. He also talks a bit about how they are being used at Google for applications in image and speech recognition, and language modelling.

#### Contents

- Pre-training with autoencoders
- Convolutional NN (Convnets): Local receptive field, Why are Convnets useful?, Image classification, General Pipeline
- Recurrent NN: Word Vectors
- Applications: Google Brain and other ongoing work

## 10 Responses to “Quoc Le’s Lectures on Deep Learning”

## jiechen

good，studying

## Saw shen

Nice

## auraham

great job! I will watch it

## seno chow

watch it

## xu

watch it

## Four short links: 14 July 2014 - O'Reilly Radar

[…] Quoc Le’s Lectures on Deep Learning — Machine Learning Summer School videos (4k!) of the deep learning lectures by Google Brain team member Quoc Le. […]

## Four short links: 14 July 2014 | Bid Data 2014

[…] Quoc Le’s Lectures on Deep Learning — Machine Learning Summer School videos (4k!) of the deep learning lectures by Google Brain team member Quoc Le. […]

## Four short links: 14 July 2014 » Salesforce Marketing

[…] Quoc Le’s Lectures on Deep Learning — Machine Learning Summer School videos (4k!) of the deep learning lectures by Google Brain team member Quoc Le. […]

## Yikun

His introduction of word representation by Tomos Mikolov is not accurate. He should read more carefully about Tomos work before giving lecture.

## Gautam Solanki

Very nice series of lectures about Neural Networks and Deep Learning ! Thanks very much