1. Image Classification

Data-driven Approach, K-Nearest Neighbor, train/validation/test splits

2. Linear Classification

Support Vector Machine, Softmax

3. Optimization

Stochastic Gradient Descent

4. Backpropagation, Intutitions

chain rule interpretation, real-valued circuits, patterns in gradient flow

5. Neural Networks Part 1: Setting up the Architecture

model of a biological neuron, activation functions, neural net architecture, representational power

6. Neural Networks Part2 : Setting up the Data

preprocessing, weight initialization, batch normalization, regularization (L2/dropout), loss functions

7. Neural Networks Part 3 : Learning and Evaluation

gradient checks, sanity checks, babysitting the learning process, momentum(+nesterov), second-order methods, Adagrad/RMSprop, hyperparameter optimization, model ensembles

8. Convolutional Neural Networks: Architectures, Pooling Layers

layers, spatial arrangement, computational considerations

9. Convolutional Neural Networks: Layer Patterns, Case studies

layer sizing patterns, AlexNet/ZFnet/VGGNet case studies