Deep Learning

Deep Learning

1. Feed Neural Network

   • Activation function
     • Sigmoid
     • Tanh
     • (L)/ReLU
     • Sign
     • Softmax
       • Classification
   • Loss function
     • L-norm
     • Cross Entropy
     • Hinge
   • Optimization
   • Regularization
   • Fully Conected Layer
   • Training
     • Backpropagation

2. Convolutional Neural Network (CNN)

   • -Spatial context -Shared weights
   • Convolution layer
   • Non linear activation
   • Pooling
   • Fully connected

3. Recurrent Neural Network (RNN)

   • Long Short Term Memory (LSTM)
   • Gated recurrent unit (GRU)
   • Bacpropagation through time

4. Introduction

   • Rossenblatt Perceptron
   • Multilayer Perceptron
   • Universal Approximation Theorem

5. Architectures

   • LeNet
   • Alexnet
   • Network in Network
   • VGG
   • Inception
   • ResNets
   • DenseNet

6. Reinforcement Learning

   • Sequential Decision Making
   • Markov Decision Process
     • Policy
     • Q Value
   • Deep Q learning

7. Unsupervised Learning

   • Autoencoder
   • Varational autoencoder
   • Restricted Boltzmann Machines
   • Generative Adversarial Networks

8. Object Detection & Segmentation

   • Object Detection
     • Slidding window
     • Region Based
       • Regional CNN
       • Fast R-CNN
     • Single shot
       • YOLO
     • Intance recognition +localization
   • Sementation
     • Fully Convoluted
     • Upsampling
     • Context Knowledge
     • Skip Connecctions

9. Visualization

   • Architecture
   • Training
   • Params
   • Attention

10. Prior Knowledge

    • Known operator