Deep Learning

Deep Learning

1. GANS

   • Google reducing need for labelled data
     • https://ai.googleblog.com/2019/03/reducing-need-for-labeled-data-in.html
     • CNN: Predicting Image Rotations
       • https://arxiv.org/abs/1803.07728
     • Combine with feature learning
     • Compare GAN
       • https://github.com/google/compare_gan
   • Curveture regularlizarion of loss function

2. CNNs

   • Skip Layers
   • Mertic learning using Siamese Networks
     • https://arxiv.org/abs/1503.03832
   • Xception by Google

3. RNNs

   • PyTorch JIT for running faster models close to CuDnn
   • Lingvo
     • Lingvo is the international language Esperanto word for “language”. This naming alludes to the roots of the Lingvo framework — it was developed as a general deep learning framework using TensorFlow with a focus on sequence models for language-related tasks such as machine translation, speech recognition, and speech synthesis.

4. Artificial General Intelligence

   • New AGi bond

5. Infinitely customizable training loop

   • - regularization techniques - hyperparameter schedules - mixed precision training - tracking metrics

6. Tensorflow

   • SWIFT
     • FastAI

7. Hard Negative Sampling

   • Look at unlabeled data that cant be correct.

8. Areas of Growing Interest

   • N SHot Learning
   • Weak Supervision
     • Snorkel