Build a Model

Build a Model

1. Model Architecture,

   • Neural Networks
     • Neural architecture search: optimize the creation of the network
       • Threshold
     • neuron
     • the Weights of a Network
       • activation functions: from 0-1.
         • step Function: Activation function AA = 1 if y> threshold, 0 otherwise
         • Linear function: A straight line function where activation is proportional to input ( which is the weighted sum from neuron ).
           • If you are familiar with gradient descent for training, you would notice that for this function, derivative is a constant. A = cx, derivative with respect to x is c. That means, the gradient has no relationship with X. It is a constant gradient and the descent is going to be on constant gradient. If there is an error in prediction, the changes made by back propagation is constant and not depending on the change in input delta(x) !!!  ---------- (-inf, inf) of linear function. output 
           • a combination of linear functions in a linear manner is still another linear function: No matter how many layers we have, if all are linear in nature, the final activation function of last layer is nothing but just a linear function of the input of first layer!, That means these two layers ( or N layers ) can be replaced by a single layer. Ah! We just lost the ability of stacking layers this way. 
         • Sigmoid Function
           • If you notice, towards either end of the sigmoid function, the Y values tend to respond very less to changes in X.
           • vanishing gradients”.
             • If you notice, towards either end of the sigmoid function, the Y values tend to respond very less to changes in X.
         • = Tanh Function -
         • ReLu: A(x) = max(0,x)
         • leaky ReLu.: y = 0.01x for x<0
       • Model improving
         • Backpropagation: identifying the weights causing an output error,
           • looks at the output of a model and goes backwards through the nodes and layers of a network to find the source or error.
         • Updating the Weights of a Network

2. Training Data

   • Bad data = bad model
   • train the model with all types of data that will encounter the model
   • Real world Data
   • diverse set
   • Equal amount of data for different types
   • correct labeling

3. Model Evaluation

   • precision, recall, and F1 score
     • https://classroom.udacity.com/nanodegrees/nd088-1/parts/41a5bc29-ab62-47f1-b1c2-e7210c1c56f1/modules/c71fcfde-70c5-458e-9607-2769e2656169/lessons/efc39a55-6ffd-4fb1-a07c-1a2f7b6ff2c7/concepts/ed39c34a-c21e-455f-a17e-215f2dc9ecf9
   • 80% training 20% testing
     • OR 10% validation and10% test

4. Transfer Learning

   • Fine tuning

5. Automated ML

Images and files in the map