Machine Learning

Machine Learning

1. Types

   • Classification
   • Prediction

2. Learning Task

   • Supervised
     • Training data includes the desired solution(labels)
     • Common Tasks
       • Classification
       • Prediction
     • Common Algorithms
       • k-Nearest Neighbors
       • Linear Regression
       • Logistic Regression
       • Support Vector Machines(SVMs)
       • Decision Tree
       • Random Forests
       • Neural Networks
   • Unsupervised
     • Tries to learn without a labeled data
     • Common Tasks
       • Clustering
         • K-Means
         • Hierarchical Cluster Anaysis(HCA)
         • Expectation Maximization
       • Visualization and Dimensionality Reduction
         • Principal Component Analysis(PCA)
         • Kernel PCA
         • Locally-Linear Embedding(LLE)
         • t-Distributed Stochastic Neighbor Embedding(t-SNE)
       • Association Rule Learning
         • Apriori
         • Eclat
       • Anomaly Dectection
   • Semisupervised
     • Some labeled data and lots of unlabelled data
   • Reinforcement
     • Use reinforcements and penalties to train the data. Often used to make robots learn how to walk. 

3. Learning Process

   • Batch
     • AKA offline, the system is trained and then is used.
   • Online or Incremental Learning
     • Learns while it runs. It can be pre trained and then getting updated while it runs. It is also called incremental learning because it can be trained offline but in an incremental fashion.
     • Out-of-core
       • Used to train huge datasets. It chops the data in smaller batches and test it untill it's acceptable.

4. Generalization Model

   • Instance Based
     • Train by heart. Use data create similarity data to find similarities.
   • Model Based
     • Create a model from the examples and use that model to make predictions.
     • Test Functions
       • Utility Function
       • Cost Function

5. Challenges

   • Bad Data
     • Insufficient Training Data
     • Non Representative Training Data
       • The data must represent what you want to generalize.
       • Sampling Noise
         • Sample too small. (Non representative data as a result of chance)
       • Sampling Bias
         • Large number of data but sampling is flawed.
         • Nonresponsive Bias
     • Poor Quality Data
       • Data that is full of errors, outliers, and noise(e.g., due to poor quality measurements)
       • Celan up the Outliers
       • Treat the Instances that are Missing Features
     • Irrelevant Features
       • The proccess of feature engineering is used to prevent this problem.
       • Feature Engineering
         • Feature Selection
         • Features Extraction
           • Combine existing features to produce a more useful one. See Dimensionality and reduction algorithms.
         • New Features
     • Overfitting the Training Data
       • Overgeneralizing
       • Noise Attributes
         • Introducing uninformative attributes can cause noise and to include unwanted patterns.
   • Bad Algorithm
   • Overfitting
     • Performs well on training data end poorly on test data.
     • Possible Solutions
       • Simplify the Model
         • Use a model with fewer parameters, reducing the number of attributes
         • Regularization
           • Constrain the model to make it simpler.
       • Gather More Training Data
       • Reduce noise in Training Data
   • Underfitting
     • Performs bad in training and test data.
     • Reasons
       • Model is Too Simple
     • Possible Solutions
       • More Powerful Model with More Parameters
       • Feeding better Features
       • Reducing Regularization
   • Testing and Validating
     • Test on Production
       • Not a good option.
     • Use Test Set
       • A common rule is 80% for training data and 20% for test data.
       • Generalization Error
         • Error rate on new cases. Training error is low and Generalization error is high you have a overfitting problem.
     • Validation Test Set
       • Used to tune the Hyperparameters and lastly test in the test set.
     • Cross Validation
       • Split the training set into complementary subsets, and each model is trained against a different combination of these subsets and validated against the remaining parts. Once the model type and hyperparameters have been selected, a final model training is made using the full training set, and the generalization error is measured on the test set. Used to avoid "wasting" too many training data into validation tests sets.

6. Algorithm

   • Hyperparameter
     • Is a parameter whose value is set before the learning process begins.
     • Regularization

7. Model

   • Model Parameter
     • Parameters derived via training. Parameters related to the model.e.g., the parameters Theta0 and Theta1 of a linear model.