COMP2611 - Uncertainty

COMP2611 - Uncertainty

1. Basic Probability

   • Terminology
     • Experiment
     • Outcome
     • Sample Space, S
     • Event, A
     • Random Variable, X
   • Equations
   • Independence
   • Conditional Probability
     • Not Causality

2. Bayesian Reasoning

   • Bayes Rule
   • Bayes For Classification
     • To classify the unseen using Bayes Rule, we see which probability is bigger
       • Example
         • P (scary|pointed, vicious, furry) vs. P (friendly|pointed, vicious, furry)
     • Naive Bayes
       • Assume the attributes are independent
         • Example
           • P ( pointed, vicious, furry|scary ) = P ( pointed|scary ) * P ( vicious|scary ) * P ( furry|scary )
       • Laplace Smoothing
         • Avoids problems where probabilities of a certain classification aren't present in the sample ( = 0 )
       • Discretisation
         • Take a numeric random variable and divide into nominal bins
         • To deal with numerical values
     • Baseline classification
       • Assume that the response doesn't depend on the explanatory variables.
       • Assign most frequent class to any test instance
   • Bayesian Networks
     • Nodes correspond to random variables
     • Each node has a conditional probability distribution
     • Choose order for variables

3. Decision Trees

   • Each path in tree represents a rule
   • ID3 Algorithm
     • Determine attribute which best classifies, use a root.
       • Repeat for each branch
     • Information gain
       • Expected reduction in entropy caused by knowing the value of attribute A
       • Used to determine best attribute
   • Entropy
     • Random Variable
     • Response Variable
     • Measures expected amount of information in a random variable

Images and files in the map