Implementations: Real machine learning schemes

Implementations: Real machine learning schemes

1. Decision trees

   • From ID3 to C4.5 (pruning, numeric attributes, ...)
   • Extending ID3: ● to permit numeric attributes: straightforward ● to deal sensibly with missing values: trickier ● stability for noisy data: requires pruning mechanism
   • End result: C4.5 (Quinlan) ● Best-known and (probably) most widely-used learning algorithm ● Commercial successor: C5.0
   • Algorithm
     • Extending ID3: ● to permit numeric attributes: straightforward ● to deal sensibly with missing values: trickier ● stability for noisy data: requires pruning mechanism End result: C4.5 (Quinlan) ● Best-known and (probably) most widely-used learning algorithm ● Commercial successor: C5.0
   • Numeric attributes
     • s
     • Standard Method: Binary Split
       • Unlike nominal attributes, every attribute has many possible split points

2. Classification rules

   • From PRISM to RIPPER and PART (pruning, numeric data, ...)

3. Association Rules

   • Frequent-pattern trees

4. Extending linear models

   • Support vector machines and neural networks ● Instance-based learning

5. Instance-based learning

   • Pruning examples, generalized exemplars, distance functions