A Comparative Study of Reduced Error Pruning Method in Decision Tree Algorithms

A Comparative Study of Reduced Error Pruning Method in Decision Tree Algorithms

1. predictive model which can be used to represent both classifiers and regression models

2. supervised method

   • trying to find the relationship between input attributes and target attributes which represent the relationship in structure as a model    
   • The  model constructed by using input attributes to predict target attribute values where an input pattern is classified into one of several classes based on their attributes values.            

3. one of most widely used technique

   •      because of their simplicity,       intelligibility, ease of implementation, construction faster and produce better accuracy [13] then other classification algorithms          

4. Structure

   • flow chart like tree structure, where each internal node denotes a test on an attribute, each branch represents an outcome of the test, and each leaf node represents a class    
   • tree structure (hierarchical structure)
   • rules (if-then statement)
   • Complex Tree
     •  If decision tree is complicated, tree structure and rules might be wasted    
     •   pruning procedures must be developed to facilitate the interpretation    
     • Pruning
       •     pruning is a methods that reduces the size of the tree by removing parts of the tree that not meaningful to avoid unnecessary complexity and to avoid over-fitting of the dataset.    
       • Pruning decision trees is a fundamental step in optimizing the computational efficiency as well as classification accuracy of such a model. The complexity of tree is clearly controlled by the pruning method used    
       • pre-pruning (forward pruning)
         •       re-pruning works with stop growing the tree earlier based on some stopping criteria, before it classifies the training set perfectly.    
         •       advantage is not generating full tree and the disadvantage is horizon effect phenomenon    
       • post- pruning (backward pruning)
         • growing phase
           •     allows over-fitting the data    
         • pruning phase
           • post prunes the grown tree    
           • Reduced Error Pruning (REP)
             •             Reduced Error Pruning (REP) is a post-pruning method decision tree [19]. REP finds the smallest version of the most accurate subtree with respect to the pruning set    

5. Agorithm

   • REPTree
     • fast decision tree learning
     • builds a decision tree based on the information gain or reducing the variance
     • REP with back over fitting
       • The basic of pruning of this algorithm is it used REP with back overfitting