Data Preprocessing

Data Preprocessing

1. Data cleaning

   • Data cleaning is the number one problem in data warehousing
   • Missing Data
   • Noisy Data
     • Noisy ;random error or variance in a measured variable
   • Binning Methods for Data Smoothing
   • Binning Methods for Data Smoothing

2. Data integration and Data integration

   • Data Transformation
     • Smoothing
     • Normalizatio
     • Attribute/feature construction
     • Aggregation
     • Generalization
   • Data integration
     • Schema integration
     • Detecting and resolving data value conflicts
     • Removing duplicates and redundant data

3. Data Discretization

   • three attributes
     • Nomina
     • Ordinal
     • Continuous
   • Binning
   • Entropy-based

4. Data reduction

   • Dimensionality reduction
     • Feature selection
     • Heuristic methods
   • Aggregation and clustering
   • Sampling