Data Preprocessing

Data Preprocessing

1. Data Transformation

   • Smoothing
   • Aggregation
   • Generalization
   • Normalization
   • Attribute construction

2. Data Cleaning

   • Noisy Data
     • Binning
     • Regression
     • Clustering
   • Missing values
     • Ignore the tuple
     • Fill the missing values manually
     • Global constant to fill the missing values
     • Use attribute mean to fill missing value
     • Use most probable value to fill missing value
   • Removing Outliers
   • Resolve inconsistency

3. Data Reduction

   • Dimensionality Reduction
   • Numerosity Reduction
   • Data Compression

4. Data Integration

   • Entity Identification problem
   • Redundancy and correlation analysis
     • X^2 correlation test for nominal data
     • Correlation coefficient for numeric data
     • Covariance for numeric data
   • Detecting and resolving data value conflicts