Data Preprocessing

Data Preprocessing

1. Data Cleaning

   • Data Cleaning can be applied to remove noise and correct incosistencies in the data
   • Missing Values
     • Each tuples have no recorded value for several attribute.
   • Noisy Data
     • Noisy is a random error or variance in a measured variable.
   • Data Cleaning as a Process
     • Missing values,noise, and inconsistencies contribute to inaccurate data.

2. Data Integration and Transformation

   • Data Integration merges data from multiple sources into coherent data store,, such as a warehouse.  
   • Data Integration
     • There are a number of issues to consider during data integration.
   • Data Tansformation
     • The data are transformed or constructed into forms appropriate for mining.

3. Data Reduction

   • Data Reduction can reduce the data size by aggregating, eliminating redundant features, or clustering , for instance.
   • Data Cube Aggregation
     • where aggregation operations are applied to the data in the construction of a data cube.
   • Attribute Subset Selection
     • where irrelevant, weakly relevant, or redundant attributes or dimensions may be detected and removed.
   • Dimensionality Reduction
     • where encoding mechanisms are used to reduce the data set size.
   • Numerosity Reduction
     • where the data are replaced or estimated by alternative, smaller data representation.

4. Data Discretization and concept Hierarchy Generation

   • Data discretization techniques can be used to reduce the number of values for a given continuous attribute by dividing the range of the attribute into intervals
   • Discretization and Concept Hierarchy Generation for Numerical Data
     • Concept hierarchies for numerical attributes can be constructed automatically based on data discretization. 
   • Concept Hierarchy Generation for Categorical
     • Categorical data are discrete data