DATA CLEANING 3

DATA CLEANING 3

1. Why need to clean data?

   • Because Data quality may be compromised by:
     • Inconsistent data
       • What cause it?
         • Because of changes in the coding system
           • EGs
             • The coding of the products for a company may be revised at a certain date.
               • When the data before this date is merged with the recorded data, data inconsistencies result if the data before the change are not recoded to the new coding scheme.
       • What to do?
         • 1. IDENTIFY anomalous data or outliers
           • HOW to identify?
             • To examine the statistics in the Data Audit page
             • OR apply Unsupervised CLUSTERING method
               • Slides
                 • Isolated points far from cluster centres are likely to be outliers
         • 2. Subsequently correct or eliminate them