DATA CLEANING 2

DATA CLEANING 2

1. Why need to clean data?

   • Because Data quality may be compromised by:
     • Noisy or anomalous data
       • What cause it?
         • Because of data corruptions during the process of data collection, recording or transmission.
           • EGs
             • A faulty equipment generating abnormal signal
             • Human errors in recording certain meter readings.
       • What to do?
         • 1. IDENTIFY anomalous data or outliers
           • HOW to identify?
             • To examine the statistics in the Data Audit page
             • OR apply Unsupervised CLUSTERING method
               • Slides
                 • Isolated points far from cluster centres are likely to be outliers
         • 2. Subsequently correct or eliminate them