2. Big Data

2. Big Data

1. Lecture #4

   • Two definitions
     • 3V
     • Making data available for analysis
   • Big Data Resources
     • Open World Assumption: what is in the big data resource might not be correct, there are data sources that are not integrated in the data resource.
   • Lambda Architecture
     • Derivation processes in the batch layer include: data cleaning, integration, aggregation

2. Lecture #5

   • Analytics
     • Big data is about making data available for analysis, and this is what this lecture is about.
   • • 1. Discovery - visual analysis is very effective
     • 2. Data Preparation - all the processes we talked about in the context of big data resources.
     • Models
       • descriptive
       • predicitive
       • prescriptive
         • doing actions based on data analysis
       • Machine learning = writing a piece of code which will make machines learn from experience on their own in the future. 3 way to teach machines how to learn:
         • Supervised learning (labels)
           • regression
           • classification
         • Unsupervised learning (no labels)
           • clustering
             • example: k-means
         • Reinforcement learning

Images and files in the map