Schema Case Studies and Data Analytics Lifecycle

Schema Case Studies and Data Analytics Lifecycle

1. Successful companies in the use of big data

   • Amazon: Uses Big Data for personalized recommendations (machine learning), inventory management, and logistics optimization.
   • Netflix: Analyzes real-time viewing data to recommend content, produce original series, and reduce churn rates.
   • Uber: Processes millions of geolocation data points in real time to calculate dynamic fares and assign drivers
   • Airbnb: Uses Big Data to adjust dynamic pricing and predict demand across different areas.
   • BBVA (banking example): Analytics for fraud detection and customer segmentation.

2. Review the Big Data architecture for data processing

   • Data ingestion: Kafka, Flume, Sqoop.
   • Storage: HDFS (Hadoop Distributed File System), NoSQL (Cassandra, MongoDB)
   • Processing: Hadoop MapReduce, Apache Spark, Hive, Pig.
   • Visualization: Tableau, Power BI, Kibana.
   • Management: YARN, Oozie, Zookeeper

3. Implications of Big Data architecture for data processing

   • Scalability: ability to handle increasing volumes of data
   • Velocity: real-time processing capabilities.
   • Variety: integration of structured, semi-structured, and unstructured data.
   • Security: risks regarding privacy and regulatory compliance (e.g., GDPR).
   • Cost: distributed infrastructure may be expensive to maintain

4. Main definitions of Big Data architecture for data processing

   • HDFS: distributed file system for large-scale storage.
   • YARN: resource manager for Hadoop.
   • MapReduce: programming model to process large datasets in parallel.
   • Apache Spark: fast in-memory analytics engine.
   • Cluster: group of interconnected nodes used to process data.