Data Analytics Lifecycle

Data Analytics Lifecycle

1. Successful Companies in the Use of Big Data

   • Netflix
   • Amazon
   • Google
   • Arrows to specific tools/technologies they use (e.g., Hadoop, Spark, AWS)

2. Components of Big Data Architecture

   • Data Sources
     • Social media, IoT devices, transactional data, etc
   • Data Storage
     • Data Lakes (e.g., AWS S3, Azure Data Lake) and Data Warehouses (e.g., Snowflake, Google BigQuery)
   • Data Processing Frameworks
     • Hadoop, Apache Spark, Flink, et
   • Data Analysis Tools
     • SQL, Python, R, Machine Learning libraries
   • Data Visualization
     • Tableau, Power BI, QlikView, etc

3. Implications of Big Data Architecture for Data Processing

   • Scalability
   • Data Quality
   • Security and Privacy
   • Latency
   • Specialized Skills Required

4. Key Definitions

   • Data Lake:
   • ETL (Extract, Transform, Load)
   • Distributed Computing
   • Data Pipeline
   • Batch Processing vs. Real-Time Processing