Data Analysis Lifecycle

Data Analysis Lifecycle

1. BIG DATA

   • is applied by
     • Successful Companies
       • Amazon
         • “optimizes logistics and sales” (35% comes from recommendations).
       • Airbnb
         • “personalizes offers” with machine learning.
       • Netflix
         • “predicts consumption and designs content” (85% guided by its system).
       • Tesla
         • “improves autonomy and maintenance” through vehicle telemetry.
       • Starbucks
         • “defines store locations and offers” with geospatial and consumer habit data.
   • is supported by
     • Big Data Architecture
       • Key Characteristics
         • Horizontal scalability Distributed processing Distributed data, Data locality Fault tolerance
       • Architecture Layers
         • Analysis & Visualization (dashboards, reports, insights). Data Management (integration, quality, governance, security). Storage & Processing (batch and streaming).
       • Technological Components
         • Origins: GFS, MapReduce Frameworks: Hadoop (HDFS, YARN), Spark, Kafka Databases: Cassandra, MongoDB Visualization Tools: Power BI, Qlik Sense, Grafana
   • has
     • Implications in Data Processing
       • Distributed computing → “accelerates massive-scale analysis”. Data locality → “reduces latency and optimizes bandwidth”. High availability → “ensures resilience and fault tolerance”. Data governance & security → “require orchestration, auditing, and monitoring”. Operational costs → “balance on-premise vs. cloud with scalability”.
   • is defined as
     • Key Definitions of Architecture
       • UNIR → “system design to transform large volumes into decisions”. IBM → “data flow management from collection to consumption”. Tecnored → “scalable, flexible, and distributed technologies and design patterns”. LIS Data Solutions → “hybrid batch + real-time approach integrating IoT/ERP/CRM with predictive dashboards”.