Data Analytics Lifecycle

Data Analytics Lifecycle

1. Successful companies in the use of big data:

   • Amazon
   • Netflix
   • Spotify
   • Google
   • Walmart
   • He use of bigdata has made it possible to: personalize playlist, shopping experience, offer recommendations, among others.

2. Big Data architecture for data processing

   • Data Sources
     • Data preprocessing should be performed: structured and normalized data.
   • EDW ( Enterprise Data Warehouse)
     • Departamental Warehouse
       • Analysts
     • Integrates and consolidates data from across the enterprise for analysis and reporting
       • Dashboards
       • Reports
       • Alerts
   • Data Science Users

3. Implications y definitions of Big Data architecture for data processing.

   • EDWs generally limit the ability of analysts to iterate on the data in a separate nonproduction environment
   • Most traditional data architectures:
     • Inhibit data exploration and more sophisticated analysis.
     • High-value data is hard to reach and leverage
     • Predictive analytics and data mining activities are last in line for data
     • Priority is given to operational rather than analytical processes
     • Analysis may be subject to constraints of sampling, which can skew model accuracy
     • Data Science projects will remain isolated and ad hoc, rather than centrally managed.
   • Definitions
     • Data Lake (Centralized repository where large amounts of data are stored in their native format)
     • EDW (Enterprise Data Warehouse)
     • ETL (Extract, Transform, Load)
     • Hadoop (Open Source Framework)
     • NoSQL (A class of databases that do not use the traditional table-based relational model.)
     • MapReduce
     • Stream Processing (Real Time Processing)
     • Machine Learning

Imagens e arquivos do mapa