BIG DATA

BIG DATA

1. Origin

   • The concept of Big Data emerged in the late 1990s due to the difficulty of processing and managing large volumes of information. Its background includes the U.S. Census (1880) —which led to Hollerith’s tabulating machine— and the “information explosion” of the 1940s, later consolidating as a response to the challenge of massive data.
     • New Solutions: Big Data + Cloud + Artificial Intelligence Applications: consumption, health, pandemics
     • Key Factors: In ternet and digital technologies (2000s) Growth of unstructured tabulating machine - Difficulty in data (images, videos,social media)
     • New Solutions: Apache Hadoop (open source) Distributed networks
     • Popularization: Late 1990s: the term big data emerged. Difficulty in processing large volumes:
     • Background: U.S census 1880 →took 10 years Hollerith's tabulating machine 1940s: Information explosion

2. Implications of Big Data

   • Big Data transforms the way organizations manage information, enabling faster and more accurate decisions, but it also raises challenges related to privacy, security, and data quality.
     • Advantages: Data-driven decision making. Prediction of trends and behaviors. Process optimization. Innovation in products and services.
     • Challenges: Privacy and data protection. Security against cyberattacks. Complexity in management and storage. Need for specialized talent.

3. Main Definitions of Big Data

   • The main definitions of Big Data are summarized through the 5V’s, which describe its essential characteristics: volume, velocity, variety, veracity, and value. These dimensions help to understand both the complexity and the potential of managing large amounts of information in different contexts..
     • Volume: Refers to the massive amount of data generated daily by devices, sensors, social networks, and digital transactions.
     • Velocity: Indicates the speed at which data is created, transmitted, and processed, often in real time.
     • Variety: Represents the diversity of data formats: structured (databases, tables), unstructured (text, images, videos), and semi-structured (XML, JSON).
     • Veracity: Relates to the quality, reliability, and accuracy of the data, considering noise or incomplete information.
     • Value:The ability to transform data into useful insights for decision-making, innovation, and competitive advantage.

4. Big Data Motivations

   • Big Data emerges as a response to the need to transform massive amounts of information into value. The main motivations include:
     • Process optimization: Improve efficiency in industries and services.
     • Data-driven decision making: Replace intuition with quantitative evidence.
     • Innovation: Create new products, services, and business models
     • Competitiveness: Gain strategic advantages through massive data analysis.
     • Personalization: Tailor products and experiences according to user behaviors.
     • Prediction and prevention: Anticipate trends, detect fraud, and manage risks.