Sem título

1880

1997

Herman Hollerith developed punched card machines to process the US census, considered a remote antecedent of massive data processing.

US Census using punch cards (Hollerith)

John Mashey coined the term "Big Data"

In his lecture "Big Data and the Next Wave of InfraStress" (SGI), Mashey warns about the stress that the accelerated growth of data will generate in the technological infrastructure.

2001

Doug Laney defines the 3V model

NASA researchers use the term "big data" in an academic article about the problems of visualizing datasets that exceed available memory.

In his research note for META Group, Laney describes Volume, Velocity, and Variety as the three dimensions that challenge traditional data management.

Stage 1: Precedents and origin of the concept (1880s–2001)

2003

Google publishes the Google File System (GFS) paper

Describe a distributed file system capable of storing large volumes of data across multiple servers. (Analysis Development)

2004

Google publishes the MapReduce paper

It presents a programming model for processing large datasets in parallel across server clusters. (Analysis Development)

2005–2006

Doug Cutting and Mike Cafarella create Hadoop

Based on Google papers, they develop an open-source framework (on Yahoo!) for distributed storage and processing of massive datasets. (Analysis Development)

2008

Hadoop becomes a high-level Apache project

Yahoo! successfully tests a cluster of more than 4,000 nodes, solidifying Hadoop as an industry standard.

2009

Apache Spark is born at UC Berkeley (AMPLab)

It is being developed as a faster alternative to Hadoop MapReduce, using in-memory processing. (Analysis Development)

Non-relational databases (MongoDB, Cassandra) are emerging, designed to handle unstructured data on a large scale.

Stage 2: Technological development of Big Data analysis (2003–2010)

2011

IBM Watson wins the Jeopardy! competition

Demonstrate the potential of natural language processing and big data analytics applied to artificial intelligence. (Application)

Documentary on the economic impact of Big Data in sectors such as healthcare, retail, and the public sector. (Need / Application)

2012

IBM incorporates the 4th V: Truthfulness

Laney's original model is expanded to include data quality and reliability.

2013

Mayer-Schönberger and Cukier publish their book Big Data

It popularizes the concept on a massive level, explaining the shift from causality to correlation in data analysis.

Stage 3: Consolidation and applications (2011–2013)

2014

Spark becomes a top-level Apache project

It is consolidating itself as the most used processing engine alongside Hadoop.

2016 - 2018

IoT (Internet of Things)

The GDPR comes into force in the European Union

2020s

Convergence with Artificial Intelligence and Machine Learning

Stage 4: Big Data in the current era (2014–present)

3v: Volume, Speed ​​and Variety

4v: Volume, Velocity, Variety and Veracity

5v: Volume, Velocity, Variety, Veracity, and Value