big data

Big Data

• Definitions vary but there are 3 V's that define the Big Data
• VOLUME - there is too much data to fit on a single server (e.g. Banking, mobile phone call logs)
• VELOCITY - data is updated to quickly for traditional systems to keep up (e.g. Search Engines, Stock Market)
• VARIETY - data is unstructured or comes in a wide variety of forms (feedback forms)
• Big Data - too big to be stored on a single server

Solutions

• - Machine Learning
• - Fact-Based Models
• - Graph Schemas
• - Distributed Systems
• - Functional Programming

Machine learning

• - Machine learning is a very useful tool for sealing with the variety form of big data
• - developed to read and summarise vast amounts of unstructured data
• - this is used on social media platforms to promote content to the users
• -

Fact-based model

• A fact-based model is a way of structuring unstructured data
• Facts are identified within data (likely using machine learning algorithms)
• Facts are atomic – they can not be subdivided into smaller meaningful components
• Facts are timestamped to make them immutable (they never change once created)

Distributed Systems

• - solves the volume problem
• - data needs to be spread across multiple servers
• - distributed systems cannot do linear problems
• 

Graph Schema

• 
• there are no rules so the shapes could be anything
• properties connected to the facts
• facts can be connected together
• edges connect facts

Functional programming

• - this solves the velocity problem
• - functions are chained together and the data is added at the end
• - this has no states
• - Functional programming is a paradigm of programming where functions are manipulated rather than the data

Slide 8

• there are no rules so the shapes could be anything
• properties connected to the facts lines
• facts can be connected together oval
• edges connect facts rectangle