Big Data

Page 1

Examples - Volume

Banking- Billions of transactions per year take place and full details of every transaction must be tracked and stored in order to create an audit trail for possible fraud investigation.

Mobile phones- Call logs and billing information must be created for the billions of mobile phones in use around the world.

Examples- Velocity

Search Engines- Being able to quickly produce a list of web pages that contain specific keywords from the billions out there

Stock Market- Trading takes place based on second-by-second fluctuations in share prices of millions of companies

Examples- Variety

Feedback forms- These often have an "any other comments" section in which people can write whatever they'd like, how do you generalise all of these?

Describe two other characteristics (other than containing a variety) that might result in a data set being classified as Big Data

1: Volume, where there is too much data for the system to hold/store

2: Velocity, where data is uploaded/received too quickly

Page 2

Machine learning

Fact-Based model

Graph Schemas- (Schema = set of rules for doing a thing) (Property = a single item of relevant data to the fact)

Facts are linked to other facts and properties with edges

Distributed Systems-

Master/Worker dynamic. Master server takes the data or process, checks which of its worker servers has the most capacity available, then assigns the task.

Functional Programming-

Big "velocity" data can be processed using this.