Databases and Big Data

Databases and Big Data

1. Entity Relationship

   • An entity is a category of object, person, event, or thing of interest to an organisation about which data is to be recorded
   • Each entity in a database system has attributes

2. Relational Databases

   • A relational database is a collection of tables in which relationships are modelled by shared attributes.
   • Tables may be linked through the use of a common attribute, if it is the primary key is used in another table, it is called a foreign key
   • Normalisation
     • Is the process used to come up with the best possible design for a relational database
       • No data is unnecessarily duplicated
       • Data is consistent throughout the database
     • First Normal Form (1NF)
       • A table is 1NF if it contains no repeating attributes or groups of attributes
     • Second Normal Form (2NF)
       • When 1NF and it contains no partial dependecies
     • Third Normal Form (3NF)
       • When 2NF and contains no "non-key" dependencies

3. Client-Server Databases

   • Use Database Management Software (DBMS) to run the network of servers.
   • Advantages
     • The consistency of the database is maintained because only one copy of the data is held
     • An expensive resource can be made available to a large number of users
     • Access right and security can be managed and controlled centrally
     • Backup and recovery can be managed centrally
   • Potential Problems
     • Data can be comprimised and overwritten
   • Solutions to problem
     • Record Locks
       • Locks the record whenever a user is editing or updating. Denieing access to anyone attempting to edit or update the record until the transaction is complete
       • The problem with record locking is that if two users are attempting to update two records, a situation can arise which can result in a deadlock
     • Serialisation
       • This ensures that the transaction do not overlap in time so cannot interfere with each other.
       • A transaction cannot start until the previous one has finished.
       • This is implemented with timestamp ordering
     • Timestamp ordering
       • Whenever a transaction starts, it is given a timestamp. When transactions take affect, the earlier one applies first
       • To ensure that transactions are not lost, every object has a read timstamp and write timestamp
       • Before writing to an object, the read timestamp is check to be the same. If not then another transactions has taken place
     • Commitment Ordering
       • Transactions are ordered in terms of their dependencies on each other as well as the time they were initiated

4. Big Data

   • Three Aspects of Big Data are:
     • Volume - too big to fit on a single server
     • Velocity - Milliseconds or seconds to respond, particullary with streamed data
     • Variety - The data may be in many forms such as structured, unstructured, text or multimedia

Images and files in the map