O GoConqr está de visual novo! Conte para nós o que você achou.

Registre-se grátis

Flashcards

Big Data Rev

Andre Clark Inglês 53 flashcards

Flashcards do conjunto (53)

  1. 1What are the 6 stages of a Database Life Cycle1. Initial Study
    2. Database Design
    3. Implementation and Loading
    4. Testing and Evaluation
    5. Operating
    6. Maintenance and Evolution
  2. 2Explain: Stage 1: Conceptual Modelling- Expressed via ERD
    - Sometimes non key attributes are shown
    - NO FK
  3. 3Explain: Stage 2: Logical Modelling- Now specific to an implementation model
    - FK's have been added
    - Expressed visually via extended Bachman diagram
  4. 4Explain: Stage 3:Physical Modelling- Specific to relational DBMS
    - MySQL can run against the database to create table structures
  5. 5What is Data?- Basis of all information systems
    - Data turns to information once processed
  6. 6What is BIG Data?- Big data is a term for data in which the data is too large to be stored or manipulated in generic data programs. Big data is generally used by companies receiving 100's of GB of data on an everyday basis.
  7. 7what is Structured Data- Organised data
    - Computer-readable and usable
    - Data can be sourced from a number of places including databases and spreadsheets
  8. 8what is Unstructured data: Give 3 examples:- Do not conform to data models and associated metadata
    - Computers have a hard time reading it
    - Pictures
    - Videos
    - Blocks of text
  9. 9name 3 Problems with spreadsheets?- Data Redundancy
    - Changing data is difficult
    - Limited security
  10. 10What is a database?- Related data sorted in a single repository
    - Eliminate inconsistency
    - Stores data structures and relationships
    - Entities - Items we want to store (Student)
    - Attributes - Piece of data we want to store (Student Name)
    - Relationships (Relationships between entities - Student may be enrolled in a course
  11. 11what is Data Hierarchy- Wisdom
    - Knowledge
    - Information
    - Data
  12. 12what are the 3 Types of Big DataStructured
    Unstructured
    Semi-Structured
  13. 13Explain: Database Management System- Collection of Programs
    - Modify
    - Store
    - Extract
    - Database
    - Small and Large Systems
  14. 14name 5 Disadvantages of DBMS- Time consuming
    - Specialised knowledge needed
    - Cost may be large
    - File size is large
    - More difficult to recover data
  15. 15name 3 Types of DBMS:- Oracle
    - MS-Access
    - SQL Server
  16. 16what are the Steps to developing a ERD:1. List the major entities in the system and represent them graphically with a rectangle
    2. Search for relationships between the entities and represent them with a diamond
    3. Add a PK for every entity
    4. Model relationship connectivity between each pair of entities
    5. Model relationship cardinalities between each pair of entities
    6. Determine if composite entities are to be created
    7. Verify the ERD
  17. 17name and Explain 8 SQL SyntaxSELECT (Column)
    FROM (Table)
    WHERE (Conditions)
    GROUP BY (Columns)
    HAVING (Group By Conditions)
    ORDER BY (Columns)
    LIMIT (Limits rows returned)
    DISTINCT (Removes repeating data)
  18. 18LIMIT- LIMIT (Constrain the number of rows returned) SELECT ProductID, UnitPrice FROM product ORDER BY UnitPrice DESC LIMIT 0,2;
  19. 19SQL Example ExplainedSELECT * (Select all)
    FROM Product (The Table)
    WHERE P_CODE = 311452 (Will only display these)
  20. 20Comparison Operators:= Equal
    > Greater Then
    < Less Than
    >= Greater than or equal
    <= Less than or equal
    <> Not equal
  21. 21what are the 6 Characteristics Big Data:- Volume
    - Variety
    - Velocity
    - Veracity
    - Variability
    - Volatility
  22. 22name 5 different types of Big Data Analysis Techniques:- Text analytics / Mining
    - Audio Analytics
    - Video Analytics
    - Social Media Analytics
    - Predictive Analytics
  23. 23what is Text analytics / mining- Used to answer business questions and improve day to day operations
    - Text mining in a computer workshop can be used to identify component defects and inform technicians to avoid the same problem
    - Identify Patterns and Trends
  24. 24what is Audio Analytics- Audio to text
    - Recognise certain sounds
    - Could be used in police footage
    - Computer can pull things out of sound
  25. 25what is Video Analytics- IBM intelligent video analytics
    - Sorts the videos in to data that can be used for patterns
    - Can be used by police to find a suspects car
    - Learns events and interests
  26. 26what is Social Media Analytics- Gathering data from conversations had on social networks
    - Business Decisions
    - Customer Centrality
    - Facebook, Twitter etc.
  27. 27what is Predictive Analytics- Extracting data from data sets to determine patterns and predicts future outcomes and trends
    - Does not tell what will happen in the future
    - Weather
  28. 28what is a Data Warehouse- A physical repository
    - Cleansed data in standardised format
    - Integrated subject orientated databases
    - Non-volatile
  29. 29what is a Data Mart- A departmental data warehouse that stores only relevant data
    - Dependant data mart - A subset is created from a data warehouse
    - Independent data mart - A small data warehouse designed for strategic business unit or a department
  30. 30what is Data Mining- CRISP - Cross industry Standard process for data mining
    - SEMMA - Sample, Explore, Modify, Model and Assess
    - KDD - Knowledge Discovery Database
    - Sources of data are often consolidated data warehouse
    - Usually client, server or web based
    - Miner is often an end user
  31. 31what is CRISPStep 1. Business Understanding
    Step 2. Data Understanding
    Step 3. Data Preparation
    Step 4. Model Building
    Step 5. Testing and Evaluation
    Step 6. Deployment
  32. 32what does SEMMA stand forSample
    Explore
    Modify
    Model
    Assess
  33. 33what is Parallel Processing- Multiple computers work on the same task
    - Same operation simultaneously
    - Improves performance
  34. 34what are 5 Data Mining Benefits- Customer Relationship Management
    - Banking and other financial
    - Retailing and Logistic
    - Manufacturing and Maintenance
    - Brokerage and Securities
    - Insurance
    - Healthcare
    - Medicine
  35. 35Data Mining Myths- Provides instant solutions
    - Not viable for business applications
    - Is only for larger business
  36. 36Data Mining Problems- Selecting wrong problem
    - Ignoring sponsor
    - Not enough time
    - Looking at wrong data
  37. 37what is a Decision TreeEmploys the divide and conquer method
  38. 38what is Cluster Analysis- Used for automatic identification or natural grouping
    - Machine learning
    - No output variable
    - Segmentation
  39. 39what is Hadoop- Open Source
    - Data processing
    - Storage
    - Big Data
    - Clustered System
    - Batch Processing
    - Processing data uses MapReduce
  40. 40what are 5 Hadoop Advantages- Scalable
    - Cost Effective
    - Flexible
    - Fast
    - Resilient to failure
  41. 41what are 4 Hadoop disadvantages- Security
    - Not Fit for small data
    - Stability issues
    - Limitations
  42. 42explain: MapReduce- Map - Takes input data and processes to produce key value pairs
    - Reduce - Takes the key value pairs and reduces or combines them to bring final result
    - Multiple maps - Bring results to the reducer
  43. 43Text Mining Concepts- Specific Questions
    - Impossible for humans
    - Filters large amount of data and extract relevant information
    - Patterns and Trends
    - Extra Information
    - Process
    - Integrity
  44. 44Text Mining Process- Establish the Corpus
    - Create the term Documents matrix
    - Extract Knowledge
  45. 45Web Mining- Largest Repository of data
    - HRML, XML and Text format
    - Too big
    - Too Complex
    Opportunities are great
    Challenges are huge
  46. 46IOT- Network enabled physical objects
    - Any time connection
    - Any place
    - Internet of things
  47. 47IOT Driving Forces- Sensor Technology
    - Cheap Miniature computers
    - Low power connectivity
    - Capable Mobile Devices
    - Power of the cloud
  48. 48IOT Challenges- Security
    - Legal
    - Social
    - Market
    - Technical Control
  49. 49Big Data Governance- Formulates policy
    - Privacy
    - Monetisation
    - Clarity of data access
    - Integration
    - Usage
    - Management
  50. 50Many To ManyResolved By Bridging
  51. 51NullsRepresent a missing value
  52. 52A primary key that consists of more than one field is called a ____ key.Composite
  53. 53A table is perceived as a ______.two-dimensional structure

Informações

Descrição
FlashCards sobre Big Data Rev, criado por Andre Clark em 13-11-2019.
Criado por
Andre Clark · nov 2019
Atualizado
14 nov 2019
Visualizações
18
Histórico
  • Criado por Ash Z · 16 nov 2018
  • Copiado por Andre Clark · 13 nov 2019
Avaliação

Mais sobre Data Warehousing and Emerging Database Technologies

Fale Conosco

Sobre o que é sua mensagem?

O que você acha do novo visual?

Você gostou?