Module 11: Fundamental Big Data Architecture

1. 1Operational Data Store (ODS)

   1. AAs an EDW contains large amounts of data, it is of particular interest when designing an architecture for a Big Data platform. It not only serves as a data source but also as the default interface through which various BI and analysis activities are carried out.
   2. BAlthough a single EDW can house multiple ODSs, because their primary role is to facilitate near-realtime reporting, their use is optional.
   3. COn the other hand, Big Data is mostly comprised of unstructured data that has no defined structure. Unless analyzed, the data may not have any value. Big Data analysis requires data to be stored in its raw form without being modeled first. Once collected, the exploratory phase separates signal (valuable data) from noise.
   4. DEDWs contain high value data that has gone through rigorous validation and quality control checks
2. 2Enterprise Data Warehouse & Big Data

   1. Astaging area
   2. Boperational data store (ODS)
   3. Cdata mart
   4. Danalytical database
3. 3Staging Area

   1. AAlthough a single EDW can house multiple ODSs, because their primary role is to facilitate near-realtime reporting, their use is optional.
   2. BIt may not be possible to extract data from all systems at the same time because of various technical or business-related issues. Due to this, a storage buffer where data extracted from different systems at varying times with differing frequencies can be stored is required
   3. CIt is generally an insert/read-only database utilizing either shared-nothing MPP architecture or shared-everything architecture. Data is fed from the data warehouse into the analytical database on regular intervals
   4. DIt usually includes an ETL process that ferries data from source systems into a temporary storage area. This process also contains data cleansing, validation and model transformation operations
4. 4Data Warehouse

   1. AAlthough the historical data can go back up to several years, the freshness of the current data depends on an enterprise’s reporting and analysis requirements.
   2. Ba relational database that acts as the single version of truth for the enterprise by storing standardized data from across the enterprise in a denormalized form that is fit for reporting and data analysis
   3. Cstores data related to various business entities, such as products or customers. Unlike an OLTP system, data is either inserted or retrieved but not updated in a data warehouse
   4. Dthe queries are generally more complex, involving multiple tables spanning a longer range of data.
5. 5Data Mart

   1. AAlthough the historical data can go back up to several years, the freshness of the current data depends on an enterprise’s reporting and analysis requirements
   2. BSome basic level of data model transformation and denormalization may also be performed in support of efficient reporting
   3. Cprovides a particular view on the data held in the data warehouse. Although makes data analysis and reporting easier and faster because the stored data is highly customized according to the specific requirements, it does result in data redundancy.
   4. Dcontains large amounts of data, it is of particular interest when designing an architecture for a Big Data platform
6. 6Analytical Database

   1. AIt is generally an insert/read-only database utilizing either shared-nothing MPP architecture or shared-everything architecture
   2. BData is highly standardized because it has gone through data cleansing, validation, quality and de-duplication processes, further suggesting that the data is of high value
   3. CSome basic level of data model transformation and denormalization may also be performed in support of efficient reporting
   4. DThese are generally expensive and may come bundled with the required hardware and software in the form of an appliance
7. 7EDW & Big Data Comparison

   1. AEDW Contain high value data that has gone through rigorous validation and quality control checks
   2. BOn the other hand, Big Data datasets must be stored in their raw unstructured forms, and their values are unknown
   3. CBig Data requires a repository that acts as a sink for a variety of data sources where data is stored as is
   4. DStores data related to various business entities, such as products or customers
8. 8EDW & Big Data Integration
   

   1. ABig Data requires a distributed and highly scalable storage and processing architecture with scale-out support
   2. BMost implementations of the Big Data appliance enable realtime and near-realtime analytics without the need for integrating multiple disparate technologies
   3. CA batch processing engine, such as MapReduce, can be used to convert semi- and unstructured data into meaningful structured data
   4. DThe next-generation data warehouse consists of heterogeneous technologies providing support for structured as well as semi- and unstructured data storage and analysis
9. 9Series Approach
   

   1. AThe introduction of the Big Data platform in this configuration is comparatively less disruptive because the Big Data platform is essentially an add-on module for processing semi- and unstructured data
   2. BProvides a highly scalable data storage and processing environment
   3. CBI tools and other analytical applications are unable to make use of the Big Data platform directly
   4. DThe implementation and maintenance of the interconnect can become complex if it incorporates complicated data processing, such as translation between different data types
10. 10Big Data Appliance Approach
    

    1. Arelational and non-relational storage
    2. Bconfiguration, management and application development environments
    3. Csupports very low-latency data analysis and reporting
    4. Dis analogous to the parallel approach and is also known as the logical data warehouse

11. 11Data Virtualization Approach
    

    1. AIt requires complex initial configuration, which usually results in consultation costs
    2. BIt is generally implemented as Data-as-a-Service (DaaS) by applying service-orientation principles.
    3. CThis approach makes non-relational data (Big Data datasets) more accessible through the use of standardized interfaces
    4. DIs generally implemented through complex software that can be expensive to acquire
12. 12To reduce storage cost and speed up operational reporting, an online transaction processing system (OLTP) can be replaced with an operational data store (ODS).

    1. ATrue
    2. BFalse
13. 13 In a data warehouse, data is kept in a fully normalized form for easier reporting

    1. ATrue
    2. BFalse
14. 14When compared with an ODS, a data warehouse’s queries are generally more complex, involving multiple tables spanning over a longer range of data. However, data import is less frequent because a data warehouse is not used for operational reporting

    1. ATrue
    2. BFalse
15. 15 An analytical database can either be based on a columnar database or in-memory solutions for fast data access

    1. ATrue
    2. BFalse
16. 16To obtain the benefits linked with the adoption of Big Data, an EDW needs to be replaced with Big Data-specific technologies since the EDW cannot store unstructured data

    1. ATrue
    2. BFalse
17. 17The next-generation data warehouse consists of Big Data storage technologies that can store large amounts of structured as well as unstructured data

    1. ATrue
    2. BFalse
18. 18In a Big Data environment, the query workloads are generally unknown because of the adhoc nature of analytical queries

    1. ATrue
    2. BFalse
19. 19In the series approach of EDW and Big Data integration, semi-structured and unstructured data is ingested by the Big Data platform, and only structured data is ingested by the EDW

    1. ATrue
    2. BFalse
20. 20One disadvantage of the series approach is that the Big Data platform cannot be directly accessed for performing analysis on large amounts of raw data

    1. ATrue
    2. BFalse
21. 21In the parallel approach of EDW and Big Data integration, the interconnect is a one-way connector between the EDW and the Big Data platform

    1. ATrue
    2. BFalse
22. 22One of the disadvantages of the Big Data appliance is that it does not provide horizontal scalability since it is a boxed solution

    1. ATrue
    2. BFalse
23. 23The Big Data appliance approach makes on-going system maintenance easier because this approach combines the EDW and the Big Data platform into a single preconfigured system

    1. ATrue
    2. BFalse
24. 24The data virtualization approach is also known as the logical data warehouse

    1. ATrue
    2. BFalse
25. 25The data virtualization approach uses an interconnect to provide a unified view of data across multiple data sources.

    1. ATrue
    2. BFalse
26. 26One of the disadvantages of the virtualization approach is that data from all data sources still needs to be copied over into a central repository in order to create the required services

    1. ATrue
    2. BFalse
27. 27Big Data & Cloud Computing
    

    1. ACan be utilized as a technology-enabler for Big Data under such circumstances
    2. BIngested data is stored in a distributed file
    3. CA single dataset may be of interest to multiple clients developed using different technologies that require data to be available in a specific format
    4. DSpecialized form of distributed computing that introduces utilization models for remotely provisioning scalable and measured IT resources
28. 28Big Data and Cloud Computing

    1. AProcessing and storage technologies that use cluster-based processing and storage resources
    2. BThe on-demand and elastic nature provides the ability for a much quicker setup of a Big Data platform
    3. CHas the potential to provide the basic components for a Big Data solution environment, including data, storage and processing resources
    4. DWhether processing data in batch or realtime mode, the pay-per-use model can be fully utilized to build a cluster whose size can be regulated based on the volume and velocity characteristics of Big Data
29. 29Cloud Delivery Models
    

    1. AInfrastructure-as-a-Service (IaaS)
    2. BPlatform-as-a-Service (PaaS)
    3. CSoftware-as-a-Service (SaaS)
    4. DComponent-as-a-Service (CaaS)
30. 30Cloud Deployment Model
    

    1. AHeterogeneous Cloud
    2. BPrivate Cloud
    3. CManaged Cloud
    4. DHybrid Cloud
31. 31Public Cloud

    1. AIs ideal for enterprises that initially built up Big Data analytics in-house but now want to scale out.
    2. BCan be used when input datasets are already stored in the cloud
    3. CIs generally less secure but more scalable due to larger pooling of storage and processing resources
    4. DIt is also ideal when datasets reside within an enterprise’s firewall.
32. 32Private Cloud

    1. AIt is also ideal when workloads vary
    2. BIs generally less secure but more scalable due to larger pooling of storage and processing resources
    3. CIt is also ideal when datasets reside within an enterprise’s firewall
    4. DCan help develop low latency data analysis capabilities
33. 33Hybrid Cloud

    1. AIt is also ideal when workloads vary
    2. BCan be used when input datasets are already stored in the cloud
    3. CIs a suitable choice when starting a Big Data project
    4. Dcleansing public datasets using the public cloud and then joining it with sensitive data in the private cloud for performing various data analyses
34. 34Big Data and Cloud Computing Issues
    

    1. Adata privacy
    2. Bregulatory compliance
    3. Cnetwork connectivity
    4. Ddata virtualization
35. 35Cloud-Related Big Data Patterns
    

    1. ACloud-based Big Data Analysis
    2. BCloud-based Big Data Visualization
    3. CCloud-based Big Data Storage
    4. DCloud-based Big Data Processing
36. 36Cloud-Related Big Data Patterns

    1. AThis pattern can also be employed when the data sources, such as the CRM system, reside in the same cloud (faster data transfer) or a proof-of-concept is being developed
    2. BOnce stored in the cloud, who owns the data – the users who contributed the data, such as social media data, or the enterprise processing the data
    3. CCleansing public datasets using the public cloud and then joining it with sensitive data in the private cloud for performing various data analyses
    4. DRequires the use of a processing engine deployed in the cloud
37. 37Data Transformation Compound Pattern
    

    1. AThis generally involves the use of NoSQL databases such that the downstream applications can communicate directly with these databases using RESTful APIs
    2. BThe underlying idea is to be able to ingest large amounts of raw data and pre-process it in order to make it suitable for traditional enterprise systems
    3. CKeeping multiple copies of the same dataset in different formats is not only inefficient but also adds operational and storage overheads
    4. DThe involved operations can include data cleansing, validation, model transformation and format transformation, as well as the joining of disparate datasets
38. 38Application Enhancement Compound Pattern
    

    1. AIngesting large amounts of data in order to calculate certain statistics or execute a machine learning and then to feed results to enterprise systems
    2. BThis generally involves the use of NoSQL databases such that the downstream applications can communicate directly with these databases using RESTful APIs
    3. CThe underlying idea is to be able to ingest large amounts of raw data and pre-process it in order to make it suitable for traditional enterprise systems
    4. DA dedicated storage layer helps store, pre-process and further integrate data with structured data without impacting the current storage infrastructure
39. 39Canonical Data FormatPattern
    

    1. AWarrants the use of a memory-based storage device with random read and write capability.
    2. BKeeping multiple copies of the same dataset in different formats is not only inefficient but also adds operational and storage overheads
    3. CA separate connector is used to connect to a particular query engine or the storage device
    4. DThe ingested data is stored to the distributed file system, where it is enriched via batch processing and then stored on a NoSQL database
40. 40Realtime Access Storage Pattern
    

    1. AIngested data is stored to the distributed file system, where it is enriched via batch processing and then stored on a NoSQL database
    2. BExporting the data in the form of a file, importing it into a database and then connecting the analytics tool to the database is not a viable option
    3. CIs associated with the serialization engine, data transfer engine, storage device and processing engine mechanisms
    4. DThe use of disk-based storage devices can severely impact the processing time of data
41. 41Direct Data Access Pattern

    1. AGreatly helps in speeding up data analysis and reduces dependence on IT personnel for data analysis tasks
    2. BIncurs increased cost because memory-based storage devices are expensive when compared with disk-based storage devices
    3. CKeeping multiple copies of the same dataset in different formats is not only inefficient but also adds operational and storage overheads
    4. DIs generally employed by enterprises that have just embarked on a Big Data journey
42. 42Analytical Sandbox Compound Pattern

    1. AThe results are fed directly to various downstream applications, such as an e-commerce application
    2. BIs generally employed by enterprises that have just embarked on a Big Data journey
    3. CRepresents a standalone solution environment
    4. DOffloads existing databases from having to perform complex and long-running data transformation jobs on large datasets
43. 43Confidential Data Storage Pattern
    

    1. AIn the case of a clustering algorithm applied to a customer dataset for finding customer cohorts
    2. BIs generally opted for by enterprises that want to move towards predictive and prescriptive analytics by creating richer statistical and machine learning models
    3. CCan be applied in such a case to ensure that even if malicious users get access to sensitive data
    4. DThis approach provides a better alternative in terms of uploading data to the cloud as well as data security and privacy issues
44. 44Large-Scale Graph ProcessingPattern
    

    1. AA dedicated storage layer helps store, pre-process and further integrate data with structured data without impacting the current storage infrastructure
    2. BIt involves traversing through a large number of nodes (entities) via their defined edges (links).
    3. CThis approach provides a better alternative in terms of uploading data to the cloud as well as data security and privacy issues
    4. DStoring and analyzing very large amounts of structured, unstructured and semi-structured Big Data datasets
45. 45Unstructured Data Store Compound Pattern

    1. AThe analytical operations performed in support of BI, data mining and creating statistical and machine learning models do not affect the performance
    2. BThis configuration is generally opted for by enterprises that want to move towards predictive and prescriptive analytics by creating richer statistical and machine learning models
    3. CCapable of ingesting and storing large amounts of semi-structured and unstructured data to develop highfidelity statistical and machine learning models for performing predictive and prescriptive analytics
    4. DAlthough analogous to the use of a cloud, this approach provides a better alternative in terms of uploading data to the cloud as well as data security and privacy issues
46. 46Batch Data Processing Compound Pattern
    

    1. AThe ingested data is stored to the distributed file system, where it is enriched via batch processing and then stored on a NoSQL database for performing analytical queries
    2. BTheir current storage infrastructure does not allow them to store semi-structured and unstructured data
    3. CA solution environment where the sole purpose of using the Big Data platform is to offload processing of large amounts of structured data
    4. DThis approach provides a better alternative in terms of uploading data to the cloud as well as data security and privacy issues
47. 47Dataset DenormalizationPattern
    

    1. ARequires exporting data via a relational data transfer engine to the data warehouse
    2. BCan be applied in such a case to ensure that even if malicious users get access to sensitive data
    3. CIs a solution environment comprised of inexpensive storage used to store large amounts of data from both internal and external data sources in an online fashion ready for consumption by any enterprise system
    4. DEnable the processing of datasets, which requires the use of a batch processing engine
48. 48Online Data Repository Compound Pattern
    

    1. AStoring and analyzing very large amounts of structured, unstructured and semi-structured Big Data datasets
    2. BA solution environment comprised of inexpensive storage used to store large amounts of data from both internal and external data sources in an online fashion ready for consumption by any enterprise system
    3. CLarge data volumes are available and the data itself has not lost its value because it is kept unprocessed in its raw form
    4. DThe sole purpose of using the Big Data platform is to offload processing of large amounts of structured data
49. 49Big Data Warehouse Compound Pattern
    

    1. AThis configuration is generally opted for by enterprises that want to move towards predictive and prescriptive analytics by creating richer statistical and machine learning models
    2. BLarge data volumes are available and the data itself has not lost its value because it is kept unprocessed in its raw form
    3. CStoring and analyzing very large amounts of structured, unstructured and semi-structured Big Data datasets
    4. DData from structured sources and from unstructured sources can first be stored on a distributed file system
50. 50Operational Data Store Compound Pattern

    1. Aa solution environment comprised of inexpensive NoSQL storage that is utilized as ___________ where large amounts of transactional data from operational systems across the enterprise are collected for operational BI and reporting
    2. BData from structured sources and from unstructured sources can first be stored on a distributed file system
    3. CLarge data volumes are available and the data itself has not lost its value because it is kept unprocessed in its raw form
    4. DLarger amounts of data that spreads over longer time periods can be stored, thereby providing the opportunity to enrich operational BI
51. 51Indirect Data Access Pattern

    1. AThe data can be imported into fit-forpurpose NoSQL databases, where it can be easily accessed in support of BI, reporting and other analytical use cases
    2. BEnable access to pre-processed data or analysis results stored in a Big Data solution environment via existing BI tools
    3. CA solution environment comprised of inexpensive NoSQL storage
    4. DEnable the processing of such datasets, which requires the use of a batch processing engine
52. 52Realtime Data Processing Compound Pattern

    1. AThe data can be imported into fit-forpurpose NoSQL databases, where it can be easily accessed in support of BI, reporting and other analytical use cases
    2. BA solution environment capable of processing streams of data in realtime or near-realtime, such as performing analytics on machine-generated or social media data
    3. CThe streaming data can be stored in disk-based storage, such as the distributed file system, for further analysis
    4. DEnable the processing of such datasets, which requires the use of a batch processing engine
53. 53High Velocity Realtime ProcessingPattern
    

    1. AEnable the immediate export of results
    2. BScenarios where the data needs processing as it arrives to obtain immediate results
    3. CA solution environment capable of processing streams of data in realtime or near-realtime
    4. DEnable access to pre-processed data or analysis results stored in a Big Data solution environment via existing BI tools
54. 54Streaming Egress Pattern

    1. AStoring high-volume and high-variety data in order to perform various analytics in isolation from other enterprise systems
    2. BData needs processing as it arrives to obtain immediate results
    3. CProvide integration with the enterprise identity and access management systems (IAMs)
    4. DEnable the immediate export of results
55. 55Additional Big Data Patterns
    

    1. ACentralized Dataset Governance
    2. BFan-in Ingress
    3. CCentralized Dataset Management
    4. DStreaming Egress Pattern
56. 56Centralized Access ManagementPattern
    

    1. AProvides a means for performing a range of data governance tasks from a central location
    2. BProvide integration with the enterprise identity and access management systems (IAMs)
    3. CMaintain data lineage and details about operations performed on the data across multiple processing stages
    4. DEnable policy-based access to resources within the Big Data platform via a central interface
57. 57Integrated Access Pattern
    

    1. AProvides a means for performing a range of data governance tasks from a central location
    2. BEnable policy-based access to resources within the Big Data platform via a central interface
    3. CCan be used to provide integration with the enterprise identity and access management systems (IAMs)
    4. DIs associated with the processing engine, storage device, query engine and productivity portal mechanisms
58. 58Centralized Dataset Governance Pattern

    1. AA security engine is used to enable single sign-on (SSO) functionality that generally works on the basis of trusting the IAM system for user authentication via the use of tokens
    2. BProvides a means for performing a range of data governance tasks from a central location
    3. CIn order to have maximum confidence in the processing results, there needs to be a way to retrace the processing steps that were taken
    4. DData merging may be required due to reasons such as the data is too fine-grained or arrives out of order, due to network latency or due to factors that are beyond the control of the enterprise
59. 59Automated Processing Metadata Insertion Pattern
    

    1. AData merging may be required due to reasons such as the data is too fine-grained or arrives out of order, due to network latency or due to factors that are beyond the control of the enterprise
    2. BCan be applied to maintain data lineage and details about operations performed on the data across multiple processing stages
    3. CIntermediate output from each stage is persisted temporarily to a storage device until the final result is computed and validated
    4. DIf the final results are incorrect, the entire series of steps need to be executed from scratch even if the results halfway were correct
60. 60Intermediate Results Storage Pattern
    

    1. AIntermediate output from each stage is persisted temporarily to a storage device until the final result is computed and validated
    2. BCan be applied to maintain data lineage and details about operations performed on the data across multiple processing stages
    3. CIn order to have maximum confidence in the processing results, there needs to be a way to retrace the processing steps that were taken
    4. DData needs to be simultaneously processed using different sub-systems
61. 61Fan-in IngressPattern
    

    1. AThe application of this design pattern requires the automated addition of metadata, based on a machine-readable standardized structure, during each stage of data processing
    2. BProvides scalability in the context of being able to add more data consumers via a simple configuration
    3. CIs applied when data needs to be simultaneously processed using different sub-system
    4. DCan be applied to implement logic that merges data originating from multiple sources and generally applies to situations where data is acquired in realtime
62. 62Fan-out Ingress Pattern
    

    1. AIntermediate output from each stage is persisted temporarily to a storage device until the final result is computed and validated
    2. BIs applied when data needs to be simultaneously processed using different sub-systems
    3. CMaintain data lineage and details about operations performed on the data across multiple processing stages
    4. DData is copied from the source location, stored in the queue and then forwarded to the interested subscribers
63. 63John wants to perform predictive analytics using a variety of textual log files. However, the current data storage infrastructure consists of relational database technologies. John accomplishes his goal by storing and pre-processing the log files without affecting current storage. Which compound pattern did John apply?
    

    1. AOnline Data Repository
    2. BUnstructured Data Store
    3. CBig Data Warehouse
    4. DOperational Data Store
64. 64Each day ABC’s head office receives a large number of reports from each of its branches across the world. Performance data is extracted from these reports and then imported into the enterprise data warehouse, from where it is used for various reporting tasks. The reports are in XML format and are currently coerced into a relational database and then a utility is run to perform data cleansing and extraction of the required data. The entire process of ingesting and loading into the data warehouse takes a long time, and with the reports getting more detailed, it is anticipated that timely processing of reports may not be possible. Which compound pattern can be applied to address the processing of the XML reports without requiring a staging database?
    

    1. AAnalytical Sandbox
    2. BUnstructured Data Store
    3. CData Transformation
    4. DBig Data Warehouse
65. 65XYZ is enhancing its analytical capabilities by capturing large amounts of structured and unstructured data across the enterprise and enabling its data scientists to perform advanced analytics. However, the Big Data architects have been advised that doing so should not impact the current operations of the enterprise data warehouse and that any required technology infrastructure should be kept separate with respect to the current IT environment. Which compound pattern should the Big Data architects apply for setting up the required Big Data platform?
    

    1. ABatch Data Processing
    2. BOperational Data Store
    3. CAnalytical Sandbox
    4. DOnline Data Repository
66. 66A large online bookstore currently recommends a random array of books on its website to its potential customers. However, it is planning to display personalized recommendations to its customers based on a profile match and the kinds of books they have bought in the past. This process involves ingesting a large amount of customer profile data from the CRM system, joining it with customer’s shopping history and then applying a machine learning algorithm. The generated results are then embedded on the webpage that the customer is browsing. Which compound pattern can be applied to implement the required solution?

    1. ABig Data Warehouse
    2. BOnline Data Repository
    3. CApplication Enhancement
    4. DRealtime Data Processing
67. 67A large cellular company is improving its monthly billing process by introducing itemized billing. However, with more than 5 million customers, it takes a long time to complete the simple process. The company anticipates that the new feature will take twice the current time. Davon, a Big Data architect, proposes a Big Data technologies-based solution that accomplishes the new itemized billing process quickly. Which compound pattern will Davon apply to complete the task?
    

    1. AApplication Enhancement
    2. BOnline Data Repository
    3. CBatch Data Processing
    4. DRealtime Data Processing
68. 68A renowned car manufacturer, XYZ, has modernized its manufacturing facility by adding a number of sensors across the assembly line. Each sensor provides a reading every 5 seconds. XYZ needs to monitor the readings transmitted by each of the sensors as soon as they are transmitted. The monitoring process involves a comparison of related groups of sensor readings to make sure that the readings fall within a predetermined range. Which compound pattern can be applied to achieve the desired result?
    

    1. ARealtime Data Processing
    2. BBatch Data Processing
    3. CData Transformation
    4. DAnalytical Sandbox
69. 69The data scientists at ABC often require access to historical data, going back as far ten years, in its raw form for various data analyses. Jackie, the Big Data architect, needs to provide the required data in such a way that the data can be retrieved without any delays. In which configuration should Jackie deploy the Big Data platform?
    

    1. AData Transformation
    2. BApplication Enhancement
    3. CBig Data Warehouse
    4. DOnline Data Repository
70. 70The business intelligence team at a large retail store has been asked to integrate weekly sales figures into a dashboard that currently displays daily sales figures. The team notices that the current operational data store used for generating the daily sales figures is already operating at its maximum storage capacity. In which configuration can the team implement a solution when using a Big Data platform?

    1. ABig Data Warehouse
    2. BBatch Data Processing
    3. COperational Data Store
    4. DAnalytical Sandbox
71. 71A small toy manufacturer, ABC, has seen a steady growth in the past 5 years. ABC’s current IT landscape consists of an ERP and a CRM system. Both the systems are Open Sourcebased, as ABC can only spare a limited amount of budget for IT. Sales are monitored by generating month-end reports by executing queries again with the ERP and the CRM. However, these reports only go back as far as 6 months, as older data is archived to a tape drive. Which compound pattern can be applied that enables ABC to keep a large amount of transactional data online, from which detailed sales reports can be generated more frequently?

    1. AOnline Data Repository
    2. BOperational Data Store
    3. CBig Data Warehouse
    4. DUnstructured Data Store
72. 72Lambda Architecture
    

    1. AThe intermediate output from each stage is persisted temporarily to a storage device until the final result is computed and validated. Persisting intermediate results helps find the stage where the error occurred, thereby enabling the recomputation of results from only that stage onward.
    2. BType of Big Data solution architecture that is comprised of multiple layers and forms the basis for developing highly scalable, available, eventually consistent, fault tolerant and low latency realtime Big Data solutions
    3. CUses a combination of both realtime and batch components that operate in parallel to process data without any delay
    4. DAdditional processing is generally required to put the data in the correct structure
73. 73Terminology of Lambda Architecture
    

    1. AView
    2. BModel
    3. CIndexed View
    4. DIndexing
74. 74[blank_start]Normalization[blank_end] is the process of storing data in a form that removes data duplication and ensures consistency

    1. ANormalization
    2. BDenormalization
    3. CPolyglot Persistence
    4. DCAP
75. 75[blank_start]Denormalization[blank_end] is the process of storing data in a form that introduces redundancy for faster querying

    1. APolyglot Persistence
    2. BCAP
    3. CSCV
    4. DDenormalization
76. 76[blank_start]Polyglot persistence[blank_end] is the practice of using more than one fit-for-purpose storage device for persisting data

    1. ACAP
    2. BPolyglot persistence
    3. CSCV
    4. DRecomputation Algorithm
77. 77[blank_start]CAP[blank_end] is a theorem that states a distributed storage system is only able to support two of the following constraints at any point in time: consistency, availability and partition-tolerance

    1. ARecomputation Algorithm
    2. BSCV
    3. CCAP
    4. DIncremental/Approximate Algorithm
78. 78[blank_start]SCV[blank_end] is a principle that states that a processing system is only capable of supporting two of the following: speed, consistency and volume at any point in time

    1. ASCV
    2. BRecomputation Algorithm
    3. CIncremental/Approximate Algorithm
    4. DSharding
79. 79The [blank_start]recomputation algorithm[blank_end] is an algorithm that processes the complete dataset to generate the result

    1. AIncremental/Approximate Algorithm
    2. BSharding
    3. CReplication
    4. Drecomputation algorithm
80. 80The [blank_start]incremental algorithm[blank_end] is an algorithm that only processes new data. It may use probability-based techniques and may generate results that are not fully reliable/accurate

    1. AReplication
    2. BSharding
    3. CRecomputation Algorithm
    4. Dincremental algorithm
81. 81[blank_start]Sharding[blank_end] is a method of achieving scalability by horizontally partitioning a large dataset across multiple nodes

    1. AReplication
    2. BDenormalization
    3. CSharding
    4. DNormalization
82. 82[blank_start]Replication[blank_end] is a method of achieving fault-tolerance by storing multiple copies of a dataset across multiple nodes

    1. ASharding
    2. BSCV
    3. CReplication
    4. DDenormalization
83. 83Purpose of the Lambda Architecture

    1. AThis not only helps process voluminous data faster but also helps cater to infrequent or ad-hoc data processing requests that require above-average storage and processing resources
    2. BData architectures are becoming difficult to design and maintain due to the ever-increasing volume, velocity and variety of data.
    3. CEfficient data storage and efficient querying have incompatible requirements that require following different strategies
    4. DData is either stored in a disk-based NoSQL or a memory-based storage device, which can be a NoSQL or some other cluster-based storage technology, that enables low latency data access to perform realtime or near-realtime analytics
84. 84Lambda Architecture Characteristics
    

    1. AProcesses raw data by employing both realtime and batch data processing techniques in parallel
    2. BMaintain data lineage and details about operations performed on the data across multiple processing stages
    3. CThe results generated by realtime processing are based on incremental algorithms that may not be consistent/accurate
    4. DBatch data processing eliminates the complexity of maintaining data consistency across nodes by storing only immutable data
85. 85Lambda Architecture Layers

    1. ABatch
    2. BServing
    3. CSpeed
    4. DQuery
86. 86Batch Layer

    1. AProcessing of raw data
    2. BStorage of raw data
    3. CAd hoc reporting
    4. DCalculation of views
87. 87Lambda Architecture Batch Layer

    1. AUses incremental algorithms and processes comparatively smaller amounts of data to provide low latency results
    2. BConsists of a storage device (distributed file system), batch processing engine and a workflow engine
    3. CUses a recomputation algorithm to provide consistent accurate views and further provides fault-tolerance when compared with an incremental algorithm
    4. DComprises an enhanced version of the query engine with logic that can automatically and intelligently combine serving and speed views based on the query criteria
88. 88Lambda Architecture Serving Layer

    1. AAlthough raw data is stored, for achieving consistency, some structure needs to be applied to the data before storage
    2. BThe storage device used in this layer only needs to support batch write (no random write) with random read capabilities
    3. CAs the layer follows the mutable storage model and the processing results are generated more frequently, the storage device that stores the views needs to support random writes with random reads
    4. DFor keeping the complexity to a minimum and providing faster reads, normally a simple key-value NoSQL database is used
89. 89Lambda Architecture Speed Layer

    1. AThe use of an append-only and streaming data storage device keeps complexity to a minimum
    2. BThe views created by the batch layer are not amenable to random querying, as these are generally stored in the distributed file system
    3. CA memory-based storage device for the storage of raw data and a memory or disk-based NoSQL storage device for the storage of views is generally used
    4. DEvent data is captured using the event data transfer engine and is processed in memory via the realtime processing engine to create indexed views that are generally stored inside a NoSQL database
90. 90Lambda Architecture Query Layer

    1. AFor easier integration, the speed and serving views should be constructed in a modular manner
    2. BMerging the results from views residing in the speed and serving layers for successfully executing a query
    3. COnce the latest batch view is available via the batch layer, the corresponding results in the realtime views can be ignored or flushed
    4. DIs a high latency layer such that there is a time lag before the latest version of the views, based on fresher data, is available
91. 91Lambda Architecture Layers in Action

    1. ARaw data is fed simultaneously to the batch and speed layers, generally using the same event data transfer engine
    2. BThe batch layer can be further used for deep analytics, as it contains complete datasets
    3. Cthe limitations of the SCV principle are also relaxed, as the realtime segment provides low latency and reasonably consistent results (SC), while the batch segment provides fully consistent results based on the complete dataset (CV).
    4. DThe combination of batch and realtime segments helps ease the restrictions imposed by the CAP theorem by simultaneously satisfying CP and AP requirements, respectively.
92. 92Lambda Architecture Benefits
    

    1. AAlgorithms for the speed layer can be complex or might need some time to understand, as they use incremental or approximation (probability)-based techniques that the batch equivalent may not be using
    2. BThe complexity of the architecture is restricted to the speed layer, as that is where the incremental algorithms and read/write database are used
    3. CThe immutable nature of the batch layer helps re-process data as a result of a data processing logic change that may occur due to new business requirements or a bug fix
    4. DRealtime data processing capability is required with consistent results
93. 93Lambda Architecture Applicability
    

    1. ARealtime data processing capability is required with consistent results
    2. BFault-tolerance and accuracy need to be added to the existing realtime system
    3. CLoss of data is not acceptable
    4. DPolyglot persistence by employing fit-for-purpose storage devices at each layer
94. 94Lambda Architecture Limitations/Challenges
    

    1. AConfiguring the batch layer to process data in small batches reduces load on the speed layer
    2. BRaw data is fed simultaneously to the batch and speed layers, generally using the same event data transfer engine, and each layer can be implemented via a different set of technologies
    3. CComplexity is greatly increased, as two separate layers need building and maintaining while ensuring that each provides the same functionality
    4. DRequires schema adherence in the batch layer, which adds complexity, adds another step before data can actually be persisted and requires prior knowledge about the structure of the incoming data
95. 95Lambda Architecture Recommendations

    1. AEmploying the same processing engine for both the speed and batch layer, such as Spark, helps keep system complexity to a minimum
    2. BThe key-value storage model employed in the serving layer may not be sufficient for all types of query requirements
    3. CThe immutable nature of the batch layer helps re-process data as a result of a data processing logic change that may occur due to new business requirements or a bug fix
    4. DA balance is required based on the processing requirements, as the throughput obtained from employing small batches may be less than from larger batches and will further require frequent updates to the serving layer
96. 96In Lambda architecture, which layer(s) is/are responsible for creating indexed views?
    

    1. Abatch layer
    2. Bserving layer
    3. Cspeed layer
    4. Dquery layer
97. 97In Lambda CAP is a theorem that applies to

    1. ACloud computing
    2. BDistributed storage system
    3. CProcessing system
    4. DEDW
98. 98In Lambda SCV is a theorem that applies to

    1. AStorage devices
    2. BDistributed Storage System
    3. CCloud computing
    4. DProcessing System

Description

Module 11: Fundamental Big Data Architecture

Author

Architect BD

Language

Español

Views

7

Tags

big data

Rating

No ratings yet

Rate this quiz

Rate this quiz

More on Data Warehouse Architecture