Quiz
GCP - Professional Data Engineer2_2
Crie uma conta grátis para terminar este quiz
Salve seu progresso e veja sua nota final.
Já tem uma conta? Inicie sessão
Perguntas do quiz (117)
1You are building an application to share financial market data with consumers, who will receive data feeds. Data is collected from the markets in real time.
Consumers will receive the data in the following ways:
✑ Real-time event stream
✑ ANSI SQL access to real-time stream and historical data
✑ Batch historical exports
Which solution should you use?- ACloud Dataflow, Cloud SQL, Cloud Spanner
- BCloud Pub/Sub, Cloud Storage, BigQuery
- CCloud Pub/Sub, Cloud Dataproc, Cloud SQL
- DCloud Dataproc, Cloud Dataflow, BigQuery
2You have a BigQuery table that ingests data directly from a Pub/Sub subscription. The ingested data is encrypted with a Google-managed encryption key. You need to meet a new organization policy that requires you to use keys from a centralized Cloud Key Management Service (Cloud KMS) project to encrypt data at rest. What should you do?
- ACreate a new Pub/Sub topic with CMEK and use the existing BigQuery table by using Google-managed encryption key.
- BCreate a new BigQuery table and Pub/Sub topic by using customer-managed encryption keys (CMEK), and migrate the data from the old BigQuery table.
- CUse Cloud KMS encryption key with Dataflow to ingest the existing Pub/Sub subscription to the existing BigQuery table.
- DCreate a new BigQuery table by using customer-managed encryption keys (CMEK), and migrate the data from the old BigQuery table.
3Flowlogistic Case Study -
Company Overview -
Flowlogistic is a leading logistics and supply chain provider. They help businesses throughout the world manage their resources and transport them to their final destination. The company has grown rapidly, expanding their offerings to include rail, truck, aircraft, and oceanic shipping.
Company Background -
The company started as a regional trucking company, and then expanded into other logistics market. Because they have not updated their infrastructure, managing and tracking orders and shipments has become a bottleneck. To improve operations, Flowlogistic developed proprietary technology for tracking shipments in real time at the parcel level. However, they are unable to deploy it because their technology stack, based on Apache Kafka, cannot support the processing volume. In addition, Flowlogistic wants to further analyze their orders and shipments to determine how best to deploy their resources.
Solution Concept -
Flowlogistic wants to implement two concepts using the cloud:
✑ Use their proprietary technology in a real-time inventory-tracking system that indicates the location of their loads
✑ Perform analytics on all their orders and shipment logs, which contain both structured and unstructured data, to determine how best to deploy resources, which markets to expand info. They also want to use predictive analytics to learn earlier when a shipment will be delayed.
Existing Technical Environment -
Flowlogistic architecture resides in a single data center:
✑ Databases
8 physical servers in 2 clusters
- SQL Server `" user data, inventory, static data
3 physical servers
- Cassandra `" metadata, tracking messages
10 Kafka servers `" tracking message aggregation and batch insert
✑ Application servers `" customer front end, middleware for order/customs
60 virtual machines across 20 physical servers
- Tomcat `" Java services
- Nginx `" static content
- Batch servers
✑ Storage appliances
- iSCSI for virtual machine (VM) hosts
- Fibre Channel storage area network (FC SAN) `" SQL server storage
- Network-attached storage (NAS) image storage, logs, backups
✑ 10 Apache Hadoop /Spark servers
- Core Data Lake
- Data analysis workloads
✑ 20 miscellaneous servers
- Jenkins, monitoring, bastion hosts,
Business Requirements -
✑ Build a reliable and reproducible environment with scaled panty of production.
✑ Aggregate data in a centralized Data Lake for analysis
✑ Use historical data to perform predictive analytics on future shipments
✑ Accurately track every shipment worldwide using proprietary technology
✑ Improve business agility and speed of innovation through rapid provisioning of new resources
✑ Analyze and optimize architecture for performance in the cloud
✑ Migrate fully to the cloud if all other requirements are met
Technical Requirements -
✑ Handle both streaming and batch data
✑ Migrate existing Hadoop workloads
✑ Ensure architecture is scalable and elastic to meet the changing demands of the company.
✑ Use managed services whenever possible
✑ Encrypt data flight and at rest
✑ Connect a VPN between the production data center and cloud environment
SEO Statement -
We have grown so quickly that our inability to upgrade our infrastructure is really hampering further growth and efficiency. We are efficient at moving shipments around the world, but we are inefficient at moving data around.
We need to organize our information so we can more easily understand where our customers are and what they are shipping.
CTO Statement -
IT has never been a priority for us, so as our data has grown, we have not invested enough in our technology. I have a good staff to manage IT, but they are so busy managing our infrastructure that I cannot get them to do the things that really matter, such as organizing our data, building the analytics, and figuring out how to implement the CFO' s tracking technology.
CFO Statement -
Part of our competitive advantage is that we penalize ourselves for late shipments and deliveries. Knowing where out shipments are at all times has a direct correlation to our bottom line and profitability. Additionally, I don't want to commit capital to building out a server environment.
Flowlogistic's management has determined that the current Apache Kafka servers cannot handle the data volume for their real-time inventory tracking system.
You need to build a new system on Google Cloud Platform (GCP) that will feed the proprietary tracking software. The system must be able to ingest data from a variety of global sources, process and query in real-time, and store the data reliably. Which combination of GCP products should you choose?- ACloud Pub/Sub, Cloud SQL, and Cloud Storage
- BCloud Load Balancing, Cloud Dataflow, and Cloud Storage
- CCloud Pub/Sub, Cloud Dataflow, and Cloud Storage
- DCloud Pub/Sub, Cloud Dataflow, and Local SSD
4MJTelco Case Study -
Company Overview -
MJTelco is a startup that plans to build networks in rapidly growing, underserved markets around the world. The company has patents for innovative optical communications hardware. Based on these patents, they can create many reliable, high-speed backbone links with inexpensive hardware.
Company Background -
Founded by experienced telecom executives, MJTelco uses technologies originally developed to overcome communications challenges in space. Fundamental to their operation, they need to create a distributed data infrastructure that drives real-time analysis and incorporates machine learning to continuously optimize their topologies. Because their hardware is inexpensive, they plan to overdeploy the network allowing them to account for the impact of dynamic regional politics on location availability and cost.
Their management and operations teams are situated all around the globe creating many-to-many relationship between data consumers and provides in their system. After careful consideration, they decided public cloud is the perfect environment to support their needs.
Solution Concept -
MJTelco is running a successful proof-of-concept (PoC) project in its labs. They have two primary needs:
✑ Scale and harden their PoC to support significantly more data flows generated when they ramp to more than 50,000 installations.
✑ Refine their machine-learning cycles to verify and improve the dynamic models they use to control topology definition.
MJTelco will also use three separate operating environments `" development/test, staging, and production `" to meet the needs of running experiments, deploying new features, and serving production customers.
Business Requirements -
✑ Scale up their production environment with minimal cost, instantiating resources when and where needed in an unpredictable, distributed telecom user community.
✑ Ensure security of their proprietary data to protect their leading-edge machine learning and analysis.
✑ Provide reliable and timely access to data for analysis from distributed research workers
✑ Maintain isolated environments that support rapid iteration of their machine-learning models without affecting their customers.
Technical Requirements -
Ensure secure and efficient transport and storage of telemetry data
Rapidly scale instances to support between 10,000 and 100,000 data providers with multiple flows each.
Allow analysis and presentation against data tables tracking up to 2 years of data storing approximately 100m records/day
Support rapid iteration of monitoring infrastructure focused on awareness of data pipeline problems both in telemetry flows and in production learning cycles.
CEO Statement -
Our business model relies on our patents, analytics and dynamic machine learning. Our inexpensive hardware is organized to be highly reliable, which gives us cost advantages. We need to quickly stabilize our large distributed data pipelines to meet our reliability and capacity commitments.
CTO Statement -
Our public cloud services must operate as advertised. We need resources that scale and keep our data secure. We also need environments in which our data scientists can carefully study and quickly adapt our models. Because we rely on automation to process our data, we also need our development and test environments to work as we iterate.
CFO Statement -
The project is too large for us to maintain the hardware and software required for the data and analysis. Also, we cannot afford to staff an operations team to monitor so many data feeds, so we will rely on automation and infrastructure. Google Cloud's machine learning will allow our quantitative researchers to work on our high-value problems instead of problems with our data pipelines.
MJTelco is building a custom interface to share data. They have these requirements:
1. They need to do aggregations over their petabyte-scale datasets.
2. They need to scan specific time range rows with a very fast response time (milliseconds).
Which combination of Google Cloud Platform products should you recommend?- ACloud Datastore and Cloud Bigtable
- BBigQuery and Cloud Bigtable
- CBigQuery and Cloud Storage
- DCloud Bigtable and Cloud SQL
5You want to migrate your existing Teradata data warehouse to BigQuery. You want to move the historical data to BigQuery by using the most efficient method that requires the least amount of programming, but local storage space on your existing data warehouse is limited. What should you do?
- AUse BigQuery Data Transfer Service with the Teradata Parallel Transporter (TPT) tbuild utility.
- BUse BigQuery Data Transfer Service by using the Java Database Connectivity (JDBC) driver with FastExport connection.
- CCreate a script to export the historical data, and upload in batches to Cloud Storage. Set up a BigQuery Data Transfer Service instance from Cloud Storage to BigQuery.
- DCreate a Teradata Parallel Transporter (TPT) export script to export the historical data, and import to BigQuery by using the bq command-line tool.
6You are using BigQuery and Data Studio to design a customer-facing dashboard that displays large quantities of aggregated data. You expect a high volume of concurrent users. You need to optimize the dashboard to provide quick visualizations with minimal latency. What should you do?
- AUse BigQuery BI Engine with materialized views.
- BUse BigQuery BI Engine with logical views.
- CUse BigQuery BI Engine with streaming data.
- DUse BigQuery BI Engine with authorized views.
7Your organization has two Google Cloud projects, project A and project B. In project A, you have a Pub/Sub topic that receives data from confidential sources. Only the resources in project A should be able to access the data in that topic. You want to ensure that project B and any future project cannot access data in the project A topic. What should you do?
- AAdd firewall rules in project A so only traffic from the VPC in project A is permitted.
- BUse Identity and Access Management conditions to ensure that only users and service accounts in project A. can access resources in project A.
- CConfigure VPC Service Controls in the organization with a perimeter around project A.
- DConfigure VPC Service Controls in the organization with a perimeter around the VPC of project A.
8You have a data stored in BigQuery. The data in the BigQuery dataset must be highly available. You need to define a storage, backup, and recovery strategy of this data that minimizes cost. How should you configure the BigQuery table that have a recovery point objective (RPO) of 30 days?
- ASet the BigQuery dataset to be multi-regional. Create a scheduled query to make copies of the data to tables suffixed with the time of the backup. In the event of an emergency, use the backup copy of the table.
- BSet the BigQuery dataset to be multi-regional. In the event of an emergency, use a point-in-time snapshot to recover the data.
- CSet the BigQuery dataset to be regional. Create a scheduled query to make copies of the data to tables suffixed with the time of the backup. In the event of an emergency, use the backup copy of the table.
- DSet the BigQuery dataset to be regional. In the event of an emergency, use a point-in-time snapshot to recover the data.
9You need to connect multiple applications with dynamic public IP addresses to a Cloud SQL instance. You configured users with strong passwords and enforced the SSL connection to your Cloud SQL instance. You want to use Cloud SQL public IP and ensure that you have secured connections. What should you do?
- AAdd all application networks to Authorized Network and regularly update them.
- BLeave the Authorized Network empty. Use Cloud SQL Auth proxy on all applications.
- CAdd CIDR 0.0.0.0/0 network to Authorized Network. Use Cloud SQL Auth proxy on all applications.
- DAdd CIDR 0.0.0.0/0 network to Authorized Network. Use Identity and Access Management (IAM) to add users.
10After migrating ETL jobs to run on BigQuery, you need to verify that the output of the migrated jobs is the same as the output of the original. You've loaded a table containing the output of the original job and want to compare the contents with output from the migrated job to show that they are identical. The tables do not contain a primary key column that would enable you to join them together for comparison.
What should you do?- ACreate stratified random samples using the OVER() function and compare equivalent samples from each table.
- BSelect random samples from the tables using the HASH() function and compare the samples.
- CUse a Dataproc cluster and the BigQuery Hadoop connector to read the data from each table and calculate a hash from non-timestamp columns of the table after sorting. Compare the hashes of each table.
- DSelect random samples from the tables using the RAND() function and compare the samples.
11You are designing storage for 20 TB of text files as part of deploying a data pipeline on Google Cloud. Your input data is in CSV format. You want to minimize the cost of querying aggregate values for multiple users who will query the data in Cloud Storage with multiple engines. Which storage service and schema design should you use?
- AUse Cloud Storage for storage. Link as temporary tables in BigQuery for query.
- BUse Cloud Bigtable for storage. Install the HBase shell on a Compute Engine instance to query the Cloud Bigtable data.
- CUse Cloud Storage for storage. Link as permanent tables in BigQuery for query.
- DUse Cloud Bigtable for storage. Link as permanent tables in BigQuery for query.
12You need to create a SQL pipeline. The pipeline runs an aggregate SQL transformation on a BigQuery table every two hours and appends the result to another existing BigQuery table. You need to configure the pipeline to retry if errors occur. You want the pipeline to send an email notification after three consecutive failures. What should you do?
- AUse the BigQueryInsertJobOperator in Cloud Composer, set the retry parameter to three, and set the email_on_failure parameter to true.
- BCreate a BigQuery scheduled query to run the SQL transformation with schedule options that repeats every two hours, and enable notification to Pub/Sub topic. Use Pub/Sub and Cloud Functions to send an email after three failed executions.
- CCreate a BigQuery scheduled query to run the SQL transformation with schedule options that repeats every two hours, and enable email notifications.
- DUse the BigQueryUpsertTableOperator in Cloud Composer, set the retry parameter to three, and set the email_on_failure parameter to true.
13You are migrating a large number of files from a public HTTPS endpoint to Cloud Storage. The files are protected from unauthorized access using signed URLs. You created a TSV file that contains the list of object URLs and started a transfer job by using Storage Transfer Service. You notice that the job has run for a long time and eventually failed. Checking the logs of the transfer job reveals that the job was running fine until one point, and then it failed due to HTTP 403 errors on the remaining files. You verified that there were no changes to the source system. You need to fix the problem to resume the migration process. What should you do?
- ARenew the TLS certificate of the HTTPS endpoint. Remove the completed files from the TSV file and rerun the Storage Transfer Service job.
- BCreate a new TSV file for the remaining files by generating signed URLs with a longer validity period. Split the TSV file into multiple smaller files and submit them as separate Storage Transfer Service jobs in parallel.
- CSet up Cloud Storage FUSE, and mount the Cloud Storage bucket on a Compute Engine instance. Remove the completed files from the TSV file. Use a shell script to iterate through the TSV file and download the remaining URLs to the FUSE mount point.
- DUpdate the file checksums in the TSV file from using MD5 to SHA256. Remove the completed files from the TSV file and rerun the Storage Transfer Service job.
14Your company is performing data preprocessing for a learning algorithm in Google Cloud Dataflow. Numerous data logs are being are being generated during this step, and the team wants to analyze them. Due to the dynamic nature of the campaign, the data is growing exponentially every hour.
The data scientists have written the following code to read the data for a new key features in the logs.
<<image>>
You want to improve the performance of this data read. What should you do?
- ACall a transform that returns TableRow objects, where each element in the PCollection represents a single row in the table.
- BUse .fromQuery operation to read specific fields from the table.
- CSpecify the TableReference object in the code.
- DUse of both the Google BigQuery TableSchema and TableFieldSchema classes.
15You orchestrate ETL pipelines by using Cloud Composer. One of the tasks in the Apache Airflow directed acyclic graph (DAG) relies on a third-party service. You want to be notified when the task does not succeed. What should you do?
- AAssign a function with notification logic to the on_failure_callback parameter tor the operator responsible for the task at risk.
- BAssign a function with notification logic to the on_retry_callback parameter for the operator responsible for the task at risk.
- CAssign a function with notification logic to the sla_miss_callback parameter for the operator responsible for the task at risk.
- DConfigure a Cloud Monitoring alert on the sla_missed metric associated with the task at risk to trigger a notification.
16You are designing the architecture to process your data from Cloud Storage to BigQuery by using Dataflow. The network team provided you with the Shared VPC network and subnetwork to be used by your pipelines. You need to enable the deployment of the pipeline on the Shared VPC network. What should you do?
- AAssign the dataflow.admin role to the service account that executes the Dataflow pipeline.
- BAssign the dataflow.admin role to the Dataflow service agent.
- CAssign the compute.networkUser role to the Dataflow service agent.
- DAssign the compute.networkUser role to the service account that executes the Dataflow pipeline.
17You are building a streaming Dataflow pipeline that ingests noise level data from hundreds of sensors placed near construction sites across a city. The sensors measure noise level every ten seconds, and send that data to the pipeline when levels reach above 70 dBA. You need to detect the average noise level from a sensor when data is received for a duration of more than 30 minutes, but the window ends when no data has been received for 15 minutes. What should you do?
- AUse session windows with a 30-minute gap duration.
- BUse hopping windows with a 15-minute window, and a thirty-minute period.
- CUse tumbling windows with a 15-minute window and a fifteen-minute .withAllowedLateness operator.
- DUse session windows with a 15-minute gap duration.
18Your company is selecting a system to centralize data ingestion and delivery. You are considering messaging and data integration systems to address the requirements. The key requirements are:
✑ The ability to seek to a particular offset in a topic, possibly back to the start of all data ever captured
✑ Support for publish/subscribe semantics on hundreds of topics
Retain per-key ordering -
Question
Which system should you choose?- AFirebase Cloud Messaging
- BApache Kafka
- CCloud Storage
- DDataflow
19You need to create a data pipeline that copies time-series transaction data so that it can be queried from within BigQuery by your data science team for analysis.
Every hour, thousands of transactions are updated with a new status. The size of the initial dataset is 1.5 PB, and it will grow by 3 TB per day. The data is heavily structured, and your data science team will build machine learning models based on this data. You want to maximize performance and usability for your data science team. Which two strategies should you adopt? (Choose two.)- ADevelop a data pipeline where status updates are appended to BigQuery instead of updated.
- BDenormalize the data as must as possible.
- CPreserve the structure of the data as much as possible.
- DCopy a daily snapshot of transaction data to Cloud Storage and store it as an Avro file. Use BigQuery's support for external data sources to query.
- EUse BigQuery UPDATE to further reduce the size of the dataset.
20You are working on a sensitive project involving private user data. You have set up a project on Google Cloud Platform to house your work internally. An external consultant is going to assist with coding a complex transformation in a Google Cloud Dataflow pipeline for your project. How should you maintain users' privacy?
- ACreate a service account and allow the consultant to log on with it.
- BGrant the consultant the Cloud Dataflow Developer role on the project.
- CGrant the consultant the Viewer role on the project.
- DCreate an anonymized sample of the data for the consultant to work with in a different project.
21Your company has a hybrid cloud initiative. You have a complex data pipeline that moves data between cloud provider services and leverages services from each of the cloud providers. Which cloud-native service should you use to orchestrate the entire pipeline?
- ACloud Composer
- BCloud Dataproc
- CCloud Dataflow
- DCloud Dataprep
22You are building a report-only data warehouse where the data is streamed into BigQuery via the streaming API. Following Google's best practices, you have both a staging and a production table for the data. How should you design your data loading to ensure that there is only one master dataset without affecting performance on either the ingestion or reporting pieces?
- AHave a staging table that is an append-only model, and then update the production table every ninety minutes with the changes written to staging.
- BHave a staging table that moves the staged data over to the production table and deletes the contents of the staging table every thirty minutes.
- CHave a staging table that moves the staged data over to the production table and deletes the contents of the staging table every three hours.
- DHave a staging table that is an append-only model, and then update the production table every three hours with the changes written to staging.
23Your company's customer_order table in BigQuery stores the order history for 10 million customers, with a table size of 10 PB. You need to create a dashboard for the support team to view the order history. The dashboard has two filters, country_name and username. Both are string data types in the BigQuery table. When a filter is applied, the dashboard fetches the order history from the table and displays the query results. However, the dashboard is slow to show the results when applying the filters to the following query:
<<imagen>>
How should you redesign the BigQuery table to support faster access?
- ACluster the table by country and username fields.
- BPartition the table by _PARTITIONTIME.
- CPartition the table by country and username fields.
- DCluster the table by country field, and partition by username field.
24An online brokerage company requires a high volume trade processing architecture. You need to create a secure queuing system that triggers jobs. The jobs will run in Google Cloud and call the company's Python API to execute trades. You need to efficiently implement a solution. What should you do?
- AUse Cloud Composer to subscribe to a Pub/Sub topic and call the Python API.
- BUse a Pub/Sub push subscription to trigger a Cloud Function to pass the data to the Python API.
- CWrite an application that makes a queue in a NoSQL database.
- DWrite an application hosted on a Compute Engine instance that makes a push subscription to the Pub/Sub topic.
25You currently have transactional data stored on-premises in a PostgreSQL database. To modernize your data environment, you want to run transactional workloads and support analytics needs with a single database. You need to move to Google Cloud without changing database management systems, and minimize cost and complexity. What should you do?
- AMigrate to BigQuery to optimize analytics.
- BMigrate and modernize your database with Cloud Spanner.
- CMigrate your workloads to AlloyDB for PostgreSQL.
- DMigrate your PostgreSQL database to Cloud SQL for PostgreSQL.
26A shipping company has live package-tracking data that is sent to an Apache Kafka stream in real time. This is then loaded into BigQuery. Analysts in your company want to query the tracking data in BigQuery to analyze geospatial trends in the lifecycle of a package. The table was originally created with ingest-date partitioning. Over time, the query processing time has increased. You need to implement a change that would improve query performance in BigQuery. What should you do?
- AImplement clustering in BigQuery on the package-tracking ID column.
- BTier older data onto Cloud Storage files and create a BigQuery table using Cloud Storage as an external data source.
- CRe-create the table using data partitioning on the package delivery date.
- DImplement clustering in BigQuery on the ingest date column.
27MJTelco Case Study -
Company Overview -
MJTelco is a startup that plans to build networks in rapidly growing, underserved markets around the world. The company has patents for innovative optical communications hardware. Based on these patents, they can create many reliable, high-speed backbone links with inexpensive hardware.
Company Background -
Founded by experienced telecom executives, MJTelco uses technologies originally developed to overcome communications challenges in space. Fundamental to their operation, they need to create a distributed data infrastructure that drives real-time analysis and incorporates machine learning to continuously optimize their topologies. Because their hardware is inexpensive, they plan to overdeploy the network allowing them to account for the impact of dynamic regional politics on location availability and cost.
Their management and operations teams are situated all around the globe creating many-to-many relationship between data consumers and provides in their system. After careful consideration, they decided public cloud is the perfect environment to support their needs.
Solution Concept -
MJTelco is running a successful proof-of-concept (PoC) project in its labs. They have two primary needs:
✑ Scale and harden their PoC to support significantly more data flows generated when they ramp to more than 50,000 installations.
✑ Refine their machine-learning cycles to verify and improve the dynamic models they use to control topology definition.
MJTelco will also use three separate operating environments `" development/test, staging, and production `" to meet the needs of running experiments, deploying new features, and serving production customers.
Business Requirements -
✑ Scale up their production environment with minimal cost, instantiating resources when and where needed in an unpredictable, distributed telecom user community.
✑ Ensure security of their proprietary data to protect their leading-edge machine learning and analysis.
✑ Provide reliable and timely access to data for analysis from distributed research workers
✑ Maintain isolated environments that support rapid iteration of their machine-learning models without affecting their customers.
Technical Requirements -
Ensure secure and efficient transport and storage of telemetry data
Question
✑ Rapidly scale instances to support between 10,000 and 100,000 data providers with multiple flows each.
✑ Allow analysis and presentation against data tables tracking up to 2 years of data storing approximately 100m records/day
✑ Support rapid iteration of monitoring infrastructure focused on awareness of data pipeline problems both in telemetry flows and in production learning cycles.
CEO Statement -
Our business model relies on our patents, analytics and dynamic machine learning. Our inexpensive hardware is organized to be highly reliable, which gives us cost advantages. We need to quickly stabilize our large distributed data pipelines to meet our reliability and capacity commitments.
CTO Statement -
Our public cloud services must operate as advertised. We need resources that scale and keep our data secure. We also need environments in which our data scientists can carefully study and quickly adapt our models. Because we rely on automation to process our data, we also need our development and test environments to work as we iterate.
CFO Statement -
The project is too large for us to maintain the hardware and software required for the data and analysis. Also, we cannot afford to staff an operations team to monitor so many data feeds, so we will rely on automation and infrastructure. Google Cloud's machine learning will allow our quantitative researchers to work on our high-value problems instead of problems with our data pipelines.
MJTelco needs you to create a schema in Google Bigtable that will allow for the historical analysis of the last 2 years of records. Each record that comes in is sent every 15 minutes, and contains a unique identifier of the device and a data record. The most common query is for all the data for a given device for a given day.
Which schema should you use?- ARowkey: date#device_id Column data: data_point
- BRowkey: date#data_point Column data: device_id
- CRowkey: data_point Column data: device_id, date
- DRowkey: date Column data: device_id, data_point
- ERowkey: device_id Column data: date, data_point
28You have 100 GB of data stored in a BigQuery table. This data is outdated and will only be accessed one or two times a year for analytics with SQL. For backup purposes, you want to store this data to be immutable for 3 years. You want to minimize storage costs. What should you do?
- A1. Create a BigQuery table clone.
2. Query the clone when you need to perform analytics. - B1. Perform a BigQuery export to a Cloud Storage bucket with archive storage class.
2. Enable versioning on the bucket.
3. Create a BigQuery external table on the exported files. - C1. Create a BigQuery table snapshot.
2. Restore the snapshot when you need to perform analytics. - D1. Perform a BigQuery export to a Cloud Storage bucket with archive storage class.
2. Set a locked retention policy on the bucket.
3. Create a BigQuery external table on the exported files.
- A1. Create a BigQuery table clone.
29You recently deployed several data processing jobs into your Cloud Composer 2 environment. You notice that some tasks are failing in Apache Airflow. On the monitoring dashboard, you see an increase in the total workers memory usage, and there were worker pod evictions. You need to resolve these errors. What should you do? (Choose two.)
- AIncrease the maximum number of workers and reduce worker concurrency.
- BIncrease the memory available to the Airflow workers.
- CIncrease the memory available to the Airflow triggerer.
- DIncrease the directed acyclic graph (DAG) file parsing interval.
- EIncrease the Cloud Composer 2 environment size from medium to large.
30You work for a large ecommerce company. You are using Pub/Sub to ingest the clickstream data to Google Cloud for analytics. You observe that when a new subscriber connects to an existing topic to analyze data, they are unable to subscribe to older data. For an upcoming yearly sale event in two months, you need a solution that, once implemented, will enable any new subscriber to read the last 30 days of data. What should you do?
- ASet the subscriber retention policy to 30 days.
- BSet the topic retention policy to 30 days.
- CCreate a new topic, and publish the last 30 days of data each time a new subscriber connects to an existing topic.
- DAsk the source system to re-push the data to Pub/Sub, and subscribe to it.
31Your company has hired a new data scientist who wants to perform complicated analyses across very large datasets stored in Google Cloud Storage and in a
Cassandra cluster on Google Compute Engine. The scientist primarily wants to create labelled data sets for machine learning projects, along with some visualization tasks. She reports that her laptop is not powerful enough to perform her tasks and it is slowing her down. You want to help her perform her tasks.
What should you do?- AHost a visualization tool on a VM on Google Compute Engine.
- BRun a local version of Jupiter on the laptop.
- CGrant the user access to Google Cloud Shell.
- DDeploy Google Cloud Datalab to a virtual machine (VM) on Google Compute Engine.
32Your company is migrating their 30-node Apache Hadoop cluster to the cloud. They want to re-use Hadoop jobs they have already created and minimize the management of the cluster as much as possible. They also want to be able to persist data beyond the life of the cluster. What should you do?
- ACreate a Google Cloud Dataflow job to process the data.
- BCreate a Cloud Dataproc cluster that uses the Google Cloud Storage connector.
- CCreate a Hadoop cluster on Google Compute Engine that uses Local SSD disks.
- DCreate a Hadoop cluster on Google Compute Engine that uses persistent disks.
- ECreate a Google Cloud Dataproc cluster that uses persistent disks for HDFS.
33You are planning to load some of your existing on-premises data into BigQuery on Google Cloud. You want to either stream or batch-load data, depending on your use case. Additionally, you want to mask some sensitive data before loading into BigQuery. You need to do this in a programmatic way while keeping costs to a minimum. What should you do?
- AUse the BigQuery Data Transfer Service to schedule your migration. After the data is populated in BigQuery, use the connection to the Cloud Data Loss Prevention (Cloud DLP) API to de-identify the necessary data.
- BCreate your pipeline with Dataflow through the Apache Beam SDK for Python, customizing separate options within your code for streaming, batch processing, and Cloud DLP. Select BigQuery as your data sink.
- CUse Cloud Data Fusion to design your pipeline, use the Cloud DLP plug-in to de-identify data within your pipeline, and then move the data into BigQuery.
- DSet up Datastream to replicate your on-premise data on BigQuery.
34You are migrating your data warehouse to Google Cloud and decommissioning your on-premises data center. Because this is a priority for your company, you know that bandwidth will be made available for the initial data load to the cloud. The files being transferred are not large in number, but each file is 90 GB.
Additionally, you want your transactional systems to continually update the warehouse on Google Cloud in real time. What tools should you use to migrate the data and ensure that it continues to write to your warehouse?- Agsutil for both the migration and the real-time updates
- Bgsutil for the migration; Pub/Sub and Dataflow for the real-time updates
- CStorage Transfer Service for the migration; Pub/Sub and Cloud Data Fusion for the real-time updates
- DBigQuery Data Transfer Service for the migration; Pub/Sub and Dataproc for the real-time updates
35You are working on a niche product in the image recognition domain. Your team has developed a model that is dominated by custom C++ TensorFlow ops your team has implemented. These ops are used inside your main training loop and are performing bulky matrix multiplications. It currently takes up to several days to train a model. You want to decrease this time significantly and keep the cost low by using an accelerator on Google Cloud. What should you do?
- AUse Cloud GPUs after implementing GPU kernel support for your customs ops.
- BUse Cloud TPUs after implementing GPU kernel support for your customs ops.
- CUse Cloud TPUs without any additional adjustment to your code.
- DStay on CPUs, and increase the size of the cluster you're training your model on.
36You are designing a Dataflow pipeline for a batch processing job. You want to mitigate multiple zonal failures at job submission time. What should you do?
- ASpecify a worker region by using the --region flag.
- BCreate an Eventarc trigger to resubmit the job in case of zonal failure when submitting the job.
- CSet the pipeline staging location as a regional Cloud Storage bucket.
- DSubmit duplicate pipelines in two different zones by using the --zone flag.
37You are administering shared BigQuery datasets that contain views used by multiple teams in your organization. The marketing team is concerned about the variability of their monthly BigQuery analytics spend using the on-demand billing model. You need to help the marketing team establish a consistent BigQuery analytics spend each month. What should you do?
- AEstablish a BigQuery quota for the marketing team, and limit the maximum number of bytes scanned each day.
- BCreate a BigQuery reservation with a baseline of 500 slots with no autoscaling for the marketing team, and bill them back accordingly.
- CCreate a BigQuery Standard pay-as-you go reservation with a baseline of 0 slots and autoscaling set to 500 for the marketing team, and bill them back accordingly.
- DCreate a BigQuery Enterprise reservation with a baseline of 250 slots and autoscaling set to 500 for the marketing team, and bill them back accordingly.
38You are part of a healthcare organization where data is organized and managed by respective data owners in various storage services. As a result of this decentralized ecosystem, discovering and managing data has become difficult. You need to quickly identify and implement a cost-optimized solution to assist your organization with the following:
• Data management and discovery
• Data lineage tracking
• Data quality validation
How should you build the solution?- AUse BigQuery to track data lineage, and use Dataprep to manage data and perform data quality validation.
- BUse Dataplex to manage data, track data lineage, and perform data quality validation.
- CUse BigLake to convert the current solution into a data lake architecture.
- DBuild a new data discovery tool on Google Kubernetes Engine that helps with new source onboarding and data lineage tracking.
39You are building a new application that you need to collect data from in a scalable way. Data arrives continuously from the application throughout the day, and you expect to generate approximately 150 GB of JSON data per day by the end of the year. Your requirements are:
✑ Decoupling producer from consumer
✑ Space and cost-efficient storage of the raw ingested data, which is to be stored indefinitely
✑ Near real-time SQL query
✑ Maintain at least 2 years of historical data, which will be queried with SQL
Which pipeline should you use to meet these requirements?- ACreate an application that publishes events to Cloud Pub/Sub, and create Spark jobs on Cloud Dataproc to convert the JSON data to Avro format, stored on HDFS on Persistent Disk.
- BCreate an application that provides an API. Write a tool to poll the API and write data to Cloud Storage as gzipped JSON files.
- CCreate an application that writes to a Cloud SQL database to store the data. Set up periodic exports of the database to write to Cloud Storage and load into BigQuery.
- DCreate an application that publishes events to Cloud Pub/Sub, and create a Cloud Dataflow pipeline that transforms the JSON event payloads to Avro, writing the data to Cloud Storage and BigQuery.
40You are building a model to predict whether or not it will rain on a given day. You have thousands of input features and want to see if you can improve training speed by removing some features while having a minimum effect on model accuracy. What can you do?
- ACombine highly co-dependent features into one representative feature.
- BEliminate features that are highly correlated to the output labels.
- CRemove the features that have null values for more than 50% of the training records.
- DInstead of feeding in each feature individually, average their values in batches of 3.
41You are preparing an organization-wide dataset. You need to preprocess customer data stored in a restricted bucket in Cloud Storage. The data will be used to create consumer analyses. You need to comply with data privacy requirements.
What should you do?- AUse customer-managed encryption keys (CMEK) to directly encrypt the data in Cloud Storage. Use federated queries from BigQuery. Share the encryption key by following the principle of least privilege.
- BUse Dataflow and the Cloud Data Loss Prevention API to mask sensitive data. Write the processed data in BigQuery.
- CUse Dataflow and Cloud KMS to encrypt sensitive fields and write the encrypted data in BigQuery. Share the encryption key by following the principle of least privilege.
- DUse the Cloud Data Loss Prevention API and Dataflow to detect and remove sensitive fields from the data in Cloud Storage. Write the filtered data in BigQuery.
42You are administering a BigQuery dataset that uses a customer-managed encryption key (CMEK). You need to share the dataset with a partner organization that does not have access to your CMEK. What should you do?
- AExport the tables to parquet files to a Cloud Storage bucket and grant the storageinsights.viewer role on the bucket to the partner organization.
- BProvide the partner organization a copy of your CMEKs to decrypt the data.
- CCopy the tables you need to share to a dataset without CMEKs. Create an Analytics Hub listing for this dataset.
- DCreate an authorized view that contains the CMEK to decrypt the data when accessed.
43You are selecting services to write and transform JSON messages from Cloud Pub/Sub to BigQuery for a data pipeline on Google Cloud. You want to minimize service costs. You also want to monitor and accommodate input data volume that will vary in size with minimal manual intervention. What should you do?
- AUse Cloud Dataproc to run your transformations. Monitor CPU utilization for the cluster. Resize the number of worker nodes in your cluster via the command line.
- BUse Cloud Dataflow to run your transformations. Monitor the job system lag with Stackdriver. Use the default autoscaling setting for worker instances.
- CUse Cloud Dataproc to run your transformations. Use the diagnose command to generate an operational output archive. Locate the bottleneck and adjust cluster resources.
- DUse Cloud Dataflow to run your transformations. Monitor the total execution time for a sampling of jobs. Configure the job to use non-default Compute Engine machine types when needed.
44You work for a large bank that operates in locations throughout North America. You are setting up a data storage system that will handle bank account transactions. You require ACID compliance and the ability to access data with SQL. Which solution is appropriate?
- AStore transaction data in Cloud Spanner. Enable stale reads to reduce latency.
- BStore transaction data in Cloud SQL. Use a federated query BigQuery for analysis.
- CStore transaction data in BigQuery. Disabled the query cache to ensure consistency.
- DStore transaction in Cloud Spanner. Use locking read-write transactions.
45You want to store your team’s shared tables in a single dataset to make data easily accessible to various analysts. You want to make this data readable but unmodifiable by analysts. At the same time, you want to provide the analysts with individual workspaces in the same project, where they can create and store tables for their own use, without the tables being accessible by other analysts. What should you do?
- AGive analysts the BigQuery Data Viewer role at the project level. Create a dataset for each analyst, and give each analyst the BigQuery Data Editor role at the project level.
- BGive analysts the BigQuery Data Viewer role on the shared dataset. Create a dataset for each analyst, and give each analyst the BigQuery Data Editor role at the dataset level for their assigned dataset.
- CGive analysts the BigQuery Data Viewer role at the project level. Create one other dataset, and give the analysts the BigQuery Data Editor role on that dataset.
- DGive analysts the BigQuery Data Viewer role on the shared dataset. Create one other dataset and give the analysts the BigQuery Data Editor role on that dataset.
46You are migrating a table to BigQuery and are deciding on the data model. Your table stores information related to purchases made across several store locations and includes information like the time of the transaction, items purchased, the store ID, and the city and state in which the store is located. You frequently query this table to see how many of each item were sold over the past 30 days and to look at purchasing trends by state, city, and individual store. How would you model this table for the best query performance?
- ATop-level cluster by store ID first, then city, then state.
- BPartition by transaction time; cluster by store ID first, then city, then state.
- CPartition by transaction time; cluster by state first, then city, then store ID.
- DTop-level cluster by state first, then city, then store ID.
47Your business users need a way to clean and prepare data before using the data for analysis. Your business users are less technically savvy and prefer to work with graphical user interfaces to define their transformations. After the data has been transformed, the business users want to perform their analysis directly in a spreadsheet. You need to recommend a solution that they can use. What should you do?
- AUse Dataflow to clean the data, and write the results to BigQuery. Analyze the data by using Looker Studio.
- BUse Dataprep to clean the data, and write the results to BigQuery. Analyze the data by using Connected Sheets.
- CUse Dataflow to clean the data, and write the results to BigQuery. Analyze the data by using Connected Sheets.
- DUse Dataprep to clean the data, and write the results to BigQuery. Analyze the data by using Looker Studio.
48You are designing the database schema for a machine learning-based food ordering service that will predict what users want to eat. Here is some of the information you need to store:
✑ The user profile: What the user likes and doesn't like to eat
✑ The user account information: Name, address, preferred meal times
✑ The order information: When orders are made, from where, to whom
The database will be used to store all the transactional data of the product. You want to optimize the data schema. Which Google Cloud Platform product should you use?- ACloud Bigtable
- BCloud Datastore
- CBigQuery
- DCloud SQL
49You want to analyze hundreds of thousands of social media posts daily at the lowest cost and with the fewest steps.
You have the following requirements:
✑ You will batch-load the posts once per day and run them through the Cloud Natural Language API.
✑ You will extract topics and sentiment from the posts.
✑ You must store the raw posts for archiving and reprocessing.
✑ You will create dashboards to be shared with people both inside and outside your organization.
You need to store both the data extracted from the API to perform analysis as well as the raw social media posts for historical archiving. What should you do?- AFeed to social media posts into the API directly from the source, and write the extracted data from the API into BigQuery.
- BStore the social media posts and the data extracted from the API in BigQuery.
- CStore the social media posts and the data extracted from the API in Cloud SQL.
- DStore the raw social media posts in Cloud Storage, and write the data extracted from the API into BigQuery.
50You are building a data pipeline on Google Cloud. You need to prepare data using a casual method for a machine-learning process. You want to support a logistic regression model. You also need to monitor and adjust for null values, which must remain real-valued and cannot be removed. What should you do?
- AUse Cloud Dataprep to find null values in sample source data. Convert all nulls to 'none' using a Cloud Dataproc job.
- BUse Cloud Dataprep to find null values in sample source data. Convert all nulls to 0 using a Cloud Dataprep job.
- CUse Cloud Dataflow to find null values in sample source data. Convert all nulls to 0 using a custom script.
- DUse Cloud Dataflow to find null values in sample source data. Convert all nulls to 'none' using a Cloud Dataprep job.
51You store and analyze your relational data in BigQuery on Google Cloud with all data that resides in US regions. You also have a variety of object stores across Microsoft Azure and Amazon Web Services (AWS), also in US regions. You want to query all your data in BigQuery daily with as little movement of data as possible. What should you do?
- ALoad files from AWS and Azure to Cloud Storage with Cloud Shell gsutil rsync arguments.
- BUse BigQuery Data Transfer Service to load files from Azure and AWS into BigQuery.
- CUse the BigQuery Omni functionality and BigLake tables to query files in Azure and AWS.
- DCreate a Dataflow pipeline to ingest files from Azure and AWS to BigQuery.
52You work for a shipping company that has distribution centers where packages move on delivery lines to route them properly. The company wants to add cameras to the delivery lines to detect and track any visual damage to the packages in transit. You need to create a way to automate the detection of damaged packages and flag them for human review in real time while the packages are in transit. Which solution should you choose?
- AUse TensorFlow to create a model that is trained on your corpus of images. Create a Python notebook in Cloud Datalab that uses this model so you can analyze for damaged packages.
- BUse BigQuery machine learning to be able to train the model at scale, so you can analyze the packages in batches.
- CUse the Cloud Vision API to detect for damage, and raise an alert through Cloud Functions. Integrate the package tracking applications with this function.
- DTrain an AutoML model on your corpus of images, and build an API around that model to integrate with the package tracking applications.
53You currently use a SQL-based tool to visualize your data stored in BigQuery. The data visualizations require the use of outer joins and analytic functions. Visualizations must be based on data that is no less than 4 hours old. Business users are complaining that the visualizations are too slow to generate. You want to improve the performance of the visualization queries while minimizing the maintenance overhead of the data preparation pipeline. What should you do?
- ACreate materialized views for the visualization queries. Use the incremental updates capability of BigQuery materialized views to handle changed data automatically. Reference the materialized views in the data visualization tool.
- BCreate materialized views with the allow_non_incremental_definition option set to true for the visualization queries. Specify the max_staleness parameter to 4 hours and the enable_refresh parameter to true. Reference the materialized views in the data visualization tool.
- CCreate a Cloud Function instance to export the visualization query results as parquet files to a Cloud Storage bucket. Use Cloud Scheduler to trigger the Cloud Function every 4 hours. Reference the parquet files in the data visualization tool.
- DCreate views for the visualization queries. Reference the views in the data visualization tool.
54You are implementing a chatbot to help an online retailer streamline their customer service. The chatbot must be able to respond to both text and voice inquiries.
You are looking for a low-code or no-cade option, and you want to be able to easily train the chatbot to provide answers to keywords. What should you do?- AUse the Cloud Speech-to-Text API to build a Python application in App Engine.
- BUse Dialogflow to implement the chatbot, defining the intents based on the most common queries collected.
- CUse Dialogflow for simple queries and the Cloud Speech-to-Text API for complex queries.
- DUse the Cloud Speech-to-Text API to build a Python application in a Compute Engine instance.
55You are administering a BigQuery on-demand environment. Your business intelligence tool is submitting hundreds of queries each day that aggregate a large (50 TB) sales history fact table at the day and month levels. These queries have a slow response time and are exceeding cost expectations. You need to decrease response time, lower query costs, and minimize maintenance. What should you do?
- ACreate a scheduled query to build sales day and sales month aggregate tables on an hourly basis.
- BBuild materialized views on top of the sales table to aggregate data at the day and month level.
- CEnable BI Engine and add your sales table as a preferred table.
- DBuild authorized views on top of the sales table to aggregate data at the day and month level.
56You want to migrate an on-premises Hadoop system to Cloud Dataproc. Hive is the primary tool in use, and the data format is Optimized Row Columnar (ORC).
All ORC files have been successfully copied to a Cloud Storage bucket. You need to replicate some data to the cluster's local Hadoop Distributed File System
(HDFS) to maximize performance. What are two ways to start using Hive in Cloud Dataproc? (Choose two.)- ARun the gsutil utility to transfer all ORC files from the Cloud Storage bucket to HDFS. Mount the Hive tables locally.
- BRun the gsutil utility to transfer all ORC files from the Cloud Storage bucket to any node of the Dataproc cluster. Mount the Hive tables locally.
- CLoad the ORC files into BigQuery. Leverage BigQuery connector for Hadoop to mount the BigQuery tables as external Hive tables. Replicate external Hive tables to the native ones.
- DRun the gsutil utility to transfer all ORC files from the Cloud Storage bucket to the master node of the Dataproc cluster. Then run the Hadoop utility to copy them do HDFS. Mount the Hive tables from HDFS.
- ELeverage Cloud Storage connector for Hadoop to mount the ORC files as external Hive tables. Replicate external Hive tables to the native ones.
57You need to deploy additional dependencies to all nodes of a Cloud Dataproc cluster at startup using an existing initialization action. Company security policies require that Cloud Dataproc nodes do not have access to the Internet so public initialization actions cannot fetch resources. What should you do?
- AUse Resource Manager to add the service account used by the Cloud Dataproc cluster to the Network User role
- BCopy all dependencies to a Cloud Storage bucket within your VPC security perimeter
- CUse an SSH tunnel to give the Cloud Dataproc cluster access to the Internet
- DDeploy the Cloud SQL Proxy on the Cloud Dataproc master
58You need to set access to BigQuery for different departments within your company. Your solution should comply with the following requirements:
✑ Each department should have access only to their data.
✑ Each department will have one or more leads who need to be able to create and update tables and provide them to their team.
✑ Each department has data analysts who need to be able to query but not modify data.
How should you set access to the data in BigQuery?- ACreate a table for each department. Assign the department leads the role of Owner, and assign the data analysts the role of Editor on the project the table is in.
- BCreate a table for each department. Assign the department leads the role of Editor, and assign the data analysts the role of Viewer on the project the table is in.
- CCreate a dataset for each department. Assign the department leads the role of OWNER, and assign the data analysts the role of WRITER on their dataset.
- DCreate a dataset for each department. Assign the department leads the role of WRITER, and assign the data analysts the role of READER on their dataset.
59You are deploying an Apache Airflow directed acyclic graph (DAG) in a Cloud Composer 2 instance. You have incoming files in a Cloud Storage bucket that the DAG processes, one file at a time. The Cloud Composer instance is deployed in a subnetwork with no Internet access. Instead of running the DAG based on a schedule, you want to run the DAG in a reactive way every time a new file is received. What should you do?
- A1. Enable Private Google Access in the subnetwork, and set up Cloud Storage notifications to a Pub/Sub topic.
2. Create a push subscription that points to the web server URL. - B1. Enable the Cloud Composer API, and set up Cloud Storage notifications to trigger a Cloud Function.
2. Write a Cloud Function instance to call the DAG by using the Cloud Composer API and the web server URL.
3. Use VPC Serverless Access to reach the web server URL. - C1. Enable the Airflow REST API, and set up Cloud Storage notifications to trigger a Cloud Function instance.
2. Create a Private Service Connect (PSC) endpoint.
3. Write a Cloud Function that connects to the Cloud Composer cluster through the PSC endpoint. - D1. Enable the Airflow REST API, and set up Cloud Storage notifications to trigger a Cloud Function instance.
2. Write a Cloud Function instance to call the DAG by using the Airflow REST API and the web server URL.
3. Use VPC Serverless Access to reach the web server URL.
- A1. Enable Private Google Access in the subnetwork, and set up Cloud Storage notifications to a Pub/Sub topic.
60You work for a large financial institution that is planning to use Dialogflow to create a chatbot for the company's mobile app. You have reviewed old chat logs and tagged each conversation for intent based on each customer's stated intention for contacting customer service. About 70% of customer requests are simple requests that are solved within 10 intents. The remaining 30% of inquiries require much longer, more complicated requests. Which intents should you automate first?
- AAutomate a blend of the shortest and longest intents to be representative of all intents.
- BAutomate the 10 intents that cover 70% of the requests so that live agents can handle more complicated requests.
- CAutomate intents in places where common words such as 'payment' appear only once so the software isn't confused.
- DAutomate the more complicated requests first because those require more of the agents' time.
61You are developing an application on Google Cloud that will automatically generate subject labels for users' blog posts. You are under competitive pressure to add this feature quickly, and you have no additional developer resources. No one on your team has experience with machine learning. What should you do?
- ABuild and train a text classification model using TensorFlow. Deploy the model using a Kubernetes Engine cluster. Call the model from your application and process the results as labels.
- BCall the Cloud Natural Language API from your application. Process the generated Sentiment Analysis as labels.
- CBuild and train a text classification model using TensorFlow. Deploy the model using Cloud Machine Learning Engine. Call the model from your application and process the results as labels.
- DCall the Cloud Natural Language API from your application. Process the generated Entity Analysis as labels.
62You work for a farming company. You have one BigQuery table named sensors, which is about 500 MB and contains the list of your 5000 sensors, with columns for id, name, and location. This table is updated every hour. Each sensor generates one metric every 30 seconds along with a timestamp, which you want to store in BigQuery. You want to run an analytical query on the data once a week for monitoring purposes. You also want to minimize costs. What data model should you use?
- A1. Create a metrics column in the sensors table.
2. Set RECORD type and REPEATED mode for the metrics column.
3. Use an UPDATE statement every 30 seconds to add new metrics. - B1. Create a metrics table partitioned by timestamp.
2. Create a sensorId column in the metrics table, that points to the id column in the sensors table.
3. Use an INSERT statement every 30 seconds to append new metrics to the metrics table.
4. Join the two tables, if needed, when running the analytical query. - C1. Create a metrics column in the sensors table.
2. Set RECORD type and REPEATED mode for the metrics column.
3. Use an INSERT statement every 30 seconds to add new metrics. - D1. Create a metrics table partitioned by timestamp.
2. Create a sensorId column in the metrics table, which points to the id column in the sensors table.
3. Use an UPDATE statement every 30 seconds to append new metrics to the metrics table.
4. Join the two tables, if needed, when running the analytical query.
- A1. Create a metrics column in the sensors table.
63You are planning to use Cloud Storage as part of your data lake solution. The Cloud Storage bucket will contain objects ingested from external systems. Each object will be ingested once, and the access patterns of individual objects will be random. You want to minimize the cost of storing and retrieving these objects. You want to ensure that any cost optimization efforts are transparent to the users and applications. What should you do?
- ACreate two Cloud Storage buckets. Use the Standard storage class for the first bucket, and use the Coldline storage class for the second bucket. Migrate objects from the first bucket to the second bucket after 30 days.
- BCreate a Cloud Storage bucket with an Object Lifecycle Management policy to transition objects from Standard to Coldline storage class if an object is not live.
- CCreate a Cloud Storage bucket with an Object Lifecycle Management policy to transition objects from Standard to Coldline storage class if an object age reaches 30 days.
- DCreate a Cloud Storage bucket with Autoclass enabled.
64You have Google Cloud Dataflow streaming pipeline running with a Google Cloud Pub/Sub subscription as the source. You need to make an update to the code that will make the new Cloud Dataflow pipeline incompatible with the current version. You do not want to lose any data when making this update. What should you do?
- AUpdate the current pipeline and use the drain flag.
- BUpdate the current pipeline and provide the transform mapping JSON object.
- CCreate a new pipeline that has the same Cloud Pub/Sub subscription and cancel the old pipeline.
- DCreate a new pipeline that has a new Cloud Pub/Sub subscription and cancel the old pipeline.
65A data scientist has created a BigQuery ML model and asks you to create an ML pipeline to serve predictions. You have a REST API application with the requirement to serve predictions for an individual user ID with latency under 100 milliseconds. You use the following query to generate predictions: SELECT predicted_label, user_id FROM ML.PREDICT (MODEL 'dataset.model', table user_features). How should you create the ML pipeline?
- ACreate a Dataflow pipeline using BigQueryIO to read results from the query. Grant the Dataflow Worker role to the application service account.
- BCreate an Authorized View with the provided query. Share the dataset that contains the view with the application service account.
- CAdd a WHERE clause to the query, and grant the BigQuery Data Viewer role to the application service account.
- DCreate a Dataflow pipeline using BigQueryIO to read predictions for all users from the query. Write the results to Bigtable using BigtableIO. Grant the Bigtable Reader role to the application service account so that the application can read predictions for individual users from Bigtable.
66You have terabytes of customer behavioral data streaming from Google Analytics into BigQuery daily. Your customers’ information, such as their preferences, is hosted on a Cloud SQL for MySQL database. Your CRM database is hosted on a Cloud SQL for PostgreSQL instance. The marketing team wants to use your customers’ information from the two databases and the customer behavioral data to create marketing campaigns for yearly active customers. You need to ensure that the marketing team can run the campaigns over 100 times a day on typical days and up to 300 during sales. At the same time, you want to keep the load on the Cloud SQL databases to a minimum. What should you do?
- ACreate streams in Datastream to replicate the required tables from both Cloud SQL databases to BigQuery for these queries.
- BCreate a Dataproc cluster with Trino to establish connections to both Cloud SQL databases and BigQuery, to execute the queries.
- CCreate BigQuery connections to both Cloud SQL databases. Use BigQuery federated queries on the two databases and the Google Analytics data on BigQuery to run these queries.
- DCreate a job on Apache Spark with Dataproc Serverless to query both Cloud SQL databases and the Google Analytics data on BigQuery for these queries.
67You have data pipelines running on BigQuery, Dataflow, and Dataproc. You need to perform health checks and monitor their behavior, and then notify the team managing the pipelines if they fail. You also need to be able to work across multiple projects. Your preference is to use managed products or features of the platform. What should you do?
- AExport the information to Cloud Monitoring, and set up an Alerting policy
- BDevelop an App Engine application to consume logs using GCP API calls, and send emails if you find a failure in the logs
- CRun a Virtual Machine in Compute Engine with Airflow, and export the information to Cloud Monitoring
- DExport the logs to BigQuery, and set up App Engine to read that information and send emails if you find a failure in the logs
68You are managing a Cloud Dataproc cluster. You need to make a job run faster while minimizing costs, without losing work in progress on your clusters. What should you do?
- AIncrease the cluster size with preemptible worker nodes, and use Cloud Stackdriver to trigger a script to preserve work.
- BIncrease the cluster size with preemptible worker nodes, and configure them to forcefully decommission.
- CIncrease the cluster size with more non-preemptible workers.
- DIncrease the cluster size with preemptible worker nodes, and configure them to use graceful decommissioning.
69You need to create a new transaction table in Cloud Spanner that stores product sales data. You are deciding what to use as a primary key. From a performance perspective, which strategy should you choose?
- AA random universally unique identifier number (version 4 UUID)
- BThe original order identification number from the sales system, which is a monotonically increasing integer
- CThe current epoch time
- DA concatenation of the product name and the current epoch time
70You've migrated a Hadoop job from an on-prem cluster to dataproc and GCS. Your Spark job is a complicated analytical workload that consists of many shuffling operations and initial data are parquet files (on average 200-400 MB size each). You see some degradation in performance after the migration to Dataproc, so you'd like to optimize for it. You need to keep in mind that your organization is very cost-sensitive, so you'd like to continue using Dataproc on preemptibles (with 2 non-preemptible workers only) for this workload.
What should you do?- ASwitch from HDDs to SSDs, override the preemptible VMs configuration to increase the boot disk size.
- BIncrease the size of your parquet files to ensure them to be 1 GB minimum.
- CSwitch to TFRecords formats (appr. 200MB per file) instead of parquet files.
- DSwitch from HDDs to SSDs, copy initial data from GCS to HDFS, run the Spark job and copy results back to GCS.
71You are updating the code for a subscriber to a Pub/Sub feed. You are concerned that upon deployment the subscriber may erroneously acknowledge messages, leading to message loss. Your subscriber is not set up to retain acknowledged messages. What should you do to ensure that you can recover from errors after deployment?
- AEnable dead-lettering on the Pub/Sub topic to capture messages that aren't successfully acknowledged. If an error occurs after deployment, re-deliver any messages captured by the dead-letter queue.
- BUse Cloud Build for your deployment. If an error occurs after deployment, use a Seek operation to locate a timestamp logged by Cloud Build at the start of the deployment.
- CSet up the Pub/Sub emulator on your local machine. Validate the behavior of your new subscriber logic before deploying it to production.
- DCreate a Pub/Sub snapshot before deploying new subscriber code. Use a Seek operation to re-deliver messages that became available after the snapshot was created.
72Your company operates in three domains: airlines, hotels, and ride-hailing services. Each domain has two teams: analytics and data science, which create data assets in BigQuery with the help of a central data platform team. However, as each domain is evolving rapidly, the central data platform team is becoming a bottleneck. This is causing delays in deriving insights from data, and resulting in stale data when pipelines are not kept up to date. You need to design a data mesh architecture by using Dataplex to eliminate the bottleneck. What should you do?
- A1. Create one lake for each domain. Inside each lake, create one zone for each team.
2. Attach each of the BigQuery datasets created by the individual teams as assets to the respective zone.
3. Have the central data platform team manage all lakes’ data assets. - B1. Create one lake for each domain. Inside each lake, create one zone for each team.
2. Attach each of the BigQuery datasets created by the individual teams as assets to the respective zone.
3. Direct each domain to manage their own lake’s data assets. - C1. Create one lake for each team. Inside each lake, create one zone for each domain.
2. Attach each of the BigQuery datasets created by the individual teams as assets to the respective zone.
3. Direct each domain to manage their own zone’s data assets. - D1. Create one lake for each team. Inside each lake, create one zone for each domain.
2. Attach each of the BigQuery datasets created by the individual teams as assets to the respective zone.
3. Have the central data platform team manage all zones’ data assets.
- A1. Create one lake for each domain. Inside each lake, create one zone for each team.
73Your company is in a highly regulated industry. One of your requirements is to ensure individual users have access only to the minimum amount of information required to do their jobs. You want to enforce this requirement with Google BigQuery. Which three approaches can you take? (Choose three.)
- ARestrict BigQuery API access to approved users.
- BDisable writes to certain tables.
- CSegregate data across multiple tables or databases.
- DRestrict access to tables by role.
- EUse Google Stackdriver Audit Logging to determine policy violations.
- FEnsure that the data is encrypted at all times.
74You are designing an Apache Beam pipeline to enrich data from Cloud Pub/Sub with static reference data from BigQuery. The reference data is small enough to fit in memory on a single worker. The pipeline should write enriched results to BigQuery for analysis. Which job type and transforms should this pipeline use?
- AStreaming job, PubSubIO, JdbcIO, side-outputs
- BBatch job, PubSubIO, side-inputs
- CStreaming job, PubSubIO, BigQueryIO, side-outputs
- DStreaming job, PubSubIO, BigQueryIO, side-inputs
75An external customer provides you with a daily dump of data from their database. The data flows into Google Cloud Storage GCS as comma-separated values
(CSV) files. You want to analyze this data in Google BigQuery, but the data could have rows that are formatted incorrectly or corrupted. How should you build this pipeline?- ARun a Google Cloud Dataflow batch pipeline to import the data into BigQuery, and push errors to another dead-letter table for analysis.
- BUse federated data sources, and check data in the SQL query.
- CEnable BigQuery monitoring in Google Stackdriver and create an alert.
- DImport the data into BigQuery using the gcloud CLI and set max_bad_records to 0.
76Your company's data platform ingests CSV file dumps of booking and user profile data from upstream sources into Cloud Storage. The data analyst team wants to join these datasets on the email field available in both the datasets to perform analysis. However, personally identifiable information (PII) should not be accessible to the analysts. You need to de-identify the email field in both the datasets before loading them into BigQuery for analysts. What should you do?
- A1. Create a pipeline to de-identify the email field by using recordTransformations in Cloud DLP with format-preserving encryption with FFX as the de-identification transformation type.
2. Load the booking and user profile data into a BigQuery table. - B1. Load the CSV files from Cloud Storage into a BigQuery table, and enable dynamic data masking.
2. Create a policy tag with the default masking value as the data masking rule.
3. Assign the policy to the email field in both tables.
4. Assign the Identity and Access Management bigquerydatapolicy.maskedReader role for the BigQuery tables to the analysts2. Load the booking and user profile data into a BigQuery table. - C1. Load the CSV files from Cloud Storage into a BigQuery table, and enable dynamic data masking.
2. Create a policy tag with the email mask as the data masking rule.
3. Assign the policy to the email field in both tables. A
4. Assign the Identity and Access Management bigquerydatapolicy.maskedReader role for the BigQuery tables to the analysts. - D1. Create a pipeline to de-identify the email field by using recordTransformations in Cloud Data Loss Prevention (Cloud DLP) with masking as the de-identification transformations type.
2. Load the booking and user profile data into a BigQuery table.
- A1. Create a pipeline to de-identify the email field by using recordTransformations in Cloud DLP with format-preserving encryption with FFX as the de-identification transformation type.
77You store historic data in Cloud Storage. You need to perform analytics on the historic data. You want to use a solution to detect invalid data entries and perform data transformations that will not require programming or knowledge of SQL.
What should you do?- AUse Cloud Dataprep with recipes to detect errors and perform transformations.
- BUse Cloud Dataflow with Beam to detect errors and perform transformations.
- CUse federated tables in BigQuery with queries to detect errors and perform transformations.
- DUse Cloud Dataproc with a Hadoop job to detect errors and perform transformations.
78You are deploying MariaDB SQL databases on GCE VM Instances and need to configure monitoring and alerting. You want to collect metrics including network connections, disk IO and replication status from MariaDB with minimal development effort and use StackDriver for dashboards and alerts.
What should you do?- AInstall the StackDriver Logging Agent and configure fluentd in_tail plugin to read MariaDB logs.
- BInstall the OpenCensus Agent and create a custom metric collection application with a StackDriver exporter.
- CInstall the StackDriver Agent and configure the MySQL plugin.
- DPlace the MariaDB instances in an Instance Group with a Health Check.
79You have uploaded 5 years of log data to Cloud Storage. A user reported that some data points in the log data are outside of their expected ranges, which indicates errors. You need to address this issue and be able to run the process again in the future while keeping the original data for compliance reasons. What should you do?
- ACreate a Dataflow workflow that reads the data from Cloud Storage, checks for values outside the expected range, sets the value to an appropriate default, and writes the updated records to the same dataset in Cloud Storage.
- BImport the data from Cloud Storage into BigQuery. Create a new BigQuery table, and skip the rows with errors.
- CCreate a Dataflow workflow that reads the data from Cloud Storage, checks for values outside the expected range, sets the value to an appropriate default, and writes the updated records to a new dataset in Cloud Storage.
- DCreate a Compute Engine instance and create a new copy of the data in Cloud Storage. Skip the rows with errors.
80You are using Workflows to call an API that returns a 1KB JSON response, apply some complex business logic on this response, wait for the logic to complete, and then perform a load from a Cloud Storage file to BigQuery. The Workflows standard library does not have sufficient capabilities to perform your complex logic, and you want to use Python's standard library instead. You want to optimize your workflow for simplicity and speed of execution. What should you do?
- ACreate a Dataproc cluster, and use PySpark to apply the logic on your JSON file.
- BCreate a Cloud Composer environment and run the logic in Cloud Composer.
- CInvoke a subworkflow in Workflows to apply the logic on your JSON file.
- DInvoke a Cloud Function instance that uses Python to apply the logic on your JSON file.
81You need to move 2 PB of historical data from an on-premises storage appliance to Cloud Storage within six months, and your outbound network capacity is constrained to 20 Mb/sec. How should you migrate this data to Cloud Storage?
- AUse trickle or ionice along with gsutil cp to limit the amount of bandwidth gsutil utilizes to less than 20 Mb/sec so it does not interfere with the production traffic
- BUse Transfer Appliance to copy the data to Cloud Storage
- CUse gsutil cp ג€"J to compress the content being uploaded to Cloud Storage
- DCreate a private URL for the historical data, and then use Storage Transfer Service to copy the data to Cloud Storage
82You want to automate execution of a multi-step data pipeline running on Google Cloud. The pipeline includes Dataproc and Dataflow jobs that have multiple dependencies on each other. You want to use managed services where possible, and the pipeline will run every day. Which tool should you use?
- AWorkflow Templates on Dataproc
- BCloud Composer
- CCloud Scheduler
- DCron
83You have an Oracle database deployed in a VM as part of a Virtual Private Cloud (VPC) network. You want to replicate and continuously synchronize 50 tables to BigQuery. You want to minimize the need to manage infrastructure. What should you do?
- ADeploy Apache Kafka in the same VPC network, use Kafka Connect Oracle Change Data Capture (CDC), and Dataflow to stream the Kafka topic to BigQuery.
- BCreate a Datastream service from Oracle to BigQuery, use a private connectivity configuration to the same VPC network, and a connection profile to BigQuery.
- CCreate a Pub/Sub subscription to write to BigQuery directly. Deploy the Debezium Oracle connector to capture changes in the Oracle database, and sink to the Pub/Sub topic.
- DDeploy Apache Kafka in the same VPC network, use Kafka Connect Oracle change data capture (CDC), and the Kafka Connect Google BigQuery Sink Connector.
84You are configuring networking for a Dataflow job. The data pipeline uses custom container images with the libraries that are required for the transformation logic preinstalled. The data pipeline reads the data from Cloud Storage and writes the data to BigQuery. You need to ensure cost-effective and secure communication between the pipeline and Google APIs and services. What should you do?
- AEnable Cloud NAT to provide outbound internet connectivity while enforcing firewall rules.
- BLeave external IP addresses assigned to worker VMs while enforcing firewall rules.
- CDisable external IP addresses and establish a Private Service Connect endpoint IP address.
- DDisable external IP addresses from worker VMs and enable Private Google Access.
85Your organization has been collecting and analyzing data in Google BigQuery for 6 months. The majority of the data analyzed is placed in a time-partitioned table named events_partitioned. To reduce the cost of queries, your organization created a view called events, which queries only the last 14 days of data. The view is described in legacy SQL. Next month, existing applications will be connecting to BigQuery to read the events data via an ODBC connection. You need to ensure the applications can connect. Which two actions should you take? (Choose two.)
- ACreate a new partitioned table using a standard SQL query
- BCreate a new view over events using standard SQL
- CCreate a Google Cloud Identity and Access Management (Cloud IAM) role for the ODBC connection and shared ג€eventsג€
- DCreate a new view over events_partitioned using standard SQL
- ECreate a service account for the ODBC connection to use for authentication
86You are architecting a data transformation solution for BigQuery. Your developers are proficient with SQL and want to use the ELT development technique. In addition, your developers need an intuitive coding environment and the ability to manage SQL as code. You need to identify a solution for your developers to build these pipelines. What should you do?
- AUse Dataflow jobs to read data from Pub/Sub, transform the data, and load the data to BigQuery.
- BUse Data Fusion to build and execute ETL pipelines.
- CUse Cloud Composer to load data and run SQL pipelines by using the BigQuery job operators.
- DUse Dataform to build, manage, and schedule SQL pipelines.
87You are designing a cloud-native historical data processing system to meet the following conditions:
✑ The data being analyzed is in CSV, Avro, and PDF formats and will be accessed by multiple analysis tools including Dataproc, BigQuery, and Compute
Engine.
✑ A batch pipeline moves daily data.
✑ Performance is not a factor in the solution.
✑ The solution design should maximize availability.
How should you design data storage for this solution?- AStore the data in a multi-regional Cloud Storage bucket. Access the data directly using Dataproc, BigQuery, and Compute Engine.
- BStore the data in a regional Cloud Storage bucket. Access the bucket directly using Dataproc, BigQuery, and Compute Engine.
- CStore the data in BigQuery. Access the data using the BigQuery Connector on Dataproc and Compute Engine.
- DCreate a Dataproc cluster with high availability. Store the data in HDFS, and perform analysis as needed.
88You are designing storage for very large text files for a data pipeline on Google Cloud. You want to support ANSI SQL queries. You also want to support compression and parallel load from the input locations using Google recommended practices. What should you do?
- ATransform text files to compressed Avro using Cloud Dataflow. Use Cloud Storage and BigQuery permanent linked tables for query.
- BTransform text files to compressed Avro using Cloud Dataflow. Use BigQuery for storage and query.
- CCompress text files to gzip using the Grid Computing Tools. Use BigQuery for storage and query.
- DCompress text files to gzip using the Grid Computing Tools. Use Cloud Storage, and then import into Cloud Bigtable for query.
89You issue a new batch job to Dataflow. The job starts successfully, processes a few elements, and then suddenly fails and shuts down. You navigate to the
Dataflow monitoring interface where you find errors related to a particular DoFn in your pipeline. What is the most likely cause of the errors?- AInsufficient permissions
- BExceptions in worker code
- CGraph or pipeline construction
- DJob validation
90You need to migrate 1 PB of data from an on-premises data center to Google Cloud. Data transfer time during the migration should take only a few hours. You want to follow Google-recommended practices to facilitate the large data transfer over a secure connection. What should you do?
- AUse a Transfer Appliance and have engineers manually encrypt, decrypt, and verify the data.
- BEstablish a Cloud Interconnect connection between the on-premises data center and Google Cloud, and then use the Storage Transfer Service.
- CEstablish a Cloud VPN connection, start gcloud compute scp jobs in parallel, and run checksums to verify the data.
- DReduce the data into 3 TB batches, transfer the data using gsutil, and run checksums to verify the data.
91You are designing the architecture of your application to store data in Cloud Storage. Your application consists of pipelines that read data from a Cloud Storage bucket that contains raw data, and write the data to a second bucket after processing. You want to design an architecture with Cloud Storage resources that are capable of being resilient if a Google Cloud regional failure occurs. You want to minimize the recovery point objective (RPO) if a failure occurs, with no impact on applications that use the stored data. What should you do?
- AAdopt two regional Cloud Storage buckets, and update your application to write the output on both buckets.
- BAdopt two regional Cloud Storage buckets, and create a daily task to copy from one bucket to the other.
- CAdopt multi-regional Cloud Storage buckets in your architecture.
- DAdopt a dual-region Cloud Storage bucket, and enable turbo replication in your architecture.
92You are deploying a MySQL database workload onto Cloud SQL. The database must be able to scale up to support several readers from various geographic regions. The database must be highly available and meet low RTO and RPO requirements, even in the event of a regional outage. You need to ensure that interruptions to the readers are minimal during a database failover. What should you do?
- ACreate a highly available Cloud SQL instance in region A. Create a highly available read replica in region B. Scale up read workloads by creating cascading read replicas in multiple regions. Promote the read replica in region B when region A is down.
- BCreate a highly available Cloud SQL instance in region Create a highly available read replica in region B. Scale up read workloads by creating cascading read replicas in multiple regions. Backup the Cloud SQL instances to a multi-regional Cloud Storage bucket. Restore the Cloud SQL backup to a new instance in another region when Region A is down.
- CCreate a highly available Cloud SQL instance in region A. Scale up read workloads by creating read replicas in multiple regions. Promote one of the read replicas when region A is down.
- DCreate a highly available Cloud SQL instance in region A. Scale up read workloads by creating read replicas in the same region. Failover to the standby Cloud SQL instance when the primary instance fails.
93You have an Apache Kafka cluster on-prem with topics containing web application logs. You need to replicate the data to Google Cloud for analysis in BigQuery and Cloud Storage. The preferred replication method is mirroring to avoid deployment of Kafka Connect plugins.
What should you do?- ADeploy the Pub/Sub Kafka connector to your on-prem Kafka cluster and configure Pub/Sub as a Source connector. Use a Dataflow job to read from Pub/Sub and write to GCS.
- BDeploy a Kafka cluster on GCE VM Instances. Configure your on-prem cluster to mirror your topics to the cluster running in GCE. Use a Dataproc cluster or Dataflow job to read from Kafka and write to GCS.
- CDeploy a Kafka cluster on GCE VM Instances with the Pub/Sub Kafka connector configured as a Sink connector. Use a Dataproc cluster or Dataflow job to read from Kafka and write to GCS.
- DDeploy the Pub/Sub Kafka connector to your on-prem Kafka cluster and configure Pub/Sub as a Sink connector. Use a Dataflow job to read from Pub/Sub and write to GCS.
94You are designing a data mesh on Google Cloud with multiple distinct data engineering teams building data products. The typical data curation design pattern consists of landing files in Cloud Storage, transforming raw data in Cloud Storage and BigQuery datasets, and storing the final curated data product in BigQuery datasets. You need to configure Dataplex to ensure that each team can access only the assets needed to build their data products. You also need to ensure that teams can easily share the curated data product. What should you do?
- A1. Create a Dataplex virtual lake for each data product, and create a single zone to contain landing, raw, and curated data.
2. Provide the data engineering teams with full access to the virtual lake assigned to their data product. - B1. Create a single Dataplex virtual lake and create a single zone to contain landing, raw, and curated data.
2. Build separate assets for each data product within the zone.
3. Assign permissions to the data engineering teams at the zone level. - C1. Create a Dataplex virtual lake for each data product, and create multiple zones for landing, raw, and curated data.
2. Provide the data engineering teams with full access to the virtual lake assigned to their data product. - D1. Create a single Dataplex virtual lake and create a single zone to contain landing, raw, and curated data.
2. Provide each data engineering team access to the virtual lake.
- A1. Create a Dataplex virtual lake for each data product, and create a single zone to contain landing, raw, and curated data.
95Your company's customer and order databases are often under heavy load. This makes performing analytics against them difficult without harming operations.
The databases are in a MySQL cluster, with nightly backups taken using mysqldump. You want to perform analytics with minimal impact on operations. What should you do?- AConnect an on-premises Apache Hadoop cluster to MySQL and perform ETL.
- BUse an ETL tool to load the data from MySQL into Google BigQuery.
- CAdd a node to the MySQL cluster and build an OLAP cube there.
- DMount the backups to Google Cloud SQL, and then process the data using Google Cloud Dataproc.
96A live TV show asks viewers to cast votes using their mobile phones. The event generates a large volume of data during a 3-minute period. You are in charge of the "Voting infrastructure" and must ensure that the platform can handle the load and that all votes are processed. You must display partial results while voting is open. After voting closes, you need to count the votes exactly once while optimizing cost. What should you do?
- AWrite votes to a Pub/Sub topic and have Cloud Functions subscribe to it and write votes to BigQuery.
- BCreate a Cloud SQL for PostgreSQL database with high availability (HA) configuration and multiple read replicas.
- CCreate a Memorystore instance with a high availability (HA) configuration.
- DWrite votes to a Pub/Sub topic and load into both Bigtable and BigQuery via a Dataflow pipeline. Query Bigtable for real-time results and BigQuery for later analysis. Shut down the Bigtable instance when voting concludes.
97You are building a real-time prediction engine that streams files, which may contain PII (personal identifiable information) data, into Cloud Storage and eventually into BigQuery. You want to ensure that the sensitive data is masked but still maintains referential integrity, because names and emails are often used as join keys.
How should you use the Cloud Data Loss Prevention API (DLP API) to ensure that the PII data is not accessible by unauthorized individuals?- AScan every table in BigQuery, and mask the data it finds that has PII.
- BCreate a pseudonym by replacing the PII data with cryptogenic tokens, and store the non-tokenized data in a locked-down button.
- CRedact all PII data, and store a version of the unredacted data in a locked-down bucket.
- DCreate a pseudonym by replacing PII data with a cryptographic format-preserving token.
98Your company's on-premises Apache Hadoop servers are approaching end-of-life, and IT has decided to migrate the cluster to Google Cloud Dataproc. A like-for- like migration of the cluster would require 50 TB of Google Persistent Disk per node. The CIO is concerned about the cost of using that much block storage. You want to minimize the storage cost of the migration. What should you do?
- ATune the Cloud Dataproc cluster so that there is just enough disk for all data.
- BPut the data into Google Cloud Storage.
- CUse preemptible virtual machines (VMs) for the Cloud Dataproc cluster.
- DMigrate some of the cold data into Google Cloud Storage, and keep only the hot data in Persistent Disk.
99You are planning to migrate your current on-premises Apache Hadoop deployment to the cloud. You need to ensure that the deployment is as fault-tolerant and cost-effective as possible for long-running batch jobs. You want to use a managed service. What should you do?
- ADeploy a Dataproc cluster. Use an SSD persistent disk and 50% preemptible workers. Store data in Cloud Storage, and change references in scripts from hdfs:// to gs://
- BDeploy a Dataproc cluster. Use a standard persistent disk and 50% preemptible workers. Store data in Cloud Storage, and change references in scripts from hdfs:// to gs://
- CInstall Hadoop and Spark on a 10-node Compute Engine instance group with preemptible instances. Store data in HDFS. Change references in scripts from hdfs:// to gs://
- DInstall Hadoop and Spark on a 10-node Compute Engine instance group with standard instances. Install the Cloud Storage connector, and store the data in Cloud Storage. Change references in scripts from hdfs:// to gs://
100You decided to use Cloud Datastore to ingest vehicle telemetry data in real time. You want to build a storage system that will account for the long-term data growth, while keeping the costs low. You also want to create snapshots of the data periodically, so that you can make a point-in-time (PIT) recovery, or clone a copy of the data for Cloud Datastore in a different environment. You want to archive these snapshots for a long time. Which two methods can accomplish this?
- AWrite an application that uses Cloud Datastore client libraries to read all the entities. Treat each entity as a BigQuery table row via BigQuery streaming insert. Assign an export timestamp for each export, and attach it as an extra column for each row. Make sure that the BigQuery table is partitioned using the export timestamp column.
- BUse managed export, and then import the data into a BigQuery table created just for that export, and delete temporary export files.
- CUse managed export, and then import to Cloud Datastore in a separate project under a unique namespace reserved for that export.
- DUse managed export, and store the data in a Cloud Storage bucket using Nearline or Coldline class.
- EWrite an application that uses Cloud Datastore client libraries to read all the entities. Format the exported data into a JSON file. Apply compression before storing the data in Cloud Source Repositories.
101A web server sends click events to a Pub/Sub topic as messages. The web server includes an eventTimestamp attribute in the messages, which is the time when the click occurred. You have a Dataflow streaming job that reads from this Pub/Sub topic through a subscription, applies some transformations, and writes the result to another Pub/Sub topic for use by the advertising department. The advertising department needs to receive each message within 30 seconds of the corresponding click occurrence, but they report receiving the messages late. Your Dataflow job's system lag is about 5 seconds, and the data freshness is about 40 seconds. Inspecting a few messages show no more than 1 second lag between their eventTimestamp and publishTime. What is the problem and what should you do?
- AThe advertising department is causing delays when consuming the messages. Work with the advertising department to fix this.
- BMessages in your Dataflow job are processed in less than 30 seconds, but your job cannot keep up with the backlog in the Pub/Sub subscription. Optimize your job or increase the number of workers to fix this.
- CMessages in your Dataflow job are taking more than 30 seconds to process. Optimize your job or increase the number of workers to fix this.
- DThe web server is not pushing messages fast enough to Pub/Sub. Work with the web server team to fix this.
102Your company has recently grown rapidly and now ingesting data at a significantly higher rate than it was previously. You manage the daily batch MapReduce analytics jobs in Apache Hadoop. However, the recent increase in data has meant the batch jobs are falling behind. You were asked to recommend ways the development team could increase the responsiveness of the analytics without increasing costs. What should you recommend they do?
- ARewrite the job in Pig.
- BDecrease the size of the Hadoop cluster but also rewrite the job in Hive.
- CIncrease the size of the Hadoop cluster.
- DRewrite the job in Apache Spark.
103You are migrating your on-premises data warehouse to BigQuery. One of the upstream data sources resides on a MySQL. database that runs in your on-premises data center with no public IP addresses. You want to ensure that the data ingestion into BigQuery is done securely and does not go through the public internet. What should you do?
- AUse Datastream to replicate data from your on-premises MySQL database to BigQuery. Gather Datastream public IP addresses of the Google Cloud region that will be used to set up the stream. Add those IP addresses to the firewall allowlist of your on-premises data center. Use IP Allowlisting as the connectivity method and Server-only as the encryption type when setting up the connection profile in Datastream.
- BUse Datastream to replicate data from your on-premises MySQL database to BigQuery. Use Forward-SSH tunnel as the connectivity method to establish a secure tunnel between Datastream and your on-premises MySQL database through a tunnel server in your on-premises data center. Use None as the encryption type when setting up the connection profile in Datastream.
- CUse Datastream to replicate data from your on-premises MySQL database to BigQuery. Set up Cloud Interconnect between your on-premises data center and Google Cloud. Use Private connectivity as the connectivity method and allocate an IP address range within your VPC network to the Datastream connectivity configuration. Use Server-only as the encryption type when setting up the connection profile in Datastream.
- DUpdate your existing on-premises ETL tool to write to BigQuery by using the BigQuery Open Database Connectivity (ODBC) driver. Set up the proxy parameter in the simba.googlebigqueryodbc.ini file to point to your data center’s NAT gateway.
104You have a table that contains millions of rows of sales data, partitioned by date. Various applications and users query this data many times a minute. The query requires aggregating values by using AVG, MAX, and SUM, and does not require joining to other tables. The required aggregations are only computed over the past year of data, though you need to retain full historical data in the base tables. You want to ensure that the query results always include the latest data from the tables, while also reducing computation cost, maintenance overhead, and duration. What should you do?
- ACreate a materialized view to aggregate the base table data. Configure a partition expiration on the base table to retain only the last one year of partitions.
- BCreate a new table that aggregates the base table data. Include a filter clause to specify the last year of partitions. Set up a scheduled query to recreate the new table every hour.
- CCreate a materialized view to aggregate the base table data. Include a filter clause to specify the last one year of partitions.
- DCreate a view to aggregate the base table data. Include a filter clause to specify the last year of partitions.
105You are building new real-time data warehouse for your company and will use Google BigQuery streaming inserts. There is no guarantee that data will only be sent in once but you do have a unique ID for each row of data and an event timestamp. You want to ensure that duplicates are not included while interactively querying data. Which query type should you use?
- AUse the ROW_NUMBER window function with PARTITION by unique ID along with WHERE row equals 1.
- BUse GROUP BY on the unique ID column and timestamp column and SUM on the values.
- CInclude ORDER BY DESK on timestamp column and LIMIT to 1.
- DUse the LAG window function with PARTITION by unique ID along with WHERE LAG IS NOT NULL.
106You need ads data to serve AI models and historical data for analytics. Longtail and outlier data points need to be identified. You want to cleanse the data in near-real time before running it through AI models. What should you do?
- AUse Cloud Storage as a data warehouse, shell scripts for processing, and BigQuery to create views for desired datasets.
- BUse Dataflow to identify longtail and outlier data points programmatically, with BigQuery as a sink.
- CUse BigQuery to ingest, prepare, and then analyze the data, and then run queries to create views.
- DUse Cloud Composer to identify longtail and outlier data points, and then output a usable dataset to BigQuery.
107You're training a model to predict housing prices based on an available dataset with real estate properties. Your plan is to train a fully connected neural net, and you've discovered that the dataset contains latitude and longitude of the property. Real estate professionals have told you that the location of the property is highly influential on price, so you'd like to engineer a feature that incorporates this physical dependency.
What should you do?- AProvide latitude and longitude as input vectors to your neural net.
- BCreate a numeric column from a feature cross of latitude and longitude.
- CCreate a feature cross of latitude and longitude, bucketize it at the minute level and use L2 regularization during optimization.
- DCreate a feature cross of latitude and longitude, bucketize it at the minute level and use L1 regularization during optimization.
108You have a variety of files in Cloud Storage that your data science team wants to use in their models. Currently, users do not have a method to explore, cleanse, and validate the data in Cloud Storage. You are looking for a low code solution that can be used by your data science team to quickly cleanse and explore data within Cloud Storage. What should you do?
- ACreate an external table in BigQuery and use SQL to transform the data as necessary. Provide the data science team access to the external tables to explore the raw data.
- BLoad the data into BigQuery and use SQL to transform the data as necessary. Provide the data science team access to staging tables to explore the raw data.
- CProvide the data science team access to Dataprep to prepare, validate, and explore the data within Cloud Storage.
- DProvide the data science team access to Dataflow to create a pipeline to prepare and validate the raw data and load data into BigQuery for data exploration.
109You are designing a real-time system for a ride hailing app that identifies areas with high demand for rides to effectively reroute available drivers to meet the demand. The system ingests data from multiple sources to Pub/Sub, processes the data, and stores the results for visualization and analysis in real-time dashboards. The data sources include driver location updates every 5 seconds and app-based booking events from riders. The data processing involves real-time aggregation of supply and demand data for the last 30 seconds, every 2 seconds, and storing the results in a low-latency system for visualization. What should you do?
- AGroup the data by using a hopping window in a Dataflow pipeline, and write the aggregated data to Memorystore.
- BGroup the data by using a hopping window in a Dataflow pipeline, and write the aggregated data to BigQuery.
- CGroup the data by using a tumbling window in a Dataflow pipeline, and write the aggregated data to Memorystore.
- DGroup the data by using a session window in a Dataflow pipeline, and write the aggregated data to BigQuery.
110You are creating the CI/CD cycle for the code of the directed acyclic graphs (DAGs) running in Cloud Composer. Your team has two Cloud Composer instances: one instance for development and another instance for production. Your team is using a Git repository to maintain and develop the code of the DAGs. You want to deploy the DAGs automatically to Cloud Composer when a certain tag is pushed to the Git repository. What should you do?
- A1. Use Cloud Build to copy the code of the DAG to the Cloud Storage bucket of the development instance for DAG testing.
2. If the tests pass, use Cloud Build to build a container with the code of the DAG and the KubernetesPodOperator to deploy the container to the Google Kubernetes Engine (GKE) cluster of the production instance. - B1. Use Cloud Build to build a container with the code of the DAG and the KubernetesPodOperator to deploy the code to the Google Kubernetes Engine (GKE) cluster of the development instance for testing.
2. If the tests pass, use the KubernetesPodOperator to deploy the container to the GKE cluster of the production instance. - C1. Use Cloud Build to build a container and the KubernetesPodOperator to deploy the code of the DAG to the Google Kubernetes Engine (GKE) cluster of the development instance for testing.
2. If the tests pass, copy the code to the Cloud Storage bucket of the production instance. - D1. Use Cloud Build to copy the code of the DAG to the Cloud Storage bucket of the development instance for DAG testing.
2. If the tests pass, use Cloud Build to copy the code to the bucket of the production instance.
- A1. Use Cloud Build to copy the code of the DAG to the Cloud Storage bucket of the development instance for DAG testing.
111You are designing a system that requires an ACID-compliant database. You must ensure that the system requires minimal human intervention in case of a failure.
What should you do?- AConfigure a Cloud SQL for PostgreSQL instance with high availability enabled.
- BConfigure a BigQuery table with a multi-region configuration.
- CConfigure a Cloud SQL for MySQL instance with point-in-time recovery enabled.
- DConfigure a Bigtable instance with more than one cluster.
112You have Cloud Functions written in Node.js that pull messages from Cloud Pub/Sub and send the data to BigQuery. You observe that the message processing rate on the Pub/Sub topic is orders of magnitude higher than anticipated, but there is no error logged in Cloud Logging. What are the two most likely causes of this problem? (Choose two.)
- APublisher throughput quota is too small.
- BError handling in the subscriber code is not handling run-time errors properly.
- CThe subscriber code cannot keep up with the messages.
- DThe subscriber code does not acknowledge the messages that it pulls.
- ETotal outstanding messages exceed the 10-MB maximum.
113You are working on a linear regression model on BigQuery ML to predict a customer's likelihood of purchasing your company's products. Your model uses a city name variable as a key predictive component. In order to train and serve the model, your data must be organized in columns. You want to prepare your data using the least amount of coding while maintaining the predictable variables. What should you do?
- AUse TensorFlow to create a categorical variable with a vocabulary list. Create the vocabulary file and upload that as part of your model to BigQuery ML.
- BUse Cloud Data Fusion to assign each city to a region that is labeled as 1, 2, 3, 4, or 5, and then use that number to represent the city in the model.
- CUse SQL in BigQuery to transform the state column using a one-hot encoding method, and make each city a column with binary values.
- DCreate a new view with BigQuery that does not include a column with city information.
114You work for a large ecommerce company. You store your customer's order data in Bigtable. You have a garbage collection policy set to delete the data after 30 days and the number of versions is set to 1. When the data analysts run a query to report total customer spending, the analysts sometimes see customer data that is older than 30 days. You need to ensure that the analysts do not see customer data older than 30 days while minimizing cost and overhead. What should you do?
- ASchedule a job daily to scan the data in the table and delete data older than 30 days.
- BUse a timestamp range filter in the query to fetch the customer's data for a specific range.
- CSet the expiring values of the column families to 29 days and keep the number of versions to 1.
- DSet the expiring values of the column families to 30 days and set the number of versions to 2.
115You are training a spam classifier. You notice that you are overfitting the training data. Which three actions can you take to resolve this problem? (Choose three.)
- AUse a larger set of features
- BIncrease the regularization parameters
- CDecrease the regularization parameters
- DUse a smaller set of features
- EGet more training examples
- FReduce the number of training examples
116You want to encrypt the customer data stored in BigQuery. You need to implement per-user crypto-deletion on data stored in your tables. You want to adopt native features in Google Cloud to avoid custom solutions. What should you do?
- ACreate a customer-managed encryption key (CMEK) in Cloud KMS. Associate the key to the table while creating the table.
- BEncrypt your data during ingestion by using a cryptographic library supported by your ETL pipeline.
- CImplement Authenticated Encryption with Associated Data (AEAD) BigQuery functions while storing your data in BigQuery.
- DCreate a customer-managed encryption key (CMEK) in Cloud KMS. Use the key to encrypt data before storing in BigQuery.
117A shipping company has live package-tracking data that is sent to an Apache Kafka stream in real time. This is then loaded into BigQuery. Analysts in your company want to query the tracking data in BigQuery to analyze geospatial trends in the lifecycle of a package. The table was originally created with ingest-date partitioning. Over time, the query processing time has increased. You need to copy all the data to a new clustered table. What should you do?
- AImplement clustering in BigQuery on the ingest date column.
- BTier older data onto Cloud Storage files and create a BigQuery table using Cloud Storage as an external data source.
- CRe-create the table using data partitioning on the package delivery date.
- DImplement clustering in BigQuery on the package-tracking ID column.
Informações
- Descrição
- Quiz sobre GCP - Professional Data Engineer2_2, criado por raul paredes em 05-02-2025.
- Autor
- raul paredes
- Idioma
- Español
- Visualizações
- 1
- Avaliação
- Ainda sem avaliações Avalie este quiz