Certified-Data-Engineer-Professional無料問題集「Databricks Certified Data Engineer Professional」

質問 1

In order to facilitate near real-time workloads, a data engineer is creating a helper function to leverage the schema detection and evolution functionality of Databricks Auto Loader. The desired function will automatically detect the schema of the source directly, incrementally process JSON files as they arrive in a source directory, and automatically evolve the schema of the table when new fields are detected.
The function is displayed below with a blank:

Which response correctly fills in the blank to meet the specified requirements?

（A）

（B）

（C）

（D）

（E）

正解：C 解答を投票する

解説: (JPNTest メンバーにのみ表示されます)

質問 2

A data engineer is attempting to execute the following PySpark code:
df = spark.read.table("sales")
result = df.groupBy("region").agg(sum("revenue"))
However, upon inspecting the execution plan and profiling the Spark job, they observe excessive data shuffling during the aggregation phase.
Which technique should be applied to reduce shuffling during the groupBy aggregation operation?

（A）Use broadcast join.

（B）Use coalesce() after the aggregation.

（C）Caching the DataFrame df.

（D）Repartition by region before aggregation.

正解：D 解答を投票する

解説: (JPNTest メンバーにのみ表示されます)

質問 3

The business intelligence team has a dashboard configured to track various summary metrics for retail stories. This includes total sales for the previous day alongside totals and averages for a variety of time periods. The fields required to populate this dashboard have the following schema:

For Demand forecasting, the Lakehouse contains a validated table of all itemized sales updated incrementally in near real-time. This table named products_per_order, includes the following fields:

Because reporting on long-term sales trends is less volatile, analysts using the new dashboard only require data to be refreshed once daily. Because the dashboard will be queried interactively by many users throughout a normal business day, it should return results quickly and reduce total compute associated with each materialization.
Which solution meets the expectations of the end users while controlling and limiting possible costs?

（A）Use the Delta Cache to persists the products_per_order table in memory to quickly the dashboard with each query.

（B）Use Structure Streaming to configure a live dashboard against the products_per_order table within a Databricks notebook.

（C）Configure a webhook to execute an incremental read against products_per_order each time the dashboard is refreshed.

（D）Populate the dashboard by configuring a nightly batch job to save the required values as a table overwritten with each update.

（E）Define a view against the products_per_order table and define the dashboard against this view.

正解：D 解答を投票する

質問 4

A table is registered with the following code:

Both users and orders are Delta Lake tables. Which statement describes the results of querying recent_orders?

（A）All logic will execute when the table is defined and store the result of joining tables to the DBFS; this stored data will be returned when the table is queried.

（B）The versions of each source table will be stored in the table transaction log; query results will be saved to DBFS with each query.

（C）Results will be computed and cached when the table is defined; these cached results will incrementally update as new records are inserted into source tables.

（D）All logic will execute at query time and return the result of joining the valid versions of the source tables at the time the query finishes.

（E）All logic will execute at query time and return the result of joining the valid versions of the source tables at the time the query began.

正解：A 解答を投票する

解説: (JPNTest メンバーにのみ表示されます)

質問 5

A data engineer has a Delta table orders with deletion vectors enabled. The engineer executes the following command:
DELETE FROM orders WHERE status = 'cancelled';
What should be the behavior of deletion vectors when the command is executed?

（A）Files are physically rewritten without the deleted rows.

（B）Rows are marked as deleted in metadata, not in files.

（C）Delta automatically removes all cancelled orders permanently.

（D）Rows are marked as deleted both in metadata and in files.

正解：B 解答を投票する

解説: (JPNTest メンバーにのみ表示されます)

質問 6

A junior data engineer is working to implement logic for a Lakehouse table named silver_device_recordings. The source data contains 100 unique fields in a highly nested JSON structure.
The silver_device_recordings table will be used downstream to power several production monitoring dashboards and a production model. At present, 45 of the 100 fields are being used in at least one of these applications.
The data engineer is trying to determine the best approach for dealing with schema declaration given the highly-nested structure of the data and the numerous fields.
Which of the following accurately presents information about Delta Lake and Databricks that may impact their decision-making process?

（A）Because Delta Lake uses Parquet for data storage, data types can be easily evolved by just modifying file footer information in place.

（B）Because Databricks will infer schema using types that allow all observed data to be processed, setting types manually provides greater assurance of data quality enforcement.

（C）The Tungsten encoding used by Databricks is optimized for storing string data; newly-added native support for querying JSON strings means that string types are always most efficient.

（D）Human labor in writing code is the largest cost associated with data engineering workloads; as such, automating table declaration logic should be a priority in all migration workloads.

（E）Schema inference and evolution on .Databricks ensure that inferred types will always accurately match the data types used by downstream systems.

正解：B 解答を投票する

解説: (JPNTest メンバーにのみ表示されます)

質問 7

A departing platform owner currently holds ownership of multiple catalogs and controls storage credentials and external locations. A data engineer has been asked to ensure continuity: transfer catalog ownership to the platform team group, delegate ongoing privilege management, and retain the ability to receive and share data via Delta Sharing. Which role must be in place to perform these actions across the metastore?

（A）Account Admin, because account admins can only create metastores but cannot change ownership of catalogs.

（B）Workspace Admin, because workspace admins can transfer ownership of any Unity Catalog object.

（C）Catalog Owner, because catalog owners can transfer any object in any catalog in the metastore.

（D）Metastore Admin, because metastore admins can transfer ownership and manage privileges across all metastore objects, including shares and recipients.

正解：D 解答を投票する

質問 8

A junior data engineer has been asked to develop a streaming data pipeline with a grouped aggregation using DataFrame df. The pipeline needs to calculate the average humidity and average temperature for each non-overlapping five-minute interval. Events are recorded once per minute per device.
Streaming DataFrame df has the following schema:
"device_id INT, event_time TIMESTAMP, temp FLOAT, humidity FLOAT"
Code block:

Choose the response that correctly fills in the blank within the code block to complete this task.

（A）"event_time"

（B）lag("event_time", "10 minutes").alias("time")

（C）to_interval("event_time", "5 minutes").alias("time")

（D）window("event_time", "5 minutes").alias("time")

（E）window("event_time", "10 minutes").alias("time")

正解：D 解答を投票する

解説: (JPNTest メンバーにのみ表示されます)

質問 9

A Delta Lake table was created with the below query:

Consider the following query:
DROP TABLE prod.sales_by_store
If this statement is executed by a workspace admin, which result will occur?

（A）The table will be removed from the catalog and the data will be deleted.

（B）An error will occur because Delta Lake prevents the deletion of production data.

（C）Nothing will occur until a COMMIT command is executed.

（D）Data will be marked as deleted but still recoverable with Time Travel.

（E）The table will be removed from the catalog but the data will remain in storage.

正解：A 解答を投票する

解説: (JPNTest メンバーにのみ表示されます)

質問 10

A data engineer needs to design an efficient pipeline that automatically processes new CSV files as they arrive in S3 storage. Which Databricks feature should the data engineer use to meet these requirements?

（A）Streaming from cloud storage using standard Spark readStream with format ("csv") and format ("json")

（B）Auto Loader with schema inference and evolution enabled

（C）COPY INTO SQL command with parameters to track processed files

（D）Traditional batch processing with scheduled Databricks Jobs

正解：B 解答を投票する

解説: (JPNTest メンバーにのみ表示されます)

質問 11

A data engineer is using Structured Streaming to read in transaction data from a bronze Delta table. It was discovered that the data has quality issues where sometimes the transaction value is negative, and when that occurs, the rows need to be routed to a separate quarantine table. They have low latency requirements for the good data since it is used by downstream systems, but the bad data will only be analyzed periodically and has no production dependencies. The quarantine job needs to be implemented so that it cannot affect the production processes that depend on the good data, and the cost of the job needs to be minimized. How should the quarantine process be implemented in order to satisfy these requirements?

（A）The existing streaming job for the good data should be updated to incorporate the quarantining of the bad data. A new boolean column called "quarantine" should be added to the dataframe, and its value should be set to true if the transaction value is less than 0 and false if the transaction value is greater than or equal to 0. Processing and storing all the data together will save costs.

（B）The streaming job for the good data needs to be modified to filter out records with a transaction value less than 0 before writing. The streaming job for the quarantine data needs to filter out records with a transaction value greater than or equal to 0 before writing. Both should run as separate streams on the same cluster to minimize cost.

（C）The existing streaming job for the good data should be updated to incorporate the quarantining of the bad data. Inside a foreachBatch function, the dataframe should be filtered so that records with a transaction value greater than or equal to 0 are written to the good data table and records with a transaction value less than 0 are written to a quarantine table. Try/Catch can be added around the writes in the foreachBatch function so that the stream can't fail.

（D）The streaming job for the good data needs to be modified to filter out records with a transaction value less than 0 before writing, and should not share compute with other processes. The streaming job for the quarantine data needs to filter out records with a transaction value greater than or equal to 0 before writing, and should be implemented on a separate small cluster and only run once a day to minimize cost.

正解：D 解答を投票する

解説: (JPNTest メンバーにのみ表示されます)

質問 12

A data engineer is creating a data ingestion pipeline to understand where customers are taking their rented bicycles during use. The engineer noticed that, over time, data being transmitted from the bicycle sensors fail to include key details like latitude and longitude. Downstream analysts need both the clean records and the quarantined records available for separate processing.
The data engineer already has this code:
import dlt
from pyspark.sql.functions import expr
rules = {
"valid_lat": "(lat IS NOT NULL)",
"valid_long": "(long IS NOT NULL)"
}
quarantine_rules = "NOT({})".format(" AND ".join(rules.values()))
@dlt.view
def raw_trips_data():
return spark.readStream.table("ride_and_go.telemetry.trips")
How should the data engineer meet the requirements to capture good and bad data?

（A）@dlt.view
@dlt.expect_or_drop("lat_long_present", "(lat IS NOT NULL AND long IS NOT NULL)") def trips_data_quarantine():
return spark.readStream.table("ride_and_go.telemetry.trips")

（B）@dlt.table(name="trips_data_quarantine")
def trips_data_quarantine():
return (
spark.readStream.table("raw_trips_data")
.filter(expr(quarantine_rules))
)

（C）@dlt.table
@dlt.expect_all_or_drop(rules)
def trips_data_quarantine():
return spark.readStream.table("raw_trips_data")

（D）@dlt.table(partition_cols=["is_quarantined", ])
@dlt.expect_all(rules)
def trips_data_quarantine():
return (
spark.readStream.table("raw_trips_data")
.withColumn("is_quarantined", expr(quarantine_rules))
)

正解：B 解答を投票する

解説: (JPNTest メンバーにのみ表示されます)

質問 13

The data architect has mandated that all tables in the Lakehouse should be configured as external (also known as "unmanaged") Delta Lake tables.
Which approach will ensure that this requirement is met?

（A）When the workspace is being configured, make sure that external cloud object storage has been mounted.

（B）When a database is being created, make sure that the LOCATION keyword is used.

（C）When configuring an external data warehouse for all table storage, leverage Databricks for all ELT.

（D）When tables are created, make sure that the EXTERNAL keyword is used in the CREATE TABLE statement.

（E）When data is saved to a table, make sure that a full file path is specified alongside the Delta format.

正解：D 解答を投票する

解説: (JPNTest メンバーにのみ表示されます)

質問 14

A DLT pipeline includes the following streaming tables:
Raw_lot ingest raw device measurement data from a heart rate tracking device.
Bpm_stats incrementally computes user statistics based on BPM measurements from raw_lot.
How can the data engineer configure this pipeline to be able to retain manually deleted or updated records in the raw_iot table while recomputing the downstream table when a pipeline update is run?

（A）Set the pipelines, reset, allowed property to false on bpm_stats

（B）Set the SkipChangeCommits flag to true raw_lot

（C）Set the skipChangeCommits flag to true on bpm_stats

（D）Set the pipelines, reset, allowed property to false on raw_iot

正解：D 解答を投票する

解説: (JPNTest メンバーにのみ表示されます)

質問 15

The data engineering team has configured a Databricks SQL query and alert to monitor the values in a Delta Lake table. The recent_sensor_recordings table contains an identifying sensor_id alongside the timestamp and temperature for the most recent 5 minutes of recordings.
The below query is used to create the alert:

The query is set to refresh each minute and always completes in less than 10 seconds. The alert is set to trigger when mean (temperature) > 120. Notifications are triggered to be sent at most every 1 minute.
If this alert raises notifications for 3 consecutive minutes and then stops, which statement must be true?

（A）The average temperature recordings for at least one sensor exceeded 120 on three consecutive executions of the query

（B）The maximum temperature recording for at least one sensor exceeded 120 on three consecutive executions of the query

（C）The total average temperature across all sensors exceeded 120 on three consecutive executions of the query

（D）The source query failed to update properly for three consecutive minutes and then restarted

（E）The recent_sensor_recordingstable was unresponsive for three consecutive runs of the query

正解：A 解答を投票する

解説: (JPNTest メンバーにのみ表示されます)

Certified-Data-Engineer-Professional 無料問題集「Databricks Certified Data Engineer Professional」

弊社を連絡する

関連リンク

トップ試験