[2024年11月] 今すぐダウンロード！リアルDatabricks Databricks-Certified-Professional-Data-Engineer試験問題集テストエンジン試験問題
最新Databricks-Certified-Professional-Data-Engineerテスト問題集を試そう！最新Databricks試験合格させます
認定試験は、複数選択問題から構成されるコンピューターベースのテストです。受験者は2時間以内に試験を完了し、合格するために最低70％のスコアを取得する必要があります。試験は監督され、受験者は信頼できるインターネット接続とウェブカメラ、マイクを備えたコンピューターを持っている必要があります。
質問 # 17
Which of the following locations hosts the driver and worker nodes of a Databricks-managed clus-ter?
- A. Control plane
- B. JDBC data source
- C. Data plane
- D. Databricks Filesystem
- E. Databricks web application
正解：C
解説：
Explanation
See the Databricks high-level architecture
質問 # 18
A Databricks SQL dashboard has been configured to monitor the total number of records present in a collection of Delta Lake tables using the following query pattern:
SELECT COUNT (*) FROM table -
Which of the following describes how results are generated each time the dashboard is updated?
- A. The total count of records is calculated from the Hive metastore
- B. The total count of records is calculated from the Delta transaction logs
- C. The total count of records is calculated from the parquet file metadata
- D. The total count of rows will be returned from cached results unless REFRESH is run
- E. The total count of rows is calculated by scanning all data files
正解：B
解説：
https://delta.io/blog/2023-04-19-faster-aggregations-metadata/#:~:text=You%20can%20get%20the%20number,a%20given%20Delta%20table%20version.
質問 # 19
Two of the most common data locations on Databricks are the DBFS root storage and external object storage mounted with dbutils.fs.mount().
Which of the following statements is correct?
- A. By default, both the DBFS root and mounted data sources are only accessible to workspace administrators.
- B. The DBFS root is the most secure location to store data, because mounted storage volumes must have full public read and write permissions.
- C. Neither the DBFS root nor mounted storage can be accessed when using %sh in a Databricks notebook.
- D. The DBFS root stores files in ephemeral block volumes attached to the driver, while mounted directories will always persist saved data to external storage between sessions.
- E. DBFS is a file system protocol that allows users to interact with files stored in object storage using syntax and guarantees similar to Unix file systems.
正解：E
解説：
DBFS is a file system protocol that allows users to interact with files stored in object storage using syntax and guarantees similar to Unix file systems1. DBFS is not a physical file system, but a layer over the object storage that provides a unified view of data across different data sources1. By default, the DBFS root is accessible to all users in the workspace, and the access to mounted data sources depends on the permissions of the storage account or container2. Mounted storage volumes do not need to have full public read and write permissions, but they do require a valid connection string or access key to be provided when mounting3. Both the DBFS root and mounted storage can be accessed when using %sh in a Databricks notebook, as long as the cluster has FUSE enabled4. The DBFS root does not store files in ephemeral block volumes attached to the driver, but in the object storage associated with the workspace1. Mounted directories will persist saved data to external storage between sessions, unless they are unmounted or deleted3. Reference: DBFS, Work with files on Azure Databricks, Mounting cloud object storage on Azure Databricks, Access DBFS with FUSE
質問 # 20
you are currently working on creating a spark stream process to read and write in for a one-time micro batch, and also rewrite the existing target table, fill in the blanks to complete the below command sucesfully.
1.spark.table("source_table")
2..writeStream
3..option("____", "dbfs:/location/silver")
4..outputMode("____")
5..trigger(Once=____)
6..table("target_table")
- A. checkpointlocation, complete, True
- B. checkpointlocation, True, complete
- C. checkpointlocation, overwrite, True
- D. checkpointlocation, True, overwrite
- E. targetlocation, overwrite, True
正解：A
質問 # 21
When working with AUTO LOADER you noticed that most of the columns that were inferred as part of loading are string data types including columns that were supposed to be integers, how can we fix this?
- A. Provide the schema of the source table in the cloudfiles.schemalocation
- B. Provide schema hints
- C. Provide the schema of the target table in the cloudfiles.schemalocation
- D. Update the checkpoint location
- E. Correct the incoming data by explicitly casting the data types
正解：B
解説：
Explanation
The answer is, Provide schema hints.
1.spark.readStream \
2.format("cloudFiles") \
3.option("cloudFiles.format", "csv") \
4.option("header", "true") \
5.option("cloudFiles.schemaLocation", schema_location) \
6.option("cloudFiles.schemaHints", "id int, description string")
7.load(raw_data_location)
8.writeStream \
9.option("checkpointLocation", checkpoint_location) \
10.start(target_delta_table_location)option("cloudFiles.schemaHints", "id int, description string")
# Here we are providing a hint that id column is int and the description is a string When cloudfiles.schemalocation is used to store the output of the schema inference during the load process, with schema hints you can enforce data types for known columns ahead of time.
質問 # 22
An external object storage container has been mounted to the location/mnt/finance_eda_bucket.
The following logic was executed to create a database for the finance team:
After the database was successfully created and permissions configured, a member of the finance team runs the following code:
If all users on the finance team are members of thefinancegroup, which statement describes how thetx_sales table will be created?
- A. A logical table will persist the physical plan to the Hive Metastore in the Databricks control plane.
- B. A logical table will persist the query plan to the Hive Metastore in the Databricks control plane.
- C. An managed table will be created in the storage container mounted to /mnt/finance eda bucket.
- D. An external table will be created in the storage container mounted to /mnt/finance eda bucket.
- E. A managed table will be created in the DBFS root storage container.
正解：C
解説：
Explanation
https://docs.databricks.com/en/lakehouse/data-objects.html
質問 # 23
When scheduling Structured Streaming jobs for production, which configuration automatically recovers from query failures and keeps costs low?
- A. Cluster: Existing All-Purpose Cluster;
Retries: Unlimited;
Maximum Concurrent Runs: 1 - B. Cluster: New Job Cluster;
Retries: None;
Maximum Concurrent Runs: 1 - C. Cluster: Existing All-Purpose Cluster;
Retries: Unlimited;
Maximum Concurrent Runs: 1 - D. Cluster: Existing All-Purpose Cluster;
Retries: None;
Maximum Concurrent Runs: 1 - E. Cluster: New Job Cluster;
Retries: Unlimited;
Maximum Concurrent Runs: Unlimited
正解：C
解説：
The configuration that automatically recovers from query failures and keeps costs low is to use a new job cluster, set retries to unlimited, and set maximum concurrent runs to 1. This configuration has the following advantages:
* A new job cluster is a cluster that is created and terminated for each job run. This means that the cluster resources are only used when the job is running, and no idle costs are incurred. This also ensures that the cluster is always in a clean state and has the latest configuration and libraries for the job1.
* Setting retries to unlimited means that the job will automatically restart the query in case of any failure, such as network issues, node failures, or transient errors. This improves the reliability and availability of the streaming job, and avoids data loss or inconsistency2.
* Setting maximum concurrent runs to 1 means that only one instance of the job can run at a time. This prevents multiple queries from competing for the same resources or writing to the same output location, which can cause performance degradation or data corruption3.
Therefore, this configuration is the best practice for scheduling Structured Streaming jobs for production, as it ensures that the job is resilient, efficient, and consistent.
References: Job clusters, Job retries, Maximum concurrent runs
質問 # 24
A data engineer wants to create a relational object by pulling data from two tables. The relational object must
be used by other data engineers in other sessions. In order to save on storage costs, the data engineer wants to
avoid copying and storing physical data.
Which of the following relational objects should the data engineer create?
- A. Spark SQL Table
- B. Temporary view
- C. Database
- D. View
- E. Delta Table
正解：D
質問 # 25
Which of the following Auto loader structured streaming commands successfully performs a hop from the landing area into Bronze?
- A. 1.spark\
2..readStream\
3..format("csv")\
4..option("cloudFiles.schemaLocation", checkpoint_directory)\
5..load("landing")\
6..writeStream.option("checkpointLocation", checkpoint_directory)\
7..table(raw) - B. 1.spark\
2..readStream\
3..load(rawSalesLocation)\
4..writeStream \
5..option("checkpointLocation", checkpointPath).outputMode("append")\
6..table("uncleanedSales") - C. 1.spark\
2..readStream\
3..format("cloudFiles")\
4..option("cloudFiles.format","csv")\
5..option("cloudFiles.schemaLocation", checkpoint_directory)\
6..load("landing")\
7..writeStream.option("checkpointLocation", checkpoint_directory)\
8..table(raw)
(Correct) - D. 1.spark\
2..read\
3..load(rawSalesLocation) \
4..writeStream\
5..option("checkpointLocation", checkpointPath) \
6..outputMode("append")\
7..table("uncleanedSales") - E. 1.spark\
2..read\
3..format("cloudFiles")\
4..option("cloudFiles.format","csv")\
5..option("cloudFiles.schemaLocation", checkpoint_directory)\
6..load("landing")\
7..writeStream.option("checkpointLocation", checkpoint_directory)\
8..table(raw)
正解：C
解説：
Explanation
The answer is
1.spark\
2..readStream\
3..format("cloudFiles") \# use Auto loader
4..option("cloudFiles.format","csv") \ # csv format files
5..option("cloudFiles.schemaLocation", checkpoint_directory)\
6..load('landing')\
7..writeStream.option("checkpointLocation", checkpoint_directory)\
8..table(raw)
Note: if you chose the below option which is incorrect because it does not have readStream
1.spark.read.format("cloudFiles")
2..option("cloudFiles.format","csv")
3....
4...
5...
Exam focus: Please review the below image and understand the role of each layer(bronze, silver, gold) in medallion architecture, you will see varying questions targeting each layer and its purpose.
Sorry I had to add the watermark some people in Udemy are copying my content.
A diagram of a house Description automatically generated with low confidence
質問 # 26
A team member is leaving the team and he/she is currently the owner of the few tables, instead of transfering the ownership to a user you have decided to transfer the ownership to a group so in the future anyone in the group can manage the permissions rather than a single individual, which of the following commands help you accomplish this?
- A. GRANT OWNER On table_name to 'group'
- B. TRANSFER OWNER table_name to 'group'
- C. GRANT OWNER table_name to 'group'*
- D. ALTER OWNER ON table_name to 'group'
- E. ALTER TABLE table_name OWNER to 'group'
正解：E
解説：
Explanation
The answer is ALTER TABLE table_name OWNER to 'group'
Assign owner to object
質問 # 27
What is the purpose of gold layer in Multi hop architecture?
- A. Eliminate duplicate records
- B. Optimized query performance for business-critical data
- C. Optimizes ETL throughput and analytic query performance
- D. Data quality checks and schema enforcement
- E. Preserves grain of original data, without any aggregations
正解：B
解説：
Explanation
Medallion Architecture - Databricks
Gold Layer:
1. Powers Ml applications, reporting, dashboards, ad hoc analytics
2. Refined views of data, typically with aggregations
3. Reduces strain on production systems
4. Optimizes query performance for business-critical data
Exam focus: Please review the below image and understand the role of each layer(bronze, silver, gold) in medallion architecture, you will see varying questions targeting each layer and its purpose.
Sorry I had to add the watermark some people in Udemy are copying my content.
質問 # 28
What is the underlying technology that makes the Auto Loader work?
- A. Structured Streaming
- B. Live DataFames
- C. DataFrames
- D. Loader
- E. Delta Live Tables
正解：A
質問 # 29
All records from an Apache Kafka producer are being ingested into a single Delta Lake table with the following schema:
key BINARY, value BINARY, topic STRING, partition LONG, offset LONG, timestamp LONG There are 5 unique topics being ingested. Only the "registration" topic contains Personal Identifiable Information (PII). The company wishes to restrict access to PII. The company also wishes to only retain records containing PII in this table for 14 days after initial ingestion. However, for non-PII information, it would like to retain these records indefinitely.
Which of the following solutions meets the requirements?
- A. Because the value field is stored as binary data, this information is not considered PII and no special precautions should be taken.
- B. Data should be partitioned by the topic field, allowing ACLs and delete statements to leverage partition boundaries.
- C. Separate object storage containers should be specified based on the partition field, allowing isolation at the storage level.
- D. Data should be partitioned by the registration field, allowing ACLs and delete statements to be set for the PII directory.
- E. All data should be deleted biweekly; Delta Lake's time travel functionality should be leveraged to maintain a history of non-PII information.
正解：B
解説：
Explanation
Partitioning the data by the topic field allows the company to apply different access control policies and retention policies for different topics. For example, the company can use the Table Access Control feature to grant or revoke permissions to the registration topic based on user roles or groups. The company can also use the DELETE command to remove records from the registration topic that are older than 14 days, while keeping the records from other topics indefinitely. Partitioning by the topic field also improves the performance of queries that filter by the topic field, as they can skip reading irrelevant partitions. References:
Table Access Control: https://docs.databricks.com/security/access-control/table-acls/index.html DELETE: https://docs.databricks.com/delta/delta-update.html#delete-from-a-table
質問 # 30
The data science team has requested assistance in accelerating queries on free form text from user reviews.
The data is currently stored in Parquet with the below schema:
item_id INT, user_id INT, review_id INT, rating FLOAT, review STRING
The review column contains the full text of the review left by the user. Specifically, the data science team is looking to identify if any of 30 key words exist in this field.
A junior data engineer suggests converting this data to Delta Lake will improve query performance.
Which response to the junior data engineer s suggestion is correct?
- A. Delta Lake statistics are not optimized for free text fields with high cardinality.
- B. Text data cannot be stored with Delta Lake.
- C. ZORDER ON review will need to be run to see performance gains.
- D. The Delta log creates a term matrix for free text fields to support selective filtering.
- E. Delta Lake statistics are only collected on the first 4 columns in a table.
正解：A
解説：
Converting the data to Delta Lake may not improve query performance on free text fields with high cardinality, such as the review column. This is because Delta Lake collects statistics on the minimum and maximum values of each column, which are not very useful for filtering or skipping data on free text fields.
Moreover, Delta Lake collectsstatistics on the first 32 columns by default, which may not include the review column if the table has more columns. Therefore, the junior data engineer's suggestion is not correct. A better approach would be to use a full-text search engine, such as Elasticsearch, to index and query the review column. Alternatively, you can use natural language processing techniques, such as tokenization, stemming, and lemmatization, to preprocess the review column and create a new column with normalized terms that can be used for filtering or skipping data. References:
* Optimizations: https://docs.delta.io/latest/optimizations-oss.html
* Full-text search with Elasticsearch: https://docs.databricks.com/data/data-sources/elasticsearch.html
* Natural language processing: https://docs.databricks.com/applications/nlp/index.html
質問 # 31
You were asked to identify number of times a temperature sensor exceed threshold temperature (100.00) by each device, each row contains 5 readings collected every 5 minutes, fill in the blank with the appropriate functions.
Schema: deviceId INT, deviceTemp ARRAY<double>, dateTimeCollected TIMESTAMP
SELECT deviceId, __ (__ (__(deviceTemp], i -> i > 100.00)))
FROM devices
GROUP BY deviceId
- A. SUM, COUNT, SIZE
- B. SUM, SIZE, ARRAY_FILTER
- C. SUM, SIZE, ARRAY_CONTAINS
- D. SUM, SIZE, FILTER
- E. SUM, SIZE, SLICE
正解：D
解説：
Explanation
FILER function can be used to filter an array based on an expression
SIZE function can be used to get size of an array
SUM is used to calculate to total by device
Diagram Description automatically generated
質問 # 32
An upstream system has been configured to pass the date for a given batch of data to the Databricks Jobs API as a parameter. The notebook to be scheduled will use this parameter to load data with the following code:
df = spark.read.format("parquet").load(f"/mnt/source/(date)")
Which code block should be used to create the date Python variable used in the above code block?
- A. input_dict = input()
date= input_dict["date"] - B. dbutils.widgets.text("date", "null")
date = dbutils.widgets.get("date") - C. date = spark.conf.get("date")
- D. import sys
date = sys.argv[1] - E. date = dbutils.notebooks.getParam("date")
正解：E
解説：
Explanation
This is the correct way to get a parameter passed to a notebook by the Databricks Jobs API. The dbutils.notebooks.getParam method returns the value of a parameter passed to a notebook as a string. If no parameter with that name is passed, it returns None by default. You can also specify a default value as a second argument. Verified References: Databricks Certified Data Engineer Professional, under "Databricks Tooling" section; Databricks Documentation, under "Pass parameters to a notebook" section.
質問 # 33
Which configuration parameter directly affects the size of a spark-partition upon ingestion of data into Spark?
- A. spark.sql.files.openCostInBytes
- B. spark.sql.adaptive.advisoryPartitionSizeInBytes
- C. spark.sql.files.maxPartitionBytes
- D. spark.sql.adaptive.coalescePartitions.minPartitionNum
- E. spark.sql.autoBroadcastJoinThreshold
正解：C
解説：
This is the correct answer because spark.sql.files.maxPartitionBytes is a configuration parameter that directly affects the size of a spark-partition upon ingestion of data into Spark. This parameter configures the maximum number of bytes to pack into a single partition when reading files from file-based sources such as Parquet, JSON and ORC. The default value is 128 MB, which means each partition will be roughly 128 MB in size, unless there are too many small files or only one large file. Verified References: [Databricks Certified Data Engineer Professional], under "Spark Configuration" section; Databricks Documentation, under "Available Properties - spark.sql.files.maxPartitionBytes" section.
質問 # 34
Which of the following are stored in the control pane of Databricks Architecture?
- A. Databricks Web Application
- B. All Purpose Clusters
- C. Job Clusters
- D. Databricks Filesystem
- E. Delta tables
正解：A
解説：
Explanation
The answer is Databricks Web Application
Azure Databricks architecture overview - Azure Databricks | Microsoft Docs Databricks operates most of its services out of a control plane and a data plane, please note serverless features like SQL Endpoint and DLT compute use shared compute in Control pane.
Control Plane: Stored in Databricks Cloud Account
* The control plane includes the backend services that Databricks manages in its own Azure account.
Notebook commands and many other workspace configurations are stored in the control plane and encrypted at rest.
Data Plane: Stored in Customer Cloud Account
* The data plane is managed by your Azure account and is where your data resides. This is also where data is processed. You can use Azure Databricks connectors so that your clusters can connect to external data sources outside of your Azure account to ingest data or for storage.
Timeline Description automatically generated
Bottom of Form
Top of Form
質問 # 35
......
あなたを必ず合格させるDatabricks-Certified-Professional-Data-Engineer問題集PDF2024年最新のに更新された122問あります：https://www.jpntest.com/shiken/Databricks-Certified-Professional-Data-Engineer-mondaishu
検証済み！Databricks-Certified-Professional-Data-Engineer問題集と解答でDatabricks-Certified-Professional-Data-Engineerテストエンジン正確解答付き：https://drive.google.com/open?id=1vkKkezuiDRy7QvAW2wkdlfcMg9BIUDlP