[2023年更新]Databricks-Certified-Professional-Data-Engineerリアルな試験問題集でDatabricks-Certified-Professional-Data-Engineer練習テスト
Databricks-Certified-Professional-Data-Engineer問題集でDatabricks Certification高確率練習問題集
Databricks Certified Professional Data Engineer になるには、データエンジニアリングに関連する幅広いトピックをカバーする厳しい試験に合格する必要があります。この試験は、Databricks ツールやテクノロジーを使用してデータパイプラインを設計、構築、維持する能力をテストするとともに、データモデリング、データウェアハウジング、データ統合の理解もテストします。
Databricks Certified Professional Data Engineer試験は、ビッグデータとクラウドコンピューティング技術に携わる個人のスキルと知識をテストするために設計されています。この試験は、候補者のApache Sparkプラットフォームを使用してビッグデータソリューションを設計、構築、維持する能力を主に評価します。この認定資格は、業界で非常に高く評価され、個人がビッグデータプロジェクトを管理する能力を証明するのに役立ちます。
Databricks 認定プロフェッショナルデータエンジニア認定は、データエンジニアリングに特化し、Databricks の技術における専門知識を証明したい個人にとって優れた選択肢です。また、Databricks を使用する企業にとっても、大量のデータを効果的に管理・分析するために必要なスキルを従業員が持っていることを保証するための貴重な資格です。
質問 # 101
A DELTA LIVE TABLE pipelines can be scheduled to run in two different modes, what are these two different modes?
- A. Once, Incremental
- B. Once, Continuous
- C. Triggered, Incremental
- D. Continuous, Incremental
- E. Triggered, Continuous
正解：E
解説：
Explanation
The answer is Triggered, Continuous
https://docs.microsoft.com/en-us/azure/databricks/data-engineering/delta-live-tables/delta-live-tables-concepts#-
*Triggered pipelines update each table with whatever data is currently available and then stop the cluster running the pipeline. Delta Live Tables automatically analyzes the dependencies between your tables and starts by computing those that read from external sources. Tables within the pipeline are updated after their dependent data sources have been updated.
*Continuous pipelines update tables continuously as input data changes. Once an update is started, it continues to run until manually stopped. Continuous pipelines require an always-running cluster but ensure that downstream consumers have the most up-to-date data.
質問 # 102
Which of the following commands can be used to run one notebook from another notebook?
- A. execute.utils.run("full notebook path")
- B. only job clusters can run notebook
- C. dbutils.notebook.run("full notebook path")
- D. spark.notebook.run("full notebook path")
- E. notebook.utils.run("full notebook path")
正解：C
解説：
Explanation
The answer is dbutils.notebook.run(" full notebook path ")
Here is the full command with additional options.
run(path: String, timeout_seconds: int, arguments: Map): String
1.dbutils.notebook.run("ful-notebook-name", 60, {"argument": "data", "argument2": "data2", ...})
質問 # 103
A data architect is designing a data model that works for both video-based machine learning work-loads and
highly audited batch ETL/ELT workloads.
Which of the following describes how using a data lakehouse can help the data architect meet the needs of
both workloads?
- A. A data lakehouse requires very little data modeling
- B. A data lakehouse provides autoscaling for compute clusters
- C. A data lakehouse stores unstructured data and is ACID-compliant
- D. A data lakehouse fully exists in the cloud
- E. A data lakehouse combines compute and storage for simple governance
正解：C
質問 # 104
While investigating a performance issue, you realized that you have too many small files for a given table, which command are you going to run to fix this issue
- A. SHRINK table_name
- B. MERGE table_name
- C. VACUUM table_name
- D. OPTIMIZE table_name
- E. COMPACT table_name
正解：D
解説：
Explanation
The answer is OPTIMIZE table_name,
Optimize compacts small parquet files into a bigger file, by default the size of the files are determined based on the table size at the time of OPTIMIZE, the file size can also be set manually or adjusted based on the workload.
https://docs.databricks.com/delta/optimizations/file-mgmt.html
Tune file size based on Table size
To minimize the need for manual tuning, Databricks automatically tunes the file size of Delta tables based on the size of the table. Databricks will use smaller file sizes for smaller tables and larger file sizes for larger tables so that the number of files in the table does not grow too large.
Table Description automatically generated
Bottom of Form
Top of Form
質問 # 105
A data engineer is designing a data pipeline. The source system generates files in a shared directory that is also
used by other processes. As a result, the files should be kept as is and will accumulate in the directory. The
data engineer needs to identify which files are new since the previous run in the pipeline, and set up the
pipeline to only ingest those new files with each run.
Which of the following tools can the data engineer use to solve this problem?
- A. Databricks SQL
- B. Unity Catalog
- C. Auto Loader
- D. Delta Lake
- E. Data Explorer
正解：C
質問 # 106
Which of the following locations hosts the driver and worker nodes of a Databricks-managed clus-ter?
- A. Databricks web application
- B. Control plane
- C. Databricks Filesystem
- D. JDBC data source
- E. Data plane
正解：E
解説：
Explanation
See the Databricks high-level architecture
質問 # 107
A data engineer has created a Delta table as part of a data pipeline. Downstream data analysts now need
SELECT permission on the Delta table.
Assuming the data engineer is the Delta table owner, which part of the Databricks Lakehouse Plat-form can
the data engineer use to grant the data analysts the appropriate access?
- A. Repos
- B. Jobs
B Dashboards - C. Databricks Filesystem
- D. Data Explorer
正解：A
質問 # 108
You are looking to process the data based on two variables, one to check if the department is supply chain or check if process flag is set to True
- A. if department == "supply chain" or process = TRUE:
- B. if department == "supply chain" | if process == TRUE:
- C. if department == "supply chain" | process == TRUE:
- D. if department == "supply chain" or process:
- E. if department = "supply chain" | process:
正解：D
質問 # 109
What is the main difference between the bronze layer and silver layer in a medallion architecture?
- A. Bad data is filtered in Bronze, silver is a copy of bronze data
- B. Bronze is raw copy of ingested data, silver contains data with production schema and optimized for ELT/ETL throughput
- C. Silver may contain aggregated data
- D. Duplicates are removed in bronze, schema is applied in silver
正解：B
解説：
Explanation
Medallion Architecture - Databricks
Exam focus: Please review the below image and understand the role of each layer(bronze, silver, gold) in medallion architecture, you will see varying questions targeting each layer and its purpose.
Sorry I had to add the watermark some people in Udemy are copying my content.
A diagram of a house Description automatically generated with low confidence
質問 # 110
You noticed that a team member started using an all-purpose cluster to develop a notebook and used the same all-purpose cluster to set up a job that can run every 30 mins so they can update un-derlying tables which are used in a dashboard. What would you recommend for reducing the overall cost of this approach?
- A. Change the cluster all-purpose to job cluster when scheduling the job
- B. Reduce the number of nodes and enable auto scale
- C. Reduce the size of the cluster
- D. Enable auto termination after 30 mins
- E. Change the cluster mode from all-purpose to single-mode
正解：A
解説：
Explanation
While using an all-purpose cluster is ok during development but anytime you don't need to interact with a notebook, especially for a scheduled job it is less expensive to use a job cluster. Using an all-purpose cluster can be twice as expensive as a job cluster.
Please note: The compute cost you pay the cloud provider for the same cluster type and size be-tween an all-purpose cluster and job cluster is the same the only difference is the DBU cost.
The total cost of cluster = Total cost of VM compute(Azure or AWS or GCP) + Cost per DBU The per DBU cost varies between all-purpose and Job Cluster Here is the recent cost estimate from AWS between Jobs Cluster and all-purpose Cluster, for jobs compute its
$0.15 cents per DBU v$0.55 cents per DBU for all-purpose
Graphical user interface Description automatically generated
How do I check how much the DBU cost for my cluster?
When you click on an exister cluster or when you look at the cluster details you will see this in the top right corner Graphical user interface, text, application, email Description automatically generated
質問 # 111
You are currently working on a notebook that will populate a reporting table for downstream process consumption, this process needs to run on a schedule every hour, what type of cluster are you going to use to set up this job?
- A. Use Azure VM to read and write delta tables in Python
- B. Use delta live table pipeline to run in continuous mode
- C. Since it's just a single job and we need to run every hour, we can use an all-purpose cluster
- D. The job cluster is best suited for this purpose.
正解：D
解説：
Explanation
The answer is, The Job cluster is best suited for this purpose.
Since you don't need to interact with the notebook during the execution especially when it's a scheduled job, job cluster makes sense. Using an all-purpose cluster can be twice as expensive as a job cluster.
FYI,
When you run a job scheduler with option of creating a new cluster when the job is complete it terminates the cluster. You cannot restart a job cluster.
質問 # 112
Which of the following table constraints that can be enforced on Delta lake tables are supported?
- A. Default, Not Null, Check Constraints
- B. Primary key, Not Null, Check Constraints
- C. Primary key, foreign key, Not Null, Check Constraints
- D. Not Null, Check Constraints
- E. Unique, Not Null, Check Constraints
正解：D
解説：
Explanation
The answer is Not Null, Check Constraints
https://docs.microsoft.com/en-us/azure/databricks/delta/delta-constraints
* CREATE TABLE events( id LONG,
* date STRING,
* location STRING,
* description STRING
* ) USING DELTA;
ALTER TABLE events CHANGE COLUMN id SET NOT NULL;
ALTER TABLE events ADD CONSTRAINT dateWithinRange CHECK (date > '1900-01-01'); Note: Databricks as of DBR 11.1 added support for Primary Key and Foreign Key when Unity Catalog is enabled but this is for information purposes only these are not actually enforced. You may ask then why are we defining these if they are not enforced, so especially these information constraints are very helpful if you have a BI tool that can benefit from knowing the relationship between the tables, so it will be easy when creating reports/dashboards or understanding the data model when using any Data modeling tool.
Primary and Foreign Key
Graphical user interface, text, application, email Description automatically generated
質問 # 113
When investigating a data issue you realized that a process accidentally updated the table, you want to query the same table with yesterday's version of the data so you can review what the prior version looks like, what is the best way to query historical data so you can do your analysis?
- A. SELECT * FROM TIME_TRAVEL(table_name) WHERE time_stamp = 'timestamp'
- B. TIME_TRAVEL FROM table_name WHERE time_stamp = date_sub(current_date(), 1)
- C. SELECT * FROM table_name TIMESTAMP AS OF date_sub(current_date(), 1)
- D. SHOW HISTORY table_name AS OF date_sub(current_date(), 1)
- E. DISCRIBE HISTORY table_name AS OF date_sub(current_date(), 1)
正解：C
解説：
Explanation
The answer is SELECT * FROM table_name TIMESTAMP as of date_sub(current_date(), 1) FYI, Time travel supports two ways one is using timestamp and the second way is using version number, Timestamp:
1.SELECT count(*) FROM my_table TIMESTAMP AS OF "2019-01-01"
2.SELECT count(*) FROM my_table TIMESTAMP AS OF date_sub(current_date(), 1)
3.SELECT count(*) FROM my_table TIMESTAMP AS OF "2019-01-01 01:30:00.000" Version Number:
1.SELECT count(*) FROM my_table VERSION AS OF 5238
2.SELECT count(*) FROM my_table@v5238
3.SELECT count(*) FROM delta.`/path/to/my/table@v5238`
https://databricks.com/blog/2019/02/04/introducing-delta-time-travel-for-large-scale-data-lakes.html
質問 # 114
The sales team has asked the Data engineering team to develop a dashboard that shows sales per-formance for all stores, but the sales team would like to use the dashboard but would like to select individual store location, which of the following approaches Data Engineering team can use to build this functionality into the dashboard.
- A. Use Dynamic views to filter the data based on the location
- B. Use Databricks REST API to create a dashboard for each location
- C. Use SQL UDF function to filter the data based on the location
- D. Use query Parameters which then allow user to choose any location
- E. Currently dashboards do not support parameters
正解：D
解説：
Explanation
The answer is
Databricks supports many types of parameters in the dashboard, a drop-down list can be created based on a query that has a unique list of store locations.
Here is a simple query that takes a parameter for
SELECT * FROM sales WHERE field IN ( {{ Multi Select Parameter }} )
Or
SELECT * FROM sales WHERE field = {{ Single Select Parameter }}
Query parameter types
*Text
*Number
*Dropdown List
*Query Based Dropdown List
*Date and Time
質問 # 115
You are currently working on reloading customer_sales tables using the below query
1. INSERT OVERWRITE customer_sales
2. SELECT * FROM customers c
3. INNER JOIN sales_monthly s on s.customer_id = c.customer_id
After you ran the above command, the Marketing team quickly wanted to review the old data that was in the table. How does INSERT OVERWRITE impact the data in the customer_sales table if you want to see the previous version of the data prior to running the above statement?
- A. Overwrites the current version of the data but clears all historical versions of the data, so you can not time travel to previous versions.
- B. By default, overwrites the data and schema, you cannot perform time travel
- C. Overwrites the data in the table but preserves all historical versions of the data, you can time travel to previous versions
- D. Overwrites the data in the table, all historical versions of the data, you can not time travel to previous versions
- E. Appends the data to the current version, you can time travel to previous versions
正解：C
解説：
Explanation
The answer is, INSERT OVERWRITE Overwrites the current version of the data but preserves all historical versions of the data, you can time travel to previous versions.
1.INSERT OVERWRITE customer_sales
2.SELECT * FROM customers c
3.INNER JOIN sales s on s.customer_id = c.customer_id
Let's just assume that this is the second time you are running the above statement, you can still query the prior version of the data using time travel, and any DML/DDL except DROP TABLE creates new PARQUET files so you can still access the previous versions of data.
SQL Syntax for Time travel
SELECT * FROM table_name as of [version number]
with customer_sales example
SELECT * FROM customer_sales as of 1 -- previous version
SELECT * FROM customer_sales as of 2 -- current version
You see all historical changes on the table using DESCRIBE HISTORY table_name Note: the main difference between INSERT OVERWRITE and CREATE OR REPLACE TABLE(CRAS) is that CRAS can modify the schema of the table, i.e it can add new columns or change data types of existing columns. By default INSERT OVERWRITE only overwrites the data.
INSERT OVERWRITE can also be used to update the schema when
spark.databricks.delta.schema.autoMerge.enabled is set true if this option is not enabled and if there is a schema mismatch command INSERT OVERWRITEwill fail.
Any DML/DDL operation(except DROP TABLE) on the Delta table preserves the historical ver-sion of the data.
質問 # 116
The Delta Live Table Pipeline is configured to run in Production mode using the continuous Pipe-line Mode.
what is the expected outcome after clicking Start to update the pipeline?
- A. All datasets will be updated once and the pipeline will shut down. The compute resources will be terminated
- B. All datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing
- C. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped
- D. All datasets will be updated continuously and the pipeline will not shut down. The compute resources will persist with the pipeline (Correct)
- E. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist after the pipeline is stopped to allow for additional testing
正解：D
解説：
Explanation
The answer is,
All datasets will be updated continuously and the pipeline will not shut down. The compute re-sources will persist with the pipeline until it is shut down since the execution mode is chosen to be continuous. It does not matter if the pipeline mode is development or production, pipeline mode only matters during the pipeline initialization.
DLT pipeline supports two modes Development and Production, you can switch between the two based on the stage of your development and deployment lifecycle.
Development and production modes
Development:
When you run your pipeline in development mode, the Delta Live Tables system:
*Reuses a cluster to avoid the overhead of restarts.
*Disables pipeline retries so you can immediately detect and fix errors.
Production:
In production mode, the Delta Live Tables system:
*Restarts the cluster for specific recoverable errors, including memory leaks and stale cre-dentials.
*Retries execution in the event of specific errors, for example, a failure to start a cluster.
Use the buttons in the Pipelines UI to switch between develop-ment and production modes. By default,
pipelines run in development mode.
Switching between development and production modes only controls cluster and pipeline execution behavior.
Storage locations must be configured as part of pipeline settings and are not affected when switching between modes.
Delta Live Tables supports two different modes of execution:
Triggered pipelines update each table with whatever data is currently available and then stop the cluster running the pipeline. Delta Live Tables automatically analyzes the dependencies between your tables and starts by computing those that read from external sources. Tables within the pipe-line are updated after their dependent data sources have been updated.
Continuous pipelines update tables continuously as input data changes. Once an update is started, it continues to run until manually stopped. Continuous pipelines require an always-running cluster but ensure that downstream consumers have the most up-to-date data Please review additional DLT concepts using the below link
https://docs.databricks.com/data-engineering/delta-live-tables/delta-live-tables-concepts.html#delta-live-tables-c
質問 # 117
What is the underlying technology that makes the Auto Loader work?
- A. Delta Live Tables
- B. Loader
- C. DataFrames
- D. Live DataFames
- E. Structured Streaming
正解：E
質問 # 118
......
Databricks-Certified-Professional-Data-Engineerリアルな問題と知能問題集：https://www.jpntest.com/shiken/Databricks-Certified-Professional-Data-Engineer-mondaishu
合格できるDatabricks-Certified-Professional-Data-Engineer試験と最新Databricks-Certified-Professional-Data-Engineer試験問題集PDF2023：https://drive.google.com/open?id=159vdVUHS9E8MqH9FsTgl115Zk1bVVHxQ