[2025年02月13日]Databricks-Certified-Data-Engineer-Associate試験問題集で100%合格率Databricks-Certified-Data-Engineer-Associate試験！
試験問題集リアルDatabricks Certification問題集102解答を試そう！
GAQM Databricks-Certified-Data-Engineer-Associate（Databricks Certified Data Engineer Associate）認定試験は、データエンジニアリングの分野で働く個人のスキルと知識を評価する専門の認定プログラムです。この認定は、データエンジニアがDatabricksテクノロジーを使用してデータパイプラインとデータウェアハウスを設計、構築、および維持する能力を検証します。試験には、データ取込み、データ変換、データモデリング、データウェアハウジング、データ品質など、さまざまなトピックが含まれています。
Databricks Certified Data Engineer Associate 資格認定は、特定の技術やベンダーに結びつかないベンダーニュートラルな資格認定です。これは、さまざまなデータエンジニアリングツールや技術を使用したい専門家にとって優れた資格認定です。
質問 # 13
A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table.
The cade block used by the data engineer is below:
If the data engineer only wants the query to execute a micro-batch to process data every 5 seconds, which of the following lines of code should the data engineer use to fill in the blank?
- A. trigger(continuous="5 seconds")
- B. trigger("5 seconds")
- C. trigger()
- D. trigger(processingTime="5 seconds")
- E. trigger(once="5 seconds")
正解：D
解説：
The processingTime option specifies a time-based trigger interval for fixed interval micro-batches. This means that the query will execute a micro-batch to process data every 5 seconds, regardless of how much data is available. This option is suitable for near-real time processing workloads that require low latency and consistent processing frequency. The other options are either invalid syntax (A, C), default behavior (B), or experimental feature (E). References: Databricks Documentation - Configure Structured Streaming trigger intervals, Databricks Documentation - Trigger.
質問 # 14
A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table.
The cade block used by the data engineer is below:
If the data engineer only wants the query to execute a micro-batch to process data every 5 seconds, which of the following lines of code should the data engineer use to fill in the blank?
- A. trigger(continuous="5 seconds")
- B. trigger("5 seconds")
- C. trigger()
- D. trigger(processingTime="5 seconds")
- E. trigger(once="5 seconds")
正解：D
解説：
Explanation
# ProcessingTime trigger with two-seconds micro-batch interval
df.writeStream \
format("console") \
trigger(processingTime='2 seconds') \
start()
https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#triggers
質問 # 15
Which of the following describes the relationship between Gold tables and Silver tables?
- A. Gold tables are more likely to contain truthful data than Silver tables.
- B. Gold tables are more likely to contain valuable data than Silver tables.
- C. Gold tables are more likely to contain more data than Silver tables.
- D. Gold tables are more likely to contain aggregations than Silver tables.
- E. Gold tables are more likely to contain a less refined view of data than Silver tables.
正解：D
解説：
According to the medallion lakehouse architecture, gold tables are the final layer of data that powers analytics, machine learning, and production applications. They are often highly refined and aggregated, containing data that has been transformed into knowledge, rather than just information. Silver tables, on the other hand, are the intermediate layer of data that represents a validated, enriched version of the raw data from the bronze layer.
They provide an enterprise view of all its key business entities, concepts and transactions, but they may not have all the aggregations and calculations that are required for specific use cases. Therefore, gold tables are more likely to contain aggregations than silver tables. References:
* What is the medallion lakehouse architecture?
* What is a Medallion Architecture?
質問 # 16
A data analyst has created a Delta table sales that is used by the entire data analysis team. They want help from the data engineering team to implement a series of tests to ensure the data is clean. However, the data engineering team uses Python for its tests rather than SQL.
Which of the following commands could the data engineering team use to access sales in PySpark?
- A. spark.sql("sales")
- B. spark.table("sales")
- C. There is no way to share data between PySpark and SQL.
- D. spark.delta.table("sales")
- E. SELECT * FROM sales
正解：B
解説：
The data engineering team can use the spark.table method to access the Delta table sales in PySpark. This method returns a DataFrame representation of the Delta table, which can be used for further processing or testing. The spark.table method works for any table that is registered in the Hive metastore or the Spark catalog, regardless of the file format1. Alternatively, the data engineering team can also use the DeltaTable.forPath method to load the Delta table from its path2. Reference: 1: SparkSession | PySpark 3.2.0 documentation 2: Welcome to Delta Lake's Python documentation page - delta-spark 2.4.0 documentation
質問 # 17
Which of the following commands will return the location of database customer360?
- A. ALTER DATABASE customer360 SET DBPROPERTIES ('location' = '/user'};
- B. DESCRIBE DATABASE customer360;
- C. DROP DATABASE customer360;
- D. DESCRIBE LOCATION customer360;
- E. USE DATABASE customer360;
正解：B
解説：
Explanation
To retrieve the location of a database named "customer360" in a database management system like Hive or Databricks, you can use the DESCRIBE DATABASE command followed by the database name. This command will provide information about the database, including its location.
質問 # 18
A dataset has been defined using Delta Live Tables and includes an expectations clause:
CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION DROP ROW What is the expected behavior when a batch of data containing data that violates these constraints is processed?
- A. Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
- B. Records that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.
- C. Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
- D. Records that violate the expectation cause the job to fail.
- E. Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table.
正解：A
質問 # 19
A data engineer wants to create a data entity from a couple of tables. The data entity must be used by other data engineers in other sessions. It also must be saved to a physical location.
Which of the following data entities should the data engineer create?
- A. Table
- B. Database
- C. Function
- D. Temporary view
- E. View
正解：A
解説：
A table is a data entity that is stored in a physical location and can be accessed by other data engineers in other sessions. A table can be created from one or more tables using the CREATE TABLE or CREATE TABLE AS SELECT commands. A table can also be registered from an existing DataFrame using the spark.catalog.createTable method. A table can be queried using SQL or DataFrame APIs. A table can also be updated, deleted, or appended using the MERGE INTO command or the DeltaTable API. References:
* Create a table
* Create a table from a query result
* Register a table from a DataFrame
* [Query a table]
* [Update, delete, or merge into a table]
質問 # 20
Which of the following describes the relationship between Gold tables and Silver tables?
- A. Gold tables are more likely to contain truthful data than Silver tables.
- B. Gold tables are more likely to contain valuable data than Silver tables.
- C. Gold tables are more likely to contain more data than Silver tables.
- D. Gold tables are more likely to contain a less refined view of data than Silver tables.
- E. Gold tables are more likely to contain aggregations than Silver tables.
正解：D
質問 # 21
A data engineer is maintaining a data pipeline. Upon data ingestion, the data engineer notices that the source data is starting to have a lower level of quality. The data engineer would like to automate the process of monitoring the quality level.
Which of the following tools can the data engineer use to solve this problem?
- A. Delta Live Tables
- B. Data Explorer
- C. Unity Catalog
- D. Delta Lake
- E. Auto Loader
正解：A
解説：
Explanation
https://docs.databricks.com/delta-live-tables/expectations.html
Delta Live Tables is a tool provided by Databricks that can help data engineers automate the monitoring of data quality. It is designed for managing data pipelines, monitoring data quality, and automating workflows.
With Delta Live Tables, you can set up data quality checks and alerts to detect issues and anomalies in your data as it is ingested and processed in real-time. It provides a way to ensure that the data quality meets your desired standards and can trigger actions or notifications when issues are detected. While the other tools mentioned may have their own purposes in a data engineeringenvironment, Delta Live Tables is specifically designed for data quality monitoring and automation within the Databricks ecosystem.
質問 # 22
A data engineer and data analyst are working together on a data pipeline. The data engineer is working on the raw, bronze, and silver layers of the pipeline using Python, and the data analyst is working on the gold layer of the pipeline using SQL The raw source of the pipeline is a streaming input. They now want to migrate their pipeline to use Delta Live Tables.
Which change will need to be made to the pipeline when migrating to Delta Live Tables?
- A. The pipeline will need to be written entirely in Python.
- B. The pipeline can have different notebook sources in SQL & Python.
- C. The pipeline will need to be written entirely in SQL.
- D. The pipeline will need to use a batch source in place of a streaming source.
正解：B
解説：
When migrating to Delta Live Tables (DLT) with a data pipeline that involves different programming languages across various data layers, the migration does not require unifying the pipeline into a single language. Delta Live Tables support multi-language pipelines, allowing data engineers and data analysts to work in their preferred languages, such as Python for data engineering tasks (raw, bronze, and silver layers) and SQL for data analytics tasks (gold layer). This capability is particularly beneficial in collaborative settings and leverages the strengths of each language for different stages of data processing.
Reference:
Databricks documentation on Delta Live Tables: Delta Live Tables Guide
質問 # 23
A data organization leader is upset about the data analysis team's reports being different from the data engineering team's reports. The leader believes the siloed nature of their organization's data engineering and data analysis architectures is to blame.
Which of the following describes how a data lakehouse could alleviate this issue?
- A. Both teams would respond more quickly to ad-hoc requests
- B. Both teams would reorganize to report to the same department
- C. Both teams would autoscale their work as data size evolves
- D. Both teams would use the same source of truth for their work
- E. Both teams would be able to collaborate on projects in real-time
正解：D
解説：
A data lakehouse is a data management architecture that combines the flexibility, cost-efficiency, and scale of data lakes with the data management and ACID transactions of data warehouses, enabling business intelligence (BI) and machine learning (ML) on all data12. By using a data lakehouse, both the data analysis and data engineering teams can access the same data sources and formats, ensuring data consistency and quality across their reports. A data lakehouse also supports schema enforcement and evolution, data validation, and time travel to old table versions, which can help resolve data conflicts and errors1. References: 1: What is a Data Lakehouse? - Databricks 2: What is a data lakehouse? | IBM
質問 # 24
Which of the following commands will return the location of database customer360?
- A. ALTER DATABASE customer360 SET DBPROPERTIES ('location' = '/user'};
- B. DESCRIBE DATABASE customer360;
- C. DROP DATABASE customer360;
- D. DESCRIBE LOCATION customer360;
- E. USE DATABASE customer360;
正解：B
解説：
The command DESCRIBE DATABASE customer360; will return the location of the database customer360, along with its comment and properties. This command is an alias for DESCRIBE SCHEMA customer360;, which can also be used to get the same information. The other commands will either drop the database, alter its properties, or use it as the current database, but will not return its location12. Reference:
DESCRIBE DATABASE | Databricks on AWS
DESCRIBE DATABASE - Azure Databricks - Databricks SQL
質問 # 25
A data architect has determined that a table of the following format is necessary:
Which of the following code blocks uses SQL DDL commands to create an empty Delta table in the above format regardless of whether a table already exists with this name?
- A. Option B
- B. Option D
- C. Option E
- D. Option A
- E. Option C
正解：C
解説：
References: Create a table using SQL | Databricks on AWS, Create a table using SQL - Azure Databricks, Delta Lake Quickstart - Azure Databricks
質問 # 26
A data engineer is designing a data pipeline. The source system generates files in a shared directory that is also used by other processes. As a result, the files should be kept as is and will accumulate in the directory. The data engineer needs to identify which files are new since the previous run in the pipeline, and set up the pipeline to only ingest those new files with each run.
Which of the following tools can the data engineer use to solve this problem?
- A. Databricks SQL
- B. Data Explorer
- C. Auto Loader
- D. Unity Catalog
- E. Delta Lake
正解：C
解説：
Explanation
Auto Loader incrementally and efficiently processes new data files as they arrive in cloud storage without any additional setup.https://docs.databricks.com/en/ingestion/auto-loader/index.html
質問 # 27
Which of the following can be used to simplify and unify siloed data architectures that are specialized for specific use cases?
- A. None of these
- B. All of these
- C. Data warehouse
- D. Data lake
- E. Data lakehouse
正解：E
解説：
A data lakehouse is a new paradigm that can be used to simplify and unify siloed data architectures that are specialized for specific use cases. A data lakehouse combines the best of both data lakes and data warehouses, providing a single platform that supports diverse data types, open standards, low-cost storage, high-performance queries, ACID transactions, schema enforcement, and governance. A data lakehouse enables data engineers to build reliable and scalable data pipelines that can serve various downstream applications and users, such as data science, machine learning, analytics, and reporting. A data lakehouse leverages the power of Delta Lake, a storage layer that brings reliability and performance to data lakes. References: What is a data lakehouse?, Delta Lake, Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics
質問 # 28
A data engineer has three tables in a Delta Live Tables (DLT) pipeline. They have configured the pipeline to drop invalid records at each table. They notice that some data is being dropped due to quality concerns at some point in the DLT pipeline. They would like to determine at which table in their pipeline the data is being dropped.
Which of the following approaches can the data engineer take to identify the table that is dropping the records?
- A. They can set up separate expectations for each table when developing their DLT pipeline.
- B. They can navigate to the DLT pipeline page, click on the "Error" button, and review the present errors.
- C. They can navigate to the DLT pipeline page, click on each table, and view the data quality statistics.
- D. They cannot determine which table is dropping the records.
- E. They can set up DLT to notify them via email when records are dropped.
正解：C
解説：
One of the features of DLT is that it provides data quality metrics for each dataset in the pipeline, such as the number of records that pass or fail expectations, the number of records that are dropped, and the number of records that are written to the target. These metrics can be accessed from the DLT pipeline page, where the data engineer can click on each table and view the data quality statistics for the latest update or any previous update. This way, they can identify which table is dropping the records and why. References:
* Monitor Delta Live Tables pipelines
* Manage data quality with Delta Live Tables
質問 # 29
Which of the following Git operations must be performed outside of Databricks Repos?
- A. Pull
- B. Clone
- C. Commit
- D. Merge
- E. Push
正解：D
解説：
Explanation
For following tasks, work in your Git provider:
Create a pull request.
Resolve merge conflicts.
Merge or delete branches.
Rebase a branch.
https://docs.databricks.com/repos/index.html
質問 # 30
Which of the following tools is used by Auto Loader process data incrementally?
- A. Databricks SQL
- B. Spark Structured Streaming
- C. Data Explorer
- D. Unity Catalog
- E. Checkpointing
正解：B
質問 # 31
Which of the following describes a benefit of creating an external table from Parquet rather than CSV when using a CREATE TABLE AS SELECT statement?
- A. CREATE TABLE AS SELECT statements cannot be used on files
- B. Parquet files have the ability to be optimized
- C. Parquet files will become Delta tables
- D. Parquet files have a well-defined schema
- E. Parquet files can be partitioned
正解：D
解説：
Option C is the correct answer because Parquet files have a well-defined schema that is embedded within the data itself. This means that the data types and column names of the Parquet files are automatically detected and preserved when creating an external table from them. This also enables the use of SQL and other structured query languages to access and analyze the data. CSV files, on the other hand, do not have a schema embedded in them, and require specifying the schema explicitly or inferring it from the data when creating an external table from them. This can lead to errors or inconsistencies in the data types and column names, and also increase the processing time and complexity.
References: CREATE TABLE AS SELECT, Parquet Files, CSV Files, Parquet vs. CSV
質問 # 32
A data engineer is attempting to drop a Spark SQL table my_table. The data engineer wants to delete all table metadata and data.
They run the following command:
DROP TABLE IF EXISTS my_table
While the object no longer appears when they run SHOW TABLES, the data files still exist.
Which of the following describes why the data files still exist and the metadata files were deleted?
- A. The table's data was larger than 10 GB
- B. The table's data was smaller than 10 GB
- C. The table was managed
- D. The table did not have a location
- E. The table was external
正解：E
質問 # 33
A data analyst has developed a query that runs against Delta table. They want help from the data engineering team to implement a series of tests to ensure the data returned by the query is clean. However, the data engineering team uses Python for its tests rather than SQL.
Which of the following operations could the data engineering team use to run the query and operate with the results in PySpark?
- A. spark.sql
- B. There is no way to share data between PySpark and SQL.
- C. spark.delta.table
- D. SELECT * FROM sales
- E. spark.table
正解：A
解説：
The spark.sql operation allows the data engineering team to run a SQL query and return the result as a PySpark DataFrame. This way, the data engineering team can use the same query that the data analyst has developed and operate with the results in PySpark. For example, the data engineering team can use spark.sql("SELECT * FROM sales") to get a DataFrame of all the records from the sales Delta table, and then apply various tests or transformations using PySpark APIs. The other options are either not valid operations (A, D), not suitable for running a SQL query (B, E), or not returning a DataFrame (A). References: Databricks Documentation - Run SQL queries, Databricks Documentation - Spark SQL and DataFrames.
質問 # 34
Which of the following is a benefit of the Databricks Lakehouse Platform embracing open source technologies?
- A. Simplified governance
- B. Avoiding vendor lock-in
- C. Ability to scale workloads
- D. Cloud-specific integrations
- E. Ability to scale storage
正解：B
解説：
One of the benefits of the Databricks Lakehouse Platform embracing open source technologies is that it avoids vendor lock-in. This means that customers can use the same open source tools and frameworks across different cloud providers, and migrate their data and workloads without being tied to a specific vendor. The Databricks Lakehouse Platform is built on open source projects such as Apache Spark™, Delta Lake, MLflow, and Redash, which are widely used and trusted by millions of developers. By supporting these open source technologies, the DatabricksLakehouse Platform enables customers to leverage the innovation and community of the open source ecosystem, and avoid the risk of being locked into proprietary or closed solutions. The other options are either not related to open source technologies (A, B, C, D), or not benefits of the Databricks Lakehouse Platform (A, B). References: Databricks Documentation - Built on open source, Databricks Documentation - What is the Lakehouse Platform?, Databricks Blog - Introducing the Databricks Lakehouse Platform.
質問 # 35
Which of the following must be specified when creating a new Delta Live Tables pipeline?
- A. A location of a target database for the written data
- B. The preferred DBU/hour cost
- C. At least one notebook library to be executed
- D. A key-value pair configuration
- E. A path to cloud storage location for the written data
正解：C
解説：
Option E is the correct answer because it is the only mandatory requirement when creating a new Delta Live Tables pipeline. A pipeline is a data processing workflow that contains materialized views and streaming tables declared in Python or SQL source files. Delta Live Tables infers the dependencies between these tables and ensures updates occur in the correct order. To create a pipeline, you need to specify at least one notebook library to be executed, which contains the Delta Live Tables syntax. You can also specify multiple libraries of different languages within your pipeline. The other options are optional or not applicable for creating a pipeline. Option A is not required, but you can optionally provide a key-value pair configuration to customize the pipeline settings, such as the storage location, the target schema, the notifications, and the pipeline mode. Option B is not applicable, as the DBU/hour cost is determined by the cluster configuration, not the pipeline creation. Option C is not required, but you can optionally specify a storage location for the output data from the pipeline. If you leave it empty, the system uses a default location. Option D is not required, but you can optionally specify a location of a target database for the written data, either in the Hive metastore or the Unity Catalog.
質問 # 36
A data engineering team has noticed that their Databricks SQL queries are running too slowly when they are submitted to a non-running SQL endpoint. The data engineering team wants this issue to be resolved.
Which of the following approaches can the team use to reduce the time it takes to return results in this scenario?
- A. They can turn on the Auto Stop feature for the SQL endpoint.
- B. They can increase the cluster size of the SQL endpoint.
- C. They can turn on the Serverless feature for the SQL endpoint.
- D. They can increase the maximum bound of the SQL endpoint's scaling range
- E. They can turn on the Serverless feature for the SQL endpoint and change the Spot Instance Policy to
"Reliability Optimized."
正解：B
解説：
Explanation
https://www.databricks.com/blog/2022/03/10/top-5-databricks-performance-tips.html
質問 # 37
......
GAQM Databricks-Certified-Data-Engineer-Associate試験は、Databricksを使用したデータパイプラインの設計と構築において、自分の熟練度を証明したいデータエンジニア向けの認定試験です。この試験は、データエンジニアリング、ビッグデータ処理と分析、Databricksを使用したクラウドコンピューティングの知識と技能を評価するように設計されています。
あなたを余裕でDatabricks-Certified-Data-Engineer-Associate試験合格させます！100%高合格率保証：https://www.jpntest.com/shiken/Databricks-Certified-Data-Engineer-Associate-mondaishu
Databricks-Certified-Data-Engineer-Associate問題集本日限定！無料アクセス可能に！：https://drive.google.com/open?id=1r6XFoWBbfeKOqUYWYkXvY-IsitVYZp4E