Databricks Databricks-Certified-Data-Engineer-Associate日常練習試験は2024年最新のに更新された102問あります [Q35-Q57]

Databricks Databricks-Certified-Data-Engineer-Associate日常練習試験は2024年最新のに更新された102問あります

有効問題を試そう！Databricks-Certified-Data-Engineer-Associate試験で実際の試験問題と解答

GAQM Databricks-Certified-Data-Engineer-Associate（Databricks Certified Data Engineer Associate）認定試験は、データの専門家にとって高度に尊敬される認定です。この試験では、DataBricksを使用してデータパイプラインの設計、構築、維持における個人の能力を測定します。この認証は、データエンジニアリングの基礎、データビックのアーキテクチャ、データモデリング、データ処理など、幅広いトピックをカバーしています。

Databricks認定データエンジニアアソシエイト認定試験の準備には、データエンジニアリングの概念をしっかりと理解し、Databricksでの作業経験が必要です。GAQMは、オンラインコース、練習問題、学習ガイドなど、試験に備えるためのさまざまな学習資料を提供しています。また、試験に完全に備えるためには、Databricksでデータパイプラインを構築および維持する実践的な経験が必要です。

質問 # 35
Which of the following describes a scenario in which a data team will want to utilize cluster pools?

A. An automated report needs to be made reproducible.
B. An automated report needs to be version-controlled across multiple collaborators.
C. An automated report needs to be tested to identify errors.
D. An automated report needs to be runnable by all stakeholders.
E. An automated report needs to be refreshed as quickly as possible.

正解：E

解説：
Explanation
Cluster pools are typically used in distributed computing environments, such as cloud-based data platforms like Databricks. They allow you to pre-allocate a set of compute resources (a cluster) for specific tasks or workloads. In this case, if an automated report needs to be refreshed as quickly as possible, you can allocate a cluster pool with sufficient resources to ensure fast data processing and report generation. This helps ensure that the report is generated with minimal latency and can be delivered to stakeholders in a timely manner.
Cluster pools allow you to optimize resource allocation for high-demand, time-sensitive tasks like real-time report generation.

質問 # 36
A data engineer is running code in a Databricks Repo that is cloned from a central Git repository. A colleague of the data engineer informs them that changes have been made and synced to the central Git repository. The data engineer now needs to sync their Databricks Repo to get the changes from the central Git repository.
Which of the following Git operations does the data engineer need to run to accomplish this task?

A. Clone
B. Merge
C. Push
D. Pull
E. Commit

正解：D

解説：
To sync a Databricks Repo with the changes from a central Git repository, the data engineer needs to run the Git pull operation. This operation fetches the latest updates from the remote repository and merges them with the local repository. The data engineer can use the Pull button in the Databricks Repos UI, or use the git pull command in a terminal session. The other options are not relevant for this task, as they either push changes to the remote repository (Push), combine two branches (Merge), save changes to the local repository (Commit), or create a new local repository from a remote one (Clone). References:
* Run Git operations on Databricks Repos
* Git pull

質問 # 37
Which of the following approaches should be used to send the Databricks Job owner an email in the case that the Job fails?

A. MLflow Model Registry Webhooks
B. Setting up an Alert in the Notebook
C. Setting up an Alert in the Job page
D. Manually programming in an alert system in each cell of the Notebook
E. There is no way to notify the Job owner in the case of Job failure

正解：C

解説：
Explanation
https://docs.databricks.com/en/workflows/jobs/job-notifications.html

質問 # 38
A data engineer has been given a new record of data:
id STRING = 'a1'
rank INTEGER = 6
rating FLOAT = 9.4
Which of the following SQL commands can be used to append the new record to an existing Delta table my_table?

A. UPDATE my_table VALUES ('a1', 6, 9.4)
B. INSERT VALUES ( 'a1' , 6, 9.4) INTO my_table
C. UPDATE VALUES ('a1', 6, 9.4) my_table
D. INSERT INTO my_table VALUES ('a1', 6, 9.4)
E. my_table UNION VALUES ('a1', 6, 9.4)

正解：D

質問 # 39
A data engineer needs to create a table in Databricks using data from a CSV file at location /path/to/csv.
They run the following command:

Which of the following lines of code fills in the above blank to successfully complete the task?

A. FROM CSV
B. USING DELTA
C. None of these lines of code are needed to successfully complete the task
D. FROM "path/to/csv"
E. USING CSV

正解：D

解説：
A data lakehouse is a new paradigm that can be used to simplify and unify siloed data architectures that are specialized for specific use cases. A data lakehouse combines the best of both data lakes and data warehouses, providing a single platform that supports diverse data types, open standards, low-cost storage, high-performance queries, ACID transactions, schema enforcement, and governance. A data lakehouse enables data engineers to build reliable and scalable data pipelines that can serve various downstream applications and users, such as data science, machine learning, analytics, and reporting. A data lakehouse leverages the power of Delta Lake, a storage layer that brings reliability and performance to data lakes. References: What is a data lakehouse?, Delta Lake, Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics

質問 # 40
In which of the following scenarios should a data engineer select a Task in the Depends On field of a new Databricks Job Task?

A. When another task needs to fail before the new task begins
B. When another task needs to use as little compute resources as possible
C. When another task has the same dependency libraries as the new task
D. When another task needs to be replaced by the new task
E. When another task needs to successfully complete before the new task begins

正解：E

解説：
A data engineer can create a multi-task job in Databricks that consists of multiple tasks that run in a specific order. Each task can have one or more dependencies, which are other tasks that must run before the current task. The Depends On field of a new Databricks Job Task allows the data engineer to specify the dependencies of the task. The data engineer should select a task in the Depends On field when they want the new task to run only after the selected task has successfully completed. This can help the data engineer to create a logical sequence of tasks that depend on each other's outputs or results. For example, a data engineer can create a multi-task job that consists of the following tasks:
Task A: Ingest data from a source using Auto Loader
Task B: Transform the data using Spark SQL
Task C: Write the data to a Delta Lake table
Task D: Analyze the data using Spark ML
Task E: Visualize the data using Databricks SQL
In this case, the data engineer can set the dependencies of each task as follows:
Task A: No dependencies
Task B: Depends on Task A
Task C: Depends on Task B
Task D: Depends on Task C
Task E: Depends on Task D
This way, the data engineer can ensure that each task runs only after the previous task has successfully completed, and the data flows smoothly from ingestion to visualization.
The other options are incorrect because they do not describe valid scenarios for selecting a task in the Depends On field. The Depends On field does not affect the following aspects of a task:
Whether the task needs to be replaced by another task
Whether the task needs to fail before another task begins
Whether the task has the same dependency libraries as another task
Whether the task needs to use as little compute resources as possible Reference: Create a multi-task job, Run tasks conditionally in a Databricks job, Databricks Jobs.

質問 # 41
Which of the following tools is used by Auto Loader process data incrementally?

A. Databricks SQL
B. Data Explorer
C. Unity Catalog
D. Spark Structured Streaming
E. Checkpointing

正解：D

解説：
Explanation
The Auto Loader process in Databricks is typically used in conjunction with Spark Structured Streaming to process data incrementally. Spark Structured Streaming is a real-time data processing framework that allows you to process data streams incrementally as new data arrives. The Auto Loader is a feature in Databricks that works with Structured Streaming to automatically detect and process new data files as they are added to a specified data source location. It allows for incremental data processing without the need for manual intervention.
How does Auto Loader track ingestion progress? As files are discovered, their metadata is persisted in a scalable key-value store (RocksDB) in the checkpoint location of your Auto Loader pipeline. This key-value store ensures that data is processed exactly once. In case of failures, Auto Loader can resume from where it left off by information stored in the checkpoint location and continue to provide exactly-once guarantees when writing data into Delta Lake. You don't need to maintain or manage any state yourself to achieve fault tolerance or exactly-once semantics.https://docs.databricks.com/ingestion/auto-loader/index.html

質問 # 42
Which of the following data lakehouse features results in improved data quality over a traditional data lake?

A. A data lakehouse stores data in open formats.
B. A data lakehouse provides storage solutions for structured and unstructured data.
C. A data lakehouse enables machine learning and artificial Intelligence workloads.
D. A data lakehouse supports ACID-compliant transactions.
E. A data lakehouse allows the use of SQL queries to examine data.

正解：D

解説：
A data lakehouse is a data management architecture that combines the flexibility, cost-efficiency, and scale of data lakes with the data management and ACID transactions of data warehouses, enabling business intelligence (BI) and machine learning (ML) on all data12. One of the key features of a data lakehouse is that it supports ACID-compliant transactions, which means that it ensures data integrity, consistency, and isolation across concurrent read and write operations3. This feature results in improved data quality over a traditional data lake, which does not support transactions and may suffer from data corruption, duplication, or inconsistency due to concurrent or streaming data ingestion and processing . References: 1: What is a Data Lakehouse? - Databricks 2: What is a Data Lakehouse? Definition, features & benefits. - Qlik 3: ACID Transactions - Databricks : [Data Lake vs Data Warehouse: Key Differences] : [Data Lakehouse: The Future of Data Engineering]

質問 # 43
Which of the following benefits of using the Databricks Lakehouse Platform is provided by Delta Lake?

A. The ability to set up alerts for query failures
B. The ability to support batch and streaming workloads
C. The ability to distribute complex data operations
D. The ability to manipulate the same data using a variety of languages
E. The ability to collaborate in real time on a single notebook

正解：B

解説：
Explanation
Delta Lake is a key component of the Databricks Lakehouse Platform that provides several benefits, and one of the most significant benefits is its ability to support both batch and streaming workloads seamlessly. Delta Lake allows you to process and analyze data in real-time (streaming) as well as in batch, making it a versatile choice for various data processing needs. While the other options may be benefits or capabilities of Databricks or the Lakehouse Platform in general, they are not specifically associated with Delta Lake.

質問 # 44
A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table.
The code block used by the data engineer is below:

If the data engineer only wants the query to process all of the available data in as many batches as required, which of the following lines of code should the data engineer use to fill in the blank?

A. trigger(continuous="once")
B. trigger(processingTime="once")
C. processingTime(1)
D. trigger(parallelBatch=True)
E. trigger(availableNow=True)

正解：E

解説：
Explanation
https://stackoverflow.com/questions/71061809/trigger-availablenow-for-delta-source-streaming-queries-in-pyspa

質問 # 45
A data engineer has developed a data pipeline to ingest data from a JSON source using Auto Loader, but the engineer has not provided any type inference or schema hints in their pipeline. Upon reviewing the data, the data engineer has noticed that all of the columns in the target table are of the string type despite some of the fields only including float or boolean values.
Which of the following describes why Auto Loader inferred all of the columns to be of the string type?

A. Auto Loader only works with string data
B. All of the fields had at least one null value
C. Auto Loader cannot infer the schema of ingested data
D. JSON data is a text-based format
E. There was a type mismatch between the specific schema and the inferred schema

正解：D

解説：
JSON data is a text-based format that represents data as a collection of name-value pairs. By default, when Auto Loader infers the schema of JSON data, it treats all columns as strings. This is because JSON data can have varying data types for the same column across different files or records, and Auto Loader does not attempt to reconcile these differences. For example, a column named "age" may have integer values in some files, but string values in others. To avoid data loss or errors, Auto Loader infers the column as a string type. However, Auto Loader also provides an option to infer more precise column types based on the sample data. This option is called cloudFiles.inferColumnTypes and it can be set to true or false. When set to true, Auto Loader tries to infer the exact data types of the columns, such as integers, floats, booleans, or nested structures. When set to false, Auto Loader infers all columns as strings. The default value of this option is false. Reference: Configure schema inference and evolution in Auto Loader, Schema inference with auto loader (non-DLT and DLT), Using and Abusing Auto Loader's Inferred Schema, Explicit path to data or a defined schema required for Auto loader.

質問 # 46
Which of the following Structured Streaming queries is performing a hop from a Silver table to a Gold table?

正解：B

解説：
The best practice is to use "Complete" as output mode instead of "append" when working with aggregated tables. Since gold layer is work final aggregated tables, the only option with output mode as complete is option E.

質問 # 47
A data engineer wants to create a relational object by pulling data from two tables. The relational object does not need to be used by other data engineers in other sessions. In order to save on storage costs, the data engineer wants to avoid copying and storing physical data.
Which of the following relational objects should the data engineer create?

A. View
B. Delta Table
C. Database
D. Spark SQL Table
E. Temporary view

正解：E

解説：
Explanation
Temp view : session based Create temp view view_name as query All these are termed as session ended:
Opening a new notebook Detaching and reattaching a cluster Installing a python package Restarting a cluster

質問 # 48
A data analyst has created a Delta table sales that is used by the entire data analysis team. They want help from the data engineering team to implement a series of tests to ensure the data is clean. However, the data engineering team uses Python for its tests rather than SQL.
Which of the following commands could the data engineering team use to access sales in PySpark?

A. spark.delta.table("sales")
B. SELECT * FROM sales
C. spark.sql("sales")
D. spark.table("sales")
E. There is no way to share data between PySpark and SQL.

正解：D

解説：
The data engineering team can use the spark.table method to access the Delta table sales in PySpark. This method returns a DataFrame representation of the Delta table, which can be used for further processing or testing. The spark.table method works for any table that is registered in the Hive metastore or the Spark catalog, regardless of the file format1. Alternatively, the data engineering team can also use the DeltaTable.forPath method to load the Delta table from its path2. References: 1: SparkSession | PySpark
3.2.0 documentation 2: Welcome to Delta Lake's Python documentation page - delta-spark 2.4.0 documentation

質問 # 49
A data engineer is attempting to drop a Spark SQL table my_table and runs the following command:
DROP TABLE IF EXISTS my_table;
After running this command, the engineer notices that the data files and metadata files have been deleted from the file system.
Which of the following describes why all of these files were deleted?

A. The table was managed
B. The table's data was larger than 10 GB
C. The table did not have a location
D. The table's data was smaller than 10 GB
E. The table was external

正解：E

解説：
Explanation
managed tables files and metadata are managed by metastore and will be deleted when the table is dropped .
while external tables the metadata is stored in a external location.hence when a external table is dropped you clear off only the metadata and the files (data) remain.

質問 # 50
An engineering manager wants to monitor the performance of a recent project using a Databricks SQL query.
For the first week following the project's release, the managerwants the query results to be updated every minute. However, the manager is concerned that the compute resources used for the query will be left running and cost the organization a lot of money beyond the first week of the project's release.
Which of the following approaches can the engineering team use to ensure the query does not cost the organization any money beyond the first week of the project's release?

A. They can set a limit to the number of DBUs that are consumed by the SQL Endpoint.
B. They can set a limit to the number of individuals that are able to manage the query's refresh schedule.
C. They can set the query's refresh schedule to end on a certain date in the query scheduler.
D. They can set the query's refresh schedule to end after a certain number of refreshes.
E. They cannot ensure the query does not cost the organization money beyond the first week of the project's release.

正解：C

質問 # 51
A data engineer is maintaining a data pipeline. Upon data ingestion, the data engineer notices that the source data is starting to have a lower level of quality. The data engineer would like to automate the process of monitoring the quality level.
Which of the following tools can the data engineer use to solve this problem?

A. Auto Loader
B. Delta Lake
C. Data Explorer
D. Delta Live Tables
E. Unity Catalog

正解：D

質問 # 52
A data engineer needs to create a table in Databricks using data from their organization's existing SQLite database.
They run the following command:

Which of the following lines of code fills in the above blank to successfully complete the task?

A. org.apache.spark.sql.jdbc
B. DELTA
C. autoloader
D. org.apache.spark.sql.sqlite
E. sqlite

正解：E

解説：
1: In the given command, a data engineer is trying to create a table in Databricks using data from an SQLite database. The correct option to fill in the blank is "sqlite" because it specifies the type of database being connected to in a JDBC connection string. The USING clause should be followed by the format of the data, and since we are connecting to an SQLite database, "sqlite" would be appropriate here. Reference:
Create a table using JDBC
JDBC connection string
SQLite JDBC driver

質問 # 53
A data engineer has realized that they made a mistake when making a daily update to a table. They need to use Delta time travel to restore the table to a version that is 3 days old. However, when the data engineer attempts to time travel to the older version, they are unable to restore the data because the data files have been deleted.
Which of the following explains why the data files are no longer present?

A. The DELETE HISTORY command was run on the table
B. The OPTIMIZE command was nun on the table
C. The HISTORY command was run on the table
D. The VACUUM command was run on the table
E. The TIME TRAVEL command was run on the table

正解：D

解説：
The VACUUM command is used to remove files that are no longer referenced by a Delta table and are older than the retention threshold1. The default retention period is 7 days2, but it can be changed by setting the delta.logRetentionDuration and delta.deletedFileRetentionDuration configurations3. If the VACUUM command was run on the table with a retention period shorter than 3 days, then the data files that were needed to restore the table to a 3-day-old version would have been deleted. The other commands do not delete data files from the table. The TIME TRAVEL command is used to query a historical version of the table4. The DELETE HISTORY command is not a valid command in Delta Lake. The OPTIMIZE command is used to improve the performance of the table by compacting small files into larger ones5. The HISTORY command is used to retrieve information about the operations performed on the table. Reference: 1: VACUUM | Databricks on AWS 2: Work with Delta Lake table history | Databricks on AWS 3: [Delta Lake configuration | Databricks on AWS] 4: Work with Delta Lake table history - Azure Databricks 5: [OPTIMIZE | Databricks on AWS] : [HISTORY | Databricks on AWS]

質問 # 54
A data engineer needs access to a table new_table, but they do not have the correct permissions. They can ask the table owner for permission, but they do not know who the table owner is.
Which of the following approaches can be used to identify the owner of new_table?

A. Review the Owner field in the table's page in Data Explorer
B. There is no way to identify the owner of the table
C. Review the Permissions tab in the table's page in Data Explorer
D. All of these options can be used to identify the owner of the table
E. Review the Owner field in the table's page in the cloud storage solution

正解：A

解説：
he approach that can be used to identify the owner of new_table is to review the Owner field in the table's page in Data Explorer. Data Explorer is a web-based interface that allows users to browse, create, and manage data objects such as tables, views, and functions in Databricks1. The table's page in Data Explorer provides various information about the table, such as its schema, partitions, statistics, history, and permissions2. The Owner field shows the name and email address of the user who created or owns the table3. The data engineer can use this information to contact the table owner and request for permission to access the table.
The other options are not correct or reliable for identifying the owner of new_table. Reviewing the Permissions tab in the table's page in Data Explorer can show the users and groups who have access to the table, but not necessarily the owner4. Reviewing the Owner field in the table's page in the cloud storage solution can be misleading, as the owner of the data files may not be the same as the owner of the table5.
There is a way to identify the owner of the table, as explained above, so option E is false.
References:
* 1: Data Explorer | Databricks on AWS
* 2: Table details | Databricks on AWS
* 3: Set owner when creating a view in databricks sql - Databricks - 9978
* 4: Table access control | Databricks on AWS
* 5: External tables | Databricks on AWS

質問 # 55
A data engineer has a Python variable table_name that they would like to use in a SQL query. They want to construct a Python code block that will run the query using table_name.
They have the following incomplete code block:
____(f"SELECT customer_id, spend FROM {table_name}")
Which of the following can be used to fill in the blank to successfully complete the task?

A. spark.delta.sql
B. spark.sql
C. spark.table
D. dbutils.sql
E. spark.delta.table

正解：B

質問 # 56
A data engineer has developed a data pipeline to ingest data from a JSON source using Auto Loader, but the engineer has not provided any type inference or schema hints in their pipeline. Upon reviewing the data, the data engineer has noticed that all of the columns in the target table are of the string type despite some of the fields only including float or boolean values.
Which of the following describes why Auto Loader inferred all of the columns to be of the string type?

A. Auto Loader only works with string data
B. All of the fields had at least one null value
C. Auto Loader cannot infer the schema of ingested data
D. JSON data is a text-based format
E. There was a type mismatch between the specific schema and the inferred schema

正解：D

解説：
Explanation
JSON data is a text-based format that uses strings to represent all values. When Auto Loader infers the schema of JSON data, it assumes that all values are strings. This is because Auto Loader cannot determine the type of a value based on its string representation. https://docs.databricks.com/en/ingestion/auto-loader/schema.html Forexample, the following JSON string represents a value that is logically a boolean: JSON "true" Use code with caution. Learn more However, Auto Loader would infer that the type of this value is string. This is because Auto Loader cannot determine that the value is a boolean based on its string representation. In order to get Auto Loader to infer the correct types for columns, the data engineer can provide type inference or schema hints. Type inference hints can be used to specify the types of specific columns. Schema hints can be used to provide the entire schema of the data.
Therefore, the correct answer is B. JSON data is a text-based format.

質問 # 57
......

テストエンジンに練習Databricks-Certified-Data-Engineer-Associateテスト問題：https://www.jpntest.com/shiken/Databricks-Certified-Data-Engineer-Associate-mondaishu

Databricks-Certified-Data-Engineer-Associateリアル試験問題でテストエンジン問題集トレーニングには102問あります：https://drive.google.com/open?id=1UdW7VyxQbg9pooEd5JP_QpYz_qJMxXa5