無料更新されたDatabricks Databricks-Machine-Learning-Associateテストエンジン問題には76問あります
ベストな問題集を使おうML Data Scientist Databricks-Machine-Learning-Associate専門試験問題
質問 # 22
A machine learning engineer wants to parallelize the training of group-specific models using the Pandas Function API. They have developed the train_model function, and they want to apply it to each group of DataFrame df.
They have written the following incomplete code block:
Which of the following pieces of code can be used to fill in the above blank to complete the task?
- A. applyInPandas
- B. mapInPandas
- C. train_model
- D. predict
- E. groupedApplyIn
正解：B
解説：
The function mapInPandas in the PySpark DataFrame API allows for applying a function to each partition of the DataFrame. When working with grouped data, groupby followed by applyInPandas is the correct approach to apply a function to each group as a separate Pandas DataFrame. However, if the function should apply across each partition of the grouped data rather than on each individual group, mapInPandas would be utilized. Since the code snippet indicates the use of groupby, the intent seems to be to apply train_model on each group specifically, which aligns with applyInPandas. Thus, applyInPandas is a better fit to ensure that each group generated by groupby is processed through the train_model function, preserving the partitioning and grouping integrity.
Reference
PySpark Documentation on applying functions to grouped data: https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.GroupedData.applyInPandas.html
質問 # 23
A data scientist is developing a machine learning pipeline using AutoML on Databricks Machine Learning.
Which of the following steps will the data scientist need to perform outside of their AutoML experiment?
- A. Exploratory data analysis
- B. Model tuning
- C. Model deployment
- D. Model evaluation
正解：A
解説：
AutoML platforms, such as the one available in Databricks Machine Learning, streamline various stages of the machine learning pipeline including feature engineering, model selection, hyperparameter tuning, and model evaluation. However, exploratory data analysis (EDA) is typically performed outside the AutoML process. EDA involves understanding the dataset, visualizing distributions, identifying anomalies, and gaining insights into data before feeding it into a machine learning pipeline. This step is crucial for ensuring that the data is clean and suitable for model training but is generally done manually by the data scientist.
Reference
Databricks documentation on AutoML: https://docs.databricks.com/applications/machine-learning/automl.html
質問 # 24
A data scientist is using the following code block to tune hyperparameters for a machine learning model:
Which change can they make the above code block to improve the likelihood of a more accurate model?
- A. Change sparkTrials() to Trials()
- B. Change fmin() to fmax()
- C. Increase num_evals to 100
- D. Change tpe.suggest to random.suggest
正解：C
解説：
To improve the likelihood of a more accurate model, the data scientist can increase num_evals to 100. Increasing the number of evaluations allows the hyperparameter tuning process to explore a larger search space and evaluate more combinations of hyperparameters, which increases the chance of finding a more optimal set of hyperparameters for the model.
Reference:
Databricks documentation on hyperparameter tuning: Hyperparameter Tuning
質問 # 25
A machine learning engineer has identified the best run from an MLflow Experiment. They have stored the run ID in the run_id variable and identified the logged model name as "model". They now want to register that model in the MLflow Model Registry with the name "best_model".
Which lines of code can they use to register the model associated with run_id to the MLflow Model Registry?
- A. mlflow.register_model(f"runs:/{run_id}/best_model", "model")
- B. millow.register_model(f"runs:/{run_id)/model")
- C. mlflow.register_model(run_id, "best_model")
- D. mlflow.register_model(f"runs:/{run_id}/model", "best_model")
正解：D
解説：
To register a model that has been identified by a specific run_id in the MLflow Model Registry, the appropriate line of code is:
mlflow.register_model(f"runs:/{run_id}/model", "best_model")
This code correctly specifies the path to the model within the run (runs:/{run_id}/model) and registers it under the name "best_model" in the Model Registry. This allows the model to be tracked, managed, and transitioned through different stages (e.g., Staging, Production) within the MLflow ecosystem.
Reference
MLflow documentation on model registry: https://www.mlflow.org/docs/latest/model-registry.html#registering-a-model
質問 # 26
Which of the following machine learning algorithms typically uses bagging?
- A. K-means
- B. Gradient boosted trees
- C. Decision tree
- D. Linear regression
- E. Random forest
正解：E
解説：
Random Forest is a machine learning algorithm that typically uses bagging (Bootstrap Aggregating). Bagging involves training multiple models independently on different random subsets of the data and then combining their predictions. Random Forests consist of many decision trees trained on random subsets of the training data and features, and their predictions are averaged to improve accuracy and control overfitting. This method enhances model robustness and predictive performance.
Reference:
Ensemble Methods in Machine Learning (Understanding Bagging and Random Forests).
質問 # 27
A machine learning engineer is using the following code block to scale the inference of a single-node model on a Spark DataFrame with one million records:
Assuming the default Spark configuration is in place, which of the following is a benefit of using an Iterator?
- A. The model will be limited to a single executor preventing the data from being distributed
- B. The data will be limited to a single executor preventing the model from being loaded multiple times
- C. The data will be distributed across multiple executors during the inference process
- D. The model only needs to be loaded once per executor rather than once per batch during the inference process
正解：D
解説：
Using an iterator in the pandas_udf ensures that the model only needs to be loaded once per executor rather than once per batch. This approach reduces the overhead associated with repeatedly loading the model during the inference process, leading to more efficient and faster predictions. The data will be distributed across multiple executors, but each executor will load the model only once, optimizing the inference process.
Reference:
Databricks documentation on pandas UDFs: Pandas UDFs
質問 # 28
Which of the following approaches can be used to view the notebook that was run to create an MLflow run?
- A. Click the "Source" link in the row corresponding to the run in the MLflow experiment page
- B. Click the "Start Time" link in the row corresponding to the run in the MLflow experiment page
- C. Click the "Models" link in the row corresponding to the run in the MLflow experiment paqe
- D. Open the MLmodel artifact in the MLflow run paqe
正解：A
解説：
To view the notebook that was run to create an MLflow run, you can click the "Source" link in the row corresponding to the run in the MLflow experiment page. The "Source" link provides a direct reference to the source notebook or script that initiated the run, allowing you to review the code and methodology used in the experiment. This feature is particularly useful for reproducibility and for understanding the context of the experiment.
Reference:
MLflow Documentation (Viewing Run Sources and Notebooks).
質問 # 29
The implementation of linear regression in Spark ML first attempts to solve the linear regression problem using matrix decomposition, but this method does not scale well to large datasets with a large number of variables.
Which of the following approaches does Spark ML use to distribute the training of a linear regression model for large data?
- A. Spark ML cannot distribute linear regression training
- B. Least-squares method
- C. Singular value decomposition
- D. Logistic regression
- E. Iterative optimization
正解：E
質問 # 30
A data scientist has produced three new models for a single machine learning problem. In the past, the solution used just one model. All four models have nearly the same prediction latency, but a machine learning engineer suggests that the new solution will be less time efficient during inference.
In which situation will the machine learning engineer be correct?
- A. When the new solution requires if-else logic determining which model to use to compute each prediction
- B. When the new solution's models have an average latency that is larger than the size of the original model
- C. When the new solution requires the use of fewer feature variables than the original model
- D. When the new solution's models have an average size that is larger than the size of the original model
- E. When the new solution requires that each model computes a prediction for every record
正解：E
解説：
If the new solution requires that each of the three models computes a prediction for every record, the time efficiency during inference will be reduced. This is because the inference process now involves running multiple models instead of a single model, thereby increasing the overall computation time for each record.
In scenarios where inference must be done by multiple models for each record, the latency accumulates, making the process less time efficient compared to using a single model.
Reference:
Model Ensemble Techniques
質問 # 31
A data scientist wants to efficiently tune the hyperparameters of a scikit-learn model in parallel. They elect to use the Hyperopt library to facilitate this process.
Which of the following Hyperopt tools provides the ability to optimize hyperparameters in parallel?
- A. fmin
- B. quniform
- C. objective_function
- D. SparkTrials
- E. search_space
正解：D
解説：
The SparkTrials class in the Hyperopt library allows for parallel hyperparameter optimization on a Spark cluster. This enables efficient tuning of hyperparameters by distributing the optimization process across multiple nodes in a cluster.
from hyperopt import fmin, tpe, hp, SparkTrials search_space = { 'x': hp.uniform('x', 0, 1), 'y': hp.uniform('y', 0, 1) } def objective(params): return params['x'] ** 2 + params['y'] ** 2 spark_trials = SparkTrials(parallelism=4) best = fmin(fn=objective, space=search_space, algo=tpe.suggest, max_evals=100, trials=spark_trials) Reference:
Hyperopt Documentation
質問 # 32
A machine learning engineer is converting a decision tree from sklearn to Spark ML. They notice that they are receiving different results despite all of their data and manually specified hyperparameter values being identical.
Which of the following describes a reason that the single-node sklearn decision tree and the Spark ML decision tree can differ?
- A. Spark ML decision trees automatically prune overfit trees
- B. Spark ML decision trees test every feature variable in the splitting algorithm
- C. Spark ML decision trees test a random sample of feature variables in the splitting algorithm
- D. Spark ML decision trees test more split candidates in the splitting algorithm
- E. Spark ML decision trees test binned features values as representative split candidates
正解：E
解説：
One reason that results can differ between sklearn and Spark ML decision trees, despite identical data and hyperparameters, is that Spark ML decision trees test binned feature values as representative split candidates. Spark ML uses a method called "quantile binning" to reduce the number of potential split points by grouping continuous features into bins. This binning process can lead to different splits compared to sklearn, which tests all possible split points directly. This difference in the splitting algorithm can cause variations in the resulting trees.
Reference:
Spark MLlib Documentation (Decision Trees and Quantile Binning).
質問 # 33
A data scientist has developed a random forest regressor rfr and included it as the final stage in a Spark MLPipeline pipeline. They then set up a cross-validation process with pipeline as the estimator in the following code block:
Which of the following is a negative consequence of including pipeline as the estimator in the cross-validation process rather than rfr as the estimator?
- A. The process will be unable to parallelize tuning due to the distributed nature of pipeline
- B. The process will leak data prep information from the validation sets to the training sets for each model
- C. The process will leak data from the training set to the test set during the evaluation phase
- D. The process will have a longer runtime because all stages of pipeline need to be refit or retransformed with each mode
正解：D
解説：
Including the entire pipeline as the estimator in the cross-validation process means that all stages of the pipeline, including data preprocessing steps like string indexing and vector assembling, will be refit or retransformed for each fold of the cross-validation. This results in a longer runtime because each fold requires re-execution of these preprocessing steps, which can be computationally expensive.
If only the random forest regressor (rfr) were included as the estimator, the preprocessing steps would be performed once, and only the model fitting would be repeated for each fold, significantly reducing the computational overhead.
Reference:
Databricks documentation on cross-validation: Cross Validation
質問 # 34
Which of the following tools can be used to distribute large-scale feature engineering without the use of a UDF or pandas Function API for machine learning pipelines?
- A. PyTorch
- B. Keras
- C. Scikit-learn
- D. Spark ML
正解：D
解説：
Spark MLlib is a machine learning library within Apache Spark that provides scalable and distributed machine learning algorithms. It is designed to work with Spark DataFrames and leverages Spark's distributed computing capabilities to perform large-scale feature engineering and model training without the need for user-defined functions (UDFs) or the pandas Function API. Spark MLlib provides built-in transformations and algorithms that can be applied directly to large datasets.
Reference:
Databricks documentation on Spark MLlib: Spark MLlib
質問 # 35
A machine learning engineer has grown tired of needing to install the MLflow Python library on each of their clusters. They ask a senior machine learning engineer how their notebooks can load the MLflow library without installing it each time. The senior machine learning engineer suggests that they use Databricks Runtime for Machine Learning.
Which of the following approaches describes how the machine learning engineer can begin using Databricks Runtime for Machine Learning?
- A. They can add a line enabling Databricks Runtime ML in their init script when creating their clusters.
- B. They can set the runtime-version variable in their Spark session to "ml".
- C. They can select a Databricks Runtime ML version from the Databricks Runtime Version dropdown when creating their clusters.
- D. They can check the Databricks Runtime ML box when creating their clusters.
正解：C
解説：
The Databricks Runtime for Machine Learning includes pre-installed packages and libraries essential for machine learning and deep learning, including MLflow. To use it, the machine learning engineer can simply select an appropriate Databricks Runtime ML version from the "Databricks Runtime Version" dropdown menu while creating their cluster. This selection ensures that all necessary machine learning libraries, including MLflow, are pre-installed and ready for use, avoiding the need to manually install them each time.
Reference
Databricks documentation on creating clusters: https://docs.databricks.com/clusters/create.html
質問 # 36
A machine learning engineer has created a Feature Table new_table using Feature Store Client fs. When creating the table, they specified a metadata description with key information about the Feature Table. They now want to retrieve that metadata programmatically.
Which of the following lines of code will return the metadata description?
- A. fs.get_table("new_table").load_df()
- B. There is no way to return the metadata description programmatically.
- C. fs.get_table("new_table").description
- D. fs.get_table("new_table")
- E. fs.create_training_set("new_table")
正解：C
解説：
To retrieve the metadata description of a feature table created using the Feature Store Client (referred here as fs), the correct method involves calling get_table on the fs client with the table name as an argument, followed by accessing the description attribute of the returned object. The code snippet fs.get_table("new_table").description correctly achieves this by fetching the table object for "new_table" and then accessing its description attribute, where the metadata is stored. The other options do not correctly focus on retrieving the metadata description.
Reference:
Databricks Feature Store documentation (Accessing Feature Table Metadata).
質問 # 37
A machine learning engineering team has a Job with three successive tasks. Each task runs a single notebook. The team has been alerted that the Job has failed in its latest run.
Which of the following approaches can the team use to identify which task is the cause of the failure?
- A. Change each Task's setting to use a dedicated cluster
- B. Migrate the Job to a Delta Live Tables pipeline
- C. Run each notebook interactively
- D. Review the matrix view in the Job's runs
正解：D
解説：
To identify which task is causing the failure in the job, the team should review the matrix view in the Job's runs. The matrix view provides a clear and detailed overview of each task's status, allowing the team to quickly identify which task failed. This approach is more efficient than running each notebook interactively, as it provides immediate insights into the job's execution flow and any issues that occurred during the run.
Reference:
Databricks documentation on Jobs: Jobs in Databricks
質問 # 38
A data scientist has been given an incomplete notebook from the data engineering team. The notebook uses a Spark DataFrame spark_df on which the data scientist needs to perform further feature engineering. Unfortunately, the data scientist has not yet learned the PySpark DataFrame API.
Which of the following blocks of code can the data scientist run to be able to use the pandas API on Spark?
- A. import pyspark.pandas as ps
df = ps.to_pandas(spark_df) - B. spark_df.to_pandas()
- C. import pyspark.pandas as ps
df = ps.DataFrame(spark_df) - D. import pandas as pd
df = pd.DataFrame(spark_df)
正解：C
解説：
To use the pandas API on Spark, the data scientist can run the following code block:
import pyspark.pandas as ps df = ps.DataFrame(spark_df)
This code imports the pandas API on Spark and converts the Spark DataFrame spark_df into a pandas-on-Spark DataFrame, allowing the data scientist to use familiar pandas functions for further feature engineering.
Reference:
Databricks documentation on pandas API on Spark: pandas API on Spark
質問 # 39
An organization is developing a feature repository and is electing to one-hot encode all categorical feature variables. A data scientist suggests that the categorical feature variables should not be one-hot encoded within the feature repository.
Which of the following explanations justifies this suggestion?
- A. One-hot encoding is not supported by most machine learning libraries.
- B. One-hot encoding is a potentially problematic categorical variable strategy for some machine learning algorithms.
- C. One-hot encoding is dependent on the target variable's values which differ for each application.
- D. One-hot encoding is not a common strategy for representing categorical feature variables numerically.
- E. One-hot encoding is computationally intensive and should only be performed on small samples of training sets for individual machine learning problems.
正解：B
解説：
One-hot encoding transforms categorical variables into a format that can be provided to machine learning algorithms to better predict the output. However, when done prematurely or universally within a feature repository, it can be problematic:
Dimensionality Increase: One-hot encoding significantly increases the feature space, especially with high cardinality features, which can lead to high memory consumption and slower computation.
Model Specificity: Some models handle categorical variables natively (like decision trees and boosting algorithms), and premature one-hot encoding can lead to inefficiency and loss of information (e.g., ordinal relationships).
Sparse Matrix Issue: It often results in a sparse matrix where most values are zero, which can be inefficient in both storage and computation for some algorithms.
Generalization vs. Specificity: Encoding should ideally be tailored to specific models and use cases rather than applied generally in a feature repository.
Reference
"Feature Engineering and Selection: A Practical Approach for Predictive Models" by Max Kuhn and Kjell Johnson (CRC Press, 2019).
質問 # 40
Which of the following statements describes a Spark ML estimator?
- A. An estimator is a trained ML model which turns a DataFrame with features into a DataFrame with predictions
- B. An estimator is an evaluation tool to assess to the quality of a model
- C. An estimator is an alqorithm which can be fit on a DataFrame to produce a Transformer
- D. An estimator is a hyperparameter arid that can be used to train a model
- E. An estimator chains multiple alqorithms toqether to specify an ML workflow
正解：C
解説：
In the context of Spark MLlib, an estimator refers to an algorithm which can be "fit" on a DataFrame to produce a model (referred to as a Transformer), which can then be used to transform one DataFrame into another, typically adding predictions or model scores. This is a fundamental concept in machine learning pipelines in Spark, where the workflow includes fitting estimators to data to produce transformers.
Reference
Spark MLlib Documentation: https://spark.apache.org/docs/latest/ml-pipeline.html#estimators
質問 # 41
A data scientist has developed a linear regression model using Spark ML and computed the predictions in a Spark DataFrame preds_df with the following schema:
prediction DOUBLE
actual DOUBLE
Which of the following code blocks can be used to compute the root mean-squared-error of the model according to the data in preds_df and assign it to the rmse variable?
- A.
- B.
- C.
- D.
正解：A
解説：
To compute the root mean-squared-error (RMSE) of a linear regression model using Spark ML, the RegressionEvaluator class is used. The RegressionEvaluator is specifically designed for regression tasks and can calculate various metrics, including RMSE, based on the columns containing predictions and actual values.
The correct code block to compute RMSE from the preds_df DataFrame is:
regression_evaluator = RegressionEvaluator( predictionCol="prediction", labelCol="actual", metricName="rmse" ) rmse = regression_evaluator.evaluate(preds_df) This code creates an instance of RegressionEvaluator, specifying the prediction and label columns, as well as the metric to be computed ("rmse"). It then evaluates the predictions in preds_df and assigns the resulting RMSE value to the rmse variable.
Options A and B incorrectly use BinaryClassificationEvaluator, which is not suitable for regression tasks. Option D also incorrectly uses BinaryClassificationEvaluator.
Reference:
PySpark ML Documentation
質問 # 42
A machine learning engineer is trying to scale a machine learning pipeline by distributing its feature engineering process.
Which of the following feature engineering tasks will be the least efficient to distribute?
- A. Imputing missing feature values with the mean
- B. Creating binary indicator features for missing values
- C. One-hot encoding categorical features
- D. Imputing missing feature values with the true median
- E. Target encoding categorical features
正解：D
解説：
Among the options listed, calculating the true median for imputing missing feature values is the least efficient to distribute. This is because the true median requires knowledge of the entire data distribution, which can be computationally expensive in a distributed environment. Unlike mean or mode, finding the median requires sorting the data or maintaining a full distribution, which is more intensive and often requires shuffling the data across partitions.
Reference
Challenges in parallel processing and distributed computing for data aggregation like median calculation: https://www.apache.org
質問 # 43
......
Databricks Databricks-Machine-Learning-Associate 認定試験の出題範囲：
| トピック | 出題範囲 |
|---|---|
| トピック 1 |
|
| トピック 2 |
|
| トピック 3 |
|
| トピック 4 |
|
100%の合格率を試そう！更新されたのはDatabricks-Machine-Learning-Associate試験問題 [2025年更新]：https://www.jpntest.com/shiken/Databricks-Machine-Learning-Associate-mondaishu
合格させるDatabricks-Machine-Learning-Associate試験にはリアル問題解答：https://drive.google.com/open?id=1E4-1nNrJt4APvKFQfCxYPeJiCxb93hQa