Free Scikit-learn Expert Practitioner Certification Sample Questions

Correct answer: A — class_weight='balanced'

• A. Setting class_weight='balanced' automatically adjusts weights inversely proportional to class frequencies, increasing the penalty for misclassifying minority class samples. This is the standard parameter for handling class imbalance in scikit-learn's LogisticRegression.
• B. Incorrect. loss='hinge' is not a valid parameter for LogisticRegression; it is associated with SVM-style classifiers such as SGDClassifier or LinearSVC. It does not address class imbalance in logistic regression.
• C. Incorrect. penalty='l1' controls the type of regularization (L1) applied to the model coefficients, encouraging sparsity. It does not change the relative cost of misclassifying one class versus another.
• D. Incorrect. C is the inverse of regularization strength; lowering it increases regularization and raising it decreases regularization. It does not specifically penalize minority class errors more.

Correct answer: A — Enable early stopping with a validation set.

• A. Correct. HistGradientBoostingClassifier supports early stopping via the `early_stopping` parameter set to True and providing a validation set (e.g., using `validation_fraction`). This monitors validation performance and stops training when improvement stalls, directly reducing overfitting.
• B. Incorrect. Increasing max_depth makes trees more complex, which typically increases overfitting. To prevent overfitting, you would reduce max_depth or use other regularization, not increase it.
• C. Incorrect. HistGradientBoostingClassifier does not support 'categorical_crossentropy'; it uses 'log_loss' (cross-entropy) for multi-class. Changing to an unsupported loss is not a valid overfitting-control technique.
• D. Incorrect. Decreasing learning_rate can sometimes improve generalization, but it is not the most direct built-in overfitting control. Early stopping is a more explicit mechanism, as the estimator does not automatically adjust learning_rate for overfitting prevention.

Correct answers: A, B, C — ('var', VarianceThreshold()); ('select', SelectKBest(f_classif, k=10)); ('select', SelectPercentile(f_classif, percentile=10))

• A. Correct. VarianceThreshold is a simple unsupervised feature selection method that removes low-variance features.
• B. Correct. SelectKBest with f_classif is a supervised univariate feature selection method that selects the top k features based on ANOVA F-scores.
• C. Correct. SelectPercentile with f_classif is a supervised univariate feature selection method that selects a percentile of features based on F-scores.
• D. Incorrect. RandomForestClassifier is a predictive model, not a feature selection method. It can provide feature importances for use with SelectFromModel.
• E. Incorrect. FeatureUnion concatenates outputs of multiple transformers; it is not a feature selection method.
• F. Incorrect. PCA is a dimensionality reduction technique that creates new components, not a feature selection method.

Correct answer: F — Brier score

• A. Incorrect. Accuracy does not measure calibration; it simply counts correct predictions and depends on a decision threshold, ignoring the quality of predicted probabilities.
• B. Incorrect. AUC-ROC evaluates the ranking ability of a classifier across thresholds and is robust to class imbalance, but it does not assess calibration. A model with perfect ranking can still have poorly calibrated probabilities.
• C. Incorrect. Average precision summarizes the precision-recall curve, focusing on the positive class ranking. It is useful for imbalanced data but does not measure probability calibration.
• D. Incorrect. F1 score combines precision and recall at a specific threshold, balancing false positives and false negatives. It does not evaluate the quality of predicted probabilities or calibration.
• E. Incorrect. Log loss (cross-entropy) is a proper scoring rule that penalizes confident wrong predictions, but it conflates calibration and refinement. It is not a pure calibration metric.
• F. Correct. Brier score measures the mean squared difference between predicted probabilities and actual outcomes, directly assessing probabilistic calibration. It can be decomposed into calibration and refinement components.

Correct answer: B — ECE

• A. Incorrect. AUC (Area Under the ROC Curve) measures the ranking power of a model by evaluating how well it separates positive and negative classes across thresholds. It does not compare predicted confidence to observed accuracy in bins, and thus does not assess calibration directly.
• B. Correct. Expected Calibration Error (ECE) is computed by partitioning predictions into bins, computing the average predicted confidence and the actual accuracy within each bin, and then taking the weighted average of the absolute differences between them. This directly quantifies calibration quality.
• C. Incorrect. The Brier score measures the mean squared difference between predicted probabilities and actual binary outcomes. Although it reflects both calibration and refinement, it does not involve binning predictions or computing absolute differences between average confidence and accuracy.
• D. Incorrect. Log loss (logarithmic loss) calculates the negative log-likelihood of the predicted probabilities for the true labels. It penalizes overconfident incorrect predictions but does not rely on binning or weighted absolute differences between confidence and accuracy.

Correct answer: B — It assumes the base estimator is already fitted, using the data for calibration.

• A. Incorrect. This describes the standard cross-validation calibration workflow, where the estimator is trained and calibrated across folds. With cv='prefit', the base estimator is not retrained inside CalibratedClassifierCV; it assumes the base estimator is already fitted.
• B. Correct. When cv='prefit', CalibratedClassifierCV assumes the base estimator is already fitted and uses the provided data solely for calibration, without refitting the base model. This is useful for calibrating a pre-trained model efficiently.
• C. Incorrect. 'prefit' is a valid setting for CalibratedClassifierCV. It does not raise an error; it changes the behavior to assume the estimator is already trained, skipping the cross-validation fitting step.
• D. Incorrect. While calibration ideally requires data separate from training data, cv='prefit' does not itself create or enforce a separate validation split. The user must provide appropriate calibration data, and the calibrator is fitted directly on that input data.

Correct answer: C — An overconfident model produces probability estimates that are too extreme.

• A. Incorrect. A perfectly calibrated model would have its reliability curve lie exactly on the diagonal across all probability ranges. The described curve deviates systematically from the diagonal, indicating miscalibration.
• B. Incorrect. Underconfidence would produce the opposite pattern: the curve would be below the diagonal for low probabilities and above for high probabilities, because estimated probabilities are too close to 0.5. The described pattern is the reverse.
• C. Correct. Being above the diagonal at low predicted probabilities means the true fraction of positives is higher than predicted, and being below the diagonal at high probabilities means the true fraction is lower. This indicates the model is overconfident, assigning probabilities too extreme (too close to 0 or 1).
• D. Incorrect. The reliability diagram assesses calibration, not discrimination. The described curve clearly indicates miscalibration, so the model does not have good calibration. Poor discrimination may coexist, but the curve does not speak to discrimination.

Correct answer: B — The bias-variance trade-off in calibration.

• A. Precision and recall measure classification performance at a specific threshold, not the behavior of calibration curves under different bin counts. The trade-off illustrated is about how binning granularity affects the estimated calibration, not about decision-threshold trade-offs.
• B. Increasing the number of bins reduces bias (the curve fits the diagonal more closely) but increases variance (higher variability in the estimated probabilities). This is the classic bias-variance trade-off applied to calibration estimation: fewer bins yield a smoother but potentially underfit curve, while more bins yield a more accurate but noisy curve.
• C. Training and inference time are computational performance considerations, not the trade-off observed in calibration curves. While more bins might slightly increase computation, the question focuses on the stability and fidelity of the calibration curve, not runtime.
• D. Histogram binning does not directly correspond to a choice between linear and non-linear calibration methods. The observed effect of bin count on curve smoothness and variability is better described by the bias-variance trade-off, not the form of the calibration function.

Correct answers: A, E — Inherit from `ClusterMixin` for default `fit_predict`.; Use `check_estimator` to verify API compliance.

• A. Correct. Inheriting from `ClusterMixin` provides a default `fit_predict` implementation and signals to scikit-learn that the estimator is a clusterer, which enables interoperability with utilities like `check_estimator` and meta-estimators.
• B. Incorrect. In scikit-learn, the `fit` method should return `self` to allow method chaining. Cluster labels are stored as an attribute (e.g., `labels_`) after fitting, not returned directly.
• C. Incorrect. A `predict` method is optional for clustering estimators; many algorithms (e.g., DBSCAN, AgglomerativeClustering) only support `fit` and `fit_predict`. It is not a requirement of the sklearn API.
• D. Incorrect. The `cluster_centers_` attribute is only appropriate for algorithms that compute centroids (e.g., KMeans). It is not a general requirement for all clustering estimators.
• E. Correct. `check_estimator` is the standard tool for validating that a custom estimator complies with scikit-learn's API and conventions, including method signatures, attribute naming, and cloning behavior.

Correct answer: C — To route metadata like `sample_weight` to specific estimators and methods in a composite estimator.

• A. Incorrect. Metadata routing is not used for hyperparameter tuning optimization or sharing information across cross-validation folds. Its purpose is to control how extra metadata is passed to the right methods in a composed estimator.
• B. Incorrect. Preventing shape mismatches when concatenating transformer outputs is handled by feature union or pipeline output management, not metadata routing. Metadata routing concerns auxiliary arguments such as sample weights, groups, or validation data, not feature dimensions.
• C. Correct. Metadata routing ensures that metadata (e.g., sample_weight, target, or custom metadata) is correctly passed to the appropriate estimators and methods in a composite estimator or pipeline. It allows estimators to explicitly declare which extra inputs they consume.
• D. Incorrect. Metadata routing does not aim to reduce memory usage by deduplicating data across pipeline steps. It is an API mechanism for requesting, accepting, and forwarding metadata arguments safely through estimator hierarchies, not memory optimization.

Correct answer: A — Metadata routing must be explicitly enabled via `sklearn.set_config(enable_metadata_routing=True)` before using it.

• A. Correct. Metadata routing is not active by default in scikit-learn, even in recent versions. Users must explicitly enable it using `sklearn.set_config(enable_metadata_routing=True)` before it can be used with estimators and meta-estimators.
• B. Incorrect. Metadata routing is not enabled by default; users must opt in via the global configuration setting. The statement is false.
• C. Incorrect. Metadata routing is available for pipelines, transformers, and other meta-estimators, not just cross-validation routines. It is designed for the entire scikit-learn ecosystem.
• D. Incorrect. Enabling metadata routing does not automatically forward parameters like `sample_weight` to every step. Each estimator must explicitly declare which metadata it requests, and only those are routed.

Correct answer: B — Gaussian Naive Bayes under naive independence assumption

• A. Incorrect. Logistic regression directly models class probabilities and typically yields well-calibrated estimates, even with default L2 regularization. It is one of the better-calibrated classifiers.
• B. Correct. Gaussian Naive Bayes is often poorly calibrated because its strong conditional independence assumption is frequently violated in real data, leading to overconfident (extreme) probability estimates. Post-calibration techniques like isotonic regression or Platt scaling are often beneficial.
• C. Incorrect. While random forests can be miscalibrated in some settings, with many trees (e.g., 500) and sufficient data (e.g., 10,000 samples), they often provide reasonably stable probability estimates. They are not as consistently poorly calibrated as Gaussian Naive Bayes.
• D. Incorrect. Setting probability=True for a linear SVM enables Platt scaling, which explicitly calibrates the decision scores into probability estimates. Therefore, the classifier is already calibrated.

Correct answer: B — A logistic regression model on raw outputs of the base estimator.

• A. Incorrect. Platt scaling (sigmoid method) does not fit a linear regression on log-odds. Instead, it learns a logistic regression (sigmoid) mapping from the base estimator's outputs to calibrated probabilities.
• B. Correct. With method='sigmoid', CalibratedClassifierCV uses Platt scaling, which fits a logistic regression model on the base estimator's raw outputs (decision function or predicted probabilities) to produce calibrated probabilities.
• C. Incorrect. Isotonic regression is used with method='isotonic', not 'sigmoid'. The sigmoid method employs a parametric logistic regression, not a non-parametric isotonic regression with any kernel.
• D. Incorrect. A support vector machine with a sigmoid kernel is unrelated to calibration. The calibration step in CalibratedClassifierCV fits a separate sigmoid mapping, not an SVM.

Correct answer: A — `importances_mean`, `importances_std`, and `importances` (the raw scores for each repeat).

• A. Correct. According to scikit-learn documentation, the `Bunch` object returned by `permutation_importance` contains `importances_mean` (the mean of the importance scores), `importances_std` (the standard deviation across repeats), and `importances` (the raw scores for each individual shuffle/repeat).
• B. Incorrect. The standard `Bunch` output for `permutation_importance` does not include `feature_names`, `importance_scores`, or `p_values`. Feature names are typically maintained via the input X (e.g., if it is a pandas DataFrame).
• C. Incorrect. These terms are more closely associated with impurity-based feature importance found in tree-based estimators (like Random Forests). `feature_importances_` is an attribute of the model, not a return value of this inspection utility.
• D. Incorrect. The function returns summary statistics and raw importances for the specific dataset (X, y) provided. It does not internally handle or return separate buckets for train, test, or cross-validation importance.

Correct answers: A, C — Passing the entire pipeline to `permutation_importance` will permute the raw text strings before TF-IDF vectorization.; Permuting the raw text column evaluates the importance of the entire text feature as a whole, rather than individual words.

• A. Correct. `permutation_importance` operates by shuffling the columns of the input data `X` provided to the function. Since the pipeline starts with a `TfidfVectorizer` that accepts raw strings, the permutation happens on these strings before they are transformed into a numerical representation.
• B. Incorrect. Shuffling occurs on the input features passed to the `permutation_importance` function. Because the tokens (words) are generated internally by the vectorizer within the pipeline, they are not seen as independent features by the permutation function. To permute individual tokens, one would need to pass the pre-transformed sparse matrix to the classifier directly.
• C. Correct. By shuffling the entire string within the text column, the relationship between that text field and the target is broken. This allows the user to evaluate the cumulative predictive value of the text field as a single feature relative to other columns like metadata.
• D. Incorrect. `permutation_importance` is agnostic to data types as long as the provided estimator (in this case, the pipeline) can handle them. Since the pipeline includes a `TfidfVectorizer`, it is perfectly capable of processing string data during the importance calculation.
• E. Incorrect. There is no specific requirement to use `n_jobs=1` for text data. While text processing and large sparse matrices can be memory-intensive, `n_jobs` is a standard parameter for parallelization and does not cause inherent memory corruption.

Correct answer: A — The count of grid points per feature for evaluating partial dependence.

• A. Correct. The `grid_resolution` parameter specifies the number of grid points per feature used to evaluate the partial dependence. A larger value makes the grid finer and the plot smoother, at the cost of more computation.
• B. Incorrect. `grid_resolution` does not control any rounding of feature values or plotting precision. It only affects the number of evaluation points used for the partial dependence calculation.
• C. Incorrect. The size of the dataset used for partial dependence is not controlled by `grid_resolution`. The dataset size is determined by the input data, and any sampling is handled by other parameters.
• D. Incorrect. DPI is a figure rendering property and is unrelated to `grid_resolution`. The parameter concerns the computational grid for feature values, not image output quality.

Correct answer: D — A feature with a constant value (variance = 0).

• A. Incorrect. Variance ≈ 0.0099 is below the default threshold of 0.01, so it would be dropped, but the constant feature (variance=0) is even more likely to be removed.
• B. Incorrect. Variance ≈ 0.09 is well above any common threshold (0.01 or 0.05), so it would be kept.
• C. Incorrect. Variance = 0.25 is the maximum for a Bernoulli variable and far above thresholds, so it would be kept.
• D. Correct. Variance = 0 is below any positive threshold, so it will always be removed by variance thresholding.
• E. Incorrect. Variance ≈ 0.0475 is above 0.01 but below 0.05; it may be dropped depending on threshold, but not as certainly as the constant feature.

Correct answer: B — It transforms the text documents into TF-IDF feature vectors.

• A. Incorrect. The code does not perform PCA or dimensionality reduction. TF-IDF produces weighted term features, not latent components like sparse PCA. Reducing from 5000 to 100 dimensions is not what TfidfVectorizer does.
• B. Correct. TfidfVectorizer converts text documents into numerical TF-IDF feature vectors. With max_features=100, it limits the vocabulary to the top 100 terms. This is a standard preprocessing step for text classification.
• C. Incorrect. TF-IDF does not regularize logistic regression or compress the feature space directly. Regularization is a property of the logistic regression model itself, not the vectorizer.
• D. Incorrect. Extracting interpretable topics is done by topic models like LDA or NMF, not TF-IDF. TF-IDF highlights important words but does not generate topic summaries.

Correct answer: B — Set the `memory` parameter in the pipeline to cache intermediate transformations.

• A. Incorrect. Setting `n_jobs=-1` is not a general approach for parallelizing transformers; it only works if the specific transformer supports it, and many do not. `FeatureUnion` itself has an `n_jobs` parameter, but that only parallelizes the transformers if they are n_jobs-aware.
• B. Correct. The `memory` parameter caches intermediate transformation results, avoiding redundant computations when `fit` or `transform` is called multiple times, especially during cross-validation or grid search.
• C. Incorrect. Replacing `FeatureUnion` with a single custom transformer would remove composability and modularity, and is not a recommended performance optimization.
• D. Incorrect. While keeping `fit` light is good practice, it is not a specific technique for improving pipeline performance in this context; the primary optimization is caching via `memory`.
• E. Incorrect. `make_union` is just a convenience wrapper for creating a `FeatureUnion` with default parameters; it does not change execution speed.
• F. Incorrect. `Pipeline` is designed to be efficient and does not introduce unnecessary overhead; avoiding it would lead to less maintainable code and potential data leakage.

Correct answers: A, B — Impurity-based importance is computed on the training set, while permutation importance can be computed on a held-out test set.; Impurity-based importance tends to inflate the importance of high-cardinality features, whereas permutation importance mitigates this bias when evaluated on unseen data.

• A. Correct. Impurity-based importance (MDI) is derived directly from the tree statistics gathered during the training process. Permutation importance is a model-agnostic post-hoc method that can be computed on any dataset, such as a held-out test set, to measure how much the model performance drops when a feature's values are shuffled.
• B. Correct. Impurity-based importance is notoriously biased toward high-cardinality features because these features offer more potential split points, allowing the model to reduce impurity more easily on the training set. Permutation importance, particularly when evaluated on a validation or test set, provides a more unbiased estimate of a feature's actual predictive power.
• C. Incorrect. Permutation importance is computationally much more expensive than impurity-based importance. While impurity importance is available immediately after training, permutation importance requires K * N passes of model predictions (where K is the number of features and N is the number of repeats).
• D. Incorrect. Both methods are versatile and support both classification and regression tasks. Impurity measures change from Gini/Entropy to Mean Squared Error/Variance depending on the task, but the concept remains the same.
• E. Incorrect. Permutation importance does not involve retraining the model. It uses the already-fitted model to make predictions on data where a specific feature's column has been randomly shuffled. Retraining for each feature would describe 'Leave-One-Feature-Out' (LOFO) importance, not permutation importance.

Correct answer: B — The model's predictions on the test set actually improved (or the error decreased) when this feature was randomly shuffled, suggesting the feature is uninformative or the model is overfitting to it.

• A. Permutation importance measures the decrease in model performance when a feature's values are shuffled. A feature that has an inversely proportional relationship with the target (like a negative coefficient in a linear model) is still predictive; shuffling it would decrease model performance, resulting in a positive importance score.
• B. Correct. Permutation importance is calculated as (baseline_score - permuted_score). A negative value means the model performed better on the test set after the feature was shuffled (permuted_score > baseline_score). This suggests the feature provides no useful signal or, more likely, that the model has overfit to noise or spurious correlations in that feature that do not generalize.
• C. While perfect collinearity can split importance between features and make individual importance scores less reliable or lower, it does not specifically cause a negative importance score. Negative scores are a result of the model's performance increasing upon the removal of the feature's signal.
• D. A negative importance score is a mathematically valid and expected outcome in permutation importance when a feature is harmful to the model's generalization; it is not indicative of a technical bug like integer overflow or a random state error.

Correct answer: B — The model's predictions are systematically inverted (predicting the negative class when it should predict positive).

• A. Incorrect. A perfectly random model would produce an ROC curve close to the diagonal line of no-discrimination with an AUC ≈ 0.5. Falling below the diagonal indicates a systematic relationship, just not the intended one.
• B. Correct. If the model's predictions are systematically inverted (e.g., swapping class labels or reversing the decision rule), the ROC curve will lie below the diagonal. In such cases, flipping the prediction labels or probabilities would yield an AUC > 0.5.
• C. Incorrect. Heavy overfitting typically yields high performance on training data and poor performance on held-out data, but it does not by itself produce systematically inverted predictions. Overfitting usually increases variance rather than causing a consistent negative correlation with true labels.
• D. Incorrect. While class imbalance affects many metrics like precision and recall, the ROC curve and AUC are largely insensitive to class distribution. Imbalance alone does not explain a systematically inverted ROC curve.

Correct answer: C — Feature_A has strong interaction effects with other features, causing its marginal effect to average out to zero.

• A. Incorrect. While a flat PDP line suggests that the marginal effect of Feature_A averages out to zero, the ICE lines clearly show that Feature_A affects predictions conditionally. The lack of a marginal trend does not mean the feature is unimportant; it simply means its influence is not consistent across the dataset.
• B. Incorrect. If Feature_A had a strong, independent linear effect, both the ICE lines and the PDP would exhibit consistent, similar slopes. Heterogeneous slopes in ICE indicate that the effect of the feature is not independent of other variables.
• C. Correct. A flat PDP combined with ICE lines that show both strong positive and negative slopes indicates that the effect of Feature_A is highly dependent on the values of other features (heterogeneous effects). These interactions cause the conditional effects to cancel each other out when averaged across the dataset, resulting in a flat marginal plot.
• D. Incorrect. The presence of varied and strong slopes in ICE lines indicates the model is capturing complex relationships and interactions. Underfitting would more likely result in simplified, flat, or uniform ICE lines across all instances due to the model's inability to capture complexity.

Correct answers: A, C — Use `pyarrow.parquet.ParquetFile.iter_batches()` to yield manageable chunks of data.; Call the `partial_fit` method on the `SGDClassifier` for each loaded chunk.

• A. Correct. `pyarrow.parquet.ParquetFile.iter_batches()` allows you to read Parquet files in manageable row batches. This streaming approach avoids loading the entire 100GB dataset into memory, making it the standard way to handle large-scale Parquet data for out-of-core workflows.
• B. Incorrect. Loading a 100GB dataset into a 32GB RAM machine is physically impossible without compression or chunking. The specific engine used (e.g., 'fastparquet') does not reduce the memory footprint of the materialized pandas DataFrame.
• C. Correct. The `partial_fit` method is scikit-learn's incremental learning API designed for out-of-core learning. It updates model parameters chunk-by-chunk. Note that for the first call to `partial_fit`, you must provide the list of all possible classes in the `classes` parameter.
• D. Incorrect. `GridSearchCV` expects data to be available in memory and does not natively support file paths or automatic chunking of Parquet files. It cannot handle out-of-core datasets without custom wrappers or integration with tools like Dask.
• E. Incorrect. Relying on system swap space for a dataset significantly larger than RAM causes 'disk thrashing' and severe performance degradation. Furthermore, the standard `fit()` method expects all input data to be present in a single contiguous block (like a NumPy array or DataFrame), which would still exceed memory limits.

Correct answer: A — BaseEstimator and TransformerMixin

• A. Correct. Inheriting from BaseEstimator provides the automatic get_params and set_params implementation required by GridSearchCV for hyperparameter tuning. TransformerMixin supplies the fit_transform convenience method and signals that the class implements the transformer interface, making it fully compatible with scikit-learn Pipelines.
• B. Incorrect. While TransformerMixin is useful for transformers, Pipeline is a composite estimator designed to chain multiple steps together. Inheriting from Pipeline is a design error for a single transformer and does not provide the parameter handling logic found in BaseEstimator.
• C. Incorrect. RegressorMixin is intended for regression estimators (predictors) and provides a specific score method. It does not provide the transform or fit_transform methods required for a transformer, and it is inappropriate for a feature engineering task.
• D. Incorrect. FeatureUnion is used to combine the outputs of multiple transformers by running them in parallel. It is a meta-estimator, not a base class intended for defining the internal logic of a standalone custom transformer.

Correct answer: C — strategy='quantile'

• A. Incorrect. The 'uniform' strategy divides the feature's range into bins of equal width. Because it focuses on interval distance rather than sample density, bins will contain different numbers of samples depending on the data distribution.
• B. Incorrect. The 'kmeans' strategy uses a 1D k-means clustering algorithm to define bins by minimizing the within-cluster variance. This produces bins that reflect natural clusters in the data but does not ensure that each bin has an equal sample count.
• C. Correct. The 'quantile' strategy uses the quantiles of the dataset to define bin edges. This ensures that each bin contains approximately the same number of samples, effectively performing equal-frequency discretization.
• D. Incorrect. 'frequency' is not a valid parameter value for the strategy argument in scikit-learn's KBinsDiscretizer. While the concept is referred to as equal-frequency binning, the implementation name is 'quantile'.

Correct answers: A, B, C — `n_knots`; `degree`; `extrapolation`

• A. `n_knots` is a valid parameter in `SplineTransformer` that specifies the number of knots for the spline basis. It directly influences the number of generated features and the overall flexibility of the spline transformation.
• B. `degree` is a fundamental parameter in `SplineTransformer` that determines the degree of the B-spline polynomial (e.g., cubic splines use `degree=3`). It affects the smoothness and complexity of the resulting basis functions.
• C. `extrapolation` is a parameter in `SplineTransformer` that defines the strategy for handling values outside the range of the knots. Supported strategies include 'constant', 'linear', 'continue', 'periodic', or 'error'.
• D. `interaction_only` is not a parameter of `SplineTransformer`. It is associated with `PolynomialFeatures`, where it determines whether only interaction terms (and not powers) should be generated.
• E. `strategy` is a parameter used in `KBinsDiscretizer` to define how bin edges are calculated (e.g., 'uniform', 'quantile'). In `SplineTransformer`, the equivalent parameter for knot placement is named `knots`.
• F. `add_indicator` is a parameter typically found in missing value imputers like `SimpleImputer` or `IterativeImputer`. It is not part of the `SplineTransformer` API.

Correct answer: B — The agreement between predicted probabilities and the actual observed frequencies of events.

• A. Incorrect. This describes the model's discrimination or ranking ability (e.g., AUC), not calibration. Calibration evaluates the accuracy of predicted probabilities, not their ordering.
• B. Correct. Calibration measures how closely predicted probabilities match observed outcome frequencies. For example, if a model predicts 0.7 for a group of samples, about 70% should actually be positive for the model to be well calibrated.
• C. Incorrect. This reflects sharpness or overconfidence, not calibration. A model can produce extreme probabilities but still be poorly calibrated if those probabilities do not align with actual event rates.
• D. Incorrect. This describes accuracy at a fixed threshold of 0.5, which depends on decision threshold selection. Calibration evaluates the reliability of probabilities across all thresholds, not just one.

Correct answer: A — The network is overfitting and becoming increasingly overconfident on misclassified examples.

• A. Correct. When a model overfits, it becomes increasingly confident in its predictions on the training data. During validation, if it misclassifies examples with high confidence, the log loss heavily penalizes these confident wrong predictions, leading to a rising validation log loss as training progresses.
• B. Incorrect. A high learning rate can cause unstable weight updates and divergence, but this typically manifests more directly in training loss. Validation log loss increasing is not a direct consequence of a high learning rate; the described behavior is better explained by overconfidence on misclassified examples.
• C. Incorrect. Mislabeled validation examples can increase log loss, but they would cause a consistent elevation rather than a systematic increase during training. A rising validation log loss over epochs is more likely due to the model becoming overconfident on mistakes.
• D. Incorrect. Although log loss can be numerically unstable with probabilities near 0 or 1, modern implementations clip probabilities to avoid infinite values. This does not explain a training-related increase in validation log loss, which is typically due to overconfidence on misclassified examples.

Correct answer: B — refit='calibration'

• A. Setting refit='recall' would select the model maximizing recall, not calibration. Since the goal is to prioritize calibration, this is not appropriate.
• B. Correct. In multi-metric scoring, refit must be set to one of the scoring dictionary keys. The key 'calibration' corresponds to 'neg_brier_score' and selecting it will choose the model with the best calibration as measured by the Brier score.
• C. The scoring dictionary does not include 'accuracy', so it is not a valid refit key. Moreover, accuracy does not directly measure calibration.
• D. While 'neg_brier_score' is the underlying scorer, refit expects a dictionary key, not the scorer name. The correct key is 'calibration'.

Correct answer: B — A CalibratedClassifierCV with method='sigmoid' around the logistic regression.

• A. Incorrect. Logistic regression probabilities are often reasonably calibrated but not guaranteed, especially with regularization, class imbalance, or model misspecification. Calibration is not inherent.
• B. Correct. Wrapping the classifier in CalibratedClassifierCV with method='sigmoid' explicitly fits a calibration model on top of the base estimator's scores, ensuring well-calibrated probabilities when optimizing a calibration-sensitive metric like Brier score.
• C. Incorrect. RandomForestClassifier does not have built-in calibration guarantees; its probabilities can be poorly calibrated and may still need calibration. The pipeline uses logistic regression, not random forest.
• D. Incorrect. Adding another PCA step does not address probability calibration. PCA is for dimensionality reduction and may affect predictive performance, but it is not a calibration method.

Correct answer: A — Specify `needs_proba=True` in `make_scorer` and confirm the model exposes `predict_proba`.

• A. Correct. Setting `needs_proba=True` in `make_scorer` instructs the scorer to use `predict_proba` instead of `predict`. The estimator must expose `predict_proba` for this to work. This is the standard way to build a scorer for metrics that consume class probabilities.
• B. Incorrect. The `needs_threshold=True` flag is used for scorers that require a continuous decision score (e.g., from `decision_function`), not specifically for probability estimates. For probability-based metrics, `needs_proba=True` is the appropriate parameter.
• C. Incorrect. While you can call `predict_proba` directly inside a custom metric function, that bypasses the `make_scorer` framework. The question asks for a valid approach using `make_scorer`, not a manual workaround.
• D. Incorrect. `make_scorer` does not have a `predict_method` parameter. The relevant parameters for controlling predictions are `needs_proba` and `needs_threshold`. Setting a non-existent parameter will raise an error.

Correct answer: B — ONNX

• A. Incorrect. joblib is natively supported by scikit-learn for model serialization, especially for models with large NumPy arrays. However, it is Python-specific and not designed for cross-language deployment, limiting its use in non-Python production environments.
• B. Correct. ONNX is an interoperable model exchange format designed to enable deployment across different runtimes and languages. With the skl2onnx library, scikit-learn models can be converted to ONNX, making it the recommended choice for moving models out of Python into non-Python production environments.
• C. Incorrect. PMML is also a model interchange format intended for portability, but it is less commonly used in the scikit-learn ecosystem compared to ONNX. While third-party libraries like jpmml-sklearn exist, ONNX has broader community support and tooling for modern deployments.
• D. Incorrect. JSON is a general-purpose data interchange format, not designed for serializing trained scikit-learn models. It cannot reliably preserve complex estimator state and numerical objects, making it unsuitable for model deployment.

Correct answer: B — skops prevents arbitrary code execution when loading

• A. Incorrect. While skops may offer compression, this is not its primary security advantage. Compression is a performance feature, not a security one. skops focuses on safer model persistence rather than storage optimization.
• B. Correct. skops is designed to prevent arbitrary code execution, a major security risk associated with pickle, by using a safer, inspectable serialization format. This makes loading models safer because the content can be validated before being reconstructed.
• C. Incorrect. skops does not automatically encrypt serialized data. Encryption is a separate security measure and is not the primary advantage provided by skops.
• D. Incorrect. skops does not inherently add digital signatures to the serialized output. While signatures could be added in a broader security workflow, the core benefit of skops is safer deserialization, not signing.

Correct answer: A — Use higher compression to reduce file size when saving.

• A. Correct. Higher compression (e.g., compress=3 in joblib.dump) reduces the serialized file size, though it may increase serialization and deserialization time. This is a direct and effective strategy for reducing storage footprint.
• B. Incorrect. Large NumPy arrays are typically part of the model's internal state and cannot be excluded without compromising the model's functionality. Serialization stores the entire object; removing required arrays would break the model.
• C. Incorrect. Memory mapping (via mmap_mode in joblib.load) helps reduce memory usage during loading by allowing on-demand access from disk, but it does not reduce the on-disk size of the serialized file.
• D. Incorrect. Loading the model in a separate thread addresses concurrency and latency during loading, but it does not affect the serialized file size or memory footprint of the stored model.