Which of the following is a common mistake in applying cross-validation?

Question

BlackTom AI · Accepted Answer

To approach this question, I’ll evaluate each option in terms of how cross-validation is typically applied in machine learning practice.
Option 1: 'Using validation set to select features and test performance' This is a common pitfall. In proper cross-validation, the feature selection process should be nested inside the cross-validation folds. If you use the validation set both to select features and to assess performance, you leak information from the validation data into the model, leading to overly optimistic performance estimates. This double-dipping is exactly what practitioners warn against.
Option 2: 'Using K-fold cross validation' This describes a standard, widely recommended cross-validation approach. K-fold cross-validation partitions the data into k subsets, trains on k-1 folds, and tests on the remaining fold, repeating to cover all folds. It is a valid and common use of cross-validation when applied correctly.
Option 3: 'Splitting data randomly' Random train/test splits are a basic evaluation method and can be a valid precursor to cross-validation when done carefully, but on its own it is not a cross-validation technique. The question concerns common mistakes in applying cross-validation, so simply splitting data randomly is not describing a cross-validation misuse.
Option 4: 'Applying cross-validation to the entire modeling process' If cross-validation is used to assess every step of modeling (including the final feature engineering, model selection, and parameter tuning) without proper nesting, it can lead to information leakage. However, when done with proper nesting (i.e., inner loops for tuning/feature selection and an outer loop for performance estimation), this is a correct and recommended use. The statement, taken alone, can imply misapplication if interpreted as applying CV to all steps without separation. Still, the core issue to identify in common mistakes is specifically using a validation set for both feature selection and performance estimation, which is a classic improper practice.
In summary, the options describe a spectrum of practices, with the most problematic and commonly cited error being using the validation set to both select features and test performance, because it contaminates the evaluation and inflates performance estimates. The other items outline standard, or conditional, uses of cross-validation that are acceptable when implemented correctly.

Which of the following is a common mistake in applying cross-validation?单项选择题

类似问题

What is the purpose of cross validation?

Among the following training and evaluation strategies, which of them divides the dataset into k subsets and, at each iteration, a different subset is reserved for testing and all the others, used for training?

What is the purpose of performing cross-validation?

When training models using machine learning, a very powerful technique for tuning and optimizing the best model is:

Why was model-building strategy #1 used to estimate generalization performance on the income data for the decision tree, logistic regression, and random forest models?

In k-fold cross-validation, what happens in each iteration?

What is one advantage of k-fold cross-validation over a single holdout set?

Which of the following is TRUE about cross validation?

更多留学生实用工具

智能学习助手

风格化写作助手

论文查重助手

文献引用助手

课堂转译助手

课堂笔记助手

Quiz搜索助手

学校历年真题

智能刷题助手

智能匹配练习

希望你的学习变得更简单