A supervised machine learning task has x1 and x2 as inputs (features), and y as its output (label). In the training phase, we use an 80-20 train-test split in a 10-fold cross-validation process, with a total of 1,000 labelled data records available. Which one of the following statements is FALSE?

Question

BlackTom AI · Accepted Answer

The question asks which statement is FALSE about a supervised learning setup with x1, x2 as inputs, y as the label, using an 80-20 train-test split within a 10-fold cross-validation, with 1,000 labeled records.
Option: 'In each cross-validation iteration, we employ a different ML model and eventually select the model with the best accuracy among the 10 iterations.'
This claim implies that across the 10 folds, a different model is tried in each iteration and then the best-performing model is chosen based on accuracy. In standard cross-validation, the model type and its hyperparameters are fixed for all folds, and the cross-validated performance is used to estimate how well that fixed model would generalize. The process of selecting among multiple models or hyperparameters would require either comparing several models outside of cross-validation or using nested cross-validation to avoid bias from using the same data both to select and to evaluate models. Therefore, this statement describes a practice that is not inherent to conventional cross-validation and is typically considered incorrect in standard CV setups.

Additional nuance to consider: the phrasing about an 80-20 train-test split within a 10-fold CV is itself inconsistent with the standard CV protocol. In 10-fold cross-validation, the data are partitioned into 10 folds of roughly equal size, and in each iteration the model is trained on 9 folds and tested on the remaining 1 fold (i.e., a 90-10 split for each iteration). An explicit 80-20 split would not align with typical 10-fold CV and would suggest a misalignment between the stated CV scheme and how folds are used.

If we were to evaluate why other potential interpretations of the statement are problematic: (1) If one actually used different models per fold, that would confound the cross-validated estimate with model selection bias unless properly nested or separated. (2) Selecting the best model across folds could lead to overoptimistic performance estimates because the selection leverages the test data from the folds. (3) The intended purpose of 10-fold CV is to average performance across folds for a single model, not to pick the best of several models per fold.

In summary, the core issue is that cross-validation does not inherently entail trying a different model in each of the 10 iterations and then selecting the best one; and the stated 80-20 split per fold further conflicts with the standard 90-10 allocation used in 10-fold CV. This makes the option false in the described setup.

类似问题

What is the purpose of cross validation?

Among the following training and evaluation strategies, which of them divides the dataset into k subsets and, at each iteration, a different subset is reserved for testing and all the others, used for training?

What is the purpose of performing cross-validation?

When training models using machine learning, a very powerful technique for tuning and optimizing the best model is:

Why was model-building strategy #1 used to estimate generalization performance on the income data for the decision tree, logistic regression, and random forest models?

In k-fold cross-validation, what happens in each iteration?

What is one advantage of k-fold cross-validation over a single holdout set?

Which of the following is TRUE about cross validation?

更多留学生实用工具

智能学习助手

风格化写作助手

论文查重助手

文献引用助手

课堂转译助手

课堂笔记助手

Quiz搜索助手

学校历年真题

智能刷题助手

智能匹配练习

希望你的学习变得更简单