Question at position 18 How does the parameter k in a k-NN model typically affect bias and variance? Decreasing k increases bias and decreases variance.Changing k has no effect on bias, only on variance.Increasing k increases bias and increases variance.Increasing k increases bias and decreases variance.

Question

BlackTom AI · Accepted Answer

Increasing k smooths the decision boundary: it raises bias and reduces variance.

1. Relevant knowledge points and concepts
- k-NN (k-nearest neighbors) predicts based on the k closest training examples.  
- Bias measures systematic error (how far model predictions are from true underlying function on average).  
- Variance measures sensitivity of model predictions to variations in the training data (how much predictions change if we retrain on a different dataset).  
- The bias–variance tradeoff: more flexible models tend to have low bias but high variance; simpler (smoother) models tend to have higher bias but lower variance.

2. Step-by-step reasoning
- Consider extreme cases: k = 1 and k = N (all training points).  
  - k = 1: the model perfectly fits individual training points (very flexible), so training error is low → low bias. But prediction for a new point depends heavily on one neighbor; a small change in training data can change the neighbor and prediction dramatically → high variance.  
  - k = N: prediction is the global majority (classification) or average (regression) of all points → very smooth, likely underfitting → high bias. Because prediction aggregates many points, removing or adding a single point has little effect → low variance.
- As k increases from small to large, predictions are averaged over more neighbors, smoothing local fluctuations (reducing variance) but also blunting the model’s ability to capture local structure (increasing bias).

3. Intermediate steps / illustrative example
- If k changes from 1 to 25 in a dataset of 100 points, a local outlier that would have decided a 1-NN classification no longer determines the class; majority of 25 neighbors dominate → less sensitivity to noise (lower variance) but potentially wrong for fine local patterns (higher bias).

4. Why the correct answer is correct
- Increasing k forces more averaging across neighbors: this reduces sensitivity to training-sample noise (variance down) while making the model less able to follow complex patterns (bias up).

5. Why other options are incorrect
- “Decreasing k increases bias and decreases variance.” — reversed: decreasing k actually decreases bias and increases variance.  
- “Changing k has no effect on bias, only on variance.” — incorrect because k affects both bias and variance.  
- “Increasing k increases bias and increases variance.” — incorrect because variance decreases as k grows.

6. Conclusion
- Therefore the correct statement is: Increasing k increases bias and decreases variance. This matches the provided answer.

类似问题

Question at position 37 Regularization increases bias and reduces variance.TrueFalse

The model showed ______ variance, suggesting instability in the predictions.

As the number of training examples goes to infinity, your model trained on that data will have:

Which statement is correct about bias, variance, and generalization error evaluated on the same problem?

Which of the following are true about model variance?

Select the following trends of errors that occur as a neural network grows in complexity

Question at position 13 Multiple Choice Question: A fitting curve plots Generalization performance vs. model complexityTrue positive rate vs. false negative rateTrue positive rate vs. false positive rateGeneralization performance vs. size of training set

更多留学生实用工具

智能学习助手

风格化写作助手

论文查重助手

文献引用助手

课堂转译助手

课堂笔记助手

Quiz搜索助手

学校历年真题

智能刷题助手

智能匹配练习

希望你的学习变得更简单