Question at position 18 How does the parameter k in a k-NN model typically affect bias and variance? Decreasing k increases bias and decreases variance.Changing k has no effect on bias, only on variance.Increasing k increases bias and increases variance.Increasing k increases bias and decreases variance.

Question

BlackTom AI · Accepted Answer

Increasing k smooths the decision boundary: it raises bias and reduces variance.

1. Relevant knowledge points and concepts
- k-NN (k-nearest neighbors) predicts based on the k closest training examples.  
- Bias measures systematic error (how far model predictions are from true underlying function on average).  
- Variance measures sensitivity of model predictions to variations in the training data (how much predictions change if we retrain on a different dataset).  
- The bias–variance tradeoff: more flexible models tend to have low bias but high variance; simpler (smoother) models tend to have higher bias but lower variance.

2. Step-by-step reasoning
- Consider extreme cases: k = 1 and k = N (all training points).  
  - k = 1: the model perfectly fits individual training points (very flexible), so training error is low → low bias. But prediction for a new point depends heavily on one neighbor; a small change in training data can change the neighbor and prediction dramatically → high variance.  
  - k = N: prediction is the global majority (classification) or average (regression) of all points → very smooth, likely underfitting → high bias. Because prediction aggregates many points, removing or adding a single point has little effect → low variance.
- As k increases from small to large, predictions are averaged over more neighbors, smoothing local fluctuations (reducing variance) but also blunting the model’s ability to capture local structure (increasing bias).

3. Intermediate steps / illustrative example
- If k changes from 1 to 25 in a dataset of 100 points, a local outlier that would have decided a 1-NN classification no longer determines the class; majority of 25 neighbors dominate → less sensitivity to noise (lower variance) but potentially wrong for fine local patterns (higher bias).

4. Why the correct answer is correct
- Increasing k forces more averaging across neighbors: this reduces sensitivity to training-sample noise (variance down) while making the model less able to follow complex patterns (bias up).

5. Why other options are incorrect
- “Decreasing k increases bias and decreases variance.” — reversed: decreasing k actually decreases bias and increases variance.  
- “Changing k has no effect on bias, only on variance.” — incorrect because k affects both bias and variance.  
- “Increasing k increases bias and increases variance.” — incorrect because variance decreases as k grows.

6. Conclusion
- Therefore the correct statement is: Increasing k increases bias and decreases variance. This matches the provided answer.

Similar Questions

Question at position 37 Regularization increases bias and reduces variance.TrueFalse

The model showed ______ variance, suggesting instability in the predictions.

As the number of training examples goes to infinity, your model trained on that data will have:

Which statement is correct about bias, variance, and generalization error evaluated on the same problem?

Which of the following are true about model variance?

Select the following trends of errors that occur as a neural network grows in complexity

Question at position 13 Multiple Choice Question: A fitting curve plots Generalization performance vs. model complexityTrue positive rate vs. false negative rateTrue positive rate vs. false positive rateGeneralization performance vs. size of training set

More Practical Tools for Students Powered by AI Study Helper

Homework AI Solver

Stylized AI Paper Writer

Plagiarism Checker Assistant

Citation AI Academic Writing Tool

In-Class Translation Assistant

AI Note Generator

AI Quiz Answers

Past Exam Questions from University Test Bank

Smart Practice Assistant

Adaptive Practice

Making Your Study Simpler