Question at position 13 What activation function should we use for the output layer for the MNIST dataset?SoftmaxReLU单项选择题
A
Softmax
B
ReLU

Question

Question at position 13 What activation function should we use for the output layer for the MNIST dataset?SoftmaxReLU

BlackTom AI · Accepted Answer

MNIST is a 10-class classification task, so the output layer must produce a probability distribution over the 10 digit classes. Softmax is the appropriate activation for that purpose.

Relevant knowledge points:
- Softmax converts raw logits z = (z1,...,zK) into probabilities p_i = exp(z_i) / sum_j exp(z_j). The outputs are non-negative and sum to 1, suitable for multi-class prediction and for use with categorical cross-entropy loss.
- ReLU(x) = max(0,x) is useful in hidden layers to introduce nonlinearity, but it does not normalize outputs or produce probabilities; it can be zero for many entries and has no upper bound.
- Training typically pairs softmax with categorical cross-entropy for stable gradients and correct probabilistic interpretation.

Step-by-step reasoning and a numerical illustration:
1. Suppose a network produces logits for three classes: z = [2.0, 1.0, 0.1].
2. Compute exponentials: exp(2.0) ≈ 7.389, exp(1.0) ≈ 2.718, exp(0.1) ≈ 1.105.
3. Sum = 7.389 + 2.718 + 1.105 ≈ 11.212.
4. Softmax probabilities: p1 = 7.389/11.212 ≈ 0.659, p2 ≈ 0.242, p3 ≈ 0.099. These sum to 1 and can be interpreted as class probabilities.
5. Prediction = argmax p_i (class with highest probability).

Why softmax is correct:
- MNIST requires assigning one of 10 mutually exclusive classes per input. Softmax provides normalized probabilities for each class and interacts correctly with cross-entropy loss to produce meaningful gradients for training.
- The model’s predicted class is obtained with argmax on the softmax outputs; training minimizes the cross-entropy between the true one-hot labels and the predicted distribution.

Why ReLU is incorrect for the output layer:
- ReLU does not normalize outputs, cannot produce a probability distribution (values are unbounded and non-negative), and is not designed for mutually exclusive multi-class outputs. It would not work properly with categorical cross-entropy and would not give interpretable class probabilities.

Therefore, the correct choice is Softmax, matching the provided answer.

Question at position 13 What activation function should we use for the output layer for the MNIST dataset?SoftmaxReLU单项选择题

类似问题

Which statements are correct regarding "Activation" Function?

Question at position 17 Which activation function is commonly used in the target layer when modeling a binary target?exponentiallogisticidentitytanh

Question at position 16 Which of following activation function can’t be used at output layer of a neural network to classify an image?TanhSigmoidIf(x>5,1,0)ReLUNone of the above

Which of the followings is not a widely used activation function?

Question at position 13 What activation function should we use for the output layer for the MNIST dataset?SoftmaxReLU

Question at position 13 What activation function should we use for the output layer for the MNIST dataset?SoftmaxReLU

Question at position 11 Which of the following computation adds nonlinearity to an ANN?Performing activation functionTaking weighed sum of inputs

Question at position 11 Which of the following computation adds nonlinearity to an ANN?Performing activation functionTaking weighed sum of inputs

更多留学生实用工具

智能学习助手

风格化写作助手

论文查重助手

文献引用助手

课堂转译助手

课堂笔记助手

Quiz搜索助手

学校历年真题

智能刷题助手

智能匹配练习

希望你的学习变得更简单