In solving multi-class classification problems with
𝑛
classes using a neural network the output of the network will be output of
𝑛
neurons (logits) which correspond to probabilities of the output to belong to the corresponding classes (one-hot encoding).

To compute the loss (cross-entropy loss) the output of neurons is evaluated by one of nonlinear activation functions tanh, sigmoid, ReLU, softmax before the loss is computed.

Question

In solving  multi-class classification problems with 
𝑛
 classes using a neural network the output of the network will be output of 
𝑛
 neurons (logits) which correspond to probabilities of the output to belong to the corresponding classes (one-hot encoding).

To compute the loss (cross-entropy loss)  the output of  neurons is evaluated by one of nonlinear activation functions tanh, sigmoid, ReLU, softmax before the loss is computed.

BlackTom AI · Accepted Answer

Begin by unpacking the statement and the standard approach to multiclass classification with neural networks.
Option analysis:
Option: The statement claims that for n-class classification the network outputs are n neurons (logits) that correspond to probabilities of belonging to each class (one-hot encoding).
- This is partially misleading: those n outputs are typically called logits, which are unnormalized scores, not probabilities. Probabilities are obtained by applying a softmax (or an equivalent) to the logits, and they sum to 1 across classes. Therefore, calling the raw outputs "probabilities" is incorrect.
Option: It then says the loss (cross-entropy) is computed after applying one of the nonlinear activations tanh, sigmoid, ReLU, or softmax to the outputs.
- In multiclass classification, the conventional pipeline uses softmax activation to convert logits into a probability distribution over classes before computing cross-entropy. ReLU, tanh, and sigmoid are not appropriate for producing a proper probability distribution across all classes in the typical cross-entropy setup. While some frameworks compute cross-entropy with logits directly (combining softmax and cross-entropy for numerical stability), the activation applied before loss is usually softmax (not the other listed activations). If you apply softmax, you obtain class probabilities rather than raw logits. If you apply any of the other activations (tanh, sigmoid, ReLU) before cross-entropy, you would not obtain a proper probability distribution across all classes, and the loss formulation would be incorrect.
- Practically, the sigmoid activation would treat each class independently (binary relevance), which is a different setup (one-vs-rest) and not the standard multi-class cross-entropy with a softmax over all classes. ReLU and tanh do not produce a valid multi-class probability distribution and can lead to incorrect loss values.
Thus, the statement contains inaccuracies:
- It conflates logits with probabilities.
- It suggests applying multiple possible activations before the loss, including ones that do not yield proper class probabilities for cross-entropy in multiclass setup, whereas softmax is the correct activation to use, if any activation is applied at all (and many implementations combine softmax with cross-entropy for numerical stability).
In summary, the claim is false because logits are not probabilities, and the typical and correct approach is to apply softmax (or a softmax-with-cross-entropy combination) rather than tanh, sigmoid, or ReLU before computing the multiclass cross-entropy loss.

类似问题

Which statements are correct regarding "Activation" Function?

Question at position 17 Which activation function is commonly used in the target layer when modeling a binary target?exponentiallogisticidentitytanh

Question at position 16 Which of following activation function can’t be used at output layer of a neural network to classify an image?TanhSigmoidIf(x>5,1,0)ReLUNone of the above

Which of the followings is not a widely used activation function?

Question at position 13 What activation function should we use for the output layer for the MNIST dataset?SoftmaxReLU

Question at position 13 What activation function should we use for the output layer for the MNIST dataset?SoftmaxReLU

Question at position 13 What activation function should we use for the output layer for the MNIST dataset?SoftmaxReLU

Question at position 11 Which of the following computation adds nonlinearity to an ANN?Performing activation functionTaking weighed sum of inputs

更多留学生实用工具

智能学习助手

风格化写作助手

论文查重助手

文献引用助手

课堂转译助手

课堂笔记助手

Quiz搜索助手

学校历年真题

智能刷题助手

智能匹配练习

希望你的学习变得更简单