Question at position 19 Which of the following functions can be used as an activation function in the output layer of a neural network if we wish to predict the probabilities of n classes (p1, p2,...,pn) such that sum of p over all n equals to 1?ReLuSigmoidTanhSoftmaxSingle choice
A
ReLu
B
Sigmoid
C
Tanh
D
Softmax

Question

Question at position 19 Which of the following functions can be used as an activation function in the output layer of a neural network if we wish to predict the probabilities of n classes (p1, p2,...,pn) such that sum of p over all n equals to 1?ReLuSigmoidTanhSoftmaxSingle choice

A

ReLu

B

Sigmoid

C

Tanh

D

Softmax

BlackTom AI · Accepted Answer

Softmax converts a vector of raw scores (logits) into nonnegative values that sum to 1, so it is the appropriate activation for producing a categorical probability distribution across n classes.

1) Relevant concepts:
- Softmax: for logits z = (z1,...,zn), softmax output pi = exp(zi) / sum_j exp(zj). Each pi ∈ (0,1) and sum_i pi = 1.
- Sigmoid: σ(z) = 1 / (1+e^{-z}) produces outputs in (0,1) but treats each unit independently — it does not enforce that outputs across units sum to 1.
- Tanh: tanh(z) ∈ (-1,1), not suitable for probabilities.
- ReLU: max(0,z) ∈ [0,∞), not normalized or bounded above.

2) Step-by-step reasoning:
- Requirement: produce probabilities p1,...,pn such that each pi ≥ 0 and sum_i pi = 1.
- Apply the softmax formula to logits (z1,...,zn):
  Step A: compute exponentials e^{zi} for each class i.
  Step B: compute normalization constant S = Σ_{j=1..n} e^{zj}.
  Step C: pi = e^{zi} / S. By construction each pi ≥ 0 and Σ_i pi = (Σ_i e^{zi}) / S = 1.

3) Example with intermediate steps:
- Let logits = [2, 1, 0]. Exponentials: [e^2, e^1, e^0] ≈ [7.389, 2.718, 1.000].
- Sum S ≈ 7.389 + 2.718 + 1.000 = 11.107.
- Probabilities: p ≈ [7.389/11.107, 2.718/11.107, 1/11.107] ≈ [0.665, 0.245, 0.090], which sum to 1.

4) Why Softmax is correct:
- It simultaneously normalizes across all classes, producing a valid categorical distribution and is differentiable for gradient-based training (e.g., with cross-entropy loss).

5) Why other options are incorrect:
- ReLU: outputs nonnegative but unbounded and not normalized — cannot guarantee sum = 1.
- Sigmoid: yields values in (0,1) independently per unit; does not enforce mutual exclusivity or normalization across classes (suitable for binary or multi-label tasks, not single-label multiclass).
- Tanh: outputs can be negative and are not probabilities.

6) Conclusion:
- The correct activation is Softmax, which matches the provided answer.

Question at position 19 Which of the following functions can be used as an activation function in the output layer of a neural network if we wish to predict the probabilities of n classes (p1, p2,...,pn) such that sum of p over all n equals to 1?ReLuSigmoidTanhSoftmaxSingle choice

Similar Questions

Softmax outputs independent probabilities for each class, so multiple classes can simultaneously receive high probability (e.g., 0.9 and 0.9) for the same input.

Which of the following is true about the softmax function in the context of multinomial logistic regression?

Which of the following is TRUE about softmax (mark all that apply).

Question textFill in the gap with ONE word from the text. Research can help to reduce the amount of Answer 1 Question 6[input] between fauna and humans.

Decide if the following statement is TRUE, FALSE or NOT GIVEN according to the text. Compared to the past, zoos regularly provide education in schools.

More Practical Tools for Students Powered by AI Study Helper

Homework AI Solver

Stylized AI Paper Writer

Plagiarism Checker Assistant

Citation AI Academic Writing Tool

In-Class Translation Assistant

AI Note Generator

AI Quiz Answers

Past Exam Questions from University Test Bank

Smart Practice Assistant

Adaptive Practice

Making Your Study Simpler