Which of the following is an example of a commonly used activation function in deep learning?

Question

BlackTom AI · Accepted Answer

Question restatement: Which of the following is an example of a commonly used activation function in deep learning?

Option 1: Linear function. While a linear activation preserves linearity, stacking linear activations across layers still yields a linear model and cannot capture nonlinear patterns, limiting the network's expressive power. This makes linear activations unsuitable as the activation function in hidden layers of deep networks.

Option 2: Sigmoid function. The sigmoid squashes inputs to the range (0, 1), and has historically been a standard activation used in neural networks. It introduces nonlinearity, allowing modeling of complex relationships. However, it can suffer from vanishing gradient issues for large magnitude inputs, which is a known drawback in deep architectures.

Option 3: Polynomial function. Although a polynomial form can be nonlinear, using a polynomial activation in deep networks is impractical due to issues like exploding/vanishing gradients, kinks, and increased computational load; it is not a standard activation used in practice for deep learning models.

Reasoning across options: The key criterion for an activation function in deep learning is to introduce nonlinearity while enabling effective gradient-based optimization. Linear activations fail to provide nonlinearity, so they are not suitable for deep stacks. Polynomial activations are not standard due to training and stability concerns, making them uncommon in practice. The sigmoid function, despite certain drawbacks, remains a classic and commonly referenced activation that fulfills nonlinearity and has been widely used across many architectures, particularly in earlier networks and in output layers for probabilistic interpretations.

In summary, the sigmoid function stands as the historically common activation among the options, while the other two choices do not meet the practical criteria for activation in deep learning hidden layers.

Which of the following is an example of a commonly used activation function in deep learning? 单项选择题

类似问题

Which statements are correct regarding "Activation" Function?

Question at position 17 Which activation function is commonly used in the target layer when modeling a binary target?exponentiallogisticidentitytanh

Question at position 16 Which of following activation function can’t be used at output layer of a neural network to classify an image?TanhSigmoidIf(x>5,1,0)ReLUNone of the above

Which of the followings is not a widely used activation function?

Question at position 13 What activation function should we use for the output layer for the MNIST dataset?SoftmaxReLU

Question at position 13 What activation function should we use for the output layer for the MNIST dataset?SoftmaxReLU

Question at position 13 What activation function should we use for the output layer for the MNIST dataset?SoftmaxReLU

Question at position 11 Which of the following computation adds nonlinearity to an ANN?Performing activation functionTaking weighed sum of inputs

更多留学生实用工具

智能学习助手

风格化写作助手

论文查重助手

文献引用助手

课堂转译助手

课堂笔记助手

Quiz搜索助手

学校历年真题

智能刷题助手

智能匹配练习

希望你的学习变得更简单