Live briefing
Everything about Softmax function
Smooth approximation of one-hot arg max
The softmax function, also known as softargmax or normalized exponential function, converts a tuple of K real numbers into a probability distribution over K possible outcomes. It is a generalization of the logistic function to multiple dimensions and is used in multinomial logistic regression. The softmax function is often used as the last activation function of a neural network to normalize the output of a network to a probability distribution over predicted output classes. Wikipedia →
🧭 Start here: a 6-step reading path
- Read the overview: Softmax functionA 5-minute orientation
- Skim a recent paper: An Empirical Evaluation of Enhanced Performance Softmax Function in Deep LearningInfluential work since 2022
- Pick up a classic book: Fundamentals of Deep LearningThe most-reprinted book on the subject
- Read the landmark paper: A novel enhanced softmax loss function for brain tumour detection using deep learningCited 97 times
- Go sideways: Gumbel distributionA closely related topic
- Go sideways: Multinomial logistic regressionA closely related topic
📖 Definitions
No dictionary entry for this exact phrase.
🏆 Landmark papers (most cited)
A novel enhanced softmax loss function for brain tumour detection using deep learning
Efficient Precision-Adjustable Architecture for Softmax Function in Deep Learning
A pseudo-softmax function for hardware-based high speed image classification
Determining the optimal temperature parameter for Softmax function in reinforcement learning
Base-2 Softmax Function: Suitability for Training and Efficient Hardware Implementation
Improving Classification Performance of Softmax Loss Function Based on Scalable Batch-Normalization
Incrementally Learning the Hierarchical Softmax Function for Neural Language Models
Hardware Implementation of a Softmax-Like Function for Deep Learning
🆕 Influential since 2022
An Empirical Evaluation of Enhanced Performance Softmax Function in Deep Learning
Aggressive Approximation of the SoftMax Function for Power-Efficient Hardware Implementations
Automatic Modulation Classification for OFDM Signals Based on CNN With $\alpha$-Softmax Loss Function
Deep Classification with Linearity-Enhanced Logits to Softmax Function
High-Precision Method and Architecture for Base-2 Softmax Function in DNN Training
📚 Books worth reading
Fundamentals of Deep LearningNikhil Buduma · 2017
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
🔗 Related topics
- Gumbel distribution In probability theory and statistics, the Gumbel distribution (also known as the type-I generalized extreme value distribution) is used to model the distributio…
- Multinomial logistic regression In statistics, multinomial logistic regression is a classification method that generalizes logistic regression to multiclass problems, i.e. with more than…
- Rectified linear unit In the context of artificial neural networks, the rectifier or ReLU (rectified linear unit) activation function is an activation function defined as the…
- Logistic regression In statistics, a logistic model (or logit model) is a statistical model that models the log-odds of an event as a linear combination of one or more independent…
- Point estimation In statistics, point estimation involves the use of sample data to calculate a single value (known as a point estimate, since it identifies a point rather…
- Information bottleneck method The information bottleneck method is a technique in information theory introduced by Naftali Tishby, Fernando C. Pereira, and William Bialek. It is designed…
- Probit model In statistics, a probit model is a type of regression where the dependent variable can take only two values. The word is a portmanteau, coming from probability…
- Generalized logistic distribution The term generalized logistic distribution is used as the name for several different families of probability distributions. For example, Johnson et al…