Live briefing
Everything about Rectified linear unit
Type of activation function
In the context of artificial neural networks, the rectifier or ReLU activation function is an activation function defined as the non-negative part of its argument, i.e., the ramp function: Wikipedia →
🧭 Start here: a 6-step reading path
- Read the overview: Rectified linear unitA 5-minute orientation
- Skim a recent paper: Using the softplus function to construct alternative link functions in generalized linear models and beyondInfluential work since 2022
- Read the landmark paper: Prediction of dissolved oxygen content in aquaculture using Clustering-based Softplus Extreme Learning MachineCited 74 times
- Go sideways: SoftplusA closely related topic
- Go sideways: Swish functionA closely related topic
📖 Definitions
Noun A smooth approximation to the rectifier in neural networks; it is the analytic function f(x)=ln(1+ex).
🗂️ The subject at a glance
🏆 Landmark papers (most cited)
Prediction of dissolved oxygen content in aquaculture using Clustering-based Softplus Extreme Learning Machine
A novel softplus linear unit for deep convolutional neural networks
Brain tumor classification using modified kernel based softplus extreme learning machine
Softplus INGARCH Model
Improving the Antinoise Ability of DNNs via a Bio-Inspired Noise Adaptive Activation Function Rand Softplus
ACO-KELM: Anti Coronavirus Optimized Kernel-based Softplus Extreme Learning Machine for classification of skin cancer
Rethinking semantic-visual alignment in zero-shot object detection via a softplus margin focal loss
Subsampling Generative Adversarial Networks: Density Ratio Estimation in Feature Space With Softplus Loss
🆕 Influential since 2022
Using the softplus function to construct alternative link functions in generalized linear models and beyond
Softplus negative binomial network autoregression
HFBO-KSELM: Hybrid Flash Butterfly Optimization-based Kernel Softplus Extreme Learning Machine for Classification of Chronic Kidney Disease
Local influence analysis in the softplus INGARCH model
Laser-Induced Breakdown Spectroscopy Quantitative Analysis Using a Bayesian Optimization-Based Tunable Softplus Backpropagation Neural Network
📚 Books worth reading
Open Library didn't respond; try again in a moment.
🔗 Related topics
- Softplus In mathematics and machine learning, the softplus function is f(x)=ln(1+ex). It is a smooth approximation (in fact, an analytic function) to the ramp…
- Swish function The swish function is a family of mathematical function defined as follows: swish β ( x ) = x sigmoid ( β x ) = x 1 + e − β x . {\displaystyle \operatorname…
- Activation function In artificial neural networks, the activation function of a node is a function that calculates the output of the node based on its individual inputs and…
- Multilayer perceptron In deep learning, a multilayer perceptron (MLP) is a kind of modern feedforward neural network consisting of fully connected neurons with nonlinear activation…
- Universal approximation theorem In the field of machine learning, the universal approximation theorems (UATs) state that neural networks with a certain structure can, in principle, approximate…
- Weight initialization In deep learning, weight initialization or parameter initialization describes the initial step in creating a neural network. A neural network contains…
- Gating mechanism In neural networks, the gating mechanism is an architectural motif for controlling the flow of activation and gradient signals. They are most prominently…
- Artificial neuron An artificial neuron is a mathematical function conceived as a model of a biological neuron in a neural network. The artificial neuron is the elementary…