Live briefing
Everything about Rectified linear unit
Type of activation function
In the context of artificial neural networks, the rectifier or ReLU activation function is an activation function defined as the non-negative part of its argument, i.e., the ramp function: Wikipedia →
🧭 Start here: a 6-step reading path
- Read the overview: Rectified linear unitA 5-minute orientation
- Skim a recent paper: Domain Adaptation Networks With Parameter-Free Adaptively Rectified Linear Units for Fault Diagnosis Under Variable Operating ConditionsInfluential work since 2022
- Pick up a classic book: Practical Electronics for InventorsThe most-reprinted book on the subject
- Read the landmark paper: Classification of Alzheimer’s Disease Based on Eight-Layer Convolutional Neural Network with Leaky Rectified Linear Unit and Max PoolingCited 302 times
- Go sideways: Activation functionA closely related topic
- Go sideways: Swish functionA closely related topic
📖 Definitions
Noun A unit (artificial neuron) in an artificial neural network that employs a rectifier.
Noun The rectifier function itself.
🗂️ The subject at a glance
🏆 Landmark papers (most cited)
Classification of Alzheimer’s Disease Based on Eight-Layer Convolutional Neural Network with Leaky Rectified Linear Unit and Max Pooling
Abnormal breast identification by nine-layer convolutional neural network with parametric rectified linear unit and rank-based stochastic pooling
Deep neural networks with Elastic Rectified Linear Units for object recognition
Deep Learning with S-Shaped Rectified Linear Activation Units
Fruit category classification via an eight-layer convolutional neural network with parametric rectified linear unit and dropout technique
Properties of the Geometry of Solutions and Capacity of Multilayer Neural Networks with Rectified Linear Unit Activations
Voxelwise detection of cerebral microbleed in CADASIL patients by leaky rectified linear unit and early stopping
Weight initialization based‐rectified linear unit activation function to improve the performance of a convolutional neural network model
🆕 Influential since 2022
Domain Adaptation Networks With Parameter-Free Adaptively Rectified Linear Units for Fault Diagnosis Under Variable Operating Conditions
Self-gated rectified linear unit for performance improvement of deep neural networks
Fractional Rectified Linear Unit Activation Function and Its Variants
DPReLU: Dynamic Parametric Rectified Linear Unit and Its Proper Weight Initialization Method
Improved Variational Mode Decomposition and One-Dimensional CNN Network with Parametric Rectified Linear Unit (PReLU) Approach for Rolling Bearing Fault Diagnosis
📚 Books worth reading
Practical Electronics for InventorsPaul Scherz · 2000
Direction and position finding by wirelessRonald Keen · 1920
Control of electric machine drive systemSeung-Ki Sul · 2011
Practical electronics for inventorsPaul Scherz · 2013
Analog Circuit DesignPeter D. Hiscocks · 2010- 📕ICTE 2015China) International Conference On Transportation Engineering (5th 2015 Dalian Shi · 2015
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
🔗 Related topics
- Activation function In artificial neural networks, the activation function of a node is a function that calculates the output of the node based on its individual inputs and…
- Swish function The swish function is a family of mathematical function defined as follows: swish β ( x ) = x sigmoid ( β x ) = x 1 + e − β x . {\displaystyle \operatorname…
- Softplus In mathematics and machine learning, the softplus function is f(x)=ln(1+ex). It is a smooth approximation (in fact, an analytic function) to the ramp…
- Multilayer perceptron In deep learning, a multilayer perceptron (MLP) is a kind of modern feedforward neural network consisting of fully connected neurons with nonlinear activation…
- Universal approximation theorem In the field of machine learning, the universal approximation theorems (UATs) state that neural networks with a certain structure can, in principle, approximate…
- Weight initialization In deep learning, weight initialization or parameter initialization describes the initial step in creating a neural network. A neural network contains…
- Gating mechanism In neural networks, the gating mechanism is an architectural motif for controlling the flow of activation and gradient signals. They are most prominently…
- Feedforward neural network A feedforward neural network is an artificial neural network in which information flows in a single direction – inputs are multiplied by weights to obtain…