Live briefing
Everything about Backpropagation
Optimization algorithm for artificial neural networks
In machine learning, backpropagation is a gradient computation method commonly used for training a neural network in computing parameter updates. Wikipedia →
🧭 Start here: a 6-step reading path
- Read the overview: BackpropagationA 5-minute orientation
- Skim a recent paper: Deep physical neural networks trained with backpropagationInfluential work since 2022
- Pick up a classic book: Deep LearningThe most-reprinted book on the subject
- Read the landmark paper: Backpropagation Applied to Handwritten Zip Code RecognitionCited 9,948 times
- Go sideways: Multilayer perceptronA closely related topic
- Go sideways: Feedforward neural networkA closely related topic
📖 Definitions
Noun An error correction technique used in neural networks.
Noun A phenomenon in which the action potential of a neuron creates a voltage spike both at the end of the axon, as normally, and also back through to the dendrites from which much of the original input current originated.
🏆 Landmark papers (most cited)
Backpropagation Applied to Handwritten Zip Code Recognition
Backpropagation through time: what it does and how to do it
A Practical Bayesian Framework for Backpropagation Networks
30 years of adaptive neural networks: perceptron, Madaline, and backpropagation
Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks
Compensation of Dispersion and Nonlinear Impairments Using Digital Backpropagation
Backpropagation and stochastic gradient descent method
Landslide susceptibility assessment and factor effect analysis: backpropagation artificial neural networks and their comparison with frequency ratio and bivariate logistic regression modelling
🆕 Influential since 2022
Deep physical neural networks trained with backpropagation
Experimentally realized in situ backpropagation for deep learning in photonic neural networks
Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks
Backpropagation-free training of deep physical neural networks
Backpropagation-Based Learning Techniques for Deep Spiking Neural Networks: A Survey
📚 Books worth reading
Deep LearningJohn D. Kelleher · 2019
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
🔗 Related topics
- Multilayer perceptron In deep learning, a multilayer perceptron (MLP) is a kind of modern feedforward neural network consisting of fully connected neurons with nonlinear activation…
- Feedforward neural network A feedforward neural network is an artificial neural network in which information flows in a single direction – inputs are multiplied by weights to obtain…
- Mathematics of neural networks in machine learning An artificial neural network (ANN) or neural network combines biological principles with advanced statistics to solve problems in domains such as pattern…
- ADALINE ADALINE (Adaptive Linear Neuron or later Adaptive Linear Element) is an early single-layer artificial neural network and the name of the physical device…
- Delta rule In machine learning, the delta rule is a gradient descent learning rule for updating the weights of the inputs to artificial neurons in a single-layer…
- Types of artificial neural networks Types of neural networks (NN) include a family of techniques. The simplest types have static components, including number of units, number of layers, unit…
- Vanishing gradient problem In machine learning, the vanishing gradient problem is the problem of greatly diverging gradient magnitudes between earlier and later layers encountered…
- Artificial neuron An artificial neuron is a mathematical function conceived as a model of a biological neuron in a neural network. The artificial neuron is the elementary…