Live briefing
Everything about Weight initialization
Technique for setting initial values of trainable parameters in a neural network
In deep learning, weight initialization or parameter initialization describes the initial step in creating a neural network. A neural network contains trainable parameters that are modified during training: weight initialization is the pre-training step of assigning initial values to these parameters. Wikipedia →
🧭 Start here: a 6-step reading path
- Read the overview: Weight initializationA 5-minute orientation
- Skim a recent paper: Physics-informed interpretable wavelet weight initialization and balanced dynamic adaptive threshold for intelligent fault diagnosis of rolling bearingsInfluential work since 2022
- Pick up a classic book: The practical guideThe most-reprinted book on the subject
- Read the landmark paper: A review on weight initialization strategies for neural networksCited 312 times
- Go sideways: Vanishing gradient problemA closely related topic
- Go sideways: Normalization (machine learning)A closely related topic
📖 Definitions
No dictionary entry for this exact phrase.
🗂️ The subject at a glance
🏆 Landmark papers (most cited)
A review on weight initialization strategies for neural networks
A weight initialization method for improving training speed in feedforward neural network
Statistically controlled activation weight initialization (SCAWI)
Mutual information based weight initialization method for sigmoidal feedforward neural networks
A weight initialization method based on neural network with asymmetric activation function
Solving the linear interval tolerance problem for weight initialization of neural networks
Suitable CNN Weight Initialization and Activation Function for Javanese Vowels Classification
Interval based Weight Initialization Method for Sigmoidal Feedforward Artificial Neural Networks
🆕 Influential since 2022
Physics-informed interpretable wavelet weight initialization and balanced dynamic adaptive threshold for intelligent fault diagnosis of rolling bearings
An analysis of weight initialization methods in connection with different activation functions for feedforward neural networks
Enhancing Disease Detection With Weight Initialization and Residual Connections Using LeafNet for Groundnut Leaf Diseases
Improved weight initialization for deep and narrow feedforward neural network
DPReLU: Dynamic Parametric Rectified Linear Unit and Its Proper Weight Initialization Method
📚 Books worth reading
- 📕The practical guideLung, and Blood Institute National Heart · 2000
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
🔗 Related topics
- Vanishing gradient problem In machine learning, the vanishing gradient problem is the problem of greatly diverging gradient magnitudes between earlier and later layers encountered…
- Normalization (machine learning) In machine learning, normalization is a statistical technique with various applications. There are two main forms of normalization, namely data normalization…
- Activation function In artificial neural networks, the activation function of a node is a function that calculates the output of the node based on its individual inputs and…
- Rectified linear unit In the context of artificial neural networks, the rectifier or ReLU (rectified linear unit) activation function is an activation function defined as the…
- Multilayer perceptron In deep learning, a multilayer perceptron (MLP) is a kind of modern feedforward neural network consisting of fully connected neurons with nonlinear activation…
- Residual neural network A residual neural network (also referred to as a residual network or ResNet) is a deep learning architecture in which the layers learn residual functions…
- Batch normalization In artificial neural networks, batch normalization (also known as batch norm) is a normalization technique used to make training faster and more stable…
- Outline of deep learning The following outline is provided as an overview of, and topical guide to, deep learning: Deep learning is a subfield of machine learning and artificial…