Live briefing
Everything about Learning rate
Tuning parameter (hyperparameter) in optimization
In machine learning and statistics, the learning rate is a tuning parameter in an optimization algorithm that determines the step size at each iteration while moving toward a minimum of a loss function. Since it influences to what extent newly acquired information overrides old information, it metaphorically represents the speed at which a machine learning model "learns". In the adaptive control literature, the learning rate is commonly referred to as gain. Wikipedia →
🧭 Start here: a 6-step reading path
- Read the overview: Learning rateA 5-minute orientation
- Skim a recent paper: CheckM2: a rapid, scalable and accurate tool for assessing microbial genome quality using machine learningInfluential work since 2022
- Pick up a classic book: What Expert Teachers DoThe most-reprinted book on the subject
- Read the landmark paper: Federated Machine LearningCited 5,912 times
- Go sideways: Stochastic gradient descentA closely related topic
- Go sideways: Gradient descentA closely related topic
📖 Definitions
No dictionary entry for this exact phrase.
🗂️ The subject at a glance
🏆 Landmark papers (most cited)
Federated Machine Learning
Multiagent learning using a variable learning rate
Defending against Backdoors in Federated Learning with Robust Learning Rate
Learning improves growth rate in grasshoppers
Efficient Backpropagation Learning Using Optimal Learning Rate and Momentum
Can learning affect exchange-rate behavior?
Deep learning with adaptive learning rate using laplacian score
Machine Learning Quantum Reaction Rate Constants
🆕 Influential since 2022
CheckM2: a rapid, scalable and accurate tool for assessing microbial genome quality using machine learning
Towards Personalized Federated Learning
Edge-IIoTset: A New Comprehensive Realistic Cyber Security Dataset of IoT and IIoT Applications for Centralized and Federated Learning
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
A survey on federated learning: challenges and applications
📚 Books worth reading
What Expert Teachers DoJohn Loughran · 2010- 📕Teacher Educators Rethink Self-Assessment in Higher EducationJudith McVarish, Catherine Milne · 2010
Baseline Assessment Curriculum and Target Setting for Pupils with Profound and Multiple Learning DifficultiesSonia Maskell · 2000
Data collectionPatricia Pulliam Phillips, Cathy Stawarski · 2008
The learning portfolioJohn Zubizarreta · 2004
Learning DisabilitiesJeffrey P. Bakken, Festus E. Obiakor · 2013
Teaching for quality learning at universityJohn B. Biggs · 1999
Learning Assessment TechniquesElizabeth F. Barkley, Claire Howell Major · 2015
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
🔗 Related topics
- Stochastic gradient descent Stochastic gradient descent (often abbreviated SGD) is an iterative method for optimizing an objective function with suitable smoothness properties (e…
- Gradient descent Gradient descent is a method for unconstrained mathematical optimization. It is a first-order iterative algorithm for minimizing a differentiable multivariate…
- Backtracking line search In (unconstrained) mathematical optimization, a backtracking line search is a line search method to determine the amount to move along a given search direction…
- Stochastic variance reduction (Stochastic) variance reduction is an algorithmic approach to minimizing functions that can be decomposed into finite sums. By exploiting the finite sum…
- Hyperparameter optimization In machine learning, hyperparameter optimization or tuning is the problem of choosing a set of optimal hyperparameters for a learning algorithm. A hyperparamete…
- Stochastic gradient Langevin dynamics Stochastic gradient Langevin dynamics (SGLD) is an optimization and sampling technique composed of characteristics from Stochastic gradient descent, a…
- Deep backward stochastic differential equation method Deep backward stochastic differential equation method is a numerical method that combines deep learning with Backward stochastic differential equation…
- Hyperparameter (machine learning) In machine learning, a hyperparameter is a parameter that can be set in order to define any configurable part of a model's learning process. Hyperparameters…