Everything about Overfitting
Flaw in mathematical modelling
In mathematical modeling, overfitting is the production of an analysis that corresponds too closely or exactly to a particular set of data and may therefore fail to fit to additional data or predict future observations reliably. An overfitted model is a mathematical model that contains more parameters than can be justified by the data. In the special case of a model that consists of a polynomial function, these parameters represent the degree of a polynomial. The essence of overfitting is to unknowingly extract some of the residual variation as if that variation represents the underlying model structure. Wikipedia →
🧭 Start here: a 6-step reading path
- Read the overview: OverfittingA 5-minute orientation
- Skim a recent paper: Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural NetworksInfluential work since 2022
- Pick up a classic book: Algorithms to Live ByThe most-reprinted book on the subject
- Read the landmark paper: The Problem of OverfittingCited 2,321 times
- Go sideways: Supervised learningA closely related topic
- Go sideways: Bias–variance tradeoffA closely related topic
📖 Definitions
Verb present participle and gerund of overfit
Noun The production of a statistical model that, due to an excess of parameters relative to the sample size, fits the sample data extremely well but fails to fit new data.
🗂️ The subject at a glance
🏆 Landmark papers (most cited)
The Problem of Overfitting
An Overview of Overfitting and its Solutions
Making better M axent models of species distributions: complexity, overfitting and evaluation
Prevention of overfitting in cryo-EM structure determination
High-resolution noise substitution to measure overfitting and validate resolution in 3D structure determination by single particle electron cryomicroscopy
Overfitting and undercomputing in machine learning
Model selection and overfitting
Neural network studies. 1. Comparison of overfitting and overtraining
🆕 Influential since 2022
Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural Networks
Understanding overfitting in random forest for probability estimation: a visualization and simulation study
Overfitting in Machine Learning: A Comparative Analysis of Decision Trees and Random Forests
Keeping Deep Learning Models in Check: A History-Based Approach to Mitigate Overfitting
A comparison of machine learning and statistical species distribution models: Quantifying overfitting supports model interpretation
📚 Books worth reading
Algorithms to Live ByBrian Christian, Tom Griffiths · 2016
The Elements of Statistical LearningTrevor Hastie, Robert Tibshirani · 2001
Process analytical technologyKatherine A. Bakeev · 2007
Principles of Data MiningMax Bramer · 2013
Genetic ProgrammingKrzysztof Krawiec · 2013- 📕The method and culture of comparative lawMaurice Adams, Dirk Heirbaut · 2014
Machine learning for hackersDrew Conway · 2012
Data science from scratchJoel Grus · 2015
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
🔗 Related topics
- Supervised learning In machine learning, supervised learning (SL) is a type of machine learning paradigm where an algorithm learns to map input data to a specific output based…
- Bias–variance tradeoff In statistics and machine learning, the bias–variance tradeoff describes the relationship between a model's complexity, the accuracy of its predictions…
- Regularization (mathematics) In mathematics, statistics, finance, and computer science, particularly in machine learning and inverse problems, regularization is a process that converts…
- Learning curve (machine learning) In machine learning (ML), a learning curve (or training curve) is a graphical representation that shows how a model's performance on a training set (and…
- Generalization error For supervised learning applications in machine learning and statistical learning theory, generalization error (also known as the out-of-sample error or…
- Bootstrap aggregating Bootstrap aggregating, also called bagging (from bootstrap aggregating) or bootstrapping, is a machine learning (ML) ensemble meta-algorithm designed to…
- Coefficient of determination In statistics, the coefficient of determination, denoted R2 or r2 and pronounced "R squared", is the proportion of the variation in the dependent variable…
- Symbolic regression Symbolic regression (SR) is a type of regression analysis that searches the space of mathematical expressions to find the model that best fits a given…