Everything About ___
Live briefing

Everything about Gradient boosting

Machine learning technique

Gradient boosting is a machine learning technique based on boosting in a functional space, where the target is pseudo-residuals instead of residuals as in traditional boosting. It gives a prediction model in the form of an ensemble of weak prediction models, i.e., models that make very few assumptions about the data, which are typically simple decision trees. When a decision tree is the weak learner, the resulting algorithm is called gradient-boosted trees; it usually outperforms random forest. As with other boosting methods, a gradient-boosted trees model is built in stages, but it generalizes the other methods by allowing optimization of an arbitrary differentiable loss function. Wikipedia →

🧭 Start here: a 6-step reading path

  1. Read the overview: Gradient boostingA 5-minute orientation
  2. Skim a recent paper: Predictive Performances of Ensemble Machine Learning Algorithms in Landslide Susceptibility Mapping Using Random Forest, Extreme Gradient Boosting (XGBoost) and Natural Gradient Boosting (NGBoost)Influential work since 2022
  3. Pick up a classic book: Decision Making HandbookThe most-reprinted book on the subject
  4. Read the landmark paper: Greedy function approximation: A gradient boosting machine.Cited 24,326 times
  5. Go sideways: XGBoostA closely related topic
  6. Go sideways: AdaBoostA closely related topic

📖 Definitions

No dictionary entry for this exact phrase.

🗂️ The subject at a glance

  1. History
  2. Informal introduction
  3. Algorithm
  4. Gradient tree boosting
  5. Regularization
  6. Usage
  7. Names
  8. Feature importance ranking
  9. Disadvantages

🏆 Landmark papers (most cited)

🆕 Influential since 2022

📚 Books worth reading

Ranked by number of editions on Open Library, a rough measure of how lasting a book is.

🔗 Related topics