Everything about Self-supervised learning
Machine learning paradigm
Self-supervised learning (SSL) is a paradigm in machine learning where a model is trained on a task using the data itself to generate supervisory signals, rather than relying on externally-provided labels. In the context of neural networks, self-supervised learning aims to leverage inherent structures or relationships within the input data to create meaningful training signals. SSL tasks are designed so that solving them requires capturing essential features or relationships in the data. The input data is typically augmented or transformed in a way that creates pairs of related samples, where one sample serves as the input, and the other is used to formulate the supervisory signal. This augmentation can involve introducing noise, cropping, rotation, or other transformations. Self-supervised learning more closely imitates the way humans learn to classify objects. Wikipedia →
🧭 Start here: a 6-step reading path
- Read the overview: Self-supervised learningA 5-minute orientation
- Skim a recent paper: ProtTrans: Toward Understanding the Language of Life Through Self-Supervised LearningInfluential work since 2022
- Pick up a classic book: Becoming a reflective practitionerThe most-reprinted book on the subject
- Read the landmark paper: Self-supervised Learning: Generative or ContrastiveCited 1,103 times
- Go sideways: Representation learningA closely related topic
- Go sideways: AutoencoderA closely related topic
📖 Definitions
No dictionary entry for this exact phrase.
🗂️ The subject at a glance
🏆 Landmark papers (most cited)
Self-supervised Learning: Generative or Contrastive
Self-supervised learning in medicine and healthcare
Self-Supervised Speech Representation Learning: A Review
Self-supervised Learning: A Succinct Review
Graph Self-Supervised Learning: A Survey
Self-Supervised Learning for Electroencephalography
Self-Supervised Learning for Multimedia Recommendation
Augmentation-Free Self-Supervised Learning on Graphs
🆕 Influential since 2022
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Self-Supervised Multimodal Learning: A Survey
Audio self-supervised learning: A survey
Structure-aware protein self-supervised learning
Holistically-Attracted Wireframe Parsing: From Supervised to Self-Supervised Learning
📚 Books worth reading
Becoming a reflective practitionerChristopher Johns · 2000
Neural Information ProcessingTingwen Huang · 2012
Internship, Practicum, and Field Placement Handbook, TheBrian N. Baird, Brian Baird · 1996
Advances in Computational IntelligenceIgnacio Rojas · 2013
The Elements of Statistical LearningTrevor Hastie, Robert Tibshirani · 2001
Advances in Multimedia ModelingShipeng Li · 2012
Basic personal counsellingDavid Geldard · 1989
The Art of LeadershipZach Kelehear · 2005
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
🔗 Related topics
- Representation learning In machine learning (ML), representation learning or feature learning is a set of techniques that allow a system to automatically discover the representations…
- Autoencoder An autoencoder is a type of artificial neural network used to learn efficient codings of unlabeled data (unsupervised learning). An autoencoder learns…
- Representation collapse Representation collapse is a phenomenon in machine learning and representation learning where a model maps different inputs to the same or very similar…
- Variational autoencoder In machine learning, a variational autoencoder (VAE) is an artificial neural network architecture introduced by Diederik P. Kingma and Max Welling in 2013…
- Unsupervised learning Unsupervised learning is a framework in machine learning where, in contrast to supervised learning, algorithms learn patterns exclusively from unlabeled…
- Vision transformer A vision transformer (ViT) is a transformer designed for computer vision. A ViT decomposes an input image into a series of patches (rather than text into…
- World model (artificial intelligence) A world model in artificial intelligence is a machine learning system that builds an internal representation of an environment. The model predicts how…
- Types of artificial neural networks Types of neural networks (NN) include a family of techniques. The simplest types have static components, including number of units, number of layers, unit…