Live briefing
Everything about Vision-language model
Type of artificial intelligence system
A vision–language model (VLM) is a type of artificial intelligence system that can jointly interpret and generate information from both images and text, extending the capabilities of large language models (LLMs), which are limited to text. It is an example of multimodal learning. Wikipedia →
🧠Start here: a 6-step reading path
- Read the overview: Vision-language modelA 5-minute orientation
- Skim a recent paper: Learning to Prompt for Vision-Language ModelsInfluential work since 2022
- Pick up a classic book: SmartKomThe most-reprinted book on the subject
- Read the landmark paper: RemoteCLIP: A Vision Language Foundation Model for Remote SensingCited 493 times
- Go sideways: Multimodal learningA closely related topic
- Go sideways: Transformer (deep learning)A closely related topic
đź“– Definitions
Noun A generative AI model capable of interpreting text, images and video.
🗂️ The subject at a glance
🏆 Landmark papers (most cited)
RemoteCLIP: A Vision Language Foundation Model for Remote Sensing
A vision–language foundation model for precision oncology
Vision–language foundation model for echocardiogram interpretation
RSGPT: A remote sensing vision language model and benchmark
RS5M and GeoRSCLIP: A Large-Scale Vision- Language Dataset and a Large Vision-Language Model for Remote Sensing
Vision Transformer and Language Model Based Radiology Report Generation
Vision-language foundation model for 3D medical imaging
Pedestrian Vision Language Model for Intentions Prediction
🆕 Influential since 2022
Learning to Prompt for Vision-Language Models
CLIP-Adapter: Better Vision-Language Models with Feature Adapters
Vision-Language Models for Vision Tasks: A Survey
Continual Learning of Image Classes With Language Guidance From a Vision-Language Model
DriveLLaVA: Human-Level Behavior Decisions via Vision Language Model
đź“š Books worth reading
SmartKomWolfgang Wahlster · 2006
Markov Models For Pattern Recognition From Theory To ApplicationsGernot A. Fink · 2014
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
đź”— Related topics
- Multimodal learning Multimodal learning is a type of deep learning that integrates and processes multiple types of data, referred to as modalities, such as text, audio, images…
- Transformer (deep learning) In deep learning, the transformer is a family of artificial neural network architectures based on the multi-head attention mechanism, in which input data…
- Vision transformer A vision transformer (ViT) is a transformer designed for computer vision. A ViT decomposes an input image into a series of patches (rather than text into…
- BERT (language model) Bidirectional encoder representations from transformers (BERT) is a language model introduced in October 2018 by researchers at Google. It learns to represent…
- Contrastive language–image pre-training Contrastive language–image pre-training (CLIP) is a technique for training a pair of neural network models, one for image understanding and one for text…
- Geospatial foundation model A geospatial foundation model (GFM), also referred to as an Earth observation foundation model, represents a paradigm shift in Earth system modeling by…
- Attention Is All You Need "Attention Is All You Need" is a 2017 research paper on machine learning authored by eight scientists and engineers working at Google. The paper introduced…
- Positional encoding In deep learning, the positional encoding or position embedding is a component of sequence models, especially Transformers. Input data such as text, images…