Everything about Minimum message length
Formal information theory restatement of Occam's Razor
Minimum message length (MML) is a Bayesian information-theoretic method for statistical model comparison and selection. It provides a formal information theory restatement of Occam's Razor: even when models are equal in their measure of fit-accuracy to the observed data, the one generating the most concise explanation of data is more likely to be correct. MML was invented by Chris Wallace, first appearing in the seminal paper "An information measure for classification". MML is intended not just as a theoretical construct, but as a technique that may be deployed in practice. It differs from the related concept of Kolmogorov complexity in that it does not require use of a Turing-complete language to model data. Wikipedia →
🧭 Start here: a 6-step reading path
- Read the overview: Minimum message lengthA 5-minute orientation
- Skim a recent paper: Multivariate bounded support asymmetric generalized Gaussian mixture model with model selection using minimum message lengthInfluential work since 2022
- Pick up a classic book: Statistical and Inductive Inference by Minimum Message Length (Information Science and Statistics)The most-reprinted book on the subject
- Read the landmark paper: Minimum Message Length and Kolmogorov ComplexityCited 243 times
- Go sideways: Minimum description lengthA closely related topic
- Go sideways: Universality probabilityA closely related topic
📖 Definitions
No dictionary entry for this exact phrase.
🗂️ The subject at a glance
🏆 Landmark papers (most cited)
Minimum Message Length and Kolmogorov Complexity
Online clustering via finite mixtures of Dirichlet and minimum message length
Minimum message length inference of secondary structure from protein coordinate data
Single-Factor Analysis by Minimum Message Length Estimation
Minimum message length estimation of mixtures of multivariate Gaussian and von Mises-Fisher distributions
Learning a mixture model for clustering with the completed likelihood minimum message length criterion
The Complexity of Strict Minimum Message Length Inference
Minimum message length encoding and the comparison of macromolecules
🆕 Influential since 2022
Multivariate bounded support asymmetric generalized Gaussian mixture model with model selection using minimum message length
Getting ‘ϕψχal’ with proteins: minimum message length inference of joint distributions of backbone and sidechain dihedral angles
Gaussian mixture model with local consistency: a hierarchical minimum message length-based approach
Model Selection from Multiple Model Families in Species Distribution Modeling Using Minimum Message Length
📚 Books worth reading
Statistical and Inductive Inference by Minimum Message Length (Information Science and Statistics)C.S. Wallace · 2005- 📕A data mining investigation into pavement roughness using minimum message length inferenceMatthew Byrne · 2007
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
🔗 Related topics
- Minimum description length Minimum description length (MDL) is a model selection principle where the shortest description of the data is judged to be the best model. MDL methods…
- Universality probability Universality probability is an abstruse probability measure in computational complexity theory that concerns universal Turing machines. A Turing machine…
- Solomonoff's theory of inductive inference Solomonoff's theory of inductive inference in philosophy is a method of evaluating scientific models according to their description length. According to…
- Chris Wallace (computer scientist) For other uses, see Chris Wallace (disambiguation). Christopher Stewart Wallace (26 October 1933 – 7 August 2004) was an Australian computer scientist…
- Inductive probability Inductive probability attempts to give the probability of future events based on past events. It is the basis for inductive reasoning, and gives the mathematica…
- Occam's razor In philosophy, Occam's razor (also spelled Ockham's razor or Ocham's razor; Latin: novacula Occami) is the problem-solving principle that recommends searching…
- Kolmogorov complexity In algorithmic information theory (a subfield of computer science and mathematics), the Kolmogorov complexity of an object, such as a piece of text, is…
- Bayes factor The Bayes factor (BF) is a ratio of two competing statistical models represented by their evidence, and is used to quantify the support for one model over…