Live briefing
Everything about Conditional entropy
Measure of relative information in probability theory
In information theory, the conditional entropy quantifies the amount of information needed to describe the outcome of a random variable given that the value of another random variable is known. Here, information is measured in shannons, nats, or hartleys. The "entropy of conditioned on " is denoted as. Wikipedia →
🧭 Start here: a 6-step reading path
- Read the overview: Conditional entropyA 5-minute orientation
- Skim a recent paper: Device-independent lower bounds on the conditional von Neumann entropyInfluential work since 2022
- Pick up a classic book: Entropía y procesos productivosThe most-reprinted book on the subject
- Read the landmark paper: Maximum entropy autoregressive conditional heteroskedasticity modelCited 170 times
- Go sideways: Joint entropyA closely related topic
- Go sideways: Mutual informationA closely related topic
📖 Definitions
Noun The portion of a random variable's own Shannon entropy which is independent from another, given, random variable.
🗂️ The subject at a glance
🏆 Landmark papers (most cited)
Maximum entropy autoregressive conditional heteroskedasticity model
Morphological Organization: The Low Conditional Entropy Conjecture
Maximum entropy and conditional probability
On the Conditional Rényi Entropy
Topological conditional entropy
The Conditional Entropy Bottleneck
Conditional entropy of ordinal patterns
Conditional Self-Entropy and Conditional Joint Transfer Entropy in Heart Period Variability during Graded Postural Challenge
🆕 Influential since 2022
Device-independent lower bounds on the conditional von Neumann entropy
Informational Steady States and Conditional Entropy Production in Continuously Monitored Systems
A DDoS Attack Detection Method Using Conditional Entropy Based on SDN Traffic
A two-way accelerator for feature selection using a monotonic fuzzy conditional entropy
A-unital Operations and Quantum Conditional Entropy
📚 Books worth reading
- 📕Entropía y procesos productivosMariá Caridad Araujo · 1998
The second law of economicsReiner Kümmel · 2011- 📕Investigation of computational and spectral analysis methods for aeroacoustic wave propagationFlorence O. Vanel · 1995
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
🔗 Related topics
- Joint entropy In information theory, joint entropy is a measure of the uncertainty associated with a set of variables. The joint Shannon entropy (in bits) of two discrete…
- Mutual information In probability theory and information theory, the mutual information (MI) of two random variables is a measure of the mutual dependence between the two…
- Differential entropy Differential entropy (also referred to as continuous entropy) in information theory is a property of absolutely continuous probability distributions which…
- Conditional mutual information In probability theory, particularly information theory, the conditional mutual information is, in its most basic form, the expected value of the mutual…
- Entropy (information theory) In information theory, the entropy of a random variable quantifies the average level of uncertainty or information associated with the variable's potential…
- Rényi entropy In information theory, the Rényi entropy is a quantity that generalizes various notions of entropy, including Hartley entropy, Shannon entropy, collision…
- Kullback–Leibler divergence In mathematical statistics, the Kullback–Leibler (KL) divergence (also called relative entropy and I-divergence), denoted DKL(P∥Q), is a type of statistical…
- Quantities of information The mathematical theory of information is based on probability theory and statistics, and measures information with several quantities of information,…