Joint and Conditional Entropy

You already have the von Neumann entropy of a single density operator: diagonalise, run eigenvalues through the Shannon formula, read off a number in bits. This chapter does the same job for two systems side-by-side — system A and system B, with a joint state \rho_{AB} that could be correlated, entangled, or neither. The questions are classical-looking: how much total uncertainty is there in the pair? How much uncertainty about A remains once you know B? How many bits do A and B share in common?

The answers are mostly the expected quantum generalisations of Shannon's definitions — with one sharp exception. The conditional entropy S(A | B) can be negative, and the sign flip is not a mistake. It is the formal signature that \rho_{AB} is entangled in a way no classical joint distribution could match. By the end of the chapter that negative number will have a name (coherent information), a sign-corrected interpretation (a quantum resource you can send through a channel), and a role as the central quantity of the quantum source-coding and channel-coding theorems you will meet in the next two chapters.

Joint entropy — the picture before the formula

Two systems, A and B. A joint state \rho_{AB} living on the tensor-product Hilbert space \mathcal{H}_A \otimes \mathcal{H}_B. You want one number measuring the total uncertainty of the pair.

The recipe is the same as for a single system: diagonalise \rho_{AB}, collect the eigenvalues \{\mu_k\} (a probability distribution on d_A \cdot d_B outcomes), and apply Shannon.

Reading the definition. The joint entropy forgets, momentarily, that \rho_{AB} lives on a tensor product. It treats \rho_{AB} as one big density operator on a d_A d_B-dimensional Hilbert space and asks the von Neumann question of it. The tensor structure only re-enters the moment you trace out a subsystem to get \rho_A or \rho_B — and compare the joint number to the marginals.

Two quick consequences from the definition alone:

• Pure joint states have S(A, B) = 0. If \rho_{AB} = |\psi\rangle_{AB}\langle\psi|_{AB}, the joint density is rank-1, eigenvalues are (1, 0, 0, \ldots), entropy is 0. The joint system is perfectly known.
• Product states split additively. If \rho_{AB} = \rho_A \otimes \rho_B, the eigenvalues of the product are the pairwise products of the marginals' eigenvalues: \mu_{ij} = \lambda_i^A \lambda_j^B. Then

Why the product splits: \log_2(\lambda_i^A \lambda_j^B) = \log_2\lambda_i^A + \log_2\lambda_j^B, and the double sum factorises because \sum_j \lambda_j^B = 1. This mirrors the classical fact that independent random variables have H(X, Y) = H(X) + H(Y).

For anything other than a product state, the joint entropy is strictly less than S(A) + S(B) — the gap measures correlation.

Marginals and partial trace — a two-line reminder

To talk about S(A) alone you need \rho_A, the reduced state on A. Built by partial trace:

where \{|b\rangle_B\} is any orthonormal basis of B. The operation is "sum over the B-diagonal." The density operator chapter derived it from first principles; here it is a tool. \rho_B is defined symmetrically by tracing out A.

Once you have \rho_A and \rho_B, their entropies S(A) = S(\rho_A) and S(B) = S(\rho_B) are just single-system von Neumann entropies.

Subadditivity — the easy master inequality

For any bipartite state \rho_{AB},

with equality iff \rho_{AB} = \rho_A \otimes \rho_B (the subsystems are uncorrelated). This is subadditivity, the direct quantum analogue of the classical H(X, Y) \leq H(X) + H(Y).

A short proof using relative entropy

The cleanest route is via quantum relative entropy. Define the relative entropy

Klein's inequality (1931), proved by the concavity of \log. Now set \rho = \rho_{AB} and \sigma = \rho_A \otimes \rho_B:

Since \log_2(\rho_A \otimes \rho_B) = (\log_2 \rho_A) \otimes I_B + I_A \otimes (\log_2 \rho_B), the second trace splits:

Why the trace splits this way: \text{tr}(\rho_{AB} (X_A \otimes I_B)) = \text{tr}_A(\rho_A X_A) after tracing out B using the definition of the partial trace. The \log_2 \rho_A piece only sees the A-marginal; the \log_2 \rho_B piece only sees the B-marginal. Entanglement contributes nothing to these trace expressions because they are linear in \rho_{AB}.

Combining,

Relative entropy is non-negative — so S(A) + S(B) - S(A, B) \geq 0, which is subadditivity. Equality in Klein's inequality happens iff \rho_{AB} = \rho_A \otimes \rho_B, giving the equality condition.

The quantity

is the quantum mutual information, the subject of the next chapter. It is the "gap" in subadditivity, and it measures the total amount of correlation — classical plus quantum — between A and B.

Conditional entropy — and the negative-entropy surprise

Classically, the chain rule H(X, Y) = H(X) + H(Y | X) defines conditional entropy via subtraction:

It measures the uncertainty of Y remaining after X is known. It is always non-negative, and in fact 0 \leq H(Y | X) \leq H(Y).

The quantum definition copies the formula:

Why the quantum version can go below zero

The classical proof that H(Y | X) \geq 0 runs through H(X, Y) \geq H(X), which itself rests on the fact that a joint probability distribution on (X, Y) is at least as uncertain as any marginal — knowing the pair (x, y) is at least as informative as knowing x.

Quantumly, that chain breaks. A pure entangled joint state |\psi\rangle_{AB} has S(A, B) = 0, yet tracing out one side can produce a mixed marginal with S(B) > 0. The "joint" is more determined than the "parts." No classical distribution can be like this — in classical probability, marginalising always loses information, never creates it.

Why pure-state marginals can be mixed: any pure bipartite state |\psi\rangle_{AB} has a Schmidt decomposition |\psi\rangle = \sum_i \sqrt{\lambda_i}\,|u_i\rangle_A |v_i\rangle_B. Tracing out B leaves \rho_A = \sum_i \lambda_i |u_i\rangle\langle u_i|, which has entropy -\sum_i \lambda_i \log_2 \lambda_i = S(B). Unless one \lambda_i = 1 (the state is a product state), the marginal is mixed and has strictly positive entropy. Entanglement creates marginal entropy even when the joint state is pure.

Bell state, worked

The cleanest worked example is the Bell state |\Phi^+\rangle = (|00\rangle + |11\rangle)/\sqrt 2.

• Joint entropy. The joint state is pure, so S(A, B) = 0.
• Marginal \rho_B. Tracing out A,

So S(B) = 1 bit.

• Conditional entropy. S(A | B) = S(A, B) - S(B) = 0 - 1 = -1 bit.

Negative one bit of conditional uncertainty. Classically impossible. Quantumly routine.

The pure-state formula S(A | B) = -S(A)

For any pure bipartite state |\psi\rangle_{AB}, the Schmidt decomposition gives S(A) = S(B) (same eigenvalues) and S(A, B) = 0. Therefore

Pure-state conditional entropy is exactly the negative of the marginal entropy. The more entangled the pure state (larger S(A)), the more negative the conditional entropy. Unentangled pure states have S(A) = 0, giving S(A | B) = 0, matching the classical intuition. Maximally entangled pure states saturate at S(A | B) = -\log_2 d, one negative bit per Schmidt level.

The coherent information — reading the minus sign correctly

"Negative uncertainty" sounds like nonsense. The right way to read it is to flip the sign and give the positive quantity a name.

Operational meaning. The coherent information quantifies how much quantum information about A is effectively stored in B — information that is not accessible from B alone (which would be classical mutual information) but that is jointly recoverable if A and B are combined. For the Bell state, I_c = +1 bit, which matches the fact that a shared Bell pair is worth exactly one "ebit" — one unit of quantum entanglement, enough for one qubit of teleportation [quantum-teleportation].

The coherent information will reappear in the next two chapters as:

• The Schumacher compression rate for pure quantum sources.
• The quantum channel capacity Q = \lim_{n \to \infty} \tfrac{1}{n} \max I_c(A \rangle B^n) over Kraus-extended channels (the Lloyd-Shor-Devetak theorem).

So the negative sign of S(A | B) is not pathology — it is, after sign-flipping, the number that quantifies the "qubit-pipe width" of any quantum channel.

Strong subadditivity — the master inequality

Subadditivity bounds S(A, B) in terms of the marginals. Strong subadditivity adds a third system and is drastically harder to prove — and drastically more powerful.

Reading the equivalence. Subtract S(B, C) from both sides of the first form: S(A, B, C) - S(B, C) \leq S(A, B) - S(B), i.e. S(A | B, C) \leq S(A | B). The left side is the conditional entropy of A given (B, C); the right side given only B. Adding information (C) can only reduce uncertainty about A — as it should.

Why it is hard

The classical version is one line:

follows immediately because conditioning is averaging over more refined events, and averaging reduces variance-style quantities.

The quantum version has no such short proof. In 1971, Oscar Lanford and Derek Robinson conjectured it. In 1973, Lieb and Ruskai [arXiv:math-ph/0205013] proved it using the Golden-Thompson inequality \text{tr}(e^{A + B}) \leq \text{tr}(e^A e^B) for Hermitian matrices, plus intricate convexity estimates. The proof stood alone for two decades as the deepest known result in quantum information theory. Modern proofs (Ruskai-Effros, Nielsen-Petz) have simplified it somewhat, but it remains a non-trivial theorem.

What it buys you

Nearly every structural theorem in quantum information theory descends from strong subadditivity:

• Monotonicity of relative entropy under CPTP maps: S(\Phi(\rho) \| \Phi(\sigma)) \leq S(\rho \| \sigma).
• Data processing inequality for quantum mutual information: I(A; B) \geq I(A; \Phi(B)) for any CPTP map \Phi on B.
• Holevo bound on classical information extractable from a quantum source [holevo-bound].
• Quantum channel capacity theorems (classical, quantum, entanglement-assisted).
• QKD security proofs (BB84, E91) — all ultimately rely on strong subadditivity.

If you remember one inequality from this chapter, make it strong subadditivity. Every other result sits downstream.

Worked examples

Common confusions

• "Negative conditional entropy means uncertainty below zero." No — "negative uncertainty" is not a physical quantity. The right reading is that S(A | B) is the arithmetic difference S(A, B) - S(B), and for quantum joint states that difference can go negative. The operational meaning lives in the sign-flipped coherent information I_c(A \rangle B) = -S(A | B), which measures a positive quantum resource.

• "S(A | B) = entropy of \rho_{A | B} for some conditional state \rho_{A | B}." Wrong, there is no single "conditional density operator" in quantum theory. Classical conditional entropy can be written as H(Y | X) = \sum_x p(x) H(Y | X = x), an average of entropies of conditional distributions. The quantum version has no analogous decomposition — the formula S(A | B) = S(A, B) - S(B) is the primary definition, not a consequence.

• "Strong subadditivity is just subadditivity repeated." It is not. Subadditivity applied twice gives S(A, B, C) \leq S(A) + S(B) + S(C), which is weaker than strong subadditivity. SSA involves overlapping subsystems (B appearing on both sides) and is a genuinely deeper statement requiring operator-convexity arguments.

• "Coherent information is always positive." Only for pure states is I_c \geq 0 automatic. For mixed states I_c can be positive, zero, or negative. The maximum over input states of I_c across uses of a channel is the channel's quantum capacity — so I_c being non-negative on some input is what makes a channel useful for quantum transmission at all.

• "Subadditivity says joint entropy decreases under correlation." Subadditivity is an upper bound: S(A, B) \leq S(A) + S(B). It says joint is at most the sum; it does not say joint decreases as correlation is added. In fact for a product state the equality holds; as you add classical correlation the joint drops (so the gap I(A:B) increases); as you add entanglement, the joint can drop all the way to zero while the marginals stay fixed.

• "S(A | B) depends only on \rho_A and \rho_B." No — it depends on the joint state \rho_{AB}. Different joint states with the same marginals can have very different conditional entropies. The Bell state and the classically correlated state \tfrac{1}{2}(|00\rangle\langle 00| + |11\rangle\langle 11|) both have maximally mixed marginals but conditional entropies of -1 and 0 respectively.

Going deeper

The Araki-Lieb triangle inequality

Alongside subadditivity, a lower bound on joint entropy:

The left inequality is Araki-Lieb (1970). It ensures that even for entangled states, S(A, B) cannot be more negative-relative-to-marginals than the difference of the marginals themselves. For a pure state |\psi\rangle_{AB} with S(A) = S(B) (by Schmidt symmetry), the Araki-Lieb bound is 0 \leq S(A, B) \leq 2 S(A), saturated on the left by pure entangled states and on the right by maximally mixed product states. The combination of subadditivity and Araki-Lieb is called the entropic triangle inequality.

Strong subadditivity equivalent forms

The inequality S(A, B, C) + S(B) \leq S(A, B) + S(B, C) has several equivalent statements, each useful in different contexts:

1. Conditional form: S(A | B, C) \leq S(A | B) — conditioning on more cannot increase uncertainty.
2. Mutual information form: I(A; C | B) \geq 0 — conditional mutual information is non-negative.
3. Data processing inequality (DPI): I(A; B) \geq I(A; \Phi(B)) for any CPTP map \Phi on B — information about A cannot increase under local processing of B.
4. Monotonicity of relative entropy: S(\Phi(\rho) \| \Phi(\sigma)) \leq S(\rho \| \sigma) for CPTP \Phi.

All four statements are equivalent to SSA and to each other; Lindblad (1975) proved the equivalences. The monotonicity-of-relative-entropy form is the one that generalises most smoothly to infinite-dimensional systems, quantum field theory, and operator-algebraic settings.

Continuity: the Fannes-Audenaert inequality

How much can S(\rho) change when \rho is perturbed? The Fannes-Audenaert inequality bounds the change by the trace-distance:

where T = \tfrac{1}{2}\|\rho - \sigma\|_1 is the trace distance, d is the dimension, and h is the binary entropy. Plugging in for conditional entropy gives the Alicki-Fannes inequality

ensuring that conditional entropy is a continuous function of the state. Useful when \rho_{AB} is known only approximately (as in any experiment).

Coherent information and quantum channel capacity

The Lloyd-Shor-Devetak theorem (first announced by Lloyd in 1997, proved rigorously by Shor 2002 and Devetak 2005 [arXiv:quant-ph/0304127]) identifies the quantum capacity of a channel \mathcal{N} as

This is the regularised coherent information. The regularisation (limit over many channel uses) is necessary because I_c is not additive: there exist channels \mathcal{N}_1, \mathcal{N}_2 where I_c of the tensor product strictly exceeds the sum — a phenomenon called superadditivity of coherent information. Smith and Yard (2008) showed the extreme version: two channels each with Q = 0 can combine to give Q > 0. Classical channels have no analogue.

The Petz recovery map and approximate SSA

Petz (1986) showed that equality in SSA — S(A | B, C) = S(A | B) — holds iff there is a "quantum Markov chain" structure: a CPTP map \mathcal{R}: B \to BC satisfying \mathcal{R}(\rho_{AB}) = \rho_{ABC}. This \mathcal{R} is the Petz recovery map. Fawzi-Renner (2015) [arXiv:1410.0664] proved an approximate version: if I(A; C | B) is small, the Petz map approximately recovers \rho_{ABC} from \rho_{AB}, with explicit error bounds in trace distance. This is the modern tool powering many recent results in quantum many-body physics and holography.

Indian research connections

Quantum information groups at HRI (Harish-Chandra Research Institute, Allahabad) and IIT Madras have produced significant work on entropic inequalities and multipartite coherent information. Ujjwal Sen's group at HRI has developed conditional-entropy measures for multipartite entanglement quantification. At IISc Bangalore, the quantum gravity community uses SSA routinely to derive constraints on holographic entanglement entropies (Ryu-Takayanagi surfaces must satisfy SSA, a non-trivial check on any proposed bulk geometry). The Raman Research Institute in Bengaluru has instrumented quantum-optics experiments reporting two-photon conditional entropies directly, with values in good agreement with theoretical predictions to within 10^{-3} bits.

Where this leads next

• Quantum mutual information — the non-negative gap in subadditivity, I(A;B) = S(A) + S(B) - S(A,B), as the total-correlation measure.
• Coherent information — the sign-flipped conditional entropy as the operational measure of quantum communication capacity.
• Strong subadditivity — deeper treatment of the Lieb-Ruskai theorem and its many equivalent forms.
• Quantum channel capacities — how coherent information defines the quantum capacity via the Lloyd-Shor-Devetak theorem.
• Holevo bound — classical information extractable from a quantum source, a consequence of SSA.
• Von Neumann entropy — the single-system prerequisite that all of this builds on.

References

1. Elliott H. Lieb and Mary Beth Ruskai, Proof of the strong subadditivity of quantum-mechanical entropy (1973) — arXiv:math-ph/0205013 (reprint).
2. Mark M. Wilde, Quantum Information Theory (2nd ed., 2017), Ch. 11–14 (Entropy inequalities and quantum capacities) — arXiv:1106.1445.
3. John Preskill, Lecture Notes on Quantum Computation, Ch. 10 (Quantum information theory) — theory.caltech.edu/~preskill/ph229.
4. Igor Devetak, The private classical capacity and quantum capacity of a quantum channel (2005) — arXiv:quant-ph/0304127.
5. Omar Fawzi and Renato Renner, Quantum conditional mutual information and approximate Markov chains (2015) — arXiv:1410.0664.
6. Wikipedia, Strong subadditivity of quantum entropy — statements, equivalent forms, and proof sketch.