On the Conservation of Information and the Second Law of Thermodynamics
How can these fundamental laws be consistent?
The conservation of quantum information, quantified in term of von Neumann entropy, is a fundamental consequence of the linearity and unitarity of quantum mechanics. As the terms information and entropy are often used interchangeably in several branches of sciences, this may sound very strange to anyone familiar with the second law of thermodynamics, which says that entropy generally increases with time. How can these claims be consistent?
First, there is nothing quantum with this apparent contradiction. The same question could be asked in classical physics. For a Hamiltonian system, the dynamics are always reversible, so information is conserved by Liouville’s theorem. One could then wonder how entropy can increase for a classical system if entropy is a measure of information.
This article is the second part of my series about entropy and the second law of thermodynamics. You should definitly check out the first part before that one.
Short answer
The entropy (or information) constrained by Liouville’s theorem is not the same as the entropy that the Second Law is talking about. The latter is talking about the amount of hidden information (information that is inaccessible to macroscopic measurements), while the former is the total information content of a system. Or in other words, the irreversibility of thermodynamics is a statistical effect and does not conflict the reversibility of classical /quantum mechanics.
Still, there is much more depth to this question than it looks, so let’s dive in.
What are entropy and information ?
The term entropy is used in different contexts, which often results in confusion. We find entropy in information theory (Shanon entropy), statistical thermodynamics (Boltzman and Gibbs entropy) and quantum mechanics (von Neuman theory) just to name a few. While they are all defined with the same formula (up to a constant), they describe different concepts.
Briefly, Boltzmann defined entropy as the measure of the number of possible microscopic arrangements (microstates) of a system that comply with the macroscopic condition of the system (macrostates). Von Neuman later extended the concept of entropy from classical statistical mechanics to quantum information theory, and Claude Shannon developed similar statistical concepts to the problem of random losses of information in telecommunication signals.
Information and entropy are typically used interchangeably in physics and information sciences. In fact, entropy can be regarded as a measure of information that is lacking (hidden or missing information) to state the microstate of a system.
Or in other word, thermodynamic entropy reflects the observer’s ignorance about a system’s microstates compatible given his macroscopic measurements.
Fine-grained and coarse-grained entropy
A fine-grained description of a system is a detailed description of its microscopic behavior, while on the other hand, a coarse-grained description is one in which some of his fine detail has been smoothed over. Practically, this smoothing increases the observer’s ignorance about the exact microstate of the system. For example, thermodynamic quantities such as temperature and pressure are the result of coarse graining as they average many particles properties into a few macro variables.
As quantum systems are generally described by all their particles, the Von-Neuman entropy is typically fine-grained, thus constant under the quantum Liouville theorem. On the other hand, from an observer’s perspective, the amount of coarse grained information of a system may change with time. Coarse grained entropy can thus be seen as the “observational entropy” or “thermodynamic entropy” because it measures the information hidden from the observer given the macrostates he measures.
This coarse graining may seem quite abstract. To get a better idea about what it means, let’s take a simple example of particles expanding in a room, initialised in the bottom left corner. As an observer, we can separate our room into different “boxes”, and then count the number of particle in each box. Assuming that the highest entropy state would correspond to the case where all boxes contain an equal number of particle, we can compute the entropy as:
S = sum(Ni * log(Ni)) where Ni is the number of particles in each box.
Interrestingly, the computed entropy depends on the grid resolution used to define the boxes. As we increase the resolution, our knowledge of the system gradually increases, up to the point where we have perfect knowledge of all the microstates when the grid size is very small (128x128). In this situation, as the hidden information is always zero, the entropy stays constant.
From this example, it becomes clear that thermodynamic entropy is observer dependent, and that its value depends on the the information available to him. Here, the apparent increase in entropy could just be a consequence of the observer’s lack of ability to measure the particles with a grid size above 128x128.
Still, there could be some fundamental processes preventing any observer, even with the most sophisticated possible device, to ever get the fine-grained system’s information. This existence of a limit to an observer’s accessible information would define an objective fundamental coarse graining, and the second law would become a fundamental law rather than an apparent phenomenon. For example, it’s pratically impossible in quantum mechanics to measure a state without destroying it (No cloning theorem), which makes quantum entanglement a good candidate for this fundamental coarse-graining.
Second law of thermodynamics and quantum entanglement
Why does the Coarse-grained Entropy increase?
Following on the coarse graining formalism described above. One can conclude that an increase in coarse grained entropy reflects an observer’s loss of knowledge about a system. From the second law, it follows that our coarse-grained representation of the universe is becoming less and less precise about its microstates, a phenomenon known as “blurring” [2]. Strikingly, the exact nature behind this blurring mechanism remains unclear, and is still being actively debated in the scientific community.
An attractive explanation was proposed by Lloyd in 1988 [3]. He realized that quantum uncertainty, and the way it spreads as particles become increasingly entangled, could be the true source of the blurring mechanism. In his view, information becomes increasingly diffuse, but it never disappears completely. States in equilibrium are maximally entangled and the approach to equilibrium can be thought of in terms of the spread of entanglement. So, although entropy increases locally, the overall entropy of the universe stays constant at zero. Assuming the universe as a whole is a closed system, it is in a pure state. But individual pieces of it, because they are entangled with the rest of the universe, are in mixtures [4,5]. In practice, this mean that the growth of the entropies of the parts is canceled by the growing negative entanglement entropy. That entanglement entropy (Sen) has no measurable consequences and the entropy described by the second law (Sobs) is the one we can monitor:
Stot = Sobs + Sen
While technically, entropy is never negative, here Sen is the conditional entropy of the hidden states (H) given the observable states (O) , which is can take negative values for quantum systems [6].
S(OH) = S(O) + S(H|O)
This may be confusing at first, so let’s take a simple illustrative example. Quantum entanglements are useful to perform physical experiments and quantum calculations as long as they are controlled in closed systems. For statistical reasons, quantum systems in contact with an environment inevitably undergo decoherence after some time. This loss of purity (or ‘quantumness’) is mediated by entanglement between the system and its environment, which is uncontrolled and unmeasurable. As a result, part of the information about the initial system is practically lost to the observer forever, thus increasing its observable entropy and making the process practically irreversible. Still, the system state will continue to be constrained by the entanglement with its environment, so the fine grained entropy of the (system + environment) did not change after decoherence.
Wavefunction collapse and Black holes
Before concluding, I would like to discuss a few open problems still actively debated in fundamental physics that may contradict what was discussed above. We so far assumed that the principle of conservation of information always hold in our universe. However, there is a few cases where information may not be conserved, and thus even the fine-grained entropy Stot of our universe could increase irreversibly.
In quantum mechanics, wave function collapse is notorious for being a non-unitary transformation, thus destroying information. This apparent violation is at the heart of the different interpretations of quantum mechanics. For example, the GRW theory proposes to add a non unitary term to the Schrodinger equation such as the wavefunction of macroscopic objects collapse spontaneously while microscopic superpositions are left unaltered with high probability. On the other hand, other theories like the Many world interpretation postulate that the wavefunction actually never collapse, but rather every possible outcome of a quantum event exists in its own branch of the universe that can never interact with others due to being in incoherent states, thus technically not destroying any information.
Likewise, black holes may destroy information in an irreversible manner through Hawking radiations. This process, where information would permanently disappear in a black hole, is known as the black hole information paradox. Nevertheless, Quantum string theory suggest that information is conserved through the holographic principle or with a quantum description of black holes, known as fuzzball. Another theory suggests that black holes may undergo a phase transition at around their half life where they would start to spill out information.
Concluding remarks:
Entropy might be one of the most mysterious and difficult concept to grasp in physics. It is still actively debated in the scientific community, especially regarding its definition outside of thermal equilibrium [7]. Strikingly, almost all systems found in nature are not in thermodynamic equilibrium, as they are continuously subject to flux of matter and energy to and from other systems.
In this article, I introduced the concept of coarse-grained entropy and gave a possible interpretation of the second law of thermodynamics with quantum entanglement. However, not everyone agrees with these concepts. For example, from Jayne’s point of view [8], since coarse grained entropy always depends on the macro variables chosen by the observers, the increase of entropy is a subjective measurement from an observer with limited knowledge rather than a fundamental quantity. Others state that the coarse graining is a fundamental consequence of the Heisenberg uncertainty principle [9], where the fine-grained entropy can intrinsically never be calculated for macroscopic system. There is still a lot to read and discuss about the matter, And after all that research, I can only agree with what my statistical physics teacher was saying :)
There are only 4 or 5 persons in this world that truly understand entropy, and I don’t belong to them.
About me
I am a PhD candidate in AI & Healthcare currently working at IBMResearch and I hold a master in quantum physics. In my spare time, I am a blockchain (Web3) developer and the CTO of Peer2Panel, a blockchain startup focused in renewable energy.
I enjoy writing in-depth articles about diverse topics such the Universe, Blockchain and AI. Unfortunatly, writing such articles is quite demanding and I whish I could take the time to write more. If you are thinking about subscribing to Medium and enjoy reading these type of articles, please consider using my referral link ! That would support me directly with a portion of your subscription. If you do so, thank you a million times!
https://medium.com/membership/@aurelien-pelissier
References
[1] https://physics.stackexchange.com/questions/390445/why-does-the-coarse-grained-entropy-increase
[2] Alonso-Serrano, Ana, and Matt Visser. “Coarse graining Shannon and von Neumann entropies.” Entropy 19.5 (2017): 207. (https://www.mdpi.com/1099-4300/19/5/207/htm)
[3] Lloyd, Seth. Black Holes, Demons and the Loss of Coherence: How complex systems get information, and what they do with it. Diss. Rockefeller University, 1988. (https://web.archive.org/web/20120607195224/http://meche.mit.edu/documents/slloyd_thesis.pdf)
[4] https://www.quantamagazine.org/quantum-entanglement-drives-the-arrow-of-time-scientists-say-20140416
[5] https://van.physics.illinois.edu/qa/listing.php?id=24045
[6] Cerf, Nicolas J., and Chris Adami. “Negative entropy and information in quantum mechanics.” Physical Review Letters 79.26 (1997): 5194.(https://arxiv.org/pdf/quant-ph/9512022.pdf)
[7] Šafránek, Dominik, Anthony Aguirre, and J. M. Deutsch. “Classical dynamical coarse-grained entropy and comparison with the quantum version.” Physical Review E 102.3 (2020): 032106. (https://arxiv.org/pdf/1905.03841.pdf)
[8] Jaynes, Edwin T. “Gibbs vs Boltzmann entropies.” American Journal of Physics 33.5 (1965): 391–398. (https://bayes.wustl.edu/etj/articles/gibbs.vs.boltzmann.pdf)
[9] Toscano, Fabricio, et al. “Uncertainty relations for coarse–grained measurements: An overview.” Entropy 20.6 (2018): 454. (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7512973/)
