The Reformulated Law of Infodynamics

An organizing principle for the origin of life and other complex systems

Melvin Vopson and S. Lepadatu introduced the Second Law of Infodynamics (SLID) with the bold claim that it “has massive implications for future developments in genomic research, evolutionary biology, computing, big data, physics, and cosmology” [1]. The proposed law offers a governing principle of how complex systems process information, and it could guide development of a new generation of analytical models. However, Vopson and Lepadatu acknowledged important unresolved questions. This essay proposes a reformulated law of infodynamics that resolves these questions. The essay is based on my most recent paper, which is currently in peer review [2].

The SLID was so named because of its relation to the Second Law of thermodynamics. While the Second Law of thermodynamics says that total entropy irreversibly increases over time, the SLID says that information entropy decreases over time. Vopson and Lepadatu illustrated the principle by the measured decline in information entropy in the SARS Covid virus between January 2020 and October 2021.

It is tempting to interpret a decline in information entropy as a gain in information. As a virus evolves, it certainly does in some sense gain information, as evidenced by its evolving ability to evade antibodies and to manipulate its target genomes for its own reproduction. However, the SLID does not address a gain in information. So what do information entropy and the SLID signify? To address these basic questions, we need a clear understanding of what information entropy is.

Entropy was originally defined by Rudolf Clausius in 1850. He defined the change in thermodynamic entropy by a change in heat divided by its temperature. Thermodynamic entropy describes a system’s thermal randomness. The Second Law of thermodynamics says that change produces entropy. The much more intuitive interpretation is that useful energy is irreversibly dissipated to more randomized energy. Falling water has potential energy, which could do work, but without diversion to work, falling water randomizes its energy by generating heat, sound, and evaporation.

Mechanics subsequently introduced statistical entropy to describe the uncertainty of a system’s precise configuration. A system’s configuration specifies the arrangement of its parts, and it defines the system’s microstate. A Lego structure is a microstate configuration comprising colored Lego blocks. So is a random-looking assemblage of blocks. Statistical entropy describes a state as a probability distribution over the vast number of possible configurations, C₁, C₂,…Cₙ. In contrast to thermodynamics, statistical mechanical microstates are not random; they are just unknown. Statistical entropy describes the microstate probabilities, P₁, P₂,…Pₙ, as the observer’s assigned likelihood for each microstate.

To eliminate the observer’s subjective bias, statistical mechanics defines entropy with zero prior information about a system’s actual configuration. With zero information, all configurations are assigned equal probabilities. With N equal-probability microstate configurations, the entropy is given by S=log(N). Information entropy for a Lego structure simply represents the number (in log units) of structures that can be assembled, with no preference toward any individual configuration. Information entropy eliminates reference to any particular observer. This is the entropy of statistical mechanics, and it is the information entropy of the SLID.

The increase in statistical mechanical entropy is commonly referred to as MaxEnt [3]. The Second Law of thermodynamics and MaxEnt both describe spontaneous increases in entropy. However, mechanics regards thermodynamic and information entropy as properties of an observer’s incomplete information and perception. It recognizes the Second Law of thermodynamics and MaxEnt as thoroughly validated empirical principles, but it cannot formally accommodate either principle, or the SLID, as fundamental physical laws.

To recognize these principles as physical laws, we need to reformulate them within the framework of the thermocontextual interpretation (TCI). I proposed the TCI as a generalization of thermodynamics and mechanics, effectively merging them into a single framework [4−5]. The TCI resolves some of the fundamental questions of physics regarding time, causality, and quantum entanglement.

The TCI’s description of states is based on the laws of physics plus three additional postulates:

Postulate 1. Temperature is a measurable property of state;

Postulate 2. Absolute zero temperature can be approached but never reached; and

Postulate 3. There are no hidden state variables.

The first two postulates are based on the zeroth and third laws of thermodynamics, and they are well established. The TCI defines a system’s state with respect to a positive-temperature reference state in equilibrium with the ambient surroundings. The TCI recognizes exergy and thermodynamic entropy as thermocontextual properties of state. Exergy is equal to a system’s potential capacity for work, and it generalizes thermodynamics’ free energy. The first two postulates expand the framework of mechanics to include entropy and exergy as physical properties of state.

Postulate Three says that a state can be completely defined by perfect observation. A state’s physical microstate configuration is observable, and this means that the state is definite. Postulate Three does not imply that a system always exists as a definite state, however. If a system is not perfectly and completely observable, it does not exist as a state. Rather, it is in transition between states and unobservable. The TCI formally describes the transitions between states based on two additional postulates [2]:

Postulate 4. A system’s accessible energy declines over time, and

Postulate 5. A transition tends to maximize its conversion of energy to accessible energy (or work).

The TCI defines accessible energy as the energy that can be accessed for work by a fixed reference observer [2]. The reference observer defines a fixed reference with respect to which changes can be measured, whether those changes involve the system, the ambient surroundings, or the observer’s information.

Postulate Four addresses the stability of states. It says that a state of lower accessible energy is more stable than a higher-accessibility state. As water flows downhill, it transitions to a more stable state of lower accessibility. The most stable state is the equilibrium state with zero exergy and minimum accessible energy. Postulate Four establishes the thermodynamic arrow of time.

The TCI recognizes two special cases for Postulate Four. The first is the Second Law of thermodynamics, which describes the irreversible production of thermodynamic entropy. This corresponds to dissipation of exergy and declines in both exergy and accessible energy.

The other special case is MaxEnt [3]. MaxEnt describes dispersion, such as the spontaneous mixing of ink and water. Dispersion increases the number of possible configurations, and this increases the information gap between an observer and the system’s actual microstate configuration. With increasing information gap, the observer lacks information needed to fully access the system’s energy for work.

Dissipation and dispersion describe two distinct transitions to a more stable state, and each is a special case of Postulate Four.

Whereas Postulate Four describes the stability of states, Postulate Five addresses the stability of change. Postulate Five states that the most stable transition is the one with the highest conversion of energy input to accessible energy. Postulate Five provides an essential counterbalance to Postulate Four. Dissipation and dispersion destroy accessible energy (Postulate Four), but a transition conserves as much accessible energy as possible (Postulate Five).

A special case of Postulate Five is the maximum efficiency principle (MaxEff). MaxEff describes nature’s empirical tendency to utilize energy as efficiently as possible [6,7]. One way to increase efficiency is to divert useful energy to work. Work can be utilized externally to do work such as recording a measurement result, or it can be utilized internally to create dissipative structures. Dissipative structures include thermal convection and whirlpools, which are sustained by external inputs of heat or fluid. Dissipative structures also include the biosphere, which is sustained by the input of sunlight. MaxEff drives the arrow of functional complexity.

Another special case of Postulate Five is the reformulated law of infodynamics (RLID) [2]. The RLID replaces information entropy of the SLID with the information gap. A low information entropy is a measure of a state description’s precision, but a low information gap is a measure of the description’s accuracy. The RLID states that an observer has a spontaneous potential to reduce its information gap. The increased information enables greater access to the transition’s energy output for work (i.e., accessibility), and from Postulate Five, this increases the transition’s stability.

The information gap between two state descriptions is formally defined by the Kullback–Leibler divergence (Dₖₗ) [8]. The Dₖₗ information gap between a state description and the system’s actual state is given by Dₖₗ=log(1/Pₐ), where Pₐ is the observer’s expectation probability that a system exists in its actual microstate configuration ‘a’ [2]. If Pₐ equals 1 (100% certainty), the Dₖₗ information gap equals zero, reflecting perfect accuracy in the state’s description.

The RLID explains the spontaneous change in the SARS-Covid virus’s information. The virus has the role of reference observer and the virus’s target cells have the role of the virus’s energy source. The change in the virus’s RNA reflects the closing of its information gap with its target’s genome. Narrowing the information gap increases the virus’s access to its target’s energy. In the case of the virus, the narrowing information gap is achieved through random mutations and selection. The RLID provides a selection criterion for the virus to favor mutations that enable it to access its target’s energy and to increase its work of reproduction.

Figure 1 illustrates the assembly of a statistical array of ambient components by the addition of energy from an external source. The assembled array has positive energy, but if an observer-agent has zero information on it, it cannot access the energy, and the array has zero accessibility. The RLID provides an observer-agent the drive to acquire information on the array, allowing it greater access to the array’s energy.

One way to reduce the agent’s information gap is to create a template that can catalyze the creation of a known sequence. Given a template and a procedure to use it, the template can replicate the array with a known sequence and zero information gap. This maximizes the array’s accessible energy.

The RLID provides the drive to create self-replicating arrays of increasing length, energy, and information content. The origin of self-replicating templates is an essential step in the chemical origin of life, and it is a simple consequence of the RLID.

Given its very general nature, the reformulated law of infodynamics will likely have applications as an organizing principle for a wide range of complex systems involving transfers of energy or any medium of value.

1. https://pubs.aip.org/aip/adv/article/12/7/075310/2819368/Second-law-of-information-dynamics
2. Crecraft, H. The second law of infodynamics: a thermocontextual reformulation (in review by journal of physical chemistry au)
3. https://en.wikipedia.org/wiki/Principle_of_maximum_entropy
4. Time and Causality: a Thermocontextual Perspective. https://www.mdpi.com/1099-4300/23/12/1705
5. https://medium.com/science-and-philosophy/a-thermocontextual-perspective-reimagining-physics-part-3-d95313ccd709
6. Dissipation+Utilization=Self−Organization. https://www.mdpi.com/1099-4300/25/2/229
7. https://harrison-69935.medium.com/the-arrow-of-functional-complexity-reimagining-physics-part-7-de359cddfb6a
8. https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence

The Reformulated Law of Infodynamics was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.