Natural Attractive Interactions and the Contraction of Information Radius

Abstract

Mutual information MI(X_t; Y_t) describes how knowledge of one subsystem reduces the uncertainty of another. We study the time evolution of the information distance
r(t) = H(X_t) + H(Y_t) - 2·MI(X_t; Y_t),
defined in terms of the marginal entropies and the mutual information of a pair of interacting variables (X_t, Y_t). The joint distribution evolves according to a continuity equation with velocity fields v_x and v_y. Under standard smoothness and no-flux assumptions, we derive variational identities for the time derivatives of the marginal entropies and the mutual information, and combine them into a master identity for the rate of change of r(t) expressed through the velocity fields and the pointwise mutual information (PMI),
φ = log p - log p_X - log p_Y.

For canonical Gibbs models with density proportional to
exp{−β[U(x) + U(y) + W(x, y)]},
we show that the gradients of φ coincide with centered interaction forces and are orthogonal—under the L2(p) inner product—to all marginal-only drift fields. This leads to a natural decomposition of the velocity field into a PMI-gradient component and a marginal drift component, written as
v = γ·∇φ + u.
When the dynamics is driven purely by PMI gradients and the marginals are preserved, we obtain an exact contraction theorem: the information distance r(t) decreases at a rate proportional to a PMI-based “Fisher energy.” When marginal drifts are present, we derive a quantitative inequality showing that r(t) still decreases whenever the PMI-gradient energy dominates the entropy production induced by the marginal drifts. Gaussian and coupled Ornstein–Uhlenbeck examples illustrate these mechanisms and provide explicit parameter ranges under which contraction holds.