Gaussian approximation of dynamic cavity equations for linearly-coupled stochastic dynamics

(i) The work expands the dynamical cavity formalism to systems with continuous degrees of freedom, pairwise (random) interactions, defined on sparse graphs. This framework includes a variety of systems of interest and allows for several applications and perspectives, which are mentioned in the work;

(ii) Despite its technical nature, the exposition remains clear and self-contained. The division of material between the main text and appendices makes the paper accessible and readable.

The work provides a generalization of the dynamical cavity formalism to systems with continuous degrees of freedom and pairwise (random) interactions, defined on sparse graphs. Eq. (17) expresses the cavity marginals in terms of the cavity local averages, correlations, and responses, defined self-consistently as averages over the cavity marginals. This equation is derived by performing a second-order expansion in the coupling parameter α in the general Eq. (6). For linear forces and additive thermal noise, this expansion leads to Gaussian marginals, allowing the authors to derive equations for the cavity order parameters (mean, correlation, and response), which generalize the previously derived dynamical TAP equations for fully connected graphs. The paper also discusses a perturbative closure scheme to account for nonlinearity in the force or noise terms (which spoil the Gaussianity). The derivation is supported by a series of illustrative applications discussed in Sec. 3.

The content of this work is novel and potentially interesting for a rather broad audience. The content of the manuscript it technical, but I find the presentation of the material quite clear and self-contained.I recommend the publication of this work in SciPost.

A few minor questions or comments:

(i) In the derivation, Sec. 2.1, I would be more explicit on the precise reason why linear interactions allow for graphical model construction presented in the paper, maybe contrasting this with the more complicated situation that would occur for non-linear interactions ( this is briefly mentioned in the conclusions, bit I feel that this point should be stressed more in the first part of the paper).

(ii) In the abstract and introduction, the authors state that the formalism developed in this work can be applied to systems subject to global constraints, and in Sec. 3, they discuss the example of a spherical constraint. Could the authors comment on whether they could treat the case in which constraints are imposed uniformly on the continuous variables (for example, their positivity, as needed in applications to ecosystems dynamics), which are perhaps not straightforward to enforce using Lagrange multipliers?

(iii) In the conclusion, the authors claim that while extending their equations to include nonlinearities in interactions—such as those found in neural networks and learning models—is challenging and intricate, this is not the case for multi-body nonlinearities, such as those in p-spin models with p≥3. Could they provide some intuition for why this is the case and how general is this statement?

Suggested changes:

1. In the introduction, it is mentioned that “DMFT has been successfully applied to study the dynamics of spherical p-spin models for aging and glassy dynamics in disordered systems”, and some references are mentioned. I feel that the work characterizing the aging solution in spherical p-spin models should be referenced to at this point:

Cugliandolo, L. F., & Kurchan, J. (1993). Analytical solution of the off-equilibrium dynamics of a long-range spin-glass model. Physical Review Letters, 71(1), 173.

2. In the introduction, when referring to the characterization of dynamical phases in neural networks via DMFT, in addition to Ref. [38] the authors may consider the very recent work:

A Montanari, P Urbani, Dynamical Decoupling of Generalization and Overfitting in Large Two-Layer Networks, arXiv preprint arXiv:2502.21269

where transitions between different dynamical learning regimes are characterized via DMFT.

3. Eq. (10), upper index of the integral should be “curly” T

Publish (easily meets expectations and criteria for this Journal; among top 50%)

The approach is new and was tested non-only in cases where the solution is known, but it also demonstrates to outperform previous approaches for the BM in an homogeneous regular graphs

All the model considered are simple, the interaction is always linear, and when applied to disordered models, one of the approximation is not obvious to me (see the report)

Report: Gaussian approximation of dynamic cavity equations for linearly coupled stochastic dynamics

I read the paper carefully and followed the derivations (without looking for typos) up to page (15). Then I found the discussion of the results reasonable and consistent.

The manuscript is very well written and represents an original contribution to the field. The authors studied the dynamics of continuos variables in Random Regulars graphs using the simplest possible kind of intereacting models, linear interactions. They also introduced non-linearities through perturbation theory.

Besides reproducing known results from the literature, Figure 3 shows the relevance of the approach. For the Bouchaud-Mezard model, the computation of the authors clearly outperform previous approaches coinciding with the simulations.

Some points to be clarified:

In section 3.4, the authors assume that the cavity quantities do not depend on the edjes. This is not trivial at all due to the disorder, specially for small K. The authors should clarify on this.

I suggest the authors to introduce a short discussion in the introduction making explicit the difference between their approach and the ones in [44] and [48].

After these minor modifications the paper should be published.

1- In section 3.4, the authors assume that the cavity quantities do not depend on the edjes. This is not trivial at all due to the disorder, specially for small K. The authors should clarify on this.

2- I suggest the authors to introduce a short discussion in the introduction making explicit the difference between their approach and the ones in [44] and [48].

Publish (easily meets expectations and criteria for this Journal; among top 50%)

1) The paper puts forward a new method to study the dynamics of sparse interacting systems of continuous degrees of freedom.

2) The paper presents detailed results for several interesting examples.

1) The paper does not test the method on graphs with heterogeneous structures.

In this work, the authors combine ideas from generating
functional analysis with the local tree-like
structure of sparse interacting systems to put forward
cavity equations for the dynamics of systems
described by coupled stochastic differential equations. By
performing a small coupling expansion, they show that the
cavity equations are solved by a Gaussian ansatz, yielding
a system of closed dynamical equations for macroscopic
parameters. These equations apply only to systems with
linear interactions, while nonlinearities can, in
principle, be incorporated via a perturbative approach.
The method is then illustrated with four different
examples of dynamical systems on regular random graphs.

The dynamical cavity method for sparse interacting
systems with binary variables has been developed
more than ten years ago. The present work proposes
an interesting extension of the dynamical cavity method
for systems modelled by continuous degrees of freedom.
Even though the approach is restricted to linear dynamics, this
work represents and important step towards a more complete
understanding of the dynamical cavity method for
coupled differential equations.

However, I would like to raise one important point. A key
feature of the cavity approach is its versatility, as it
allows to solve problems on graphs with heterogeneous
structures, including random degrees and/or coupling
strengths. In the present work, all examples discussed
by the authors apply to regular random graphs, whose
spectral density is analytically known. To make the
approach more robust and general, I believe it would be
valuable to test the method on a linear dynamical system
with heterogeneous interactions, where
the spectral density of the interaction matrix does not
have an analytic closed form.

When resubmitting their manuscript, the authors should also address
the following points and modify the paper where necessary:

1) I might have misunderstood figure 1, but it seems that
the interactions between variables at different times do not
correspond to the graphical structure shown. I don't see why
the graph in figure 1 includes an interaction
between $\hat{x}_{i}^n$ and $x_{i}^{n-1}$ at the previous time
step, since this term is absent from eq. (5). The authors
also mention on page 5 that the local tree-like structure of
the graph is a consequence of the linear
coupling between variables. I think it'd be important to
elaborate further on this point and better explain why linear
interactions allow one to disentangle the loopy structure.

2) Regarding eq. (17), I have the following issue. If one
expands eq. (17) up to second order in $\alpha$, one should recover
the expansion of eq. (6), but this does not seem
to be the case. The reason is that the $O(\alpha^2)$ contribution
coming from the term with $\mu_{k \setminus i}^n$ in eq. (17) does
not appear to cancel out with the $O(\alpha^2)$ contribution
in eq. (6). Please clarify this point.

3) On page 8 the authors mention that $B_{i \setminus j}(t,t^{\prime})$
is zero because the dynamics is causal. In the generating functional
formalism, the moments of conjugate variables are zero because the
generating functional is normalized. It would therefore be
important to elaborate further on why causality implies that these
quantities vanish in the present context.

4) In the long time limit, eq. (50) has a stable solution if
all eigenvalues of $\boldsymbol{J}-\boldsymbol{\lambda}$ have
negative real parts. Besides that, the spectral density of, for instance, a
sparse and symmetric random matrix $\boldsymbol{J}-\boldsymbol{\lambda}$
typically has unbounded support for $N \rightarrow \infty$, which
makes the linear dynamics described by eq. (50) unstable. It is thus
important to make explicit the conditions on $\boldsymbol{J}-\boldsymbol{\lambda}$
that render the linear dynamics stable.

5) The authors mention on page 14 that the relation between random matrix
theory and the Gaussian solution for the cavity equations is double-sided.
Equations (45) and (48) for the response function are exactly the same
as the cavity equations for the diagonal elements of the resolvent
of sparse symmetric matrices (in this sense, it'd be instructive
to cite a few relevant papers where these equations have been originally
derived, e.g., works of Perez-Castillo, Rogers, Metz, Biroli, etc). On
the other hand, the spectral density of $\boldsymbol{J}-\boldsymbol{\lambda}$
determines the correlation and response of the linear dynamical
system, but this is only valid for graphical models
whose spectral density has a bounded support (see point 4). Is it
not correct to think that this restriction should also manifest
itself in the Gaussian solution of the dynamical cavity equations?

6) In section 3.1, the authors solve the equations for the Laplace
transform of the response and the correlation functions in the TTI
regime, in the case of random regular graphs. Technically
speaking, this problem is equivalent to solving the resolvent equations
for RRG's, so this section does not present new material. It would be
interesting to explore further the consequences of the solution. For
instance, how do the behaviours of the two-point functions compare with
those in the fully-connected case?

7) On page 24, it is not clear why one can assume that the local
quantities are independent of the site index. Although the graph
is regular, the coupling strengths are random variables, and local
quantities should fluctuate from site to site. Please, clarify the status
of the homogeneity assumption in this case.

8) Equations (108-111) are equivalent to those for a ferromagnetic model
with $J_{ij} = J$. The solution of eqs. (108-111) exhibits aging, which shows
that in the sparse regime there is no need for randomness in the coupling strengths
to observe aging effects. I believe this is an interesting conclusion
from eqs. (108-111). It would be also important to clarify the meaning of the
fully-connected limit in this context, because the model is ferromagnetic for
finite $K$, while we recover the Cugliandolo-Kurchan equations for $p=2$ spherical
model with Gaussian couplings in the limit $K \rightarrow \infty$.

Minor comments:

a) After eq. (7), what do the authors mean by the
term "quasi-probability distribution"?

b) In the text after eq. (9), the upper limit of the summation seems wrong.

c) Equation (10) is a functional. I suggest the authors replace the
brackets $(\dots)$ by $[\dots]$, as this is the standard notation for the
argument of a functional.

d) In eq. (54), I think it would be more consistent with the notation
in the paper to use a different type of index to identify the eigenvalues.

e) Typo on page 14: "Hermitization method".

f) Typo on page 15: "proceed as follows".

g) Typo on page 23: "bimodal distribution" instead of "binomial distribution".

h) The sentence "We can therefore safely substitute $z$..." on page 13 is
misleading, since we cannot compute the resolvent directly on the real
line (the resolvent is singular at the eigenvalues). Note also that you
keep a small imaginary part in eq. (54).

Publish (easily meets expectations and criteria for this Journal; among top 50%)