Contributor info: Prof. Silvio Franz

Fabián Aguirre-López, Silvio Franz, Mauro Pastore

SciPost Phys. 18, 039 (2025) · published 31 January 2025 | Toggle abstract · pdf

Random features models play a distinguished role in the theory of deep learning, describing the behavior of neural networks close to their infinite-width limit. In this work, we present a thorough analysis of the generalization performance of random features models for generic supervised learning problems with Gaussian data. Our approach, built with tools from the statistical mechanics of disordered systems, maps the random features model to an equivalent polynomial model, and allows us to plot average generalization curves as functions of the two main control parameters of the problem: the number of random features $N$ and the size $P$ of the training set, both assumed to scale as powers in the input dimension $D$. Our results extend the case of proportional scaling between $N$, $P$ and $D$. They are in accordance with rigorous bounds known for certain particular learning tasks and are in quantitative agreement with numerical experiments performed over many order of magnitudes of $N$ and $P$. We find good agreement also far from the asymptotic limits where $D\to ∞$ and at least one between $P/D^K$, $N/D^L$ remains finite.

Silvio Franz, Flavio Nicoletti, Giorgio Parisi, Federico Ricci-Tersenghi

SciPost Phys. 12, 016 (2022) · published 12 January 2022 | Toggle abstract · pdf

We study the energy minima of the fully-connected $m$-components vector spin glass model at zero temperature in an external magnetic field for $m\ge 3$. The model has a zero temperature transition from a paramagnetic phase at high field to a spin glass phase at low field. We study the eigenvalues and eigenvectors of the Hessian in the minima of the Hamiltonian. The spectrum is gapless both in the paramagnetic and in the spin glass phase, with a pseudo-gap behaving as $\lambda^{m-1}$ in the paramagnetic phase and as $\sqrt{\lambda}$ at criticality and in the spin glass phase. Despite the long-range nature of the model, the eigenstates close to the edge of the spectrum display quasi-localization properties. We show that the paramagnetic to spin glass transition corresponds to delocalization of the edge eigenvectors. We solve the model by the cavity method in the thermodynamic limit. We also perform numerical minimization of the Hamiltonian for $N\le 2048$ and compute the spectral properties, that show very strong corrections to the asymptotic scaling approaching the critical point.

Silvio Franz, Antonio Sclocchi, Pierfrancesco Urbani

SciPost Phys. 9, 012 (2020) · published 23 July 2020 | Toggle abstract · pdf

We show that soft spheres interacting with a linear ramp potential when overcompressed beyond the jamming point fall in an amorphous solid phase which is critical, mechanically marginally stable and share many features with the jamming point itself. In the whole phase, the relevant local minima of the potential energy landscape display an isostatic contact network of perfectly touching spheres whose statistics is controlled by an infinite lengthscale. Excitations around such energy minima are non-linear, system spanning, and characterized by a set of non-trivial critical exponents. We perform numerical simulations to measure their values and show that, while they coincide, within numerical precision, with the critical exponents appearing at jamming, the nature of the corresponding excitations is richer. Therefore, linear soft spheres appear as a novel class of finite dimensional systems that self-organize into new, critical, marginally stable, states.

Silvio Franz, Giorgio Parisi, Maksim Sevelev, Pierfrancesco Urbani, Francesco Zamponi

SciPost Phys. 2, 019 (2017) · published 2 June 2017 | Toggle abstract · pdf

Random constraint satisfaction problems (CSP) have been studied extensively using statistical physics techniques. They provide a benchmark to study average case scenarios instead of the worst case one. The interplay between statistical physics of disordered systems and computer science has brought new light into the realm of computational complexity theory, by introducing the notion of clustering of solutions, related to replica symmetry breaking. However, the class of problems in which clustering has been studied often involve discrete degrees of freedom: standard random CSPs are random K-SAT (aka disordered Ising models) or random coloring problems (aka disordered Potts models). In this work we consider instead problems that involve continuous degrees of freedom. The simplest prototype of these problems is the perceptron. Here we discuss in detail the full phase diagram of the model. In the regions of parameter space where the problem is non-convex, leading to multiple disconnected clusters of solutions, the solution is critical at the SAT/UNSAT threshold and lies in the same universality class of the jamming transition of soft spheres. We show how the critical behavior at the satisfiability threshold emerges, and we compute the critical exponents associated to the approach to the transition from both the SAT and UNSAT phase. We conjecture that there is a large universality class of non-convex continuous CSPs whose SAT-UNSAT threshold is described by the same scaling solution.