Dense Hopfield networks in the teacher-student setting
Robin Thériault, Daniele Tantari
SciPost Phys. 17, 040 (2024) · published 8 August 2024
- doi: 10.21468/SciPostPhys.17.2.040
- Submissions/Reports
-
Abstract
Dense Hopfield networks with $p$-body interactions are known for their feature to prototype transition and adversarial robustness. However, theoretical studies have been mostly concerned with their storage capacity. We derive the phase diagram of pattern retrieval in the teacher-student setting of $p$-body networks, finding ferromagnetic phases reminiscent of the prototype and feature learning regimes. On the Nishimori line, we find the critical amount of data necessary for pattern retrieval, and we show that the corresponding ferromagnetic transition coincides with the paramagnetic to spin-glass transition of $p$-body networks with random memories. Outside of the Nishimori line, we find that the student can tolerate extensive noise when it has a larger $p$ than the teacher. We derive a formula for the adversarial robustness of such a student at zero temperature, corroborating the positive correlation between number of parameters and robustness in large neural networks. Our model also clarifies why the prototype phase of $p$-body networks is adversarially robust.
Supplementary Information
External links to supplemental resources; opens in a new tab.