SciPost Submission Page
Tensorization of neural networks for improved privacy and interpretability
by José Ramón Pareja Monturiol, Alejandro Pozas-Kerstjens, David Pérez-García
This is not the latest submitted version.
Submission summary
| Authors (as registered SciPost users): | José Ramón Pareja Monturiol |
| Submission information | |
|---|---|
| Preprint Link: | scipost_202503_00007v2 (pdf) |
| Code repository: | https://github.com/joserapa98/tensorization-nns |
| Date submitted: | Aug. 7, 2025, 8:03 p.m. |
| Submitted by: | José Ramón Pareja Monturiol |
| Submitted to: | SciPost Physics |
| Ontological classification | |
|---|---|
| Academic field: | Physics |
| Specialties: |
|
| Approaches: | Theoretical, Experimental, Computational |
Abstract
We present a tensorization algorithm for constructing tensor train/matrix product state (MPS) representations of functions, drawing on sketching and cross interpolation ideas. The method only requires black-box access to the target function and a small set of sample points defining the domain of interest. Thus, it is particularly well-suited for machine learning models, where the domain of interest is naturally defined by the training dataset. We show that this approach can be used to enhance the privacy and interpretability of neural network models. Specifically, we apply our decomposition to (i) obfuscate neural networks whose parameters encode patterns tied to the training data distribution, and (ii) estimate topological phases of matter that are easily accessible from the MPS representation. Additionally, we show that this tensorization can serve as an efficient initialization method for optimizing MPS in general settings, and that, for model compression, our algorithm achieves a superior trade-off between memory and time complexity compared to conventional tensorization methods of neural networks.
Author indications on fulfilling journal expectations
- Provide a novel and synergetic link between different research areas.
- Open a new pathway in an existing or a new research direction, with clear potential for multi-pronged follow-up work
- Detail a groundbreaking theoretical/experimental/computational discovery
- Present a breakthrough on a previously-identified and long-standing research stumbling block
Author comments upon resubmission
We thank the Editor and the Referees for their time and effort dedicated to reviewing our manuscript. We acknowledge that assessing interdisciplinary works is oftentimes challenging, and we recognize the efforts made to provide a fair refereeing process. We have revised the text to address all the comments and suggestions raised during the review process.
In response to the detailed feedback from the referees, we have submitted a revised version of our manuscript that addresses all the concerns raised. We have made a particular effort to clarify the distinctions between our method and previous works, especially TT-RS and TT-CI, which was a concern shared by both referees. To this end, we have included clarifying remarks and citations throughout Section 2 and introduced two new subsections (3.1.3 and 3.2.3), along with Tables 3.1 and 3.2, which present direct performance comparisons between TT-RSS and TT-CI on representative examples. Additionally, all experiments in Sections 3.1, 3.2, and 4.2 have been recomputed using double precision, and as a result, we have updated Figures 3.1, 3.2, and 4.4. While the results remain largely consistent with those originally reported, we hope that the updated figures, together with the new tables, help clarify the potential of our approach.
We hope that these changes address the reviewers’ concerns and that our manuscript is now suitable for publication in SciPost Physics.
In response to the detailed feedback from the referees, we have submitted a revised version of our manuscript that addresses all the concerns raised. We have made a particular effort to clarify the distinctions between our method and previous works, especially TT-RS and TT-CI, which was a concern shared by both referees. To this end, we have included clarifying remarks and citations throughout Section 2 and introduced two new subsections (3.1.3 and 3.2.3), along with Tables 3.1 and 3.2, which present direct performance comparisons between TT-RSS and TT-CI on representative examples. Additionally, all experiments in Sections 3.1, 3.2, and 4.2 have been recomputed using double precision, and as a result, we have updated Figures 3.1, 3.2, and 4.4. While the results remain largely consistent with those originally reported, we hope that the updated figures, together with the new tables, help clarify the potential of our approach.
We hope that these changes address the reviewers’ concerns and that our manuscript is now suitable for publication in SciPost Physics.
List of changes
- We added clarifying remarks and citations to Section 2 to better distinguish TT-RSS from prior works (TT-RS and TT-CI), addressing a shared concern by both referees. These edits span lines 409–425, 441–442, 449–451, 458–461, 592–592, 595–599, and 609–617.
- We introduced two new subsections 3.1.3 and 3.2.3 along with Tables 3.1 and 3.2. These present direct numerical comparisons between TT-RSS and TT-CI on the representative examples discussed earlier in the section.
- All experiments in Sections 3.1, 3.2, and 4.2 have been recomputed using double precision, and as a result, we have updated Figures 3.1, 3.2, and 4.4.
- We updated the captions of Figures 3.1 and 3.2 to clarify that the x-axis represents the number of pivots N. In addition, we revised the captions of these figures, as well as that of Figure 4.4, to indicate that double precision is used only in those computations.
- In the tensorization of random TT functions, we increased the number of pivots considered from the range 10–20 to 10–35 to better illustrate how the test error decreases as the number of pivots increases.
- Minor corrections and clarifications were made throughout the manuscript for improved readability and consistency.
Current status:
Has been resubmitted
Reports on this Submission
Report
I believe that Referee 2 has a valid point: This is not a physics paper (see item 1 in "Weaknesses" of Report #2 by Referee 2 on 2025-6-24). For example, the MNIST data set discussed in section 3.2.2 definitely is not physics, neither are many of the other points discussed in this manuscript.
Furthermore, I have an issue with the context of the one point that actually is physics, namely the AKLT model discussed in section 4.2.1. Between Eqs. (4.6) and (4.7), the authors refer to their own relatively recent 2021 review [33]. However, if one looks at appendix 1.e of Ref. [33], this refers back to the original AKLT paper Phys. Rev. Lett. 59, 799 (1987). The notation used in section 4.2.1 of the present work might not be obvious from the original AKLT reference, but I understand that this $2 \times 2$ matrix-product state representation is known since at least 30 years. Take for example the review article by Hans-Jürgen Mikeska and Alexei K. Kolezhuk, Lecture Notes in Physics 645, 1 (2004), https://doi.org/10.1007/BFb0119591. There, in section 1.3.3, one not only finds a very similar representation with $2 \times 2$ $g$ matrices, but also references to early publications on the matrix-product state interpretation of the AKLT and related states such as M. Fannes, B. Nachtergaele, R. F. Werner, Europhys. Lett. 10, 633 (1989); Commun. Math. Phys. 144, 443 (1992); A. Klümper, A. Schadschneider, J. Zittartz, J. Phys. A 24. L955 (1991); Z. Phys. B 87, 281 (1992); Europhys. Lett. 24, 293 (1993).
I believe that this context issue needs to be fixed, i.e., the authors need to make it clear that the AKLT story is much older than the current presentation suggests. Once this is done, I believe that the manuscript can be published in SciPost Physics Core.
Furthermore, I have an issue with the context of the one point that actually is physics, namely the AKLT model discussed in section 4.2.1. Between Eqs. (4.6) and (4.7), the authors refer to their own relatively recent 2021 review [33]. However, if one looks at appendix 1.e of Ref. [33], this refers back to the original AKLT paper Phys. Rev. Lett. 59, 799 (1987). The notation used in section 4.2.1 of the present work might not be obvious from the original AKLT reference, but I understand that this $2 \times 2$ matrix-product state representation is known since at least 30 years. Take for example the review article by Hans-Jürgen Mikeska and Alexei K. Kolezhuk, Lecture Notes in Physics 645, 1 (2004), https://doi.org/10.1007/BFb0119591. There, in section 1.3.3, one not only finds a very similar representation with $2 \times 2$ $g$ matrices, but also references to early publications on the matrix-product state interpretation of the AKLT and related states such as M. Fannes, B. Nachtergaele, R. F. Werner, Europhys. Lett. 10, 633 (1989); Commun. Math. Phys. 144, 443 (1992); A. Klümper, A. Schadschneider, J. Zittartz, J. Phys. A 24. L955 (1991); Z. Phys. B 87, 281 (1992); Europhys. Lett. 24, 293 (1993).
I believe that this context issue needs to be fixed, i.e., the authors need to make it clear that the AKLT story is much older than the current presentation suggests. Once this is done, I believe that the manuscript can be published in SciPost Physics Core.
Requested changes
Place the AKLT story properly into its historical context.
Recommendation
Accept in alternative Journal (see Report)
Author: José Ramón Pareja Monturiol on 2025-11-16 [id 6039]
(in reply to Report 1 on 2025-10-16)We thank the referee for the constructive comment and for pointing out the need to better contextualize the AKLT model within its historical development. Following this suggestion, we have added a short clarification in Section 4.2.1 (lines 1097–1098) explicitly acknowledging that the TT/MPS representation of the AKLT state was derived in early works, and we now cite the relevant foundational references [Fannes, Nachtergaele, and Werner, Commun. Math. Phys. 144, 443–490 (1992); Klümper, Schadschneider, and Zittartz, Z. Phys. B 87, 281–287 (1992)].
We believe this addition properly places our presentation in its historical context and addresses the referee’s concern.