Learning tensor networks with tensor cross interpolation: new algorithms and libraries

We thank the referee for the detailed report. A list of responses to each request follows:

1. We discuss each example below:
   • Any tensor, and thus any function, can be factorized into a quantics tensor train representation. The failure mode for QTT is thus not one where the QTT fails to resolve the function, but one where the costs of using a QTT exceeds those of other representations. As QTT are factorizations in length scales, they are inefficient if the length scales are not separable. An example of such a function is a function $f(\mathbf{x})$ which is equal to $1$ if $\mathbf x$ is inside some contour, and equal to $0$ otherwise. If the contour has non-separable structure, the bond dimension of a QTT that approximates $f$ is close to maximal. Fairly simple curved contours, such as a sphere, have enough non-separable structure to lead to large bond dimensions $\chi \approx 1000$. We have added a paragraph that describes more examples for QTT ranks of simple analytic functions in multiple dimensions to section 6.1 (page 35).
   • Indeed, the examples can trivially be extended to higher dimensions. Whether the convergence with rank will remain fast strongly depends to the problem under consideration. This question has already been intensely debated in the context of the many-body problem. For instance MPS-based methods such as DMRG can be used to study the Hubbard model in 2D but two-dimensional tensor networks (so-called PEPS) are preferable. Nevertheless, since computing with MPS is much faster than PEPS, MPS can remain competitive even in this case. In that respect, the two above examples are very different, and we answer the question separately. Note that in general we can gain expressivity by generalizing the underlying tensor network structure to trees, as shown in arXiv:2410.03572. In examples such as the partition function in section 5.3, the dimensionality of the target function is crucial, basically because we can apply the arguments based on entanglement established in the field of tensor network wavefunction. In other words, the entanglement of the target function is an intrinsic property and TCI is just a way to reveal it. The $W_{\boldsymbol{\sigma}}$ tensor involved in the computation of a 2D partition function has entanglement properties satisfying the area law. Consequently, an exact MPS representation of this tensor will require a rank that increases exponentially with system size (irrespective of whether the MPS obtained using SVD- or CI-based compression). In contrast, we use the quantics representation to solve the heat equation. This reveals the (possibly small) entanglement between different scales, which is not necessarily related to the dimensionality, as was already shown in the literature. Quantics has been applied to partial differential equations in arbitrary dimensions, including quantum chemistry in 3D (see for example arXiv:2308.03508).
2. Indeed, the comment of the referee clarifies the general scope of the auto-MPO topic. We emphasize that our CI-based compression naturally works without the need to explicitly include specific structure of the operator. We have added a footnote about the approach of Ref. 64 in the main text (page 44).
3. Typos:
   • We have changed the sentence accordingly (page 17).
   • We have changed the main text accordingly (page 31).
   • We thank the referee for pointing out this inconsistency. The statement that the quantics Fourier transform can be represented as an MPO with low rank can be found in both references, which discovered this representation independent from each other. Therefore, we now cite both references in all places where this statement is made (pages 35, 36, 58).

We thank the referee for the detailed report. A list of responses to each request follows:

1. The explanation was indeed not as clear as intended. As the referee correctly noted, it is possible to derive Eq. (24) from Eq.(15) directly through the definition of the Green's function and the self-energy. We have rewritten the explanation given in section 3.2.4 (page 11) to show the derivation of Eq. (24) from Eq. (15) explicitly.

2. We have added an example for a fully nested configuration of pivot lists, with examples for pivots that would break nesting conditions (pages 17, 18).

3. A "$k$-dimensional slice" of some tensor $F_{\boldsymbol{\sigma}}$ is a slice with $k$ free indices, i.e. all indices except $k$ of them are constrained. We have added a remark explaining the terminology above Eq. (33) (page 16).

4. We have added an explaining sentence after Eq. (43) (page 21).

5. We have corrected the sentence accordingly (page 22).

6. We thank the referee for pointing out that the motivation for CI-canonicalization may have been unclear. A TCI is indeed an MPS, but an MPS is only a TCI if it is in CI gauge, i.e. comprised of tensors $T_\ell$ and $P_\ell^{-1}$ that are slices of the MPS itself. Some of the algorithms described in our manuscript, such as the TCI optimization algorithms and global pivot insertion, only apply to MPS in CI gauge: they rely on the property that all core tensors of a TCI are slices of the full tensor, or, equivalently, that the TCI can be fully defined through index sets $\mathcal{I_\ell}$ and $\mathcal{J}_\ell$. An arbitrary MPS does not generically fulfill this property, and thus the MPS must be transformed to CI gauge before applying the above-mentioned algorithms. We have added a paragraph detailing this motivation to section 4.5 (page 23). We give explicit examples where the conversion is advantageous.

7. Indeed, the constant prefactors are inconsistent in the sense that $\mathcal{O}(f(x)) = \mathcal{O}(3 f(x))$. The original intention was to show that achieving full nesting requires exactly 3 sweeps, and the runtime cost is therefore thrice that of a 1-site sweep. Since this comparison only works between CI-canonicalization and 1-site sweeps, we have removed the constant factors of 3 from the asymptotic complexities (now Table 2, page 27).

8. We now use two different color schemes $\beta$ and $\mathcal{L}$ in Fig. 5, as suggested by the referee.