Efficient numerical simulations with Tensor Networks: Tensor Network Python (TeNPy)

1) Well-written introduction, very pedagogical

2) Examples of codes for python TN libraries

3) Discussion about abelian symmetries in the TN libraries

1) Only focused on Matrix Product States

2) No numerical benchmarks of the libraries

3) No discussion of non-abelian symmetries

In this paper the authors offer a pedagogical introduction to some basic aspects of tensor network states and many-body entanglement theory, focusing their explanations on Matrix Product States (MPS) and their associated methods. The authors also explain the implementation of abelian symmetries in MPS, and provide a practical guide for their tensor network library in Python (TeNPy) with practical codes.

This is definitely a very useful paper, which is also nicely written It definitely deserves publication in SciPost. I have some observations though, that the authors should consider before publication:

1) Could the authors comment also (at least briefly) on the implementation of non-abelian symmetries?

2) The authors say in the title that they discuss “tensor networks”. However, in practice they only discuss MPS. I would therefore either discuss briefly other tensor networks in their paper (PEPS, MERA…), or modify the title and say “Matrix Product States” instead of Tensor Networks.

3) The paper is fully analytical, and no numerical calculation is shown. Since this is a paper about some programming libraries, it would be very helpful to the reader if the authors showed some examples of numerical benchmarks with TeNPy.

1) Comment on non-abelian symmetries

2) Modify title or discuss TN beyond 1d

3) Add numerical benchmarks

1. excellent introduction into matrix-product states and their underlying physical motivation

2. good description of the TeNPy library with example codes provided both here and in the documentation accompanying TeNPy.

3. complete description for the implementation of Abelian symmetries

4. discussion of basic TEBD and DMRG methods as provided by TeNPy

1. reliance on the Γ-Λ-notation which is not necessary in the presented applications and numerically unstable

2. the discussion of DMRG is relatively short and not describing the current state of the art

The lecture notes both provide an excellent motivation for and description of matrix-product states as well as an introduction into the presented TeNPy Python package. The Python package itself is excellently documented (outside these lecture notes), provides the examples described in the lecture notes and appears to provide reasonable performance given the design goals which focus on usability, ease of extensibility and understandability.

Modulo some minor typos and small extensions (see below), my main concerns with the lecture notes submitted are then two-fold. As soon as these are addressed in some way, I wholeheartedly suggest publication of these very readable, pedagogical and thorough lecture notes.

First, the algorithms are presented using the Γ-Λ-notation/gauge for matrix product states. While reasonably straightforward, restoring this gauge after tensor updates involves the numerically unstable multiplication by the inverse singular value tensor. However, neither for the presented TEBD algorithm nor for DMRG updates is it necessary to keep the state within the Γ-Λ notation and indeed, in the section on DMRG, a different gauge is used: the much more stable alternative, the mixed canonical form with left-orhogonalised tensors (A), an orthogonality centre (either a single-site tensor M or a bond tensor C) and right-orthogonalised tensors (B).
The Γ-Λ notation is primarily useful when a) updating site tensors in parallel in TEBD (cf. Urbanek and Soldan, 2016) or b) evaluating many local expectation values. While a) is a potentially useful application, Urbanek and Soldan used an interesting trick to avoid the multiplication by the inverse singular value tensor. b) is potentially applicable, but evaluating expectation values is typically much cheaper than optimising or time-evolving tensors, so it appears unreasonable to enforce the Γ-Λ gauge also during the intermediate operations (instead of restoring it at the end). If the Γ-Λ notation provides other benefits, it would be useful to present those benefits in section 3.1 or instead only use the A-M-B form throughout the entire lecture notes.

Second, the section on DMRG should be extended to capture more of the current state of the art. In particular, the single-site update is missing completely. The primary aim of the density matrix perturbation or the subspace expansion is not so much to improve the convergence of the 2-site DMRG (2DMRG), but also to allow for a reliable single-site DMRG (1DMRG) update with its associated computational speed-up. The relative advantages of 2DMRG (slightly better convergence of other observables, less potential to get stuck) and 1DMRG (much faster, larger bond dimensions, often better energies) should be discussed in a pedagogical introduction to variational ground-state searches with MPS. A ‘staging scheme’ to incrementally increase the bond dimension and accuracy of the calculation during the sweeps is also present in the example code (I think, the example is not entirely clear) but not explained in detail. Further methods to avoid convergence problems (multi-grid DMRG, perturbation Hamiltonians, initialisation of the starting state etc.) could also be mentioned. Finally, extrapolations to the limit $\chi \to \infty$ are an essential part of large-scale DMRG calculations and should not be excluded from a pedagogical introduction.

1. Reconsidering the use of Γ-Λ gauge at least in the pedagogical introduction. If keeping it, explaining the trick used by Urbanek and Soldan (Computer Physics Communications, 2016) to avoid multiplication by the inverse singular value tensor would be useful.

2. Extending the section on DMRG to present more of the details of the algorithm and how to implement/use those within TeNPy would be worthwhile.

Additionally, I’d like to suggest the following minor corrections:

3. "the Hadane phase in quantum spin…" should read "the Haldane phase…".

4. "For each algorithm, we give a short example code show how to call" should read either "showing how to" or "to show how to call".

5. Below equation (1), $\{ | \alpha \rangle_{L(R)} \}$ are called an "orthonormal basis" of $\mathcal{H}_L$. A basis is a complete set of vectors spanning the entire vector space, which cannot be since $\mathcal{H} = \mathcal{H}_L \otimes \mathcal{H}_R$ does not ensure that the left and right spaces have the same dimension. Instead, the $\{ | \alpha \rangle_{L(R)} \}$ are the left and right Schmidt bases and form an orthonormal basis of a *subspace* of $\mathcal{H}_L$ and $\mathcal{H}_R$ respectively.

6. In section 3.1, it may be useful to also introduce the adjective "auxiliary" to describe the bond/virtual indices, as it is commonly encountered in the literature.

7. I’m not sure it is a good idea to describe an MPS on a finite system as "uniform" (above equation (9)). While strictly speaking true, it may be more appropriate to reserve the adjective "uniform" to infinite-system MPS.

8. In section 3.3, imaginary time evolution is described as "being used to find ground states of the Hamiltonian" (eq. (22) and the sentence above it). However, imaginary TEBD is not the method of choice for this task (as also explained later on) and it may confuse a novice reader. A more appropriate example application of imaginary time evolution may be the evaluation of Green’s functions.

9. At the bottom of page 14, it is said that the TEBD method can only be applied to nearest-neighbour Hamiltonians. The introduction of swap gates (Stoudenmire and White, 2010), however, avoids this problem entirely at only slightly larger computational effort.

10. The example code for TEBD with TeNPy is split over a line-break, which makes copy-pasting it a bit hard. In general, a more traditional formatting with a monospaced typeface may be worth considering (also depending on the editor’s preferences).

11. On page 16, Ref. 61 is mentioned to describe finite state machines for MPO construction. A very readable other paper on that topic is Paeckel et al 2017 (SciPost Phys. 3, 035), which is certainly worth mentioning here.

12. I’m not sure what is meant by "even if they might not be the true Schmidt basis" in footnote 3 on page 17. The DMRG procedure always keeps a Schmidt decomposition between the current left- and right environment and hence always keeps a left and right Schmidt basis.

13. Fig. 8b) is slightly misleading (as then also mentioned in the text). While the effective Hamiltonian does take this form, it is very suboptimal to compute it. Instead, the authors might consider showing $\psi_{\mathrm{eff}}$ (i.e. the two-site tensor) and the product $\hat H_{\mathrm{eff}} \cdot \psi_{\mathrm{eff}}$, which would avoid the impression that one has to explicitly evaluate $\hat H_{\mathrm{eff}}$. As an aside, in the case of quantum numbers which split the local state-space into one-dimensional blocks (i.e. a single state per quantum number), the ordering L·W·M·R instead of L·M·W·R allows precomputation of L·W prior to all Lanczos iterations and is typically somewhat faster than the asymptotically-optimal ordering, at least in 1DMRG calculations.

14. On page 19, it is mentioned that the left and right bases have to be orthonormal to ensure optimal truncation. However, these bases have to be orthonormal for the much more fundamental reason that one wishes to update the tensor using a standard instead of a generalised eigenvalue problem (as eg. explained in Schollwöck 2011, around Eq. 208). This fact should not be omitted!

15. On page 21, it may be worthwhile to mention the recent VUMPS algorithm/method as an alternative to the previous iMPS methods.

16. Section header "4.1 Infinte Time Evolving…" should read "4.1 Infinite Time Evolving…".

17. In footnote 5, the authors say that they restrict themselves to "local, abelian symmetries". In fact, the symmetries considered are *global* symmetries (i.e., not local/gauge symmetries). The restriction on locality is that the symmetry has to *act* locally; e.g. momentum conservation on a ring in real space cannot be implemented in the setting described here, one has to go to momentum space to do so.

18. I am not sure I understand the role of the total charge Q per tensor completely, in particular during tensor-tensor contractions. Do I understand correctly that Q acts as an additional one-dimensional dummy index carrying a total charge out of/into the tensor, with those legs combined into a new "pipe" leg during tensor-tensor contractions? If so, it may be worthwhile to offer this alternative explanation as well.

19. In the installation instructions (doc/INSTALL.rst) of the TeNPy package on Github, the probably aliased command "git co <tag>" is used. The authors may wish to replace that by "git checkout <tag>".

20. Finally, I would suggest to link to the excellent documentation of the TeNPy package at https://tenpy.github.io/ directly from the paper. It is absolutely worth reading and hence also pointing out to readers.