Krotov: A Python implementation of Krotov's method for quantum optimal control

1. The manuscript and library present a useful tool for quantum optimal control, which is of interest to theoretical and experimental researchers in quantum physics, and it seems particularly timely in the current development of the field of quantum computing and more broadly of quantum information processing, in which noisy intermediate-scale quantum (NISQ) technology devices need to counteract the effect of noise.

2. The manuscript and software package will make the Krotov method more accessible to a wider audience, increasing the possibility of benchmarking the performance and robustness various gradient-based and gradient-free methods and possibly their interplay.

3. The library includes cutting-edge features and care toward code optimization. The open-source software package is developed according to the best practices of open-source code development (presentation, online documentation, comments in functions, code formatting, library organization, unit-testing, example notebooks). Useful informative examples are provided online and in the manuscript, as well as code snippets in the manuscript.

4. Generally, the manuscript is very well written. The authors provide a nice introduction and bibliography for the field of quantum optimal control theory. Discussion of some applications is provided.

1. The Introduction section could flesh out more how the Krotov method and its implementation in the presented library can help a researcher in quantum optimal control. General differences with existing methods could be highlighted more, even if some specific comparisons are addressed in detail in the following Section 4 of the manuscript.

2. The inclusion of graphics could be attractive for the reader. For example, benchmarking and performance plots could be added. Also, plots relative to an example implementation could be included. Some are already present in the online examples, see https://krotov.readthedocs.io/en/latest/08_examples.html.

3. The discussion of the Krotov method and its implementation in the library are very technical, and there is a somewhat abrupt jump from the general discussion of the Introduction, or the first part of Section 2.1 and, e.g., the detailed description of the terms of Eq. (1). A somewhat more gentle introduction could be appealing to the reader. Also, Section 3 does not look extremely easy to navigate. I do not have a specific recommendation on how to act in this regard, but possibly the introduction of a simple example as that of Appendix B could first gain the attention of the reader in the first part of the manuscript, leaving the full code for the example in the Appendix. In general, a pedagogical approach could greatly benefit the reader and increase the potential user base, whereas currently the discussion becomes quite technical and detail-oriented in Sections 2, 3, and 4. Details are very useful but a bird's-eye view seems somehow missing. This could be quite easily added in the initial parts of these sections to guide the reader with a stronger narrative.

The article provides the implementation of a quantum optimal control method for quantum information processing and quantum technology. The manuscript provides details on the Python library and on the algorithms it implements, providing context on the Krotov method in the quantum optimal control field.

The krotov package enriches a scene in quantum tech research that is increasingly relying on open-source packages to implement investigations and integrate functionalities. The current library seems a timely addition to the toolbox of the quantum tech researcher, at a time when dissipation and noise hamper the ideal implementation of quantum protocols for quantum computing, quantum information processing and quantum control in general.

I am confident that the krotov library will become an important reference point in quantum optimal control theory resources, increasing the popularity of this method and possibly enhancing comparative studies benchmarking various algorithms with open-source code. I hope that the suggested modifications, once implemented in the revised manuscript, will make this tool of even greater help not only for the experts in the field, but for more researchers eager to learn about quantum optimal control with a hands-on approach.

1. Please consider addressing point (1.) of the weaknesses, i.e. more general comments about the Krotov method and the krotov library in the introductory part of the manuscript.

1.1. For example, in the abstract a line could be added about a defining feature of the Krotov method with respect to other optimal control methods. What about mentioning that Krotov is a gradient-based method?

1.2. Also consider mentioning that Krotov, according to Sec. 2.3 of the paper, assumes time-continuous control fields for assured convergence, but can be applied also for discrete controls.

1.3. With regard to integration with quantum computing experiments, the authors could consider mentioning the advantages of implementing Krotov in currently available quantum computers. If possible, for example, the authors could discuss how would this fare with the OpenPulse scheme announced by IBM, if there are enough details to address it (https://arxiv.org/pdf/1809.03452.pdf). The authors could add if they envision any interaction with existing or future quantum computing platforms, such as IBM, Google, EU's OpenSuperQ.

1.4. With regard to anticipating to the reader a bit more of the contents of the following sections, in the last paragraphs of the Introduction, which includes the structure of the article, more details could be given. Also, information about the Appendices could be included.

2. Please consider addressing point (2.) of the weaknesses, i.e. adding plots to the manuscript. Also, as an advice, the authors could consider including a table that summarizes already some of the properties of the Krotov method versus other methods.

3. Consider making Section 2.3 into a pseudo-code which will help the reader to understand the method better and also implement in C++/Julia or any faster language. In section 2.3 (Time discretization) it seems that the core of the implementation is described. This section could be conveniently converted into a pseudo-code of the implementation with some indication of the variable types and explicitly writing down how they should be calculated (not just referencing the equations) e.g.,

"- Construct XT (\xik^{i} (T)), a state vector or density matrix according to Eq (8) as, . math:: \xi_k^{i} (T) = - \frac{\delta J}{\delta \bra {\phi_k }}
- Perform a backward propagation over the full-time step using Eq (.) as .. math:: \dot x = - \frac{i}{\hbar}
- Store each step in memory (as a list/array of vectors or density matrices)."
and so on.
This will facilitate an easy understanding of the code behind the implementation, which could help anyone to rewrite the whole tool in a compiled language or otherwise for faster implementation as the authors suggest in the introduction.

4. Please consider theses additional references:

4.1. On the Krotov method, these references from Ref. [20] seem to be missing and relevant in the discussion:
[36] J. P. Palao and R. Kosloff, Phys. Rev. Lett. 89, 188301 (2002).
[47] V. F. Krotov and I. N. Feldman, Eng. Cybern. 21, 123 (1983), Russian original: Izv. Akad. Nauk. SSSR Tekh. Kibern. 52 (1983), 162–167.
[48] A. I. Konnov and V. F. Krotov, Autom. Remote Control 60, 1427 (1999), Russian original: Avtom. Telemekh. 1999, 77–88.

4.2. Please add information about a recently introduced gradient-based optimal control technique, the GOAT method,
"Tunable, Flexible, and Efficient Optimization of Control Pulses for Practical Qubits"
Shai Machnes, Elie Assémat, David Tannor, and Frank K. Wilhelm
Phys. Rev. Lett. 120, 150401 – Published 9 April 2018
https://doi.org/10.1103/PhysRevLett.120.150401
A discussion of differences and similarities with Krotov would help the interested reader.

4.3. Please consider citing both QuTiP papers when citing that library:
J.R. Johansson, P.D. Nation, F. Nori
QuTiP: An open-source Python framework for the dynamics of open quantum systems
Comp. Phys. Comm. 183, 1760-1772 (2012).
J.R. Johansson, P.D. Nation, F. Nori
https://www.sciencedirect.com/science/article/pii/S0010465512000835?via%3Dihub

QuTiP 2: A Python framework for the dynamics of open quantum systems
Comp. Phys. Comm. 184, 1234 (2013).
https://www.sciencedirect.com/science/article/pii/S0010465512003955?via%3Dihub

4.4. On feedback-controlled adiabatic quantum computation: https://journals.aps.org/pra/abstract/10.1103/PhysRevA.86.052306
On efficient quantum control for universal quantum computing: https://journals.aps.org/pra/abstract/10.1103/PhysRevA.81.040303
On quantum feedback control: https://journals.aps.org/pra/abstract/10.1103/PhysRevA.79.052102

5. About the krotov library:
The installation works fine both from conda + pip and by copying the GitHub folder.

5.1. The code snippets of Section 3, at best, should be self-contained.

5.1.1. If one copy pastes them, the code does not run, as many variables are not defined, such as "H", "L", "omega_vals", "delta_vals". Maybe adding these few lines is worth the increased clarity it will provide.

5.1.2. The "itertools" library used on page 11 is not imported or cited in the manuscript.

5.1.3. The example of Appendix B could be hosted online (or in a Zenodo datased; if it is already, it could be said explicitly, when the authors mention the other packages). Note that including the line numbering prevents the reader from easily copy-pasting the code in a script, as the line numbers get copied too.

5.2. In "Krotov.optimize", line 414, the authors could consider to allow the user to separate the exact optimization routine and the other optimization routines. The authors could think of adding other methods beyond negative gradient, see https://arxiv.org/abs/1609.04747. This is just a suggestion, but the authors could address in the manuscript this extension, maybe in the future perspectives of Sec. 5.

5.3. Library testing.
5.3.1. Possible bug report: Running the Example in a Jupyter Notebook, the line
>>> obj.reset_symbol_counters()
gave an error
>>> AttributeError: 'Objective' object has no attribute 'reset_symbol_counters'
Whereas
obj.summarize()
Provides the expected
'|(2⊗2)⟩ - {[Herm[2⊗2,2⊗2], [Herm[2⊗2,2⊗2], u3(t)], [Herm[2⊗2,2⊗2], u4(t)]]} - |(2⊗2)⟩'
This is with
"module 'krotov' from '/Users/username/miniconda3/lib/python3.6/site-packages/krotov/__init__.py'"

5.3.2. Possible bug report:
>>> obj.summarize(use_unicode=False)
Gave the error
>>> summarize() got an unexpected keyword argument 'use_unicode'
Although the documentation says this is an option
https://krotov.readthedocs.io/en/latest/_modules/krotov/objectives.html#Objective.summarize

From the API docs: https://krotov.readthedocs.io/en/latest/_modules/krotov/objectives.html#Objective.summarize, the krotov.objectives.summarize method has the following signature,
"def summarize(self, use_unicode=True, reset_symbol_counters=False)", but in the latest version present in the master branch of the online repository (https://github.com/qucontrol/krotov/blob/master/src/krotov/objectives.py) as well as the pip version 0.30, the summarize method has a different signature,"def summarize(self, ctrl_counter=None)". The respective docstrings are consistent with the change but this is not updated in readthedocs and should be updated perhaps. Adding the Krotov package to Zenodo would provide version release linked to the paper for future reference.

Please double-check the code snippets and examples contained in the manuscript and in the online library documentation using the pip distribution of the software package.

5.4. All of the the Notebooks contained in the GitHub repository online ran fine. The first cell of each requires the user to have the watermark software, which does not get installed with the krotov package from pip, and gives an error. This command could be commented, or the requirement included (beyond for contributing guidelines), or removed from the notebooks or, as a suggestion, a command like qutip.about(), also for the Krotov library, could be added. Also, just as an enquiry, I wonder what "oct" as in "oct_results", used in all notebooks, stands for.

5.5. Although this is not part of the manuscript, all notebooks found on Github ran fine, besides the following minor issues:
- notebook 3 is very computationally demanding, and was not run after cell 20;
- notebook 8 required parallel implementation as was partially run;
- notebook 9, which gave an error at cell 6 ("Invalid initial_state: must be Qobj, not ndarray"); maybe this modification is part of Krotov development updates.

6. Detailed comments on the manuscript:
6.1. Introduction
6.1.1. With regard to the sentence:
"While GRAPE is found in various software packages, there has not been an open source implementation of Krotov’s method to date."
The authors could mention, if they know it, why the Krotov method was not widely implemented so far and what made it appealing to the authors and is appealing to the community in these days.

6.1.2. "Moreover, the QuTiP library [24] exists": Please consider citing both QuTiP papers.

6.1.3. Reference [20], https://arxiv.org/abs/1011.4874, mentions Krotov. From that reference:
" We benchmark Krotov-type algorithms and grape algorithms over a selection of scenarios, giving the user of control techniques guidelines as to which algorithm is appropriate for which problem."
It seems that this is overlooked in the manuscript.

6.1.4. Regarding the sentence:
"Compared to the Fortran and C/C++ languages traditionally used for scientific computing, and more recently Julia [35], pure Python code usually performs slower by two to three orders of magnitude [36,37]. Thus, for hard optimization problems that require several thousand iterations to converge, the Python implementation provided by the krotov package may not be sufficiently fast. In this case, it may be desirable to implement the entire optimization and time propagation in a single, more efficient (compiled) language."
What about Cython, would it be of help?

6.1.5. The inclusion of optimization open-systems dynamics, as also detailed in one example, seems very interesting with regard to the realistic model of dynamics in NISQ devices. This feature, which is also present in QuTiP, see https://nbviewer.jupyter.org/github/qutip/qutip-notebooks/blob/master/examples/control-pulseoptim-Lindbladian.ipynb, could be highlighted more in the Introduction.

6.2. Section 1

6.2.1. "The control fields might be the amplitudes of a laser pulse, for the control of a molecular system, RF fields for nuclear magnetic resonance, or microwave fields for superconducting circuits."
Control fields for ion-based quantum computing, a promising emerging candidate, could be added.

6.2.2. The DYNAMO package seems to be including Krotov:
See https://github.com/shaimach/Dynamo/search?q=krotov&unscoped_q=krotov
The presented krotov library is on high level of quality in terms of best practices, documentation, unit testing. Anyhow, consider citing this implementation, which by the way is open source, although it does not rely upon an open source language as Python (is instead based on MatLAB).

6.2.3. Please consider writing explicitly that Eq. (6) is Schrödinger equation and introducing the physics problem in a more gradual way, with more general details.

6.3. Section 2
6.3.1. After Eq. (1), there is a discussion on the functional that is minimized in Krotov. The authors give an example of what the \ket{\phi_k} could look like for two qubit quantum gates. Maybe it will be better to indicate how the guess control function also looks like (is it just a one dimensional function, a pulse?). This is just to keep things pedagogical for the first time reader. Also, the index l seems to be not formally defined. Are there l controls for k states? Or is it independent of the number of states and even for a single state-to-state transfer, there could be more than one (l > 1) controls?
As a first-time reader it would be nice to have a self-sufficient picture of the functional to be minimized.

6.4. Section 3
6.4.1. As an advice, the authors can also mention that krotov makes "optimal use" of classes in Python.
6.4.2. Consider spelling once in the paper QuTiP as the Quantum Toolbox in Python.
6.4.3. After equation 17, there is a snippet of code which defines a new `sigma` class. The `S` should be in caps as class names in python are recommended to start with caps.
6.4.4. Comment: Very nice flowchart in Figure 2 to select the optimization method based on problem instance.
6.4.5. With regards to "Large Hilbert space dimensions [10,11,33,34] and open quantum systems [29] in particular require considerable numerical effort to optimize.", can we get an idea of the dimensions of the Hilbert space where Krotov starts to face issues - 32? 64? 128? (3 qubits, 5 qubits?)
6.4.6. As a suggestion of In section 3, how to use, installation instruction could be mentioned perhaps in a one-liner. This is discussed in Appendix A and could be cross-referenced here.

6.5. Consider adding a Conclusions section, either adding them to the current Section 5 or dividing the future perspectives from the conclusions of the manuscript, relative to the library, method and applications.

6.6. In Appendix B, which includes a simple example of application of Krotov, the physics should be much better introduced and the results of the notebook should be better presented and more commented. In general, including a general simple example and a more refined high-level example would appeal to the interested coder.

6.7. Referencing: As a suggestion, the authors could consider adding the Krotov package to Zenodo and cite that version release as a crystallized version for future reference; also, Zenodo provides a citeable DOI.

1. The Krotov method is well-known in the optimal control community.

2. The addition of a high-quality open-source implementation is very welcome and could be of use to a large audience of both optimal control theoreticians and experimentalists.

1. The description of the base Krotov algorithm must be self-contained. This is currently not the case.

2. The Authors intermingle the specification of the algorithm with an explanation as to why it is formulated the way it is.

The Authors present a Python implementation of the Krotov quantum optimal control method. The Krotov method is well-known in the optimal control community, and the addition of a high-quality open-source implementation is very welcome and could be of use to a large audience of both optimal control theoreticians and experimentalists. As such, the work is important and should be published.

There are, however, are some issue with the current form of the manuscript which, in my opinion, MUST be amended prior to publication. Primarily, the explanation of the Krotov algorithm in Section 2 requires revising, to make it self-contained as to clearly separate a detailed description of the algorithm from the explanation as to its construction.

13 additional issues which must be addressed (numbered 3-15), and 12 optional comments (numbered 16-27) are listed below.

The explanation of the Krotov algorithm in Section 2 requires revising, on two fronts:

1. The description of the base Krotov algorithm MUST BE SELF-CONTAINED. For example, the introduction of the Schrödinger equation, a basic feature of the method, is left to references [5,42,43], which puts an unreasonable burden on the reader. While there are many Krotov use cases and variants which can be cited away, one Krotov variant must be presented in full within the manuscript.

2. The Authors intermingle the specification of the algorithm with an explanation as to why it is formulated the way it is. The result left this reader confused. Please SPECIFY THE ALGORITHM IN A CLEAR, EXPLICIT, CONCISE AND COMPLETE FORM. One possible example is section "Algorithmic Steps" of the cited [20]. The explanation of the algorithm, a self-contained version of Section 2, can then refer to the steps of the algorithm, making it easier for the reader to follow.

Additional issues which, in this reviewer's opinion, MUST be addressed are as follows:

3. Section 1: The statement "Gradient-based methods typically converge faster, unless the number of optimization parameters can be kept small, and can be separated into methods that update the external control field concurrently or sequentially in time [20]" does not appear to be supported by the data in the cited reference.

4. Section 2.2: The sentence "It achieves this by adding a vanishing quantity ..." is unclear or incomplete. What is this quantity?
5. Section 2.2: It is unclear how Eq. (5) follows from Eq. (4).
6. Section 2.3, point "1. Construct the states ..." The statement "These typically depend on the states..." is unclear. How do they depend on the states? When don't they? It feels as if this explanation is not starting at the beginning.
7. Section 2.3, point "3. Starting from the known ..." The paragraph is unclear. Please revise. Include Eq. (5) as you wish it to be used. Explain 'This approximation of t ≈ t + dt=2 is what constitutes the "time discretization" mathematically'.

8. Section 3: "pip install krotov" from Appendix A and the URLs from appendix C belong in section 3.1 or "section 3.0", otherwise one reads about a package without knowing where to get it. The license (BSD, as I discovered on the package's site) should also be noted.

9. Section 4 and especially 4.3: The comparison made here is limited to the two other methods available in QuTip, GRAPE and CRAB. It is therefore far from a comprehensive comparison of quantum optimal control methods. Therefore, at the very least, the limited scope should be stated clearly - both in the text and in Figure 2.

10. Section 4.1, "* Krotov’s method mathematically guarantees monotonic convergence in the continuous limit": From the manuscript I understand this to be true only for time-continuous formulations, and explicitly untrue for the time-discrete implementation which is presented here. Therefore, either the manuscript has confused me, or this statement is misleading. Either way something needs to be fixed.
11. Section 4.1, "Krotov’s method does not require a line search to determine the ideal magnitude of the pulse update in the direction of the gradient.": Aren't we supposed to set \lambda manually? Again, I'm confused.
12. Section 4.2: "A possible drawback of gradient-free optimization is that it is also prone to get stuck in local optimization minima." - This statement is equally true of many gradient-driven methods, and patently untrue of some gradient-free methods, such as simulated annealing.
13. Section 4.2: Singling out Subplex in NLopt [60] is unjustified. NLopt, nevergrad [https://github.com/facebookresearch/nevergrad] and other packages contain dozens of relevant algorithms.
14. Section 4, last paragraph: The text does not clarify why "gradient free" = CRAB, and why the random choice is beneficial.
15. Section 4, discussion of "time continuous controls": The krotov package explicitly deals with discretized controls. Moreover, it is not clear why the resultant control fields would approximate a continuous control field (i.e. bandwidth constrained), as no smoothing penalty term was demonstrated. And should such a term be introduced, in fairness one must acknowledge a similar term has been used in GRAPE. So it is unclear why the Krotov method should have an advantage in discretized time-continuous controls.


Additional issues, which the Authors may choose to address, are:

16. Section 1: The paragraph starting with "Large Hilbert space dimensions" - have the authors tried cython?

17. Section 2, around Eq. (9): Is S(t) multiplied with the control fields? If so, are they bound? If they are unbounded, it is unclear why S(t) would enforce anything; and if they are bounded, the bounding mechanism is unclear.
18. Section 2.3: The statement "monotonic convergence is mathematically guaranteed" is claimed but not argued. At minimum, repeat citations appearing at the top of Section 2.2.

19. Section 3.1: How is the use of Krotov with QuTip similar / different than using GRAP or CRAB for the same task? A side-by-side or similar would be useful.
20. Section 3.1: "* objectives: a list of objectives ..." Why is H per objective? Isn't is identical for all?
21. Section 3.1: "* propagator: A routine that calculates ..." Why not use the QuTip propagator (I'm guessing there is a good reason - it would be helpful if you specify it).
22. Section 3.2: "H[1][1]: dict(lambda_a =5, shape =S)": Unclear what is H[1][1] (eps0??) and why is it a key in a dictionary.
23. Section 3.2: "The value of λa may be adjusted dynamically with respect to the rate of convergence" - How? Which function call?

24. Appendix A: The discussion of (Ana)conda is extraneous. Python environment management is a mess better left to programming forums.
25. Appendix B: Moving the complete example to the main text, would greatly help the reader in understanding the calling structure in its entirety. Then special cases can be specified.
26. Appendix B: The function "hamiltonian.functions def_control" and "S" are almost identical. Isn't there a way to guarantee the required consistency by having "hamiltonian" use "S"?
27. Appendix B: Several statements in the code are not explained, e.g. "chi_constructor = krotov . functionals . chis_re", "J_T= krotov . functionals . J_T_re".