Carlo.jl: A general framework for Monte Carlo simulations in Julia

- Simple and well-written Julia package for performing Monte Carlo simulations

- Lack of applications (e.g. different stat-mech models, quantum Monte Carlo) that would make users change practice. Ising application is too simple. To have impact, this scheduler/error analysis library needs to come with a more complete software environment
- Little added value of the paper accompanying the code

This paper is associated to a Julia library designed to help perform Monte Carlo simulations. The library contains a scheduler (to perform computations in parallel, including with checkpointing) as well as error analysis. Those are the basic blocks of every Monte Carlo simulation.

I am not a Julia practitioner, and actually couldn’t install the package (but that’s probably due to my lack of knowledge of the Julia environment). I am nevertheless convinced that the package works and is efficient.

It somehow fills a gap that was left by the C++ Alps library (somewhat popular in this community, but no longer maintained) which contained similar features but also many other features (lattice. Model libraries) and application packages, and another previous attempt in Julia (MonteCarlo.jl for which the status in unclear).

Now an important question remains: what is the target audience for this package / paper package ?
I suspect this is for Monte Carlo practitioners (mostly in the field of statistical mechanics and quantum (lattice) models and I tentatively see two possible specific target groups :
1. Monte Carlo practitioners who have not switched to Julia (: these researchers have probably their own Monte Carlo frameworks already, and it is not obvious that this paper will make them switch to Julia for the following reasons : the Ising model example is too simple, the performances (e.g. good naive parralelization in Figure 3) are certainly identical to those with their own house-made Monte Carlo, and there are no new specific non-trivial Monte Carlo aspects that are introduced in this package (see suggestion below however)
2. Julia practitcionners that may need to do Monte Carlo simulations. I see perhaps an interest for this category.

I naturally think that the readership of SciPost Physics Codebases is mostly composed of researchers in category 1 (this is my case). Let me also mention that, as a matter of fact, the paper accompanying the code is very simple, and not much more instructive than a well-made tutorial that could be included in the library.

Based on this, I think that there is perhaps more to be done to convince (me and the rest of the readership in this category 1) that this library is an added value that merits a new publication be included in the SciPost Physics Codebases. At the moment, I don’t think the simple port to Julia of standard Monte Carlo features is enough. As suggestions for improvements, I recommend to :

- Document and explain using a non-trivial example of what is the parallel run mode, which promises to offer nontrivial MPI parallelism
- Provide a more advanced applicative package (than just Ising, that is unlikely to be very useful), e.g. in quantum lattice models, that is based on this Monte Carlo Julia library. The author has publications using quantum Monte Carlo simulations with Stochastic Series Expansions (SSE), and the documentation on the github mentions quantum Monte Carlo SSE and auxiliary field codes, based on the framework. I think this would be an added value for which category 1 audience could cross the Julia Rubicon.

1. Provide at least one more advanced applicative package (than just Ising, that is unlikely to be very useful), e.g. in quantum lattice models, that is baed on this Monte Carlo Julia library. see main report for suggestions

2. Document and explain using a non-trivial example of what is the parallel run mode

Ask for major revision

1- Provides a practical scheduler to optimize CPU usage
2- Includes a basic set of typical measurements that can be used

1- Only one simple example is provided
2- Requires a large effort for the user, such that it may only be used by experts

The article introduced Carlo.jl, a Julia package that helps implement Monte Carlo methods to different problems by providing the basic functions generally needed and a task scheduler that allows to comprise all desired runs into a single script. First, it is nice to see that Julia packages are being programmed, documented, and published. The theoretical physics community is stearing in this direction and, since Julia is a rather new language, there is much room to develop and share open code.

As a Monte Carlo user, there is a point that was unclear when first reading the article. Is Carlo.jl aimed for quantum or classical Monte Carlo calculations? It later became clear that Carlo.jl only provides the core functions needed for a battery of applications, but the extent of applications should be made immediately clear to the reader. This is just a minor and personal issue.

About the code, I see the advantage of using Carlo.jl. However, since most of the model programming depends on the user, the package seems to be appealing to experts only. To say it bluntly, it is probably too much work for someone who needs a quick result to compare with either experiments or some other theory. This could be greatly alleviated with the consistent inclusion of more examples of application (not in the article, but in the library). To say something: Heisenberg, XY, Ising, classical, quantum, chain, square, triangular, cubic. Something to ease the way into Monte Carlo methods for beginners.

Otherwise, people who have already been working on Monte Carlo, probably have their code. And I find it hard to believe that someone would exchange their own code, where one knows exactly how it works and how to adapt it to each scenario, for some other code where they still need to code in some complicated model.

I recognize that this is a too personal point of view, and I cannot predict how popular the package will be. My recommendations are merely suggestions to widen the scope of posible users. That being said, I see no reason why the code (and article) should not be published as is. Ideally, we (scientists) should stop having our own codes for everything, wasting time in programming the same things over and over. Especially when they can be generalized. And this package goes in that direction.

Publish (meets expectations and criteria for this Journal)