Mott transition in a cavity-boson system: A quantitative comparison between theory and experiment

1-The use of the "MCTDH-X" method for the simulation of correlated boson dynamics is demonstrated convincingly.
2-Excellent agreement between theory and experiments.

1-Since the paper introduces a new theoretical tool, a more substantial comparison to other existing methods would be highly desirable.

The authors introduce the "Multiconfigurational Time-Dependent Hartree Method for Indistinguishable Particles" as a tool to describe self-organisation transitions of ultracold bosons coupled by cavity-mediated interactions. This method enables them to carry out simulations of the phase diagram encompassing a self-organized superfluid and a Mott-insulator phase. Comparison with experiment shows excellent agreement. As such, the paper opens a new pathway in the research on cavity-coupled many-body systems in cold atoms and condensed matter. It could there be published if the authors address my concerns.

Apart from minor changes, which I list below, my main criticism concerns the discussion of the proposed theoretical method. Clearly, from a theoretical point of view, the main problem in the considered system is the competition between local and global (cavity-mediated) interactions. The authors circumvent this problem by simulating a four-well model. This is the smallest lattice size where the self-ordering transition to a checkerboard phase can be observed, so it seems the authors need to know in advance which many-body phases to expect. It would be good if such limitations were discussed openly. It would be good to present an outlook how this theoretical tool will scale with different setups that lack this fundamental symmetry.
In the same spirit, the authors emphasize in the introduction that the presented MCTDH-X method "captures many-body effects beyond the Gross-Pitaevski mean field limit". What is the relevance of beyond-Gross-Pitaevskii effects? A direct comparison with mean field methods, however superficial, might be very helpful for readers to appreciate the relevance of these effects.

1-On page 2, the authors write "The cavity-BEC system can thus reproduce a quantum-optical version of the Bose-Hubbard model."
It's not clear what defines the "quantum-optical" version of the BH model. I suppose the authors are thinking of a global interaction akin to the cavity-dipole coupling in Jaynes-Cummings or Tavis-Cummings models? This needs to be clarified.
2-The authors write "We emphasize that once the effective contact interaction strength g2D and the calibration factor are determined, there is no more free parameter in the simulation when compared to the experimental system." How does this compare to other approaches?

1- Clear and concise presentation of the methodology and the results.
2- Comparison of numerical results obtained via a new technique with real experimental data.
3- The numerical results obtained via the MCTDH-X method agree reasonably well with the many-body quantum phases observed in the experiment. Therefore, the method could serve as a powerful tool to answer open questions in the realm of many-body cavity QED.

1- The presentation of the results can be improved in certain parts (see list of required changes below).
2- Potential additional systematic errors are not elaborated in great detail.

In their manuscript "Mott transition in a cavity-boson system: A quantitative comparison between theory and experiment" Lin et. al compare numerical results obtained via the Multiconfigurational Time-Dependent Hartree Method for indistinguishable particles (MCTDH-X) to experimental data. The focus lies on predicting the superfluid to Mott-insulator transition in the Dicke self-ordered phase of a BEC coupled to a standing wave resonator. Beyond mean-field predictions for the non-trivial dynamic nature of this hybrid many-body quantum system are very challenging. The results in this manuscript show that the MCTDH-X method is a viable approach to tackle this problem. Hence, applying the method to new problems in future can lead to new theoretical predictions in this growing research field and guide future experiments towards realizing novel many-body quantum phases.
The results and methodology are presented in great detail and the focus of the main text lies on the fundamental physics and the general methodology. Details for the more expert reader are provided in a well written appendix.
Based on these arguments I think the manuscript fulfills the acceptance criteria of SciPost physics and should be published.

Nevertheless, I would like the authors to perform some minor changes on the manuscript and address the points below. This should enhance the clarity to the non-expert reader and shed some additional light on potential additional systematic errors.

1- Fig 5: The authors should extend the $y$-axis in Fig. 5(a) such that the values of $|\alpha|^2$ in the SMI phase are shown. The experimental data show a plateau of $N_\mathrm{ph}$ beyond a certain $E_{p,\mathrm{exp}}$. I assume the numerical $|\alpha|^2$ does not exhibit this plateau because it is obtained via a mean-field treatment. While the applied self-consistent approach is perfectly fine from a methodological point of view, it seems that the mean-field $\alpha$ leads to some systematic error when comparing the numerics to experimental data. Why do the experimental data exhibit this plateau? Could it be related to heating or particle loss (see also point 4 below)? The authors should elaborate a bit more on this point in their discussion of Fig. (5).

2- I think it would be instructive to provide similar figures as Fig. 4(g)-(l) [and maybe Fig.5(a)] for the single orbital case M = 1 (mean-field) in the appendix. This would be a powerfull display of the underlying problem which the authors address (no prediction of SMI phase) with their new method.

3- In the last paragraph of section 3.2 (page 9) the authors state that the "NP-SSF boundaries collapse upon each other" in Fig. 2. I can see this for small values of the effective detuning but for larger values the numerical and experimental curves deviate quite substantially. The same holds for Fig. 6 in the appendix. This discrepancy could be related to my arguments under point 1 in this list (see above). It would imply that the value for $|\alpha|^2$ is overestimated in the applied self-consistent method in comparison to real experimental values. This means that the numerics predicts the SSF-SMI boundary for weaker pump strenghts due to this overestimation in $|\alpha|^2$. I think the authors should comment on this point and amend the paper correspondingly.

4- The authors attribute the difference in the relative widths of the central Bragg peak $\mathcal{W}$ in the experimental data and the numerical results to the different shape of the initial BEC due to the asymmetric trapping potential in the experiment. However, one thing which is not discussed throughout the manuscipt is heating of the BEC via the pump beam in the experiment. In Fig.4(a)-(f) one can clearly see the buildup of some thermal background, which of course is not captured by the employed numerical method. Could it be that heating effects also contribute to the discrepancy in $\mathcal{W}$?

5- At several stages throughout the manuscript the authors claim that the self-organization of the BEC leads to a superlattice (see for example paragraph 2 on page 2). I don't understand why the checkerboard lattice which is formed in this case should be called a superlattice. Usually the term 'superlattice' is referred to a periodic lattice with periodically varying heights of the potential wells. I don't see where this is the case in the presented results.

6- Typos to be corrected:
-abstract: quantiative -> quantitative
-first line page 11: simplfication -> simplification
- reference [41]: light$\hat{A}$ -> light

1-Serious theoretical modeling of complex experimental results
2-Introduction of new computational method, potentially applicable to many other experimatal scenarios

1-Some of the experimental points are not clearly discussed
2-A direct comparison to traditionally employed theoretical techniques is missing

I reviewed the manuscript „Mott transition in a cavity-boson system: A quantitative comparison between theory and experiment“ by Rui Lin et al.
The authors present a comparison of theoretical and experimental phase boundaries employing MCTDH-X, reducing computational complexity by reducing the atomic system to four wells of the optical lattice.
The employed theoretical tools are novel and have the potential to be employed in many other experimental scenarios, making them quite appealing for the simplicity and at the same time effectiveness of the method.
I believe the paper is well written and deserves publication. I have several comments that would improve the quality of the manuscript.

1-In section 2.2, concerning retadation effects, the authors take the strategy of measuring the different phase boundaries with different ramping times. For the NP-SSF transition, they employ a slow ramp, to avoid retardation, while for the SSF-SMI they use a faster ramp to minimize atom loss. In this case, the light field suffers from retardation as reported in the methods section. The atomic potential is altered by the presence of the cavity light field and this retardation also affects the SSF-SMI experimental phase boundary. A discussion of this effect already in this section would improve the clarity of the manuscript on this point.

2-In the same section, concerning the calibration factor applied to the effective pump strength, the authors argue for it from the point of view of the pump not being monochromatic. An indication of the laser linewidth would help the reader to understand this point.

3-The authors write that "only the component at frequency $\omega_p$ can scatter into the cavity", but such frequency is defined as the frequency of the pump itself. Do the authors imply that the spectrum of the pump is being filtered? If so, how broad is the frequency range involved in the scattering process? Does the comparison of frequency range to the laser linewidth approximately justify the value the authors report for the calibration factor?

4-In section 4, caption of figure 5, there is a mention of relatively strong background signal picked up by the detector as soon as the laser is switched on, in fact making up about half of the maximum photon number recorded during self-organization. A discussion of this point is missing in the text. Does such signal come to the detector as diffused light or does it come form the cavity? In this last case, it would induce a strong symmetry breaking on the transition, possibly substantially modifying the behaviour of the system.

5-In the conclusions, the authors compare their method to a more traditional approach based on the Bose-Hubbard model. They regard it as complementary to theirs, suffering from different systematic effecs with respect to the ones their approach suffers from. I found this point very informative, in fact it would make sense in my opinion to move the discussion to section 3, or even the introduction, in order to motivate the author’s choices in this respect. It would also make sense to have a direct comparison between the two approaches, at least in the methods section, to appreciate the differences in this regard.