SciPost Submission Page
Optimal, fast, and robust inference of reionization-era cosmology with the 21cmPIE-INN
by Benedikt Schosser, Caroline Heneka, Tilman Plehn
This is not the latest submitted version.
This Submission thread is now published as
Submission summary
| Authors (as registered SciPost users): | Tilman Plehn · Benedikt Schosser |
| Submission information | |
|---|---|
| Preprint Link: | https://arxiv.org/abs/2401.04174v2 (pdf) |
| Date submitted: | Jan. 17, 2025, 10:07 a.m. |
| Submitted by: | Schosser, Benedikt |
| Submitted to: | SciPost Physics Core |
| Ontological classification | |
|---|---|
| Academic field: | Physics |
| Specialties: |
|
| Approaches: | Computational, Phenomenological |
Abstract
Modern machine learning will allow for simulation-based inference from reionization-era 21cm observations at the Square Kilometre Array. Our framework combines a convolutional summary network and a conditional invertible network through a physics-inspired latent representation. It allows for an optimal and extremely fast determination of the posteriors of astrophysical and cosmological parameters. The sensitivity to non-Gaussian information makes our method a promising alternative to the established power spectra.
List of changes
Introduction: Highlight more how our approach differs from previous ones (page 2)
Section 2: 2.1 - Fix typo (page 4)
2.2 -Rename section to Neural Posterior Estimation (page 4) -Remove most mentions of BayesFlow (whole paper)
2.3 -Discuss higher dimensional latent space (page 7&8)
2.4 - Give more information on training scheme (page 8) - More context on how the summary is adjusted (page 9)
2.5 - Discuss SBC much more (page 10)
Section 3:
3.2 Point out that SBC is not possible with MCMC (page 14)
Current status:
Reports on this Submission
Report #1 by Anonymous (Referee 1) on 2025-3-19 (Invited Report)
- Cite as: Anonymous, Report on arXiv:2401.04174v2, delivered 2025-03-19, doi: 10.21468/SciPost.Report.10856
Strengths
- Clear presentation of simulation codes and statistical approach
- Statistical soundness of proposed analysis
- Clear description of results
Weaknesses
- It remains unclear if the proposed approach gives better or faster results than a power-spectrum based analysis.
Report
In their reply, the authors define optimality as obtaining posterior distributions that “use all of the information in the data and are well-calibrated.” They argue that SBC tests this and that passing the SBC test implies optimal information usage. However, this is not correct.
SBC only tests the consistency of the posterior given a particular data summary—here, the learned CNN summary. Even if the summary is entirely uninformative and the posterior is simply the prior, SBC would still be passed perfectly. This can be seen by replacing the posterior with the prior in the SBC algorithm cited in the paper. Thus, SBC cannot be used to claim that information is optimally used (i.e., that the CNN-derived summaries are sufficient).
Furthermore, this is distinct from the issue of underconfidence, which refers to posteriors being too wide given a specific data summary. SBC assesses whether posteriors are consistent with a given summary but does not evaluate the optimality of that summary itself.
Additionally, SBC tests posterior properties averaged over data realizations. It is possible that for some observations, posteriors exhibit positive bias, while for others, they are negatively biased—effects that may cancel out in the SBC histogram.
Otherwise, the paper is in good shape.
Requested changes
I would like to ask the authors to clarify the role of SBC in the context of optimality in both the abstract and the main text. They make strong claims about the optimality of their setup, but I do not see these fully supported by the tests presented in the paper. The paper is highly valuable and certainly deserves to be published as soon as possible. I also appreciate the use of SBC, which is an excellent method. However, SBC has specific caveats—it does not test the sufficiency of data summaries and may overlook varying biases—that should be explicitly described and properly reflected in the abstract, discussion and in the conclusions.
Recommendation
Ask for minor revision
Author: Benedikt Schosser on 2025-03-21 [id 5303]
(in reply to Report 1 on 2025-03-19)Dear Referee, thank you for the kind words, and we appreciate the feedback. We included the desired clarifications as well as reduced and clarified the usage of the word "optimal" in the paper. The exact modifications are in the list of changes.