Spey: smooth inference for reinterpretation studies

Presents a tested and accessible solution to a common and important set of problems in the field.

Is well written and clear and comes with links to well-documented open-source code.

Focusses too much on exclusion limits, at least in the introduction. More emphasis early on on potential discovery, and on fitting parameters (as shown in the later neutrino example) would improve the paper.

Reproducible interpretation and re-interpretation of statistical results based on experimental data is a topic of growing and crucial importance. If sucessfully adopted and supported, this package could facilitate significantly better communication and rigour in this area.

1. See my comment on the introduction above.
2. Eq 2 and 3 are a bit confusing in terms of the subscript/superscript on n. The index i moves from subscript to superscript, and seems to maybe sum over s and b at one point and over bins at the next. Also, lambda is said to be a function of n_s and n_b but is shown as a function of mu and theta. I realise this is pretty standard stuff but please check for clarity. 3 page 5 please clarify what "units determined by the user" means. How are these units used and reported by Spey? Does Spey even know or care what they are?

In this paper, the author presents a new software tool designed for statistical inference in reinterpretation studies. The tool's versatility enables users to create various likelihood approximations, integrate pre-existing packages, and combine these likelihoods under multiple assumptions. Additionally, the author introduces a new likelihood approximation technique to improve the precision of previously proposed simplified approaches. All techniques are rigorously tested throughout the paper, and their respective limitations are clearly delineated. The improvements and techniques provided in this paper have been showcased to enhance the limits of an MSSM scenario, demonstrating that even with existing analyses, one can extend the boundaries of current searches.

Furthermore, the author allows users to extend the coverage of the package in terms of the versatility of likelihood proposals and adds the means to cite their independent extensions through the package. This feature enhances the tool's utility and promotes collaboration and the sharing of new statistical methodologies. Hence, I believe the manuscript is valuable for HEP and any empirical sciences.

I have a minor structural observation concerning the paper. While the conclusion is well-structured, providing a concise summary and outlining future steps, a similar organisation is missing in the introduction. I recommend that the author revise the introduction to underscore the current challenges within the field and elucidate how this tool offers a viable solution to these issues.