SciPost Submission Page
SModelS v2.3: enabling global likelihood analyses
by Mohammad Mahdi Altakach, Sabine Kraml, Andre Lessa, Sahana Narasimha, Timothée Pascal, Wolfgang Waltenberger
This is not the latest submitted version.
This Submission thread is now published as
Submission summary
| Authors (as registered SciPost users): | Sabine Kraml · Andre Lessa · Timothée Pascal · Wolfgang Waltenberger |
| Submission information | |
|---|---|
| Preprint Link: | https://arxiv.org/abs/2306.17676v1 (pdf) |
| Code repository: | https://github.com/SModelS/smodels/releases |
| Data repository: | https://doi.org/10.5281/zenodo.8086949 |
| Date submitted: | 2023-07-03 15:08 |
| Submitted by: | Kraml, Sabine |
| Submitted to: | SciPost Physics |
| Ontological classification | |
|---|---|
| Academic field: | Physics |
| Specialties: |
|
| Approaches: | Computational, Phenomenological |
Abstract
We present version 2.3 of SModelS, a public tool for the fast reinterpretation of LHC searches for new physics on the basis of simplified-model results. The main new features are a database update with the latest available experimental results for full Run 2 luminosity, comprising in particular the full suit of available electroweak-ino searches, and the ability to combine likelihoods from different analyses. This enables statistically more rigorous constraints and opens the way for global likelihood analyses for LHC searches. The physics impact is demonstrated for the electroweak-ino sector of the minimal supersymmetric standard model.
Current status:
Reports on this Submission
Report #1 by Anonymous (Referee 3) on 2023-8-3 (Invited Report)
- Cite as: Anonymous, Report on arXiv:2306.17676v1, delivered 2023-08-03, doi: 10.21468/SciPost.Report.7575
Strengths
1- Improvement of the exclusion region of supersymmetric searches by combination of measurements (simplified)
2- Extended coverage of analyses compared to the last publication
Weaknesses
1- Section 2.2: the criteria to declare two analyses "approximately" uncorrelated is not defined.
2- Section 2.2: explanation of the limitations of the approximations should be clarified.
Report
The publication addresses the important task of the combination of experimental analyses in simplified scenarios. The updates of the software package itself as well as the database of useable analyses is more than an incremental step, justifying a new publication.
I recommend publication after the questions have been addressed.
Requested changes
1- Which degree of correlation is accepted in the combination and how is it evaluated?
2- The SRs are quoted as being orthogonal, but is this sufficient as the background estimation in the SR may be based on control regions? Isn't it necessary to also ensure that the CR of analysis A is orthogonal to SR of analysis B et cetera?
3- ULs depend on the statistical and systematic errors. Section 2.2 and equation 6 discuss the combination of orthogonal analyses. Couldn't orthogonal have correlated errors, eg the error on the measurement on the integrated luminosity? How does the combination ensure that such errors are not reduced by combining N analyses?
Report #2 by Anonymous (Referee 4) on 2023-8-2 (Invited Report)
- Cite as: Anonymous, Report on arXiv:2306.17676v1, delivered 2023-08-02, doi: 10.21468/SciPost.Report.7592
Report
The main purpose of the paper "SModelS v2.3: enabling global likelihood analyses" by Altakach ea. is to report about new features of a public tool for fast reinterpretation of LHC searches, SModelS. It is a valuable contribution that neatly summarises the new features. The paper is well written and the message is clear. An example of a combination analysis can be easily reproduced even for an inexperienced user.
The main new feature is a possibility to combine "orthogonal" LHC searches. Orthogonality would typically mean, according to the paper, combination of ATLAS and CMS results or hadronic and leptonic searches. The decision about validity of combination is left to a user. Perhaps it would be worthwhile to expand the recommendations regarding this point.
The combination feature is illustrated using 2 newly added searches CMS-SUS-21-002 and ATLAS-SUSY-2018-41, which target boosted hadronic decays of W/Z/H + MET . In Figs. 3-5 a result of scan is presented. A problematic thing here are the orange (yellow(khaki?)/blue in Fig. 5) points along m_chargino=m_LSP line. This is very misleading even though there is a comment in the text that these are actually Higgsino and the exclusion is in fact due to heavy winos. I think the problem is that a MSSM-parameters based model is presented in a way simplified model are being shown. Authors should consider presentation of a plot(s) in terms of M1/M2/mu parameters instead.
Regarding the example discussed in Sec. 2, I think it would be interesting to extend discussion with a pair of points marked as blue/red in Fig. 5, e.g. those at ~(700, 200) GeV. Finally, most of the dots are yellowish to me-this is something I see both in print and on screen-but the caption says "khaki" which means dull "greenish"?
Finally, the last sentence of Sec. 4 says "combination of analyses (...) leads to statistically more robust limits". I am not sure how to interpret this statement? In Fig. 5 the number of newly excluded and unexcluded points seem to be rather similar. I do not see this statement justified. Clearly, there is an obvious impact of combined SRs seen in Fig. 2, but this is quite a different effect. While I understand "robustness" argument, the chosen example does not immediately support this statement.
To summarize, the paper provides a quick reference guide to the recent changes in SModelS. The new features are illustrated with an interesting physics example. This warrants publication after minor changes in discussion and presentation of the results, as specified above.
Requested changes
1. Consider a change in presentation of results of the scan in Figs. 3-5. At least a comment in the caption is necessary. Fix coloring issue in Fig. 5.
2. Consider adding another benchmark illustrating combination of analyses.
3. "Robustness" should be elaborated or the statement modified.
Author: Sabine Kraml on 2023-08-22 [id 3915]
(in reply to Report 1 on 2023-08-03)Regarding point 1, "which degree of correlation is accepted in the combination and how is it evaluated":
We are not evaluating the degree of correlation of analyses. SModelS only provides the capability of combining analyses. The working assumption here is that the correlation of systematic uncertainties for combinable analyses are negligible. Which analyses are actually combinable under this assumption has to be defined by the user. This also applies to point 2, that one should also ensure that the CRs of analysis A are orthogonal to the SRs of analysis B, etc. We add a line of caution in the revised manuscript.
Regarding point 3, it is true that orthogonal analyses can still have correlated errors, e.g. the error on the measurement on the integrated luminosity. In general the effect of neglecting this type of uncertainty is considered negligible compared to other approximations in the whole simplified model approach. However, the user still has to apply common sense; e.g., they should not combine N analyses of very limited sensitivity. Again, the tool provides the technical capability, but it is up to the user to use it sensibly.