SciPost Submission Page
What's Anomalous in LHC Jets?
by Thorsten Buss, Barry M. Dillon, Thorben Finke, Michael Krämer, Alessandro Morandini, Alexander Mück, Ivan Oleksiyuk, Tilman Plehn
This is not the latest submitted version.
This Submission thread is now published as
|Authors (as registered SciPost users):
|Thorsten Buss · Barry Dillon · Thorben Finke · Tilman Plehn
Searches for anomalies are a significant motivation for the LHC and help define key analysis steps, including triggers. We discuss how LHC anomalies can be defined through probability density estimates, evaluated in a physics space or in an appropriate neural network latent space, and discuss the model-dependence in choosing an appropriate data parameterisation. We illustrate this for classical k-means clustering, a Dirichlet variational autoencoder, and invertible neural networks. For two especially challenging scenarios of jets from a dark sector we evaluate the strengths and limitations of each method.
Submission & Refereeing History
You are currently on this page
Reports on this Submission
- Cite as: Anonymous, Report on arXiv:scipost_202207_00012v1, delivered 2022-10-23, doi: 10.21468/SciPost.Report.5960
Thank you for taking into account my feedback and I am very sorry for the delayed response! I have now read the full paper and have a number of comments that should be addressed before I can recommend publication in SciPost Physics. Please note that I still think the intro/conclusions could be improved, in particular in relation to what specifically is studied in the paper.
- This paper includes a number of interesting comparisons, but the connection between the framing and the results could be improved. At its core, this paper studies a number of unsupervised methods (none of which are new I believe) for particular BSM signals and I think it would be helpful to frame the paper in that way, instead of over claiming some deeper/broader goal. I would state early on somewhere that the goal of this paper is to examine classical and deep machine learning methods and how preprocessing affects their sensitivity for a class of intersting/difficult BSM models.
- Sec. 2.3: Don't both your standardization and de-sparcification actually reduce the information content of the jets? What is the tradeoff between loss in sensitivity from information distortion and ease of training for the NN? (maybe the former could be quantified using fully supervised networks?)
- "Looking at LHC physics these concepts can be developed most easily for QCD jets" -> I agree these are available in large numbers and are less complex than full events, but they are more complex than everything else. Maybe this should say that jets are useful because they are so complex that it is "easy" for BSM to hide in them?
- Sec. 1, paragraph 2: this paragraph comes out of nowhere! It would be useful to give a bit more context before diving into details of AEs. For example, "supervision" is not defined. I would also avoid "is the simplest" which is a matter of opinion and I think most people would even agree it is not true (I would say things like kNN is even simpler). Paragraph 4 also seems to imply ("Based on these practical successes") that the AE work came first in the story of anomaly detection in HEP, which is not true.
- Sec. 1, paragraph 3: "is mapped to a low-dimensional latent distribution" -> this is not quite correct; for VAEs, the latent space need not be smaller than the input space (although in practice, it usually is).
- "the concept of outliers is not defined," -> this seems too strong. Just because p > 0 for everything doesn't mean you can't have outliers. Please consider rephrasing.
- "a simple, working definition of anomalous data is an event which lies in a low-density phase space region" -> this is a fine (albeit coordinate-dependent) definition, but it is not the only one. You discuss other methods later in this paragraph, but you seem to imply that they are variations of the low density assumption (which is not true). Please consider rephrasing. This is also an issue of the next paragraph, e.g. "This way, anomaly searches are fundamentally linked to density estimation" is also not true for all AD searches.
- Intro: Pythia is mentioned without a citation.
- "Dirichlet VAEs (DVAEs) outperform for example standard VAEs" -> I don't think you know this generically - please clarify that this has been shown for specific models (this is a problem with model independent searches!)
- Intro: why call them INNs? Don't you just mean normalizing flows? (also no citation here). It is fine to use a particular NF implementation of course, but it seems strange to specialize already here.
- "The bottom line of our comparison is that density estimation really is at the basis of ML- based anomaly searches in LHC jets." -> please rephrase (see above).
- Cite as: Anonymous, Report on arXiv:scipost_202207_00012v1, delivered 2022-09-12, doi: 10.21468/SciPost.Report.5687
Let me apologize to the authors for the delay in submitting this referee report. I had erroneously thought I had already submitted my report at the end of August, but clearly did not.
In any case, the authors have responded to all of my concerns in an adequate way. I am happy to recommend publication in one of the SciPost journals.
Whether it should be SciPost Physics or SciPost Physics Core, I leave as a decision to the editor. I have no concerns about the scientific content of the work, and the authors indeed show state of the art results for certain tasks. My concern about the utility of implicit hypothesis testing remains, though the authors understand this limitation of their studied methods.