SciPost logo

SciPost Submission Page

Understanding Event-Generation Networks via Uncertainties

by Marco Bellagente, Manuel Haußmann, Michel Luchmann, Tilman Plehn

This is not the latest submitted version.

This Submission thread is now published as

Submission summary

Authors (as registered SciPost users): Marco Bellagente · Tilman Plehn
Submission information
Preprint Link: https://arxiv.org/abs/2104.04543v1  (pdf)
Date submitted: 2021-05-03 15:27
Submitted by: Bellagente, Marco
Submitted to: SciPost Physics
Ontological classification
Academic field: Physics
Specialties:
  • High-Energy Physics - Phenomenology
Approaches: Computational, Phenomenological

Abstract

Following the growing success of generative neural networks in LHC simulations, the crucial question is how to control the networks and assign uncertainties to their event output. We show how Bayesian normalizing flow or invertible networks capture uncertainties from the training and turn them into an uncertainty on the event weight. Fundamentally, the interplay between density and uncertainty estimates indicates that these networks learn functions in analogy to parameter fits rather than binned event counts.

Current status:
Has been resubmitted

Reports on this Submission

Report #1 by Anonymous (Referee 1) on 2021-6-4 (Invited Report)

  • Cite as: Anonymous, Report on arXiv:2104.04543v1, delivered 2021-06-03, doi: 10.21468/SciPost.Report.3020

Strengths

1. Detailed explanations for the features seen
2. Thorough investigation of the uncertainties in BINNs

Weaknesses

1. Lots of small grammatical errors, in terms of punctuation and proper use of plural

Report

The authors study the uncertainties associated with invertible neural networks (INN) through the use of promoting the deterministic weights to distributions. These weights are then updated through the use of Bayesian statistics. This study opens a new approach to study the accuracy of the INNs, and to help pinpoint regions of phase space that need more data to accurately reproduce the probability distribution.

Overall, the article is for the most part very well written, with a few places in which the authors are unclear detracting from the reader's understanding of the work. Additionally, there are a fair number grammatical errors, but these do not subtract from the overall presentation of the material.

The authors take the time to study the features that are seen in their toy models in such a way to help gain some insight for real world results. The authors provide sufficient details of the hyper-parameters to allow the reader to reproduce their work.

The requested changes can be found in the section below. With the changes below, I believe that this article is suitable for publication in SciPost Physics.

Requested changes

1. Page 3, in the paragraph starting: "One contribution to the error budget are systematic ...", the authors should change this to read "Two contributions to the ...," and the authors should switch the subscripts for $\sigma_{th/sys}$ to $\sigma_{sys/th}$ to be consistent with the previous part of the sentence.
2. In the same paragraph as 1, the sentence starting, "For example accounting for large, momentum-dependent logarithms ..." is unclear in what the authors are trying to convey. I would recommend reworking this sentence.
3. On page 4 and page 15, the authors use the word "effectively," but the word "efficiently" would be better suited in these situations.
4. Page 8, the authors state that the "always remove the phase space boundaries," but never specify what they consider the boundary. Is it just the exact endpoint, or some region around each endpoint, or something else?
5. Page 9, in the sentence "If we now assume..." the authors have the sentence fragment "as it will turn out useful," in which the meaning is unclear. Do they mean that the fit will be useful or the interpretation of the network acting like a fit? The authors should clarify this sentence fragment.
6. Page 10, the two sentences after equation 23 can be combined into a single sentence to make the meaning more clear. The first sentence is clear enough, but the second sentence is not clear.
7. Page 12 and page 13, the authors reference the lower panels of Fig. 4 and Fig. 6 respectively, but the Figures are laid out horizontally and not vertically. The text should be changed to reflect this.
8. Page 12, in the same sentence as point 7, the authors mention a "2-parameter fig," I believe this should read "2-parameter fit."
9. Page 13, in the same sentence as point 7, the authors show a one-parameter fit, but spend the previous few sentences discussing the two different contributions and when different ones dominate. They should either do a two-parameter fit, or give some justification on why they do not include the second parameter.
10. Page 17, the authors mention in two separate places a comparison between Fig. 8 and Fig. 4. It would be beneficial for the reader for the authors to overlay the curves from Fig. 4 on Fig. 8 to make their points more clear and easy for the reader to see.
11. Overall, the paper could benefit from a quick read through to check for appropriate use of plural verbs with plural nouns, and other minor grammatical errors that do not take away from the overall message.

  • validity: high
  • significance: good
  • originality: high
  • clarity: high
  • formatting: excellent
  • grammar: good

Author:  Marco Bellagente  on 2021-10-01  [id 1795]

(in reply to Report 1 on 2021-06-04)
Category:
remark
question
answer to question
correction

We kindly answer to the requested changes:

  1. Page 3, in the paragraph starting: "One contribution to the error budget are systematic ...", the authors should change this to read "Two contributions to the ...," and the authors should switch the subscripts for sigma_{} to be consistent with the previous part of the sentence.

------> Corrected

  1. In the same paragraph as 1, the sentence starting, "For example accounting for large, momentum-dependent logarithms ..." is unclear in what the authors are trying to convey. I would recommend reworking this sentence.

------>The sentence has been rephrased in a clearer way

  1. On page 4 and page 15, the authors use the word "effectively," but the word "efficiently" would be better suited in these situations.

------> Corrected

  1. Page 8, the authors state that the "always remove the phase space boundaries," but never specify what they consider the boundary. Is it just the exact endpoint, or some region around each endpoint, or something else?

------> The exact meaning has been written explicitly

  1. Page 9, in the sentence "If we now assume..." the authors have the sentence fragment "as it will turn out useful," in which the meaning is unclear. Do they mean that the fit will be useful or the interpretation of the network acting like a fit? The authors should clarify this sentence fragment.

------> Corrected

  1. Page 10, the two sentences after equation 23 can be combined into a single sentence to make the meaning more clear. The first sentence is clear enough, but the second sentence is not clear.

------> Corrected

  1. Page 12 and page 13, the authors reference the lower panels of Fig. 4 and Fig. 6 respectively, but the Figures are laid out horizontally and not vertically. The text should be changed to reflect this.

------> Corrected

  1. Page 12, in the same sentence as point 7, the authors mention a "2-parameter fig," I believe this should read "2-parameter fit."

------> Corrected the typo

  1. Page 13, in the same sentence as point 7, the authors show a one-parameter fit, but spend the previous few sentences discussing the two different contributions and when different ones dominate. They should either do a two-parameter fit, or give some justification on why they do not include the second parameter.

------> We added a sentence specifying that we originally performed a 2-parameters fit, but as it performed very similarly to a single-parameter fit, eventually we decided to focus on the dominant effect

  1. Page 17, the authors mention in two separate places a comparison between Fig. 8 and Fig. 4. It would be beneficial for the reader for the authors to overlay the curves from Fig. 4 on Fig. 8 to make their points more clear and easy for the reader to see.

------> We attach this figure, but we believe that our current set of plots is easier to understand.

  1. Overall, the paper could benefit from a quick read through to check for appropriate use of plural verbs with plural nouns, and other minor grammatical errors that do not take away from the overall message.

------> Thank you for pointing this out, we went through the draft carefully.

Best regards

Attachment:

Fig_8_modifed.pdf

Login to report or comment