Generative Networks for Precision Enthusiasts

Major:

• "To increase the efficiency, we stick to the same network for the common..." -> I found this paragraph to be quite confusing. Do you train a mumu+j network and then use that as input to the mumu + jj/jjj training? Aren't there correlations between the jets such that p(j1 j2) != p(j2|j1) x p(j1)? -> We rewrote the paragaph to make it more clear. Your comment is correct, and these correlations are included through the conditioning structure, we hope this is now clear from the text.
• What is P_{ref} in Eq. 6? it is never used again nor explained. Is it just to make the argument of the log dimensionless? Maybe it would be easier /clearer to drop it? -> Sorry for not being clear, this is the density encoded in the training data, while P(x,c) is the learned model density. We changed ref' todata' to make that clear. We keep it, because it is needed later.
• What is \psi in Eq. 8? It is called "the network", but I thought you are predicting the full density P? -> We changed the formulas and the discussion, hopefully making things more clear now.
• "We emphasize that in our case this is not a problem with the non-trivial phase space topology" -> I did not understand this - isn't there actually a hole (in the topological sense) in the phase space? -> This is correct, but we find that this is not the problem of the network, instead, the network just ignores the subleading structure. We changed the sentence, hopefully this is clear now.
• Performance: Figs. 4,5,7 don't look so different, although it is hard to compare them directly since they are spread across many pages. Can you comment on this? -> We changed the discussion at the end of 3.1, to make this more clear. The results of Fig.4 are included in Fig.5, and for the Delta R distributions we point to the visible difference. In the discussion of Fig.7 we point out the advantage that DiscFlow provides unweighted events, unlike the two other methods.
• Eq. 18: This is mysterious to me - would you please provide some explanation? I understand that this downweights the w_D term when D is close to 1/2, as it should be if the learned and reference are similar. But why this functional form? Does adding the weight term and the D-dependence of alpha introduce a bias? Eq. 17 is no longer maximum likelihood so it is not obvious to me that its optimum is actuall the target density. -> We agree, and also say that the DiscFlow loss is indeed no more maximum likelihood globally. We changed the discussion after, now Eq(17) to explain the idea, and this functional form is just the simplest possible version.
• The paragraph afer Fig. 7 makes it sound like the discriminator and generator networks are not trained at the same time. Why not? -> The text was unclear, we train them together eventually after separate pre-training or burn-in.
• Fig. 10, right plot: Why does this also show sqrt(n) scaling? -> The black line on the right is shown for comparison to make it clear that the total uncertainty is now the dominating contribution to the total uncertainty, as described at the bottom of p.17.
• Fig. 12, left plot: Why does adding more data sometimes make the uncertainty worse? -> If the network does not learn the density, as it happens for the blue line, it also does not estimate the uncertainty correctly. That is the main problem with Bayesian network and their coupled extraction of density and uncertainty. We clarified this in the text.

Minor:

Introduction

• first paragraph: "analyses and inference" -> seems redundant; doing analysis is performing inference. -> removed inference.
• Please spell out LHC the first time it is used. Same for GANs and VAEs in the second paragraph. -> So we assume the referee is ATLAS or CMS? We would argue that someone who does not understand those abbreviations should stop reading our paper right there, but we are happy to oblige.
• second paragraph: "...potential to improve LHC simulations [1]" -> if I understand correctly, [1] is only about "physics generators", but does not include e.g. detector simulation (so I'm not sure it stands as a catch-all for this statement). -> We agree and moved Ref.[1] away, since we are not aware of a similar introduction to detector simulations.
• second paragraph: seems odd that [33-37] and [38-48] are not in the previous paragraph, which is about the full range of steps in the simulation chain. -> We shifted those references to the earlier paragraph.
• third paragraph: GANs and VAEs are not invertible. -> We were not clear in the wording, the simulation is inverted, the networks are not, they are conditional. We clarified this in the text.
• Eq. 3: jets are not particles - perhaps "each physics object is..." would be more precise? -> Thank you, changed it.
• "Magic Transformation" -> I find this a bit unsatisfying because this is something that is not automatically optimized, but manually put in by hand to fix an issue with the out of the box approach. What if your phase space is much bigger and there are more holes? Perhaps you could comment on how this generalizes? It may also be prudent to call it something other than "Magic" since that implies that it comes out of nowhere when in fact it is based on your physics intuition (!) -> We added a sentence to warn readers. We like magic because it is physics-driven, but we would not recommend anyone to rely on its systematic application or generalization.
• Eq. 15 is confusing - y_i is usually a label, but I think here it has the same structure as Eq. 16? What is B? I would think the first sum would be the INN output and the second sum would be over the physics simulator output? -> We adjusted the notation, one is generated and the other is training data.
• Fig. 10/12: Why is the x-axis mu and not n (the symbol used for per-bin event counts in the text) -> We use mu for the mean and sigma for the standard deviation of the per-bin event counts. We added the definition of mu to Eq. 18

1) The authors should more clearly state the envisioned application scenario for their proposed method, thereby addressing both strengths and limitations of their technique. To give an example, the process of choice is quite inclusive dilepton production, simulated with Sherpa. However, the authors then only consider the projection on fixed R=0.4 and pT>20 GeV jets and in fact only on exclusive jet multiplicities ... an horrendous simplification of the initial task. -> We noticed that we did not show any inclusive distributions, because they are less challenging that the exclusive ones, but we now state that out generator is jet-inclusive. We are not sure what aspects of an R-cut and a pT-cut is oversimplifying? We would argue that our method will work on all kinds of processes at the reconstruction level. We modified the beginning of 2.1 accordingly.

2) The authors claim to account for the right admixture of the different jet multiplicities, however, results are only presented for exclusive jet bins. It would be good to show observables that receive contributions from all channels, for example the HT distribution, i.e. the scalar sum of jet pT's for the inclusive sample. -> We added a comment on inclusive observables to the end of Sec.2 and included corresponding panels to Figs.4 and 7.

3) At several places the authors mention that their INN-generator is very fast, it would be nice to have a statement about the overall resources needed to generate events in their setup, though the comparison to the original generator might be misleading, see point 1. -> With our setup, generating 1M events takes roughly 2.5min on a GPU. However, because a comparison of the generation speed between different methods on different hardware might be misleading, and the focus of this work was exploring the toolbox provided by INNs for event generation, we did not include a benchmark of the generation time in the text.

Minor corrections:

• p11: '5-jet' -> '5-particle/object' -> done
• p17: 'correlation of the ... correlations' ??? -> done -p20: I do not see the claimed analogy to HO corrections, that needs further explanation -> We included a slightly longer discussion.