Loop Amplitudes from Precision Networks

Report 1:

1- In section 2, the authors describe the dataset they use, which is taken from [8]. A set of cuts is then applied, but no justification for them is presented. I understand that one must make a choice in the application of cuts to avoid singularities. However, since later on the paper shows that the performance deteriorates for large amplitude, i.e. presumably close to the cut boundaries, one might be led to believe that the cuts are selected to enhance performance. It would be useful if the BNN could for instance be trained on looser or tighter cuts, to find out of the qualitative conclusions would be different.

-> Originally, the cuts were meant to mimick the detector acceptance, but at least for the 2-jet case they also cut off singularities. We have added more information to the text.

As a side-question, why are the amplitudes real rather than complex numbers?

-> We corrected this, only the squared amplitued we learn are real. We corrected this in the text as well.

2- Why do the authors choose to make use of a 20-dimensional representation of the phase space, while its actual dimension is only 7-dimensional (2 initial-state momentum fractions, and 3×3−4=5 final-state components)? Would a lower-dimensional representation not improve performance?

-> The goal of the paper was to show that it is possible to learn the amplitudes precisely and at face value, for an actual application we will exploit this additional handle, and we have early studies showing that this does help. We added this discussion to the text.

3- At the beginning of section 3, it is claimed that the MSE loss limits the performance of implementations without Bayesian architectures. What is the justification for this claim? In particular, the BNN loss also has a MSE term that is responsible for incentivizing the BNN to match predicted and real amplitudes.

-> We have clarified that statement in this paragraph.

4- Under \textbf{Bayesian networks and uncertainties}, there is a sentence 'By definition, the Bayesian network includes a generalized dropout and an explicit regularization term in the loss, which stabilize the training. I believe the authors mean, respectively, the sampling over the network weights and the simultaneous output of σmodel, but this is certainly not clear to the reader at this point in the text, and may still be unclear after reading the full paper if one does not have the prerequisite domain knowledge.

-> We clarified this sentence and added a reference to a long and pedagogical introduction to our Bayesian network setup. We agree that this paper requires some domain knowledge and believe that the issue can be solved this way.

5- Further down, there is a sentence 'We implement the variational approximation as a KL divergence.' This sentence has similar issues of requiring the reader to be familiar with the lingo. I think the explanation should state that p(ω|T) is the real distribution that is unknown, so you introduce a trainable variational approximation q(w), and define a loss function that minimizes the difference.

-> We adopted the referee's suggestion for this explanation.

6- Before eq. 7, it is stated that the evidence can be dropped 'if the normalization condition is enforced another way'. I do not believe that this 'other way' is clarified elsewhere. I believe that it should just say that the evidence term does not depend on any trainable parameters, and can thus be dropped from the loss.

-> We changed the wording, to make clear that we enforce the normalization by construction.

7- While the authors already attempt to do so, I think the distinction between σmodel and σpred could be clarified further. My understanding is that...

-> We are not sure what exactly the referee means, but we made an attempt to explain this distinction more clear after Eq.(9).

8- The next paragraph then makes a link with the actual implementation of the BNN. However, it says 'This sampling uses the network-encoded amplitude and uncertainty values...', while the reader does not even know yet that the model is meant to predict those. I would reorder some elements of this section, by first saying that the BNN is essentially meant to model p(A|x,ω), ¯A and σmodel are its mean and variance, and you can thus implement it as a feed-forward network that predicts that mean and variance as a function of x,ω.

-> We have followed this suggestion and adapted the paragraph before Eq.(12).

9- Under \textbf{Network architecture}, is the uncertainty enforced to be positive after the final 40→2 linear layer through some activation? Does the mean have an activation?

-> We added additional information to the paragraph.

10- Figures 2 and 3, it took me a while to figure out what the differences were between the plots. Maybe in figure 2 add some text like '(train)' and '(test)' underneath the gg→γγg labels. In figure 3 it is especially confusing that the weight-dependent pull is shown above the weight-independent pull, while the reverse is true in the text. I would suggest splitting it into two figures, and adding some text to indicate which pull is being plotted (even though it is also shown on the x-axis).

-> We have split Fig.3 and adjusted all figure captions.

11- The results before boosting show a bias towards positive values. It is unclear to me if this is also captured by the model uncertainty. Please elaborate.

-> We added a brief discussion of this effect, now at the end of the section and a motivation for the boosting.

12- I think the reference to [14] should be supplemented by other works that use this technique, like 2009.03796, 2106.00792. The comma after [14] should also be a period.

-> We added the two references for re-weighting.

13- Above section 4.2, 'over-training though loss-boosting' → 'over-training through loss-boosting'

-> Changed.

Report 2:

1 - Section 2, what quarks run in the loop? Are top/bottom quarks included and, if so, are the masses fixed or left as free parameters of the network?

-> We use the amplitudes from Ref.[9], generated with all quark flavors, but do not attempt to infer the model parameters. Thinking about, this would be an interesting question, but beyond the simple precision-regression discussed here.

2 - Throughout the draft the authors appear to use the word "amplitudes" to mean "samples of the amplitude"/"amplitude samples". In typical usage, one might say that the gg -> gam gam g process has a single amplitude (or perhaps several helicity or color-ordered amplitudes). However, the authors refer to "90k training amplitudes". Here I suspect they mean a single (or few) amplitudes sampled at 90k phase-space points.

-> We agree that our wording is not well-defined, so we now define it on p.3. The referee is right, this is what we mean and now say.

3 - Section 2, the authors write that each data point consists of a real amplitude. Usually, an amplitude is a complex number. I suspect the authors are referring to the squared amplitude. If this is correct, this should be stated more clearly.

-> We clarified this aspect.

4 - The authors use a highly-redundant 20-dimensional parametrization. Why do the authors not use, for example, just the independent Mandelstam invariants? Can the authors demonstrate that a lower dimensional parametrization is not better for their training (as one might naively expect)?

-> We now mention in the paper that the goal was to learn the squared transition amplitude as precisely as possible, and with minimal pre-processing. An optimized pre-processing will most likely be the topic of a follow-up paper, and we do have initial results indicating that a lower dimensionality helps quantitatively, but does not change the qualitative results shown in this paper.

5 - Below Eq(6) the authors write "if we enforce the normalization condition another way" in order to justify dropping the final term in Eq(6). The exact procedure is not clear to me, where is the normalization condition enforced? Perhaps this sentence can be reworded for clarity.

-> As requested also be Referee 1 we re-worded this paragraph to make it clear.

6 - In the first line of Eq(8) the authors refer to p(A|w,T). Perhaps I am misunderstanding or missing a step, is this a typo or does this indeed follow from Eq(4)?

-> Thank you for pointing this out, we corrected the typo.

7 - In the second paragraph of "Network architecture" the authors write "The 2->3 part of the reference process in Eq(25)", do they mean to reference instead Eq(1)?

-> Again, thank you for pointing this out, it should be Eq.(1).

8 - Comparing the \delta^(test) panel of Figure 2 with that of Figure 6 it naively appears that significantly more test points fall into the overflow bin (for the largest 0.1% and 1% of amplitude values) after loss-based boosting. Could the authors please comment further on this and to what extent do they consider it a problem?

-> We consider this a problem and one of the motivations to move towards the preformance boosting, now clarified in the text.

9 - In Figure 8, although the \delta^(test) performance does seem to be broadly improved, again, significantly more test points fall into the overflow bin than in Figure 2 or Figure 6. Could the authors comment further on this?

-> Our main point is that performance boosting provides comparable precision for all amplitudes, not just the majority of amplitudes. We clarify this point in the discussion of (now) Fig.9.

10 - Sec 5, final paragraph, the authors write that "The uncertainties for the training data still cover the deviations from the truth, but unlike the central values this uncertainty estimate does not generalize correctly to the test data". If I have understood Figure 10 correctly, this fact is visible in the bottom (model/test) plot of the lower panel, where the green band no longer reflects the true uncertainty (which is presumably ~the grey band). One of the strengths of the authors' approach is that it provides not only an estimate of the amplitude but also of the uncertainty of the estimate. The authors write "This structural issue with process boosting could be ameliorated by alternating between loss-boosting and performance-boosting", can the authors demonstrate that an additional loss-boosting step would improve the quality of the uncertainty estimate without reversing the improvement in the performance? This would be a very strong and convincing argument for using their proposed procedure.

-> We agree with the referee, and we have evidence that for individual trainings the alternating training works, but unfortunately we do not have a reliable algorithm or method which we could present in this paper. We are at it, through...

11 - Sec 6, there is a typo "boosteing"

-> Corrected, thank you!