Class Imbalance Techniques for High Energy Physics

I have made the following changes based on the referee report of the anonymous referee:
1.) I am now decaying the $W$ bosons
2.) The threshold on $m_{jj}$ has been raised to 850 GeV

I have made the following changes based on the referee report of Tilman Plehn:
1.) I revised a paragraph in Section 2 discussing the precison-recall curve so that it goes into more detail.
2.) I added several paragraphs to Section 2 going into greater detail on focal loss and what it is useful for. In a nutshell it down weights easy-to-classify examples in the loss function so that training is more focused on the hard-to-classify examples.
3.) What was footnote 1 has been removed
4.) Feynman diagrams have been added, see Figure 1
5.) Figure 2 has been added, which gives both the precision-recall curve and the ROC curve for the models in Table 1.
6.) The cuts and feature engineering have been updated to match for Ref. [46] as closely as possible. Good agreement is found when a fair comparison can be made. I find the AUC to be higher for $\Delta\phi_{jj}$ than Ref. [46]. However this is due to my treatment of partons from the hard scattering process as jets.
7.) There is no problem with the signal-background separation in (what is now) Figure 3. The mean predicted probability for a classifier with an unweighted loss function trained on an imbalanced dataset is $r$, the imbalance ratio. I would take complete signal-background separation in the training dataset to be a sign of overfitting if such behavior was not also observed in the validation dataset, which it’s not in this case. Balancing the training set moves the mean value to 0.5. This can be seen in the upper right panel from the balanced random forest. Weighting the loss function with the inverse of the class frequencies also moves the mean value to 0.5. Focal loss is intermediate between these two scenarios, $r$ and 0.5, as can be seen in the bottom right panel.
8.) Performance is reported as the mean $\pm$ the standard deviation of the five folds of the cross validation. I added this to the caption of the table. It was already in the text. My apologies for the confusion around the use of the word significant. This is certainly not a Higgs boson level discovery, nor even 3-sigma evidence for something. Apparently I’ve been out of physics for too long. I was approaching from of the point of view how large of an effect would need to be observed to convince my business stakeholders it would be worth rolling this out. The manuscript has been revised accordingly.
9.) The global fit reference now includes citations for 9 papers, Ref. [56-64], and I added a footnote about 0904.3866.
10.) I added a paragraph to end of Section 4.1 attempting to clarify these questions.
11.) The precision-recall curves for the tests in Table 2 have been added in Figure 3. The curves are consistent with the finding in Table 2, and provide another way of demonstrating that class imbalance techniques can lead to a performance in charm tagging efficiency.
12.) A direct comparison against (what is now) Ref. [76] is not possible given the backgrounds considered in the two works are different.

Additional Notes:
- 3 figures have been added
- 10 references have been added