SciPost Submission Page

How to GAN away Detector Effects

by Marco Bellagente, Anja Butter, Gregor Kasieczka, Tilman Plehn, Ramon Winterhalder

Submission summary

As Contributors: Tilman Plehn · Ramon Winterhalder
Arxiv Link: https://arxiv.org/abs/1912.00477v2
Date submitted: 2020-01-07
Submitted by: Winterhalder, Ramon
Submitted to: SciPost Physics
Discipline: Physics
Subject area: High-Energy Physics - Phenomenology
Approaches: Theoretical, Computational

Abstract

LHC analyses directly comparing data and simulated events bear the danger of using first-principle predictions only as a black-box part of event simulation. We show how simulations, for instance, of detector effects can instead be inverted using generative networks. This allows us to reconstruct parton level information from measured events. Our results illustrate how, in general, fully conditional generative networks can statistically invert Monte Carlo simulations. As a technical by-product we show how a maximum mean discrepancy loss can be staggered or cooled.

Current status:
Editor-in-charge assigned


Submission & Refereeing History

Submission 1912.00477v2 on 7 January 2020

Login to report


Comments

Andy Buckley on 2020-01-18
Category:
remark
objection
suggestion for further work

I read this with interest, although did not have the time to do so in full detail and hence this is an informal comment rather than official review report. It's technically rather nice, although I was under the impression that unfolding has been attempted and achieved with ML methods several times before, and it would be good to highlight the distinctions of this version.

It would also be good to engage with the physics and the fundamental statistics of event calculation and detector interaction. The claim to unfold to parton level will ring alarm bells for many who note that the degrees of freedom at parton level in an event generator record are usually not 100% physical, and so any such extraction will unavoidably bear biases from calculation schemes, as well as any semi-arbitrary choices in how a given generator represents its intermediate stages. A more careful choice of unfolding target would be a safely reconstructed event topology from final-state objects, avoiding the physical-ambiguity implications of a parton-level target. The hadronization process is usually recognised as the level that unfolding should not attempt to extrapolate beyond, the decay and detector interaction stages following it being regarded as sufficiently classical to be safe (as opposed to the QM dynamics of the partonic bit that precedes it). Maybe this is all known and the parton level used for convenience despite its issues -- which seems a trivial reason to introduce a whole avenue of criticism --but if so then please make the straw-man nature of the target clear in the text.

Of course, using final-state MC observables still introduces model dependence via the generator (which, by the way, isn't clearly specified beyond "using MadGraph5" -- with merging? how many extra partons? what parton shower and tune?) and the detector model. In practice, neither is a perfect match to data to be unfolded, and most time spent on unfolding goes into assessing non-closures due to model variations, rather than the estimation of a migration matrix that is the equivalent of the method discussed here. It seems like GANs could potentially help with parametrising systematics, too, but these issues at least need to be acknowledge if not actually addressed.

Finally, it seems that the GAN approach generates distributions that "look right", which is fine probabilistically... indeed to my mind it would be better to explicitly generate hundreds or thousands of GAN events per reco event, to map its likelihood distribution. But a single mapping of reco to truth event through the GAN is not "the answer" -- in general such a thing cannot be known by any method, even without the "parton level" issue. Maybe it represents the maximum-likelihood truth configuration for that reco event, or is just a single sample from the conditional likelihood distribution? Since there is room for confusion here, and potentially for calamity if a one-for-one GANning of reco events were attempted in a low-stats search region, it would be good also to discuss this "philosophical" issue of what unfolding means in the context of your method.