SciPost Submission Page
Applying reinforcement learning to optical cavity locking tasks: considerations on actor-critic architectures and real-time hardware implementation
by Mateusz Bawaj, Andrea Svizzeretto
Submission summary
| Authors (as registered SciPost users): | Mateusz Bawaj |
| Submission information | |
|---|---|
| Preprint Link: | https://arxiv.org/abs/2509.14884v1 (pdf) |
| Date submitted: | Sept. 19, 2025, 8:54 a.m. |
| Submitted by: | Mateusz Bawaj |
| Submitted to: | SciPost Physics Proceedings |
| Proceedings issue: | The 2nd European AI for Fundamental Physics Conference (EuCAIFCon2025) |
| Ontological classification | |
|---|---|
| Academic field: | Physics |
| Specialties: |
|
| Approach: | Computational |
The author(s) disclose that the following generative AI tools have been used in the preparation of this submission:
Spell checking and grammar correction. Gen AI products have been verified by the authors.
Abstract
This proceedings contains our considerations made during and after fruitful discussions held at EuCAIFCon 2025. We explore the use of deep reinforcement learning for autonomous locking of Fabry-Perot optical cavities in non-linear regimes, with relevance to gravitational-wave detectors. A custom Gymnasium environment with a time-domain simulator enabled training of agents such as deep deterministic policy gradient, achieving reliable lock acquisition for both low- and high-finesse cavities, including Virgo-like parameters. We also discuss possible improvements with Twin Delayed DDPG, Soft Actor Critic and meta-reinforcement learning, as well as strategies for low-latency execution and off-line policy updates to address hardware limitations. These studies lay the groundwork for future deployment of reinforcement learning-based control in real optical setups.
Current status:
Reports on this Submission
Report
Some concrete comments:
Intro:
Reference for Gymnasium enviromnet?
"The agent is tested on both low- and high-finesse cavities, including those matching the parameters of the Virgo interferometer, demonstrating reliable lock acquisition even under strong nonlinearity.": This claim might benefit from a figure or table to illustrate the achieved performance
Improvements:
"computationally too heavy for us" -> can there be put a number next to this? What is the binding factor? eg: "would require a 100x increase in training time" could be interesting for the reader, if such a statement can be made
Recommendation
Ask for minor revision