| Title |
A Physics-Based Simulator to Facilitate Reinforcement Learning in the RHIC Accelerator Complex |
| Authors |
- L.K. Nguyen, K.A. Brown, M.R. Costanzo, Y. Gao, M. Harvey, J.P. Jamilkowski, J. Morris, V. Schoefer
BNL, Upton, New York, USA
|
| Abstract |
The successful use of machine learning (ML) in particle accelerators has greatly expanded in recent years; however, the realities of operations often mean very limited machine availability for ML development, impeding its progress in many cases. This paper presents a framework for exploiting physics-based simulations, coupled with real machine data structure, to facilitate the investigation and implementation of reinforcement learning (RL) algorithms, using the longitudinal bunch-merge process in the Booster and Alternating Gradient Synchrotron (AGS) at Brookhaven National Laboratory (BNL) as examples. Here, an initial fake wall current monitor (WCM) signal is fed through a noisy physics-based model simulating the behavior of bunches in the accelerator under given RF parameters and external perturbations between WCM samples; the resulting output becomes the input for the RL algorithm and subsequent pass through the simulated ring, whose RF parameters have been modified by the RL algorithm. This process continues until an optimal policy for the RF bunch merge gymnastics has been learned for injecting bunches with the required intensity and emittance into the Relativistic Heavy Ion Collider (RHIC), according to the physics model. Robustness of the RL algorithm can be evaluated by introducing other drifts and noisy scenarios before the algorithm is deployed and final optimization occurs in the field.
|
| Funding |
Work supported by Brookhaven Science Associates, LLC under Contract No. DE-SC0012704 with the U.S. Department of Energy. |
| Paper |
download FR2AO04.PDF [1.397 MB / 7 pages] |
| Slides |
download FR2AO04_TALK.PDF [2.689 MB] |
| Cite |
download ※ BibTeX
※ LaTeX
※ Text/Word
※ RIS
※ EndNote |
| Conference |
ICALEPCS2023 |
| Series |
International Conference on Accelerator and Large Experimental Physics Control Systems (19th) |
| Location |
Cape Town, South Africa |
| Date |
09-13 October 2023 |
| Publisher |
JACoW Publishing, Geneva, Switzerland |
| Editorial Board |
Volker RW Schaa (GSI, Darmstadt, Germany); Andy Götz (ESRF, Grenoble, France); Johan Venter (SARAO, Cape Town, South Africa); Karen White (SNS, Oak Ridge, TN, USA); Marie Robichon (ESRF, Grenoble, France); Vivienne Rowland (SARAO, Cape Town, South Africa) |
| Online ISBN |
978-3-95450-238-7 |
| Online ISSN |
2226-0358 |
| Received |
04 October 2023 |
| Accepted |
05 December 2023 |
| Issued/td>
| 16 December 2023 |
| DOI |
doi:10.18429/JACoW-ICALEPCS2023-FR2AO04 |
| Pages |
1630-1636 |
| Copyright |
 Published by JACoW Publishing under the terms of the Creative Commons Attribution 4.0 International license. Any further distribution of this work must maintain attribution to the author(s), the published article's title, publisher, and DOI. |
|
