The RHIC heavy ion program relies on a series of RF bunch merge gymnastics to combine individual source pulses into bunches of suitable intensity. Intensity and emittance preservation during these gymnastics require careful setup of the voltages and phases of RF cavities operating at several different harmonic numbers. The optimum setting tends to drift over time, degrading performance and requiring operator attention to correct. We describe a reinforcement learning approach to learning and maintaining an optimum configuration, accounting for the relevant RF parameters and external perturbations (e.g., a changing main dipole field) using a physics-based simulator at Brookhaven Alternating Gradient Synchrotron (AGS).