Timing Systems, Synchronization and Real-Time Applications
Paper Title Page
THBR01 Renovation of the Trigger Distribution in CERN’s Open Analogue Signal Information System Using White Rabbit 839
 
  • D. Lampridis, T. Gingold, A. Poscia, M.H. Serans, M.R. Shukla, T.P. da Silva
    CERN, Geneva, Switzerland
  • D. Michalik
    Aalborg University, Aalborg, Denmark
 
  The Open Analogue Signal Information System (OASIS) acts as a distributed oscilloscope system that acquires signals from devices across the CERN accelerator complex and displays them in a convenient, graphical way. Today, the OASIS installation counts over 500 multiplexed digitisers, capable of digitising more than 5000 analogue signals and offers a selection of more than 250 triggers for the acquisitions. These triggers are mostly generated at a single central place and are then distributed by means of a dedicated coaxial cable per digitiser, using a "star" topology. An upgrade is currently under way to renovate this trigger distribution system and migrate it to a White Rabbit (WR) based solution. In this new system, triggers are distributed in the form of Ethernet messages over a WR network, allowing for better scalability, higher time-stamping precision, trigger latency compensation and improved robustness. This paper discusses the new OASIS trigger distribution architecture, including hardware, drivers, front-end, server and application-tier software. It then provides results from preliminary tests in laboratory installations.  
slides icon Slides THBR01 [2.229 MB]  
DOI • reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2021-THBR01  
About • Received ※ 09 October 2021       Accepted ※ 21 December 2021       Issue date ※ 06 February 2022  
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)  
 
THBR02 White Rabbit and MTCA.4 Use in the LLRF Upgrade for CERN’s SPS 847
 
  • T. Włostowski, K. Adrianek, M. Arruat, P. Baudrenghien, A.C. Butterworth, G. Daniluk, J. Egli, J.R. Gill, T. Gingold, J.D. González Cobas, G. Hagmann, P. Kuzmanović, D. Lampridis, M.M. Lipiński, S. Novel González, J.P. Palluel, M. Rizzi, A. Spierer, M. Sumiński, A. Wujek
    CERN, Geneva, Switzerland
 
  The Super Proton Synchrotron (SPS) Low-level RF (LLRF) system at CERN was completely revamped in 2020. In the old system, the digital signal processing was clocked by a submultiple of the RF. The new system uses a fixed-frequency clock derived from White Rabbit (WR). This triggered the development of an eRTM module for generating very precise clock signals to be fed to the optional RF backplane in MTCA.4 crates. The eRTM14/15 sandwich of modules implements a WR node delivering clock signals with a jitter below 100 fs. WR-clocked RF synthesis inside the FPGA makes it simple to reproduce the RF elsewhere by broadcasting the frequency-tuning words over the WR network itself. These words are received by the WR2RF-VME module and used to produce beam-synchronous signals such as the bunch clock and the revolution tick. This paper explains the general architecture of this new LLRF system, highlighting the role of WR-based synchronization. It then goes on to describe the hardware and gateware designs for both modules, along with their supporting software. A recount of our experience with the deployment of the MTCA.4 platform is also provided.  
slides icon Slides THBR02 [0.981 MB]  
DOI • reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2021-THBR02  
About • Received ※ 12 October 2021       Revised ※ 24 October 2021       Accepted ※ 03 January 2022       Issue date ※ 28 February 2022
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)  
 
THBR03 Prototype of White Rabbit Based Beam-Synchronous Timing Systems for SHINE 853
 
  • P.X. Yu, Y.B. Yan
    SSRF, Shanghai, People’s Republic of China
  • G.H. Gong
    Tsinghua University, Beijing, People’s Republic of China
  • G. Gu, Z.Y. Jiang, L. Zhao
    USTC, Hefei, Anhui, People’s Republic of China
  • Y.M. Ye
    TUB, Beijing, People’s Republic of China
 
  Shanghai HIgh repetition rate XFEL aNd Extreme light facility (SHINE) is under construction. SHINE requires precise distribution and synchronization of the 1.003086MHz timing signals over a long distance of about 3.1 km. Two prototype systems were developed, both containing three functions: beam-synchronous trigger signal distribution, random-event trigger signal distribution and data exchange between nodes. The frequency of the beam-synchronous trigger signal can be divided according to the accelerator operation mode. Each output pulse can be configured for different fill modes. A prototype system was designed based on a customized clock frequency point (64.197530MHz). Another prototype system was designed based on the standard White Rabbit protocol. The DDS (Direct Digital Synthesis) and D flip-flops (DFFs) are adopted for RF signal transfer and pulse configuration. The details of the timing system design and test results will be reported in this paper.  
slides icon Slides THBR03 [3.344 MB]  
DOI • reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2021-THBR03  
About • Received ※ 11 October 2021       Revised ※ 19 October 2021       Accepted ※ 22 December 2021       Issue date ※ 10 February 2022
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)  
 
THBR04
Nanosecond machine learning with BDT for high energy physics  
 
  • T.M. Hong, S.T. Roche
    University of Pittsburgh, Pittsburgh, Pennsylvania, USA
 
  Funding: TMH was supported by US DOE [DE-SC0007914]
We present a novel implementation of classification using boosted decision trees (BDT) on FPGA. Our BDT approach offers an alternative to existing packages, including those that implement neural networks on FPGA, with less dependence of DSP utilization that is replaced by other resources. Our design philosophy is to remove clocked operations in favor of combinatoric logic through High Level Synthesis. The firmware implementation of binary classification requiring 100 training trees with a maximum depth of 4 using four input variables gives a latency value of about 10ns at various clock speeds. We optimize the parameters using a software package, which interfaces to Xilinx Vivado through High Level Synthesis. Such a tool may enable the FPGA-based trigger systems at the Large Hadron Collider to be more sensitive to new physics at high energy experiments. The work is described in https://arxiv.org/abs/2104.03408
 
slides icon Slides THBR04 [1.380 MB]  
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)  
 
THBR05
An integrated scheme for online correction of laser focal position  
 
  • N.M. Cook, S.J. Coleman, J.P. Edelen, R. Nagler
    RadiaSoft LLC, Boulder, Colorado, USA
  • S.K. Barber, J. van Tilborg
    LBNL, Berkeley, USA
 
  Funding: This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of High Energy Physics under Award Number DE-SC 00259037.
High repetition-rate, ultrafast laser systems play a critical role in a host of modern scientific and industrial applications. We present a prototype diagnostic and correction scheme for controlling laser focal position for operation at 10s of Hz. Our strategy is to couple fast wavefront sensor measurements at multiple positions to generate a focal position prediction. We then train a neural network to predict the specific adjustments to adaptive actuators along the beamline to provide the desired correction to the focal position at 10s of ms timescales. Our initial proof-of-principle demonstrations leverage pre-compiled data and pre-trained networks operating ex-situ from the laser system. We then discuss the application of a high-level synthesis framework for generating a low-level hardware description of ML-based correction algorithms on FPGA hardware coupled directly to the beamline. Lastly, we consider the use of remote computing resources, such as the Sirepo scientific framework*, to actively update these correction schemes in the presence of new data
*M.S. Rakitin et al., ’Sirepo: an open-source cloud-based software interface for X-ray source and optics simulations," Journal of Synchrotron Radiation25, 1877-1892 (Nov 2018).
 
slides icon Slides THBR05 [1.342 MB]  
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)  
 
THPV021 TATU: A Flexible FPGA-Based Trigger and Timer Unit Created on CompactRIO for the First Sirius Beamlines 908
 
  • J.R. Piton, D. Alnajjar, D.H.C. Araujo, J.L. Brito Neto, L.P. Do Carmo, L.C. Guedes, M.A.L. Moraes
    LNLS, Campinas, Brazil
 
  In the modern synchrotron light sources, the higher brilliance leads to shorter acquisition times at the experimental stations. For most beamlines of the fourth-generation source SIRIUS, it was imperative to shift from the usual software-based synchronization of operations to the much faster triggering by hardware of some key equipment involved in the experiments. As a basis of their control system for devices, the SIRIUS beamlines have standard CompactRIO controllers and I/O modules along the hutches. Equipped with a FPGA and a hard processor running Linux Real-Time, this platform could deal with the triggers from and to other devices, in the order of ms and µs. TATU (Time and Trigger Unit) is a code running in a CompactRIO unit to coordinate multiple triggering conditions and actions. TATU can be either the master pulse generator or the follower of other signals. Complex trigger pattern generation is set from a user-friendly standardized interface. EPICS process variables (by means of LNLS Nheengatu*) are used to set parameters and to follow the execution status. The concept and first field test results in at least four SIRIUS beamlines are presented.
* D. Alnajjar, G. S. Fedel, and J. R. Piton, "Project Nheengatu: EPICS support for CompactRIO FPGA and LabVIEW-RT", ICALEPCS’19, New York, NY, USA, Oct. 2019, paper WEMPL002.
 
poster icon Poster THPV021 [0.618 MB]  
DOI • reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2021-THPV021  
About • Received ※ 10 October 2021       Accepted ※ 21 November 2021       Issue date ※ 02 February 2022  
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)  
 
THPV022 MRF Timing System Design at SARAF 912
 
  • A. Gaget
    CEA-IRFU, Gif-sur-Yvette, France
 
  CEA Saclay Irfu is in charge of an important part of the control system of the SARAF LINAC accelerator based at Soreq (Israel). This includes, among other, the control of the timing system (synchronization and timestamping). CEA has already installed and uses successfully the timing distribution with MRF on test benches for ESS or IPHI, so it has been decided to use the same technologies. The reference frequency will be distributed along the accelerator by a master oscillator Wenzel and the UTC time will be based on a Meridian II GPS, these 2 devices will be connected to the Event Master (EVM) card which is the main element of the timing system architecture. Through an optical fiber network, the MRF timing system allows to distribute downstream and upstream events with a µs propagation time. Currently, we are working on development in order to also use it for the machine protection system of the accelerator. In this paper, hardware, timing architecture, software developments and tests will be presented.  
poster icon Poster THPV022 [1.539 MB]  
DOI • reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2021-THPV022  
About • Received ※ 08 October 2021       Revised ※ 20 October 2021       Accepted ※ 23 January 2022       Issue date ※ 01 March 2022
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)  
 
THPV025 A New Timing System for PETRA IV 916
 
  • T. Wilksen, A. Aghababyan, K. Brede, H.T. Duhme, M. Fenner, U. Hurdelbrink, H. Kay, H. Lippek, H. Schlarb
    DESY, Hamburg, Germany
 
  At DESY an upgrade of the PETRA III synchrotron light source towards a fourth-generation, low emittance machine PETRA IV is currently being actively pursued. The realization of this new machine implies a new design of the timing and synchronization system since requirements on beam quality and controls will significantly change from the existing implementation at PETRA III. The technical design phase of the PETRA IV project is in mid-phase and supposed to deliver a Technical Design Report by end of next year. The conceptual layout of the timing system will follow the successful MTCA.4-based approach as in use at the European XFEL. It will be enhanced to meet the requirements of a synchrotron facility and its booster and linac pre-accelerators. We present general concepts of the timing system, its integration into the control system as well as first specifications of the MTCA.4-based hardware components.  
DOI • reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2021-THPV025  
About • Received ※ 10 October 2021       Revised ※ 21 October 2021       Accepted ※ 21 November 2021       Issue date ※ 11 January 2022
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)  
 
THPV027 Application of the White Rabbit System at SuperKEKB 919
 
  • H. Kaji
    KEK, Ibaraki, Japan
  • Y. Iitsuka
    EJIT, Hitachi, Ibaraki, Japan
 
  We employ the White Rabbit system to satisfy the increasing requests from the SuperKEKB operations. The SuperKEKB-type slave node was developed based on the SPEC board and FMC-DIO card. The firmware was customized slightly to realize the SuperKEKB needs. The device/driver for EPICS was developed. The five slave nodes have been operated since the 2021 autumn run. The delivery of the beam permission signal from the central control building to the injector linac is taken care of by new slave nodes. The timing of the abort request signal and the trigger for the abort kicker magnet are recorded with the distributed TDC system. More slave nodes will be installed in the next year to enhance the role of the distributed TDC system.  
poster icon Poster THPV027 [1.186 MB]  
DOI • reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2021-THPV027  
About • Received ※ 10 October 2021       Revised ※ 25 October 2021       Accepted ※ 21 November 2021       Issue date ※ 08 January 2022
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)  
 
THPV028 Analysis of AC Line Fluctuation for Timing System at KEK 923
 
  • D. Wang
    Sokendai, Ibaraki, Japan
  • Y. Enomoto, K. Furukawa, H. Kaji, F. Miyahara, M. Sato, H. Sugimura
    KEK, Ibaraki, Japan
 
  The timing system controls the injection procedure of the accelerator by performing signal synchronization and trigger delivery to the devices all over the installations at KEK. The trigger signals is usually generated at the same phase of an AC power line to reduce the unwanted variation of the beam quality. This requirement originates from the power supply systems. However, the AC line synchronization conflicts with the bucket selection process of SuperKEKB low energy ring (LER) which stores the positron beam. The positron beam is firstly injected into a damping ring (DR) to lower the emittance before entering desired RF bucket in LER. A long bucket selection cycle for DR and LER makes it difficult to coincide with AC line every injection pulse. This trouble is solved by grouping several injection pulses into various of injection sequences and manipulating the length of sequences to adjust the AC line arrival timing. Therefore, the timing system is sensitive to drastically AC line fluctuation. The failure of timing system caused by strong AC line fluctuation and solutions are introduced in this work.  
poster icon Poster THPV028 [1.010 MB]  
DOI • reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2021-THPV028  
About • Received ※ 17 October 2021       Revised ※ 28 October 2021       Accepted ※ 21 November 2021       Issue date ※ 09 December 2021
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)  
 
THPV029 Development of Timing Read-Back System for Stable Operation of J-PARC 927
 
  • M. Yang
    Sokendai, Ibaraki, Japan
  • N. Kamikubota
    KEK, Ibaraki, Japan
  • N. Kikuzawa
    JAEA/J-PARC, Tokai-mura, Japan
  • K.C. Sato
    J-PARC, KEK & JAEA, Ibaraki-ken, Japan
  • Y. Tajima
    Kanto Information Service (KIS), Accelerator Group, Ibaraki, Japan
 
  Since 2006, the Japan Proton Accelerator Research Complex (J-PARC) timing system has been operated successfully. However, there were some unexpected trig-ger-failure events, typically missing trigger events, during the operation over 15 years. When a trigger-failure event occurred, it was often tough to find the one with the fault among many suspected modules. To solve the problem more easily, a unique device, triggered scaler, was devel-oped for reading back accelerator signals. The performance of the module has been evaluated in 2018. In 2021, we measured and observed an LLRF sig-nal as the first signal of the read-back system for beam operation. After firmware upgrades of the module, some customized timing read-back systems were developed, and successfully demonstrated as coping strategies for past trigger-failure events. In addition, a future plan to apply the read-back system to other facilities is discussed. More details are given in the paper.  
DOI • reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2021-THPV029  
About • Received ※ 20 October 2021       Accepted ※ 21 November 2021       Issue date ※ 13 January 2022  
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)  
 
THPV031 Upgrade of Timing System at HZDR ELBE Facility 931
 
  • Ž. Oven, L. Krmpotić, U. Legat, U. Rojec
    Cosylab, Ljubljana, Slovenia
  • M. Justus, M. Kuntzsch, A. Schwarz, K. Zenker
    HZDR, Dresden, Germany
 
  The ELBE center for high power radiation sources is operating an electron linear accelerator to generate various secondary radiation like neutrons, positrons, intense THz and IR pulses and Bremsstrahlung. Timing system, that is currently in operation, has been modified and extended in the last two decades to enable new experiments. At the moment parts of this timing system are using obsolete components which makes maintenance a very challenging endeavour. To make ELBE timing system again a more homogenous system, that will allow for easier adaption to new and more complex trigger patterns, an upgrade based on Micro Research Finland (MRF) hardware platform is currently in progress. This upgrade will enable parallel operation of two electron sources and subsequent kickers to serve multiple end stations at the same time. Selected hardware enables low jitter emission of timing patterns and a long-term delay compensation of the distribution network. We are currently in the final phase of development and with plans for commissioning to be completed in 2022.  
poster icon Poster THPV031 [2.801 MB]  
DOI • reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2021-THPV031  
About • Received ※ 11 October 2021       Revised ※ 20 October 2021       Accepted ※ 21 November 2021       Issue date ※ 11 January 2022
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)  
 
THPV032 The Demonstrator of the HL-LHC ATLAS Tile Calorimeter 935
 
  • P. Tsotskolauri
    Tbilisi State University, T’bilisi, Georgia
 
  The High Luminosity Large Hadron Collider (HL-LHC) has motivated R&D to upgrade the ATLAS Tile Calorimeter. The new system consists on an optimized analogue design engineered with selected radiation-tolerant COTS and redundancy layers to avoid single points of failure. The design will provide better timing, improved energy resolution, lower noise and less sensitivity to out-of-time pileup. Multiple types of FPGAs, CERN custom rad-hard ASICs (GBTx), and multi-Gbps optical links are used to distribute LHC timing, read out fully digital data of the whole TileCal, transmit timing and calibrated energy per cell to the Trigger system at 40 MHz, and provide triggered data at 1 MHz. To test the upgraded electronics in real ATLAS conditions, a hybrid demonstrator prototype module containing the new calorimeter module electronics, but still compatible with TileCal’s legacy system was tested in ATLAS during 2019-2021. An upgraded version of the demonstrator with finalized HL-LHC electronics is being assembled to be tested in testbeam campaigns at the Super Proton Syncrotron (SPS) at CERN. We present current status and results for the different tests done with the upgraded demonstrator system.
Presented on behalf of the ATLAS Tile Calorimeter System
 
poster icon Poster THPV032 [1.041 MB]  
DOI • reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2021-THPV032  
About • Received ※ 18 October 2021       Revised ※ 29 November 2021       Accepted ※ 23 December 2021       Issue date ※ 11 February 2022
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)  
 
THPV033 Reusable Real-Time Software Components for the SPS Low Level RF Control System 939
 
  • M. Sumiński, K. Adrianek, B. Bielawski, A.C. Butterworth, J. Egli, G. Hagmann, P. Kuzmanović, S. Novel González, A. Rey, A. Spierer
    CERN, Geneva, Switzerland
 
  In 2021 the Super Proton Synchrotron has been recommissioned after a complete renovation of its low level RF system (LLRF). The new system has largely moved to digital signal processing implemented as a set of functional blocks (IP cores) in Field Programmable Gate Arrays (FPGAs) with associated software to control them. Some of these IP cores provide generic functionalities such as timing, function generation, data resampling and signal acquisition, and are reused in several components, with a potential application in other accelerators. To take full advantage of the modular approach, IP core flexibility must be complemented by the software stack. In this paper we present steps we have taken to reach this goal from the software point of view, and describe the custom tools and procedures used to implement the various software layers.  
poster icon Poster THPV033 [1.234 MB]  
DOI • reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2021-THPV033  
About • Received ※ 09 October 2021       Accepted ※ 25 February 2022       Issue date ※ 28 February 2022  
Cite • reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)