JACoW is a publisher in Geneva, Switzerland that publishes the proceedings of accelerator conferences held around the world by an international collaboration of editors.
TY - UNPB AU - Zhang, C.L. ED - Schaa, Volker R.W. ED - Bian, Lin ED - Chen, Zhichu ED - Dong, Yuhui ED - He, Ping TI - High-Throughput Data Orchestration and Streaming System for High Energy Photon Source J2 - Proc. of MEDSI2023, Beijing, China, 06-10 November 2023 CY - Beijing, China T2 - International Conference on Mechanical Engineering Design of Synchrotron Radiation Equipment and Instrumentation T3 - 12 LA - english AB - The forthcoming fourth-generation light source, the High Energy Photon Source (HEPS), is expected to generate a massive amount of data with a throughput estimated at hundreds of terabytes per day. With the advancement of synchrotron radiation technology and experimental methodologies, the data transmission pipeline has become increasingly complex, evolving from a simple detector-to-storage single-channel in the past to a complex network with multiple data generators and applications. In order to ensure a favorable experimental experience, effective data management and distribution are essential. In response to these challenges, Mamba Data Worker (MDW) has been developed, which serves the primary functions of real-time detector data acquisition, interaction with multi-terminal dataflow, metadata extraction, online assembly and writing, and light weight online data processing. This work presents the design and development plan of MDW, outlines the essential technologies involved, and illustrates its current application at the Beijing Synchrotron Radiation Facility (BSRF). PB - JACoW Publishing CP - Geneva, Switzerland ER -