The Joint Accelerator Conferences Website (JACoW) is an international collaboration that publishes the proceedings of accelerator conferences held around the world.
TY - CONF AU - Boychenko, S. AU - Galilée, M.A. AU - Garnier, J.C. AU - Zenha-Rela, M. AU - Zerlauth, M. ED - Schaa, Volker RW ED - Costa, Isidre ED - Fernández, David ED - Matilla, Ãscar TI - Multi-Criteria Partitioning on Distributed File Systems for Efficient Accelerator Data Analysis and Performance Optimization J2 - Proc. of ICALEPCS2017, Barcelona, Spain, 8-13 October 2017 C1 - Barcelona, Spain T2 - International Conference on Accelerator and Large Experimental Control Systems T3 - 16 LA - english AB - Since the introduction of the map-reduce paradigm, relational databases are being increasingly replaced by more efficient and scalable architectures, in particular in environments where a query will process TBytes or even PBytes of data in a single execution. The same tendency is observed at CERN, where data archiving systems for operational accelerator data are already working well beyond their initially provisioned capacity. Most of the modern data analysis frameworks are not optimized for heterogeneous workloads such as they arise in the dynamic environment of one of the world's largest accelerator complex. This contribution presents a Mixed Partitioning Scheme Replication (MPSR) as a solution that will outperform conventional distributed processing environment configurations for almost the entire phase-space of data analysis use cases and performance optimization challenges as they arise during the commissioning and operational phases of an accelerator. We will present results of a statistical analysis as well as the benchmarking of the implemented prototype, which allow defining the characteristics of the proposed approach and to confirm the expected performance gains. PB - JACoW CP - Geneva, Switzerland SP - 1436 EP - 1441 KW - ion KW - operation KW - data-analysis KW - distributed KW - framework DA - 2018/01 PY - 2018 SN - 978-3-95450-193-9 DO - 10.18429/JACoW-ICALEPCS2017-THPHA036 UR - http://jacow.org/icalepcs2017/papers/thpha036.pdf ER -