JACoW logo

Joint Accelerator Conferences Website

The Joint Accelerator Conferences Website (JACoW) is an international collaboration that publishes the proceedings of accelerator conferences held around the world.

Title Data Analysis Infrastructure for Diamond Light Source Macromolecular & Chemical Crystallography and Beyond
  • M. Gerstel, A. Ashton, R.J. Gildea, K. Levik, G. Winter
    DLS, Oxfordshire, United Kingdom
Abstract The Diamond Light Source data analysis infrastructure, Zocalo, is built on a messaging framework. Analysis tasks are processed by a scalable pool of workers running on cluster nodes. Results can be written to a common file system, sent to another worker for further downstream processing and/or streamed to a LIMS. Zocalo allows increased parallelization of computationally expensive tasks and makes the use of computational resources more efficient. The infrastructure is low-latency, fault-tolerant, and allows for highly dynamic data processing. Moving away from static workflows expressed in shell scripts we can easily re-trigger processing tasks in the event that an issue is found. It allows users to re-run tasks with additional input and ensures that automatically and manually triggered processing results are treated equally. Zocalo was originally conceived to cope with the additional demand on infrastructure by the introduction of Eiger detectors with up to 18 Mpixels and running at up to 560 Hz framerate on single crystal diffraction beamlines. We are now adapting Zocalo to manage processing tasks for ptychography, tomography, cryo-EM, and serial crystallography workloads.
Paper download WEMPR001.PDF [1.678 MB / 5 pages]
Export download ※ BibTeX LaTeXText/WordRISEndNote
Conference ICALEPCS2019
Series International Conference on Accelerator and Large Experimental Physics Control Systems (17th)
Location New York, NY, USA
Date 05-11 October 2019
Publisher JACoW Publishing, Geneva, Switzerland
Editorial Board Karen S. White (ORNL, Oak Ridge, TN, USA); Kevin A. Brown (BNL, Upton, NY, USA); Philip S. Dyer (BNL, Upton, NY, USA); Volker RW Schaa (GSI, Darmstadt, Germany)
Online ISBN 978-3-95450-209-7
Online ISSN 2226-0358
Received 30 September 2019
Accepted 10 October 2019
Issue Date 30 August 2020
DOI doi:10.18429/JACoW-ICALEPCS2019-WEMPR001
Pages 1031-1035
Creative Commons CC logoPublished by JACoW Publishing under the terms of the Creative Commons Attribution 3.0 International license. Any further distribution of this work must maintain attribution to the author(s), the published article's title, publisher, and DOI.