ICALEPCS2019 - List of Authors (Gerstel, M.)

Paper	Title	Page
WEMPR001	Data Analysis Infrastructure for Diamond Light Source Macromolecular & Chemical Crystallography and Beyond	1031
WEPHA094	use link to see paper's listing under its alternate paper code
	M. Gerstel, A. Ashton, R.J. Gildea, K. Levik, G. Winter DLS, Oxfordshire, United Kingdom
	The Diamond Light Source data analysis infrastructure, Zocalo, is built on a messaging framework. Analysis tasks are processed by a scalable pool of workers running on cluster nodes. Results can be written to a common file system, sent to another worker for further downstream processing and/or streamed to a LIMS. Zocalo allows increased parallelization of computationally expensive tasks and makes the use of computational resources more efficient. The infrastructure is low-latency, fault-tolerant, and allows for highly dynamic data processing. Moving away from static workflows expressed in shell scripts we can easily re-trigger processing tasks in the event that an issue is found. It allows users to re-run tasks with additional input and ensures that automatically and manually triggered processing results are treated equally. Zocalo was originally conceived to cope with the additional demand on infrastructure by the introduction of Eiger detectors with up to 18 Mpixels and running at up to 560 Hz framerate on single crystal diffraction beamlines. We are now adapting Zocalo to manage processing tasks for ptychography, tomography, cryo-EM, and serial crystallography workloads.
DOI •	reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2019-WEMPR001
About •	paper received ※ 30 September 2019 paper accepted ※ 10 October 2019 issue date ※ 30 August 2020
Export •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

Paper

Title

Page

WEMPR001

Data Analysis Infrastructure for Diamond Light Source Macromolecular & Chemical Crystallography and Beyond

1031

WEPHA094

use link to see paper's listing under its alternate paper code

M. Gerstel, A. Ashton, R.J. Gildea, K. Levik, G. Winter
DLS, Oxfordshire, United Kingdom

The Diamond Light Source data analysis infrastructure, Zocalo, is built on a messaging framework. Analysis tasks are processed by a scalable pool of workers running on cluster nodes. Results can be written to a common file system, sent to another worker for further downstream processing and/or streamed to a LIMS. Zocalo allows increased parallelization of computationally expensive tasks and makes the use of computational resources more efficient. The infrastructure is low-latency, fault-tolerant, and allows for highly dynamic data processing. Moving away from static workflows expressed in shell scripts we can easily re-trigger processing tasks in the event that an issue is found. It allows users to re-run tasks with additional input and ensures that automatically and manually triggered processing results are treated equally. Zocalo was originally conceived to cope with the additional demand on infrastructure by the introduction of Eiger detectors with up to 18 Mpixels and running at up to 560 Hz framerate on single crystal diffraction beamlines. We are now adapting Zocalo to manage processing tasks for ptychography, tomography, cryo-EM, and serial crystallography workloads.

DOI •

reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2019-WEMPR001

About •

paper received ※ 30 September 2019 paper accepted ※ 10 October 2019 issue date ※ 30 August 2020

Export •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)