JACoW logo

Joint Accelerator Conferences Website

The Joint Accelerator Conferences Website (JACoW) is an international collaboration that publishes the proceedings of accelerator conferences held around the world.


https://doi.org/10.18429/JACoW-ICALEPCS2017-THPHA043
Title Lightflow - a Lightweight, Distributed Workflow System
Authors
  • A. Moll, R. Clarken, P. Martinpresenter, S.T. Mudie
    SLSA-ANSTO, Clayton, Australia
Abstract The Australian Synchrotron, located in Clayton, Melbourne, is one of Australia's most important pieces of research infrastructure. After more than 10 years of operation, the beamlines at the Australian Synchrotron are well established and the demand for automation of research tasks is growing. Such tasks routinely involve the reduction of TB-scale data, online (realtime) analysis of the recorded data to guide experiments, and fully automated data management workflows. In order to meet these demands, a generic, distributed workflow system was developed. It is based on well-established Python libraries and tools. The individual tasks of a workflow are arranged in a directed acyclic graph and one or more directed acyclic graphs form a workflow. Workers consume the tasks, allowing the processing of a workflow to scale horizontally. Data can flow between tasks and a variety of specialised tasks is available. Lightflow has been released as open source on the Australian Synchrotron GitHub page
Paper download THPHA043.PDF [0.166 MB / 3 pages]
Poster download THPHA043_POSTER.PDF [0.582 MB]
Export download ※ BibTeX LaTeXText/WordRISEndNote
Conference ICALEPCS2017, Barcelona, Spain
Series International Conference on Accelerator and Large Experimental Control Systems (16th)
Proceedings Link to full ICALEPCS2017 Proccedings
Session Poster Session
Date 12-Oct-17   16:45–19:00
Main Classification Data Management and Processing
Keywords ion, synchrotron, distributed, EPICS, experiment
Publisher JACoW, Geneva, Switzerland
Editors Volker RW Schaa (GSI, Darmstadt, Germany); Isidre Costa (ALBA-CELLS, Cerdanyola del Vallès, Spain); David Fernández (ALBA-CELLS, Cerdanyola del Vallès, Spain); Óscar Matilla (ALBA-CELLS, Cerdanyola del Vallès, Spain)
ISBN 978-3-95450-193-9
Published January 2018
Copyright
Copyright © 2018 by JACoW, Geneva, Switzerland     CC-BY Creative Commons License
cc Creative Commons Attribution 3.0