ICALEPCS2017 - List of Authors (Clarken, R.)

Paper	Title	Page
THPHA042	ASCI: A Compute Platform for Researchers at the Australian Synchrotron	1455
	J. Marcou, R.R.I. Bosworth ASCo, Clayton, Victoria, Australia R. Clarken SLSA-ANSTO, Clayton, Australia P. Martin, A. Moll SLSA, Clayton, Australia
	The volume and quality of scientific data produced at the Australian Synchrotron continues to grow rapidly due to advancements in detectors, motion control and automation. This makes it critical that researchers have access to computing infrastructure that enables them to efficiently process and extract insight from their data. To facilitate this, we have developed a compute platform to enable researchers to analyse their data in real time while at the beamline as well as post-experiment by logging in remotely. This system, named ASCI, provides a convenient web-based interface to launch Linux desktops running inside Docker containers on high-performance compute hardware. Each session has the user's data mounted and is preconfigured with the software required for their experiment. This poster will present the architecture of the system and explain the design decisions in building this platform.
	Poster THPHA042 [1.402 MB]
DOI •	reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2017-THPHA042
Export •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

THPHA043	Lightflow - a Lightweight, Distributed Workflow System	1457
	A. Moll, R. Clarken, P. Martin, S.T. Mudie SLSA-ANSTO, Clayton, Australia
	The Australian Synchrotron, located in Clayton, Melbourne, is one of Australia's most important pieces of research infrastructure. After more than 10 years of operation, the beamlines at the Australian Synchrotron are well established and the demand for automation of research tasks is growing. Such tasks routinely involve the reduction of TB-scale data, online (realtime) analysis of the recorded data to guide experiments, and fully automated data management workflows. In order to meet these demands, a generic, distributed workflow system was developed. It is based on well-established Python libraries and tools. The individual tasks of a workflow are arranged in a directed acyclic graph and one or more directed acyclic graphs form a workflow. Workers consume the tasks, allowing the processing of a workflow to scale horizontally. Data can flow between tasks and a variety of specialised tasks is available. Lightflow has been released as open source on the Australian Synchrotron GitHub page
	Poster THPHA043 [0.582 MB]
DOI •	reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2017-THPHA043
Export •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

Paper

Title

Page

ASCI: A Compute Platform for Researchers at the Australian Synchrotron

1455

J. Marcou, R.R.I. Bosworth
ASCo, Clayton, Victoria, Australia
R. Clarken
SLSA-ANSTO, Clayton, Australia
P. Martin, A. Moll
SLSA, Clayton, Australia

The volume and quality of scientific data produced at the Australian Synchrotron continues to grow rapidly due to advancements in detectors, motion control and automation. This makes it critical that researchers have access to computing infrastructure that enables them to efficiently process and extract insight from their data. To facilitate this, we have developed a compute platform to enable researchers to analyse their data in real time while at the beamline as well as post-experiment by logging in remotely. This system, named ASCI, provides a convenient web-based interface to launch Linux desktops running inside Docker containers on high-performance compute hardware. Each session has the user's data mounted and is preconfigured with the software required for their experiment. This poster will present the architecture of the system and explain the design decisions in building this platform.

Poster THPHA042 [1.402 MB]

DOI •

reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2017-THPHA042

Export •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

THPHA043

Lightflow - a Lightweight, Distributed Workflow System

1457

A. Moll, R. Clarken, P. Martin, S.T. Mudie
SLSA-ANSTO, Clayton, Australia

The Australian Synchrotron, located in Clayton, Melbourne, is one of Australia's most important pieces of research infrastructure. After more than 10 years of operation, the beamlines at the Australian Synchrotron are well established and the demand for automation of research tasks is growing. Such tasks routinely involve the reduction of TB-scale data, online (realtime) analysis of the recorded data to guide experiments, and fully automated data management workflows. In order to meet these demands, a generic, distributed workflow system was developed. It is based on well-established Python libraries and tools. The individual tasks of a workflow are arranged in a directed acyclic graph and one or more directed acyclic graphs form a workflow. Workers consume the tasks, allowing the processing of a workflow to scale horizontally. Data can flow between tasks and a variety of specialised tasks is available. Lightflow has been released as open source on the Australian Synchrotron GitHub page

Poster THPHA043 [0.582 MB]

DOI •

reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2017-THPHA043

Export •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)