ICALEPCS2017 - List of Authors (Brambilla, M.)

Paper	Title	Page
TUPHA177	Status of the Development of the Experiment Data Acquisition Pipeline for the European Spallation Source	835
	A.H.C. Mukai, M.J. Christensen, J.M.C. Nilsson, T.S. Richter, M. Shetty ESS, Copenhagen, Denmark F.A. Akeroyd, M.J. Clarke STFC/RAL/ISIS, Chilton, Didcot, Oxon, United Kingdom M. Brambilla, M. Könnecke, D. Werder PSI, Villigen PSI, Switzerland M.D. Jones Tessella, Abingdon, United Kingdom
	Funding: This project is partially funded by the European Union Framework Programme for Research and Innovation Horizon 2020, under grant agreement 676548. The European Spallation Source will produce more data than existing neutron facilities, due to higher accelerator power and to the fact that all data will be collected in event mode with no hardware veto. Detector data will be acquired and aggregated with metadata coming from sources such as sample environment, choppers and motion control. To aggregate data we will use Apache Kafka with FlatBuffers serialisation. A common schema repository defines the formats to be used by the data producers and consumers. The main consumers we are prototyping are a file writer for NeXus files and live reduction and visualisation via Mantid. A Jenkins-based setup using virtual machines is being used for integration tests, and physical servers are available in an integration laboratory alongside real hardware. We present the current status of the data acquisition pipeline and results from the testing and integration work going on at the ESS Data Management and Software Centre in collaboration with in-kind and BrightnESS partners.
	Poster TUPHA177 [0.434 MB]
DOI •	reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2017-TUPHA177
Export •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

THPHA167	EPICS Data Streaming and HDF File Writing for ESS Benchmarked Using the Virtual AMOR Instrument	1815
	D. Werder, M. Brambilla, M. Koennecke PSI, Villigen PSI, Switzerland F.A. Akeroyd, M.J. Clarke STFC/RAL/ISIS, Chilton, Didcot, Oxon, United Kingdom M.D. Jones Tessella, Abingdon, United Kingdom A.H.C. Mukai, J.M.C. Nilsson, T.S. Richter ESS, Copenhagen, Denmark
	Funding: This work is funded by the European Union Framework Programme for Research and Innovation Horizon 2020, under grant agreement 676548. As a contribution to the European Spallation Source as part of BrightnESS, the Paul Scherrer Institut is involved in the streaming of EPICS data and the writing of NeXus compliant HDF5 files. We combine this development with the transition of the AMOR instrument at the Paul Scherrer Institut to EPICS and a streaming based data architecture. To guide our development before ESS has operational equipment, we use a detailed simulation of the instrument AMOR at SINQ to test and integrate our data streaming components. We convert EPICS data sources to Google FlatBuffers as our message format and distribute them using Apache Kafka. On the file writing side, we combine the messages from EPICS data sources as well as from neutron events to write HDF5 files at rates up to 4.8 GiB/s using Parallel HDF. This platform will also be used for testing the experiment control software on top of EPICS.
	Poster THPHA167 [0.476 MB]
DOI •	reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2017-THPHA167
Export •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

Paper

Title

Page

Status of the Development of the Experiment Data Acquisition Pipeline for the European Spallation Source

835

A.H.C. Mukai, M.J. Christensen, J.M.C. Nilsson, T.S. Richter, M. Shetty
ESS, Copenhagen, Denmark
F.A. Akeroyd, M.J. Clarke
STFC/RAL/ISIS, Chilton, Didcot, Oxon, United Kingdom
M. Brambilla, M. Könnecke, D. Werder
PSI, Villigen PSI, Switzerland
M.D. Jones
Tessella, Abingdon, United Kingdom

Funding: This project is partially funded by the European Union Framework Programme for Research and Innovation Horizon 2020, under grant agreement 676548.
The European Spallation Source will produce more data than existing neutron facilities, due to higher accelerator power and to the fact that all data will be collected in event mode with no hardware veto. Detector data will be acquired and aggregated with metadata coming from sources such as sample environment, choppers and motion control. To aggregate data we will use Apache Kafka with FlatBuffers serialisation. A common schema repository defines the formats to be used by the data producers and consumers. The main consumers we are prototyping are a file writer for NeXus files and live reduction and visualisation via Mantid. A Jenkins-based setup using virtual machines is being used for integration tests, and physical servers are available in an integration laboratory alongside real hardware. We present the current status of the data acquisition pipeline and results from the testing and integration work going on at the ESS Data Management and Software Centre in collaboration with in-kind and BrightnESS partners.

Poster TUPHA177 [0.434 MB]

DOI •

reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2017-TUPHA177

Export •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

THPHA167

EPICS Data Streaming and HDF File Writing for ESS Benchmarked Using the Virtual AMOR Instrument

1815

D. Werder, M. Brambilla, M. Koennecke
PSI, Villigen PSI, Switzerland
F.A. Akeroyd, M.J. Clarke
STFC/RAL/ISIS, Chilton, Didcot, Oxon, United Kingdom
M.D. Jones
Tessella, Abingdon, United Kingdom
A.H.C. Mukai, J.M.C. Nilsson, T.S. Richter
ESS, Copenhagen, Denmark

Funding: This work is funded by the European Union Framework Programme for Research and Innovation Horizon 2020, under grant agreement 676548.
As a contribution to the European Spallation Source as part of BrightnESS, the Paul Scherrer Institut is involved in the streaming of EPICS data and the writing of NeXus compliant HDF5 files. We combine this development with the transition of the AMOR instrument at the Paul Scherrer Institut to EPICS and a streaming based data architecture. To guide our development before ESS has operational equipment, we use a detailed simulation of the instrument AMOR at SINQ to test and integrate our data streaming components. We convert EPICS data sources to Google FlatBuffers as our message format and distribute them using Apache Kafka. On the file writing side, we combine the messages from EPICS data sources as well as from neutron events to write HDF5 files at rates up to 4.8 GiB/s using Parallel HDF. This platform will also be used for testing the experiment control software on top of EPICS.

Poster THPHA167 [0.476 MB]

DOI •

reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2017-THPHA167

Export •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)