PCaPAC2014 - List of Authors (Vondrous, A.)

Paper

Title

Page

Data Management at the Synchrotron Radiation Facility ANKA

13

D. Ressmann, A. Kopmann, V. Mauch, W. Mexner, A. Vondrous
KIT, Eggenstein-Leopoldshafen, Germany

The complete chain from submitting a proposal, collecting meta data, performing an experiment, towards analysis of these data and finally long term archive will be described. During this process a few obstacles have to be tackled. The workflow should be transparent to the user as well as to the beamline scientists. The final data will be stored in NeXus compatible HDF5 container format. Because the transfer of one large file is more efficient than transferring many small files, container formats enable a faster transfer of experiment data. At the same time HDF5 supports to store meta data together with the experiment data. For large data sets another implication is the performance to download the files. Furthermore the analysis software might not be available at each home institution; as a result it should be an option to access the experiment data on site. The meta data allows to find, analyse, preserve and curate the data in a long term archive, which will become a requirement fairly soon.

Slides WCO202 [2.380 MB]

FPO034

Beamline Data Management at the Synchrotron ANKA

231

A. Vondrous, T. Jejkal, W. Mexner, D. Ressmann, R. Stotzka
KIT, Eggenstein-Leopoldshafen, Germany

We present an architecture consisting of measurement devices, beamline data management and data repository to enable data management at the synchrotron facility ANKA. The operators perform some data management tasks manually and individually for each measurement method. In order to provide the functionality of a data repository it is necessary to collect the data, aggregate metadata and to perform the ingests into the data repository. The data management layer between the measurement devices and the data repository is referred to beamline data management (BLDM), which performs data collection, metadata aggregation and data ingest. Shared libraries contain functionality like migration, ingest or metadata aggregation and form the basis of the BLDM. The workflows and the current state of execution are persisted to enable monitoring and error handling. After data ingest into the data repository, implemented with the KIT Data Manager, archiving, content preservation or bit preservation services are provided for the ingested data. BLDM can connect the existing infrastructure with the data repository without major changes of routine processes to build a data repository for a synchrotron.

Paper	Title	Page
WCO202	Data Management at the Synchrotron Radiation Facility ANKA	13
	D. Ressmann, A. Kopmann, V. Mauch, W. Mexner, A. Vondrous KIT, Eggenstein-Leopoldshafen, Germany
	The complete chain from submitting a proposal, collecting meta data, performing an experiment, towards analysis of these data and finally long term archive will be described. During this process a few obstacles have to be tackled. The workflow should be transparent to the user as well as to the beamline scientists. The final data will be stored in NeXus compatible HDF5 container format. Because the transfer of one large file is more efficient than transferring many small files, container formats enable a faster transfer of experiment data. At the same time HDF5 supports to store meta data together with the experiment data. For large data sets another implication is the performance to download the files. Furthermore the analysis software might not be available at each home institution; as a result it should be an option to access the experiment data on site. The meta data allows to find, analyse, preserve and curate the data in a long term archive, which will become a requirement fairly soon.
	Slides WCO202 [2.380 MB]

FPO034	Beamline Data Management at the Synchrotron ANKA	231
	A. Vondrous, T. Jejkal, W. Mexner, D. Ressmann, R. Stotzka KIT, Eggenstein-Leopoldshafen, Germany
	We present an architecture consisting of measurement devices, beamline data management and data repository to enable data management at the synchrotron facility ANKA. The operators perform some data management tasks manually and individually for each measurement method. In order to provide the functionality of a data repository it is necessary to collect the data, aggregate metadata and to perform the ingests into the data repository. The data management layer between the measurement devices and the data repository is referred to beamline data management (BLDM), which performs data collection, metadata aggregation and data ingest. Shared libraries contain functionality like migration, ingest or metadata aggregation and form the basis of the BLDM. The workflows and the current state of execution are persisted to enable monitoring and error handling. After data ingest into the data repository, implemented with the KIT Data Manager, archiving, content preservation or bit preservation services are provided for the ingested data. BLDM can connect the existing infrastructure with the data repository without major changes of routine processes to build a data repository for a synchrotron.