ICALEPCS2011 - List of Authors (Xiong, N.)

Paper

Title

Page

Common Data Model ; A Unified Layer to Access Data from Data Analysis Point of View

1220

N. Hauser, T.K. Lam, N. Xiong
ANSTO, Menai, New South Wales, Australia
A. Buteau, M. Ounsy, S. Poirier
SOLEIL, Gif-sur-Yvette, France
C. Rodriguez
ALTEN, Boulogne-Billancourt, France

For almost 20 years, the scientific community of neutrons and synchrotron facilities has been dreaming of using a common data format to be able to exchange experimental results and applications to analyse them. If using HDF5 as a physical container for data quickly raised a large consensus, the big issue is the standardisation of data organisation. By introducing a new level of indirection for data access, the CommonDataModel (CDM) framework offers a solution and allows to split development efforts and responsibilities between institutes. The CDM is made of a core API that accesses data through a data format plugins mechanism and scientific applications definitions (i.e. sets of logically organized keywords defined by scientists for each experimental technique). Using a innovative "mapping" system between applications definitions and physical data organizations, the CDM allows to develop data reduction applications regardless of data files formats AND organisations. Then each institute has to develop data access plugins for its own files formats along with the mapping between application definitions and its own data files organisation. Thus, data reduction applications can be developed from a strictly scientific point of view and are natively able to process data coming from several institutes. A concrete example on a SAXS data reduction application, accessing NeXus and EDF (ESRF Data Format) file will be commented.

Slides THCHAUST03 [36.889 MB]

Paper	Title	Page
THCHAUST03	Common Data Model ; A Unified Layer to Access Data from Data Analysis Point of View	1220
	N. Hauser, T.K. Lam, N. Xiong ANSTO, Menai, New South Wales, Australia A. Buteau, M. Ounsy, S. Poirier SOLEIL, Gif-sur-Yvette, France C. Rodriguez ALTEN, Boulogne-Billancourt, France
	For almost 20 years, the scientific community of neutrons and synchrotron facilities has been dreaming of using a common data format to be able to exchange experimental results and applications to analyse them. If using HDF5 as a physical container for data quickly raised a large consensus, the big issue is the standardisation of data organisation. By introducing a new level of indirection for data access, the CommonDataModel (CDM) framework offers a solution and allows to split development efforts and responsibilities between institutes. The CDM is made of a core API that accesses data through a data format plugins mechanism and scientific applications definitions (i.e. sets of logically organized keywords defined by scientists for each experimental technique). Using a innovative "mapping" system between applications definitions and physical data organizations, the CDM allows to develop data reduction applications regardless of data files formats AND organisations. Then each institute has to develop data access plugins for its own files formats along with the mapping between application definitions and its own data files organisation. Thus, data reduction applications can be developed from a strictly scientific point of view and are natively able to process data coming from several institutes. A concrete example on a SAXS data reduction application, accessing NeXus and EDF (ESRF Data Format) file will be commented.
	Slides THCHAUST03 [36.889 MB]