| Title |
Enabling Data Analytics as a Service for Large Scale Facilities |
| Authors |
- K. Woods, R.J. Clegg, N.S. Cook, R. Millward
Tessella, Abingdon, United Kingdom
- F. Barnsely, C. Jones
STFC/RAL, Chilton, Didcot, Oxon, United Kingdom
|
| Abstract |
The Ada Lovelace Centre (ALC) at STFC is an integrated, cross-disciplinary data intensive science centre, for better exploitation of research carried out at large scale UK Facilities including the Diamond Light Source, the ISIS Neutron and Muon Facility, the Central Laser Facility and the Culham Centre for Fusion Energy. ALC will provide on-demand, data analysis, interpretation and analytics services to worldwide users of these research facilities. Using open-source components, ALC and Tessella have together created a software infrastructure to support the delivery of that vision. The infrastructure comprises a Virtual Machine Manager, for managing pools of VMs across distributed compute clusters; components for automated provisioning of data analytics environments across heterogeneous clouds; a Data Movement System, to efficiently transfer large datasets; a Kubernetes cluster to manage on demand submission of Spark jobs. In this paper, we discuss the challenges of creating an infrastructure to meet the differing analytics needs of multiple facilities and report the architecture and design of the infrastructure that enables Data Analytics as a Service.
|
| Funding |
UK Research and Innovation - Science & Technology Facilities Council (UK SBS IT18160) |
| Paper |
download MOPHA160.PDF [0.356 MB / 3 pages] |
| Poster |
download MOPHA160_POSTER.PDF [1.665 MB] |
| Export |
download ※ BibTeX
※ LaTeX
※ Text/Word
※ RIS
※ EndNote |
| Conference |
ICALEPCS2019 |
| Series |
International Conference on Accelerator and Large Experimental Physics Control Systems (17th) |
| Location |
New York, NY, USA |
| Date |
05-11 October 2019 |
| Publisher |
JACoW Publishing, Geneva, Switzerland |
| Editorial Board |
Karen S. White (ORNL, Oak Ridge, TN, USA); Kevin A. Brown (BNL, Upton, NY, USA); Philip S. Dyer (BNL, Upton, NY, USA); Volker RW Schaa (GSI, Darmstadt, Germany) |
| Online ISBN |
978-3-95450-209-7 |
| Online ISSN |
2226-0358 |
| Received |
30 September 2019 |
| Accepted |
10 October 2019 |
| Issue Date |
30 August 2020 |
| DOI |
doi:10.18429/JACoW-ICALEPCS2019-MOPHA160 |
| Pages |
614-616 |
| Copyright |
 Published by JACoW Publishing under the terms of the Creative Commons Attribution 3.0 International license. Any further distribution of this work must maintain attribution to the author(s), the published article's title, publisher, and DOI. |
|
