Tessella, Abingdon, United Kingdom
STFC/RAL, Chilton, Didcot, Oxon, United Kingdom
The Ada Lovelace Centre (ALC) at STFC provides on-demand, data analysis, interpretation and analytics services to scientists using UK research facilities. ALC and Tessella have built software systems to scale analysis environments to handle peaks and troughs in demand as well as to reduce latency by provision environments closer to scientists around the world. The systems can automatically provision infrastructure and supporting systems within compute resources around the world and in different cloud types (including commercial providers). The system then uses analytics to dynamically provision and configure virtual machines in various locations ahead of demand so that users experience as little delay as possible. In this poster, we report on the architecture and complex software engineering used to automatically scale analysis environments to heterogeneous clouds, make them secure and easy to use. We then discuss how analytics was used to create intelligent systems in order to allow a relatively small team to focus on innovation rather than operations.
Tessella, Abingdon, United Kingdom
STFC/RAL, Chilton, Didcot, Oxon, United Kingdom
The Ada Lovelace Centre (ALC) at STFC is an integrated, cross-disciplinary data intensive science centre, for better exploitation of research carried out at large scale UK Facilities including the Diamond Light Source, the ISIS Neutron and Muon Facility, the Central Laser Facility and the Culham Centre for Fusion Energy. ALC will provide on-demand, data analysis, interpretation and analytics services to worldwide users of these research facilities. Using open-source components, ALC and Tessella have together created a software infrastructure to support the delivery of that vision. The infrastructure comprises a Virtual Machine Manager, for managing pools of VMs across distributed compute clusters; components for automated provisioning of data analytics environments across heterogeneous clouds; a Data Movement System, to efficiently transfer large datasets; a Kubernetes cluster to manage on demand submission of Spark jobs. In this paper, we discuss the challenges of creating an infrastructure to meet the differing analytics needs of multiple facilities and report the architecture and design of the infrastructure that enables Data Analytics as a Service.