ICALEPCS2019 - List of Authors (Flucke, G.)

Paper	Title	Page
MOMPR004	Control and Analysis Software Development at the European XFEL	158
MOPHA126	use link to see paper's listing under its alternate paper code
	H. Santos, M. Beg, M. Bergemann, V. Bondar, S. Brockhauser, C. Carinan, R. Costa, F. Dall’Antonia, C. Danilevski, W. Ehsan, S.G. Esenov, R. Fabbri, H. Fangohr, G. Flucke, D. Fulla Marsa, G. Giovanetti, D. Goeries, S. Hauf, D.G. Hickin, T. Jarosiewicz, E. Kamil, Y. Kirienko, A. Klimovskaia, T.A. Kluyver, D. Mamchyk, T. Michelat, I. Mohacsi, A. Parenti, R. Rosca, D.B. Rück, R. Schaffer, A. Silenzi, M. Spirzewski, S. Trojanowski, C. Youngman, J. Zhu EuXFEL, Schenefeld, Germany S. Brockhauser BRC, Szeged, Hungary H. Fangohr University of Southampton, Southampton, United Kingdom
	Agile Project Management (Agile PM), coupled with the DevOps concept, has been worked out as a fundamental approach in a highly uncertain and unpredictable environment to achieve mature software development and to efficiently support concurrent operation. At the European XFEL, Agile PM and DevOps have been applied to provide adaptability and efficiency in the development and operation of its control system: Karabo. In this context, the Control and Analysis Software Group (CAS) has developed in-house a management platform composed of the following macro-artefacts: (1) Agile Process; (2) Release Planning; (3) Testing Infrastructure; (4) Roll-out and Deployment Strategy; (5) Automated tools for Monitoring Control Points (i.e. Configuration Items) and; (6) Incident Management**. The software engineering management platform is also integrated with User Relationship Management to establish and maintain a proper feedback loop with our scientists who set up the requirements. This article aims to briefly describe the above points and show how agile project management has guided the software strategy, development and operation of the Karabo control system at the European XFEL. Toward Project Management 2.0 The European X-ray Free Electron Laser technical design report *Karabo:An integrated software framework combining control, data management, and scientific comp.
	Poster MOMPR004 [0.871 MB]
DOI •	reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2019-MOMPR004
About •	paper received ※ 27 September 2019 paper accepted ※ 10 October 2019 issue date ※ 30 August 2020
Export •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MOPHA040	Beam Position Feedback System Supported by Karabo at European XFEL	281
	V. Bondar, M. Beg, M. Bergemann, S. Brockhauser, C. Carinan, R. Costa, F. Dall’Antonia, C. Danilevski, W. Ehsan, S.G. Esenov, R. Fabbri, H. Fangohr, G. Flucke, D. Fulla Marsa, A. Galler, G. Giovanetti, D. Goeries, J. Grünert, S. Hauf, D.G. Hickin, T. Jarosiewicz, E. Kamil, Y. Kirienko, A. Klimovskaia, T.A. Kluyver, D. Mamchyk, T. Michelat, I. Mohacsi, A. Parenti, D.B. Rück, H. Santos, R. Schaffer, A. Silenzi, C. Youngman, P. Zalden, J. Zhu EuXFEL, Schenefeld, Germany S. Brockhauser BRC, Szeged, Hungary H. Fangohr University of Southampton, Southampton, United Kingdom
	The XrayFeed device of Karabo [1, 2] is designed to provide spatial X-ray beam stability in terms of drift compensation utilizing different diagnostic components at the European XFEL (EuXFEL). Our feedback systems proved to be indispensable in cutting-edge pump-probe experiments at EuXFEL. The feedback mechanism is based on a closed loop PID control algorithm [3] to steer the beam position measured by a so-called diagnostic devices to the desired centered position via defined actuator adjusting the alignment of X-ray optical elements, in our case a flat X-ray mirror system. Several diagnostic devices and actuators can be selected according to the specific experimental area where a beam position feedback is needed. In this contribution, we analyze the improvement of pointing stability of X-rays using different diagnostic devices as an input source for our feedback system. Different types of photon diagnostic devices such as gas-based X-ray monitors [4], quadrant detectors based on avalanche photo diodes [5] and optical cameras imaging the X-ray footprint on scintillator screens have been evaluated in our pointing stability studies.
	Poster MOPHA040 [0.963 MB]
DOI •	reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2019-MOPHA040
About •	paper received ※ 30 September 2019 paper accepted ※ 08 October 2019 issue date ※ 30 August 2020
Export •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

TUCPR02	Data Exploration and Analysis with Jupyter Notebooks	799
	H. Fangohr, M. Beg, M. Bergemann, V. Bondar, S. Brockhauser, C. Carinan, R. Costa, F. Dall’Antonia, C. Danilevski, J.C. E, W. Ehsan, S.G. Esenov, R. Fabbri, S. Fangohr, G. Flucke, C. Fortmann-Grote, D. Fulla Marsa, G. Giovanetti, D. Goeries, S. Hauf, D.G. Hickin, T. Jarosiewicz, E. Kamil, M. Karnevskiy, Y. Kirienko, A. Klimovskaia, T.A. Kluyver, M. Kuster, L. Le Guyader, A. Madsen, L.G. Maia, D. Mamchyk, L. Mercadier, T. Michelat, J. Möller, I. Mohacsi, A. Parenti, M. Reiser, R. Rosca, D.B. Rück, T. Rüter, H. Santos, R. Schaffer, A. Scherz, M. Scholz, A. Silenzi, M. Spirzewski, J. Sztuk, J. Szuba, S. Trojanowski, K. Wrona, A.A. Yaroslavtsev, J. Zhu EuXFEL, Schenefeld, Germany S. Brockhauser BRC, Szeged, Hungary A. Campbell, A. Götz, J. Kieffer ESRF, Grenoble, France H. Fangohr University of Southampton, Southampton, United Kingdom E. Fernandez-del-Castillo, G. Sipos The EGI Foundation, Amsterdam, The Netherlands J. Hall, E. Pellegrini, J.F. Perrin ILL, Grenoble, France T. Holm Rod, J.R. Selknaes, J.W. Taylor ESS, Copenhagen, Denmark J. Reppin, F. Schlünzen, M. Schuh DESY, Hamburg, Germany
	Funding: With support from EU’s H{2}020 grants 823852 (PaNOSC) and #676541 (OpenDreamKit), the Gordon and Betty Moore Foundation GBMF #4856, the EPSRC’s CDT (EP/L015382/1) and program grant (EP/N032128/1). Jupyter notebooks are executable documents that are displayed in a web browser. The notebook elements consist of human-authored contextual elements and computer code, and computer-generated output from executing the computer code. Such outputs can include tables and plots. The notebook elements can be executed interactively, and the whole notebook can be saved, re-loaded and re-executed, or converted to read-only formats such as HTML, LaTeX and PDF. Exploiting these characteristics, Jupyter notebooks can be used to improve the effectiveness of computational and data exploration, documentation, communication, reproducibility and re-usability of scientific research results. They also serve as building blocks of remote data access and analysis as is required for facilities hosting large data sets and initiatives such as the European Open Science Cloud (EOSC). In this contribution we report from our experience of using Jupyter notebooks for data analysis at research facilities, and outline opportunities and future plans.
	Slides TUCPR02 [15.943 MB]
DOI •	reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2019-TUCPR02
About •	paper received ※ 24 September 2019 paper accepted ※ 20 October 2019 issue date ※ 30 August 2020
Export •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WECPR03	Status of the Karabo Control and Data Processing Framework	936
	G. Flucke, N. Al-Qudami, M. Beg, M. Bergemann, V. Bondar, D. Boukhelef, S. Brockhauser, C. Carinan, R. Costa, F. Dall’Antonia, C. Danilevski, W. Ehsan, S.G. Esenov, R. Fabbri, H. Fangohr, D. Fulla Marsa, G. Giovanetti, D. Goeries, S. Hauf, D.G. Hickin, E. Kamil, Y. Kirienko, A. Klimovskaia, T.A. Kluyver, D. Mamchyk, T. Michelat, I. Mohacsi, A. Muennich, A. Parenti, R. Rosca, D.B. Rück, H. Santos, R. Schaffer, A. Silenzi, K. Wrona, C. Youngman, J. Zhu EuXFEL, Schenefeld, Germany S. Brockhauser BRC, Szeged, Hungary H. Fangohr University of Southampton, Southampton, United Kingdom
	To achieve a tight integration of instrument control and (online) data analysis, the European XFEL decided in 2011 to develop Karabo, a custom control and data processing system. Karabo provides control via event-driven communication. Signal/slot and request/reply patterns are implemented via a central message broker. Data pipelines for e.g. scientific workflows or detector calibration are implemented as direct TCP/IP connections. The core entities of Karabo are self-describing devices written in C++ or Python. They represent hardware, orchestrate other devices, or provide system services like data logging and configuration storage. To operate Karabo, a Python command line interface and a generic GUI written in PyQt are provided. Control and data widgets compose Karabo scenes that are provided by devices or are manually customized and stored together with device configurations in a central database. Since 2016, Karabo is used to commission and operate the currently three photon beam lines and six scientific instruments at the European XFEL. This contribution summarizes the status of Karabo, highlights achievements and lessons learned, and gives an outlook for future directions. Heisen, B., et al. (2013) In 14th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS 2013. San Francisco, CA.
	Slides WECPR03 [2.660 MB]
DOI •	reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2019-WECPR03
About •	paper received ※ 27 September 2019 paper accepted ※ 09 October 2019 issue date ※ 30 August 2020
Export •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

Paper

Title

Page

MOMPR004

Control and Analysis Software Development at the European XFEL

158

MOPHA126

use link to see paper's listing under its alternate paper code

H. Santos, M. Beg, M. Bergemann, V. Bondar, S. Brockhauser, C. Carinan, R. Costa, F. Dall’Antonia, C. Danilevski, W. Ehsan, S.G. Esenov, R. Fabbri, H. Fangohr, G. Flucke, D. Fulla Marsa, G. Giovanetti, D. Goeries, S. Hauf, D.G. Hickin, T. Jarosiewicz, E. Kamil, Y. Kirienko, A. Klimovskaia, T.A. Kluyver, D. Mamchyk, T. Michelat, I. Mohacsi, A. Parenti, R. Rosca, D.B. Rück, R. Schaffer, A. Silenzi, M. Spirzewski, S. Trojanowski, C. Youngman, J. Zhu
EuXFEL, Schenefeld, Germany
S. Brockhauser
BRC, Szeged, Hungary
H. Fangohr
University of Southampton, Southampton, United Kingdom

Agile Project Management (Agile PM), coupled with the DevOps concept, has been worked out as a fundamental approach in a highly uncertain and unpredictable environment to achieve mature software development and to efficiently support concurrent operation*. At the European XFEL**, Agile PM and DevOps have been applied to provide adaptability and efficiency in the development and operation of its control system: Karabo***. In this context, the Control and Analysis Software Group (CAS) has developed in-house a management platform composed of the following macro-artefacts: (1) Agile Process; (2) Release Planning; (3) Testing Infrastructure; (4) Roll-out and Deployment Strategy; (5) Automated tools for Monitoring Control Points (i.e. Configuration Items****) and; (6) Incident Management*****. The software engineering management platform is also integrated with User Relationship Management to establish and maintain a proper feedback loop with our scientists who set up the requirements. This article aims to briefly describe the above points and show how agile project management has guided the software strategy, development and operation of the Karabo control system at the European XFEL.
*Toward Project Management 2.0
**The European X-ray Free Electron Laser technical design report
***Karabo:An integrated software framework combining control, data management, and scientific comp.

Poster MOMPR004 [0.871 MB]

DOI •

reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2019-MOMPR004

About •

paper received ※ 27 September 2019 paper accepted ※ 10 October 2019 issue date ※ 30 August 2020

Export •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MOPHA040

Beam Position Feedback System Supported by Karabo at European XFEL

281

V. Bondar, M. Beg, M. Bergemann, S. Brockhauser, C. Carinan, R. Costa, F. Dall’Antonia, C. Danilevski, W. Ehsan, S.G. Esenov, R. Fabbri, H. Fangohr, G. Flucke, D. Fulla Marsa, A. Galler, G. Giovanetti, D. Goeries, J. Grünert, S. Hauf, D.G. Hickin, T. Jarosiewicz, E. Kamil, Y. Kirienko, A. Klimovskaia, T.A. Kluyver, D. Mamchyk, T. Michelat, I. Mohacsi, A. Parenti, D.B. Rück, H. Santos, R. Schaffer, A. Silenzi, C. Youngman, P. Zalden, J. Zhu
EuXFEL, Schenefeld, Germany
S. Brockhauser
BRC, Szeged, Hungary
H. Fangohr
University of Southampton, Southampton, United Kingdom

The XrayFeed device of Karabo [1, 2] is designed to provide spatial X-ray beam stability in terms of drift compensation utilizing different diagnostic components at the European XFEL (EuXFEL). Our feedback systems proved to be indispensable in cutting-edge pump-probe experiments at EuXFEL. The feedback mechanism is based on a closed loop PID control algorithm [3] to steer the beam position measured by a so-called diagnostic devices to the desired centered position via defined actuator adjusting the alignment of X-ray optical elements, in our case a flat X-ray mirror system. Several diagnostic devices and actuators can be selected according to the specific experimental area where a beam position feedback is needed. In this contribution, we analyze the improvement of pointing stability of X-rays using different diagnostic devices as an input source for our feedback system. Different types of photon diagnostic devices such as gas-based X-ray monitors [4], quadrant detectors based on avalanche photo diodes [5] and optical cameras imaging the X-ray footprint on scintillator screens have been evaluated in our pointing stability studies.

Poster MOPHA040 [0.963 MB]

DOI •

reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2019-MOPHA040

About •

paper received ※ 30 September 2019 paper accepted ※ 08 October 2019 issue date ※ 30 August 2020

Export •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

TUCPR02

Data Exploration and Analysis with Jupyter Notebooks

799

H. Fangohr, M. Beg, M. Bergemann, V. Bondar, S. Brockhauser, C. Carinan, R. Costa, F. Dall’Antonia, C. Danilevski, J.C. E, W. Ehsan, S.G. Esenov, R. Fabbri, S. Fangohr, G. Flucke, C. Fortmann-Grote, D. Fulla Marsa, G. Giovanetti, D. Goeries, S. Hauf, D.G. Hickin, T. Jarosiewicz, E. Kamil, M. Karnevskiy, Y. Kirienko, A. Klimovskaia, T.A. Kluyver, M. Kuster, L. Le Guyader, A. Madsen, L.G. Maia, D. Mamchyk, L. Mercadier, T. Michelat, J. Möller, I. Mohacsi, A. Parenti, M. Reiser, R. Rosca, D.B. Rück, T. Rüter, H. Santos, R. Schaffer, A. Scherz, M. Scholz, A. Silenzi, M. Spirzewski, J. Sztuk, J. Szuba, S. Trojanowski, K. Wrona, A.A. Yaroslavtsev, J. Zhu
EuXFEL, Schenefeld, Germany
S. Brockhauser
BRC, Szeged, Hungary
A. Campbell, A. Götz, J. Kieffer
ESRF, Grenoble, France
H. Fangohr
University of Southampton, Southampton, United Kingdom
E. Fernandez-del-Castillo, G. Sipos
The EGI Foundation, Amsterdam, The Netherlands
J. Hall, E. Pellegrini, J.F. Perrin
ILL, Grenoble, France
T. Holm Rod, J.R. Selknaes, J.W. Taylor
ESS, Copenhagen, Denmark
J. Reppin, F. Schlünzen, M. Schuh
DESY, Hamburg, Germany

Funding: With support from EU’s H{2}020 grants 823852 (PaNOSC) and #676541 (OpenDreamKit), the Gordon and Betty Moore Foundation GBMF #4856, the EPSRC’s CDT (EP/L015382/1) and program grant (EP/N032128/1).
Jupyter notebooks are executable documents that are displayed in a web browser. The notebook elements consist of human-authored contextual elements and computer code, and computer-generated output from executing the computer code. Such outputs can include tables and plots. The notebook elements can be executed interactively, and the whole notebook can be saved, re-loaded and re-executed, or converted to read-only formats such as HTML, LaTeX and PDF. Exploiting these characteristics, Jupyter notebooks can be used to improve the effectiveness of computational and data exploration, documentation, communication, reproducibility and re-usability of scientific research results. They also serve as building blocks of remote data access and analysis as is required for facilities hosting large data sets and initiatives such as the European Open Science Cloud (EOSC). In this contribution we report from our experience of using Jupyter notebooks for data analysis at research facilities, and outline opportunities and future plans.

Slides TUCPR02 [15.943 MB]

DOI •

reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2019-TUCPR02

About •

paper received ※ 24 September 2019 paper accepted ※ 20 October 2019 issue date ※ 30 August 2020

Export •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WECPR03

Status of the Karabo Control and Data Processing Framework

936

G. Flucke, N. Al-Qudami, M. Beg, M. Bergemann, V. Bondar, D. Boukhelef, S. Brockhauser, C. Carinan, R. Costa, F. Dall’Antonia, C. Danilevski, W. Ehsan, S.G. Esenov, R. Fabbri, H. Fangohr, D. Fulla Marsa, G. Giovanetti, D. Goeries, S. Hauf, D.G. Hickin, E. Kamil, Y. Kirienko, A. Klimovskaia, T.A. Kluyver, D. Mamchyk, T. Michelat, I. Mohacsi, A. Muennich, A. Parenti, R. Rosca, D.B. Rück, H. Santos, R. Schaffer, A. Silenzi, K. Wrona, C. Youngman, J. Zhu
EuXFEL, Schenefeld, Germany
S. Brockhauser
BRC, Szeged, Hungary
H. Fangohr
University of Southampton, Southampton, United Kingdom

To achieve a tight integration of instrument control and (online) data analysis, the European XFEL decided in 2011 to develop Karabo*, a custom control and data processing system. Karabo provides control via event-driven communication. Signal/slot and request/reply patterns are implemented via a central message broker. Data pipelines for e.g. scientific workflows or detector calibration are implemented as direct TCP/IP connections. The core entities of Karabo are self-describing devices written in C++ or Python. They represent hardware, orchestrate other devices, or provide system services like data logging and configuration storage. To operate Karabo, a Python command line interface and a generic GUI written in PyQt are provided. Control and data widgets compose Karabo scenes that are provided by devices or are manually customized and stored together with device configurations in a central database. Since 2016, Karabo is used to commission and operate the currently three photon beam lines and six scientific instruments at the European XFEL. This contribution summarizes the status of Karabo, highlights achievements and lessons learned, and gives an outlook for future directions.
* Heisen, B., et al. (2013) In 14th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS 2013. San Francisco, CA.

Slides WECPR03 [2.660 MB]

DOI •

reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2019-WECPR03

About •

paper received ※ 27 September 2019 paper accepted ※ 09 October 2019 issue date ※ 30 August 2020

Export •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)