1. Staged deployment of interactive multi-application HPC workflows
- Author
-
Abigail Morrison, Alexander Peyser, Sandra Diaz-Pier, and Wouter Klijn
- Subjects
FOS: Computer and information sciences ,0303 health sciences ,business.industry ,Computer science ,Computer Science - Human-Computer Interaction ,computer.software_genre ,Supercomputer ,Human-Computer Interaction (cs.HC) ,Visualization ,03 medical and health sciences ,Task (computing) ,0302 clinical medicine ,Workflow ,Computer Science - Distributed, Parallel, and Cluster Computing ,Software deployment ,Middleware (distributed applications) ,Robot ,Use case ,Distributed, Parallel, and Cluster Computing (cs.DC) ,Software engineering ,business ,computer ,030217 neurology & neurosurgery ,030304 developmental biology - Abstract
Running scientific workflows on a supercomputer can be a daunting task for a scientific domain specialist. Workflow management solutions (WMS) are a standard method for reducing the complexity of application deployment on high performance computing (HPC) infrastructure. We introduce the design for a middleware system that extends and combines the functionality from existing solutions in order to create a high-level, staged user-centric operation/deployment model. This design addresses the requirements of several use cases in the life sciences, with a focus on neuroscience. In this manuscript we focus on two use cases: 1) three coupled neuronal simulators (for three different space/time scales) with in-transit visualization and 2) a closed-loop workflow optimized by machine learning, coupling a robot with a neural network simulation. We provide a detailed overview of the application-integrated monitoring in relationship with the HPC job. We present here a novel usage model for large scale interactive multi-application workflows running on HPC systems which aims at reducing the complexity of deployment and execution, thus enabling new science., Comment: 7 pages, 3 figures, The 2019 International Conference on High Performance Computing & Simulation
- Published
- 2019
- Full Text
- View/download PDF