Search

Complex simulations are rarely performed in clouds because of huge overheads and latencies that occur in a virtual environment. Also, in many cases, standard communication channels based on TCP/IP could not bring any performance gains if a multi-VM setup is engaged. Therefore, clouds have to be tuned and modified in order to be used as HPC computing tools. In this paper, we present a rebuild of Magpar, a perspective micromagnetic solver that became broken with recent Linux updates. We have restored it to the original version and further updated its obsolete MPI communication libraries with the latest MPI-3 standard at the time. However, in order to achieve significant performance improvements, we had to break the existing VM isolation by engaging inter-VM shared-memory or ivshmem, which comes in a form of a PCI device. For this purpose, we have used free MVAPICH2-virt software that has built-in support for ivshmem and then integrated it into Magpar. As a result, we have achieved that Magpar is run for the first time in OpenStack cloud using ivshmem as a shared memory channel. Furthermore, we have shown the relation between the number of processing elements and initial problem size with respect to the saturation of computing time speedup. Finally, our measurements showed improvements in execution time between factors of 1.6 - 6, depending on the number of vCPUs and VMs engaged, compared with the standard TCP/IP communication channel.

The MediGRID [1] project, which is part of the German e-Science initiative D-Grid, aims to provide a community Grid for researchers in the fields of bioinformatics, medical image processing, biomedical ontology, and clinical research applications. Users in the life science domain usually do not have a strong computer science background. Therefore, it is crucial to make the Grid solution as user friendly as possible. To achieve this goal, MediGRID is building up an infrastructure that follows two principles: gridification and virtualization.The basic Grid middleware used in MediGRID is the Globus Toolkit 4, which provides services for security, execution and data management, and resource discovery. On top of Globus, MediGRID employs further specialized middlewares. The first is the Grid Workflow Execution Service (GWES) [2] that orchestrates the distributed execution of workflows on Grid resources. Application workflows consist of several successive program executions and intermediate data transfers. Second, the Storage Resource Broker (SRB) [3] Data Grid middleware is utilized to handle large amounts of distributed data and corresponding metadata. We use SRB to create a global logical namespace in which file replica can be accessed by unique file names, independent of the physical locations. At the top level, a portal framework is employed to provide a convenient graphical user interface.The middleware layers allow to virtualize the MediGRID resources. This means, end users do not interact with specific clusters and storage elements but deal with a high-level abstraction of the Grid. For this, extensive gridification work is necessary to set up the Grid infrastructure and adapt the applications to Grid computing. We will demonstrate the whole process with the gene-finding application AUGUSTUS [4].

Grid computing stands for the effort undertaken mainly by computing centers to open up and combine their resources for an enhanced availability. There is a growing demand for an automatic balance of inter-infrastructure resource requests that existing middleware such as UNICORE and Globus Tool Kit is ill-suited to satisfy, as it it requires the user to provide the location of suitable resources and only facilitates the migration process. Other projects like Gridway or LSF Multicluster suffer (at least currently) from missing interoperability. We describe a distributed, failure-resilient meta-scheduling architecture that allows the automatic exchange of job requests between resource providers, aiming at improved resource utilization, automatic load-balancing, as well as reduced turn-around times. Additionally, the system tries to achieve grid-wide improvements while still preserving the autonomy of resource providers. This is accomplished by making all decisions locally.