Search

Workflow optimization is an approach to enhance the speed, robustness, and compactness of workflows by exploiting their structure, runtime, and output. This chapter initially highlights the significance of workflow optimization along with different possible levels of optimization. Further, it outlines the Taverna optimization framework over single and distributed infrastructure together with the optimization plug-ins that are validated using two scientific workflow executions.

Many scientific applications are often modeled as workflows. The data and computational resource requirements are high for such workflow applications. Cloud provides a better solution to this problem by offering the promising environment for the execution of these workflow. As it involves tremendous data computations and resources, there is a need to automate the entire process. Workflow management system serves this purpose by orchestrating workflow task and executing it on distributed resources. Pegasus is a well-known workflow management system that has been widely used in large-scale e-applications. This chapter provides an overview about the Pegasus Workflow Management System, describes the environmental setup with OpenStack and creation and execution of workflows in Pegasus, and discusses about the workflow scheduling in cloud with its issues.

Modeling and simulation of scientific workflow play a vital role in resource allocation in a distributed environment. Simulation is one of the methods to solve the complex scientific workflows in distributed environment. There are many scientific workflow simulation software frameworks that are available for grid and cloud environment. WorkflowSim is an open-source simulator. WorkflowSim Simulator extends the existing CloudSim Simulator. The architecture, components, and scheduling algorithms used and also the simulation results are explained for CloudSim Simulator and WorkflowSim Simulator.

Cloud infrastructures typically offer access to boundless virtual resources dynamically provisioned on demand for hosting, running, and managing a variety of mission-critical applications like scientific workflows, big data processing application, business intelligence-based applications, high-performance computing (HTC), and high transaction computing (HTC). Due to the surging popularity of the irresistible cloud idea, there are cloud datacenters spreading across the globe comprising heterogeneous cloud platforms and infrastructures catering to fast-evolving demands of worldwide businesses. The pervasive connectivity has enabled for the unprecedented success of the cloud concept. However, intensive automation is the key to the originally intended success of the cloud paradigm. Researchers across the world are focusing on unearthing powerful and pioneering tools and techniques for automated infrastructure life-cycle management. Similarly there are pathbreaking work-around approaches, algorithms, and architectures for workload consolidation. In short, there are many cloud-related aspects yearning for technologically sound automation, acceleration, and augmentation capabilities.

Today’s scientific application requires tremendous amount of computation-driven as well as data-driven supported resources. The scientific applications are represented as workflows. The workflow management systems are designed and developed to depict the workflows of complex nature. The workflow management systems are able to reliably and efficiently coordinate among various resources in a distributed environment. This chapter describes various workflow management software like Kepler, Taverna, Triana, Pegasus, and Askalon. The architecture and functionalities of these workflow management systems are explained in the following sections.

Workflow execution employs predictive analytics to extract significant, unidentified as well as precious insights from several stages of execution. Further, the operational analytics integrates these valuable insights directly into decision engine which enables analytical as well as machine learning-driven decision-making for an efficient workflow execution. This chapter highlights several analytical and machine learning approaches that are practiced in workflow predictions. Additionally, it explains the significance of hybrid approach which includes both analytical and machine learning models for workflow prediction. Finally, it describes the hybrid approach employed in PANORAMA architecture using two workflow applications.