Search

Scientific workflows are powerful tools for the management of scalable experiments, often composed of complex tasks running on distributed resources. Existing cyberinfrastructure provides components that can be utilized within repeatable workflows. However, data and computing advances continuously change the way scientific workflows get developed and executed, pushing the scientific activity to be more data-driven, heterogeneous, and collaborative. Workflow development today depends on the effective collaboration and communication of a cross-disciplinary team, not only with humans but also with analytical systems and infrastructure. This paper presents a collaboration-centered reference architecture to extend workflow systems with dynamic, predictable, and programmable interfaces to systems and infrastructure while bridging the exploratory and scalable activities in the scientific process. We present a conceptual design toward the development of methodologies and services for effective workflow-driven collaborations, namely the PPoDS methodology for collaborative workflow development and the SmartFlows Services for smart execution in a rapidly evolving cyberinfrastructure ecosystem. [ABSTRACT FROM AUTHOR]

We present a method to integrate multiple implementations of a machine learning algorithm in Kepler actors. This feature enables the user to compare accuracy and scalability of various implementations of a machine learning technique without having to change the workflow. These actors are based on the Execution Choice actor. They can be incorporated into any workflow to provide machine learning functionality. We describe a use case where actors that provide several implementations of k-means clustering can be used in a workflow to process sensor data from weather stations for predicting wildfire risks. [ABSTRACT FROM AUTHOR]

Abstract: With more and more workflow systems adopting cloud as their e xecution environment, it becomes increasingly challenging on how to effic iently manage various workflows, v irtual machines (VMs) and workflow e xecution on VM instances. To make the system scalable and easy-to-extend, we design a Workflow as a Service (WFaaS) a rchitecture with independent services. A core part of the architecture is how to efficiently respond continuous workflo w requests from users and schedule their e xecutions in the c loud. Based on diffe rent targets, we p ropose four heuristic workflo w scheduling a lgorith ms for the WFaaS architecture, and analy ze the differences and best usages of the algorith ms in terms of performance, cost and the price/performance ratio via experimental studies. [Copyright &y& Elsevier]

Abstract: Distributed Data-Parallel (DDP) patterns such as MapReduce have become increasingly popular as solutions to facilitate data-intensive applications, resulting in a number of systems supporting DDP workflows. Yet, applications or workflows built using these patterns are usually tightly-coupled with the underlying DDP execution engine they select. We present a framework for distributed data-parallel execution in the Kepler scientific workflow system that enables users to easily switch between different DDP execution engines. We describe a set of DDP actors based on DDP patterns and directors for DDP workflow executions within the presented framework. We demonstrate how DDP workflows can be easily composed in the Kepler graphic user interface through the reuse of these DDP actors and directors and how the generated DDP workflows can be executed in different distributed environments. Via a bioinformatics usecase, we discuss the usability of the proposed framework and validate its execution scalability. [Copyright &y& Elsevier]