Search

Correct and reliable linkage of independently produced information is a requirement to enable sophisticated applications and processing workflows. These can ultimately help address the challenges posed by complex systems (such as socio-ecological systems), whose many components can only be described through independently developed data and model products. We discuss the first outcomes of an investigation in the conceptual and methodological aspects of semantic annotation of data and models, aimed to enable a high standard of interoperability of information. The results, operationalized in the context of a long-term, active, large-scale project on ecosystem services assessment, include: A definition of interoperability based on semantics and scale; A conceptual foundation for the phenomenology underlying scientific observations, aimed to guide the practice of semantic annotation in domain communities; A dedicated language and software infrastructure that operationalizes the findings and allows practitioners to reap the benefits of data and model interoperability. The work presented is the first detailed description of almost a decade of work with communities active in socio-ecological system modeling. After defining the boundaries of possible interoperability based on the understanding of scale, we discuss examples of the practical use of the findings to obtain consistent, interoperable and machine-ready semantic specifications that can integrate semantics across diverse domains and disciplines.

Provenance of scientific workflows has been considered a mean to provide workflow reproducibility. However, the provenance approaches adopted so far are not applicable in the context of Cloud because the provenance trace lacks the Cloud information. This paper presents a novel approach that collects the Cloud-aware provenance and represents it as a graph. The workflow execution reproducibility on the Cloud is determined by comparing the workflow provenance at three levels i.e., workflow structure, execution infrastructure and workflow outputs. The experimental evaluation shows that the implemented approach can detect changes in the provenance traces and the outputs produced by the workflow. [ABSTRACT FROM AUTHOR]

Advances in sequencing techniques have led to exponential growth in biological data, demanding the development of large-scale bioinformatics experiments. Because these experiments are computation- and data-intensive, they require high-performance computing techniques and can benefit from specialized technologies such as Scientific Workflow Management Systems and databases. In this work, we present BioWorkbench, a framework for managing and analyzing bioinformatics experiments. This framework automatically collects provenance data, including both performance data from workflow execution and data from the scientific domain of the workflow application. Provenance data can be analyzed through a web application that abstracts a set of queries to the provenance database, simplifying access to provenance information. We evaluate BioWorkbench using three case studies: SwiftPhylo, a phylogenetic tree assembly workflow; SwiftGECKO, a comparative genomics workflow; and RASflow, a RASopathy analysis workflow. We analyze each workflow from both computational and scientific domain perspectives, by using queries to a provenance and annotation database. Some of these queries are available as a pre-built feature of the BioWorkbench web application. Through the provenance data, we show that the framework is scalable and achieves high-performance, reducing up to 98% of the case studies execution time. We also show how the application of machine learning techniques can enrich the analysis process. [ABSTRACT FROM AUTHOR]

Applications structured as Directed Acyclic Graphs (DAGs) of tasks occur in many domains, including popular scientific workflows. DAG scheduling has thus received an enormous amount of attention. Many of the popular DAG scheduling heuristics make scheduling decisions based on path lengths. At large scale compute platforms are subject to various types of failures with non-negligible probabilities of occurrence. Failures that have recently received increased attention are “silent errors,” which cause data corruption. Tolerating silent errors is done by checking the validity of computed results and possibly re-executing a task from scratch. The execution time of a task then becomes a random variable, and so do path lengths in a DAG. Unfortunately, computing the expected makespan of a DAG (and equivalently computing expected path lengths in a DAG) is a computationally difficult problem. Consequently, designing effective scheduling heuristics in this context is challenging. In this work, we propose an algorithm that computes a first order approximation of the expected makespan of a DAG when tasks are subject to silent errors. We find that our proposed approximation outperforms previously proposed approaches both in terms of approximation error and of speed. [ABSTRACT FROM AUTHOR]

Scientific workflows have become a standardized way for scientists to represent a set of tasks to overcome/solve a certain scientific problem. Usually these workflows consist of numerous CPU and I/O-intensive jobs that are executed using workflow management systems (WfMS), on clouds, grids, supercomputers, etc. Previously, it was shown that using k-way partitioning to distribute a workflow’s tasks between multiple machines in the cloud reduces the overall data communication and therefore lowers the cost of the bandwidth usage. A framework was built to automate this process of partitioning and execution of any workflow submitted by a scientist that is meant to be run on Pegasus WfMS, in the cloud, with ease. The framework provisions the instances in the cloud using CloudML, configures and installs all the software needed for the execution, partitions and runs the provided scientific workflow, also showing the estimated makespan and cost. [ABSTRACT FROM AUTHOR]

Cloud service providers are offering computing resources at a reasonable price as a pay-per-use model. Further, cloud service providers have also introduced different pricing models like spot, blockspot and spotfleet instances that are cost effective and user’s have to go through the bidding to balance the reliability and monetary costs. Henceforth, Scientific Workflows (SWf) that are used to model applications of high throughput, computation and complex large-scale data analysis are significantly adopting these computing resources. Nevertheless, spot instances are terminated when the market spot price exceeds the users bid price. Moreover, failures are inevitable in such a large distributed systems and often pose a challenge to design a fault-tolerant scheduling algorithm for SWf. This paper presents an efficient, low-cost and fault-tolerant scheduling algorithm and a bidding strategy to minimize the volatility and cost of resource provisioning for SWf. The proposed algorithm uses spot and blockspot instances as hybrid instances in comparison with on-demand instance to reduce the execution cost and fault-tolerant while meeting the SWf deadline. The results obtained reveal the promising potential of the proposed scheduling algorithm and are demonstrated through empirical simulation study that is robust under short deadlines with minimal makespan and cost. [ABSTRACT FROM AUTHOR]

In the modern era, workflows are adopted as a powerful and attractive paradigm for expressing/solving a variety of applications like scientific, data intensive computing, and big data applications such as MapReduce and Hadoop. These complex applications are described using high-level representations in workflow methods. With the emerging model of cloud computing technology, scheduling in the cloud becomes the important research topic. Consequently, workflow scheduling problem has been studied extensively over the past few years, from homogeneous clusters, grids to the most recent paradigm, cloud computing. The challenges that need to be addressed lies in task-resource mapping, QoS requirements, resource provisioning, performance fluctuation, failure handling, resource scheduling, and data storage. This work focuses on the complete study of the resource provisioning and scheduling algorithms in cloud environment focusing on Infrastructure as a service (IaaS). We provided a comprehensive understanding of existing scheduling techniques and provided an insight into research challenges that will be a possible future direction to the researchers. [ABSTRACT FROM AUTHOR]

Today’s computational, experimental, and observational sciences rely on computations that involve many related tasks. The success of a scientific mission often hinges on the computer automation of these workflows. In April 2015, the US Department of Energy (DOE) invited a diverse group of domain and computer scientists from national laboratories supported by the Office of Science, the National Nuclear Security Administration, from industry, and from academia to review the workflow requirements of DOE’s science and national security missions, to assess the current state of the art in science workflows, to understand the impact of emerging extreme-scale computing systems on those workflows, and to develop requirements for automated workflow management in future and existing environments. This article is a summary of the opinions of over 50 leading researchers attending this workshop. We highlight use cases, computing systems, workflow needs and conclude by summarizing the remaining challenges this community sees that inhibit large-scale scientific workflows from becoming a mainstream tool for extreme-scale science. [ABSTRACT FROM AUTHOR]

The reproduction and replication of research results has become a major issue for a number of scientific disciplines. In computer science and related computational disciplines such as systems biology, the challenges closely revolve around the ability to implement (and exploit) novel algorithms and models. Taking a new approach from the literature and applying it to a new codebase frequently requires local knowledge missing from the published manuscripts and transient project websites. Alongside this issue, benchmarking, and the lack of open, transparent and fair benchmark sets present another barrier to the verification and validation of claimed results. In this paper, we outline several recommendations to address these issues, driven by specific examples from a range of scientific domains. Based on these recommendations, we propose a high-level prototype open automated platform for scientific software development which effectively abstracts specific dependencies from the individual researcher and their workstation, allowing easy sharing and reproduction of results. This new e-infrastructure for reproducible computational science offers the potential to incentivise a culture change and drive the adoption of new techniques to improve the quality and efficiency – and thus reproducibility – of scientific exploration.

Background: Network generation tools coupled with chemical reaction rules have been mainly developed for synthesis planning and more recently for metabolic engineering. Using the same core algorithm, these tools apply a set of rules to a source set of compounds, stopping when a sink set of compounds has been produced. When using the appropriate sink, source and rules, this core algorithm can be used for a variety of applications beyond those it has been developed for. Results: Here, we showcase the use of the open source workflow RetroPath2.0. First, we mathematically prove that we can generate all structural isomers of a molecule using a reduced set of reaction rules. We then use this enumeration strategy to screen the chemical space around a set of monomers and predict their glass transition temperatures, as well as around aminoglycosides to search structures maximizing antibacterial activity. We also perform a screening around aminoglycosides with enzymatic reaction rules to ensure biosynthetic accessibility. We finally use our workflow on an E. coli model to complete E. coli metabolome, with novel molecules generated using promiscuous enzymatic reaction rules. These novel molecules are searched on the MS spectra of an E. coli cell lysate interfacing our workflow with OpenMS through the KNIME Analytics Platform. Conclusion: We provide an easy to use and modify, modular, and open-source workflow. We demonstrate its versatility through a variety of use cases including molecular structure enumeration, virtual screening in the chemical space, and metabolome completion. Because it is open source and freely available on MyExperiment.org, workflow community contributions should likely expand further the features of the tool, even beyond the use cases presented in the paper. [ABSTRACT FROM AUTHOR]

Data-driven research requires interactive systems supporting fast and intuitive data exploration. An important component is the user interface that facilitates this process. In biodiversity research, data is commonly of spatio-temporal nature. This poses unique opportunities for visual analytics approaches. In this paper we present the core concepts of the web-based front end of our vat (Visualization, Analysis and Transformation) system, a distributed geo-processing application. We present the results of two user studies and highlight unique features, among others for the management of time and the generalization of data. [ABSTRACT FROM AUTHOR]

This paper presents dispel4py, a new Python framework for describing abstract stream-based workflows for distributed data-intensive applications. These combine the familiarity of Python programming with the scalability of workflows. Data streaming is used to gain performance, rapid prototyping and applicability to live observations. dispel4py enables scientists to focus on their scientific goals, avoiding distracting details and retaining flexibility over the computing infrastructure they use. The implementation, therefore, has to map dispel4py abstract workflows optimally onto target platforms chosen dynamically. We present four dispel4py mappings: Apache Storm, message-passing interface (MPI), multi-threading and sequential, showing two major benefits: a) smooth transitions from local development on a laptop to scalable execution for production work, and b) scalable enactment on significantly different distributed computing infrastructures. Three application domains are reported and measurements on multiple infrastructures show the optimisations achieved; they have provided demanding real applications and helped us develop effective training. The dispel4py.org is an open-source project to which we invite participation. The effective mapping of dispel4py onto multiple target infrastructures demonstrates exploitation of data-intensive and high-performance computing (HPC) architectures and consistent scalability. [ABSTRACT FROM AUTHOR]

Autoscaling strategies achieve efficient and cheap executions of scientific workflows running in the cloud by determining appropriate type and amount of virtual machine instances to use while scheduling the tasks/data. Current strategies only consider on-demand instances ignoring the advantages of a mixed cloud infrastructure comprising also spot instances. Although the latter type of instances are subject to failures and therefore provide an unreliable infrastructure, they potentially offer significant cost and time improvements if used wisely. This paper discusses a novel autoscaling strategy with two features. First, it combines both types of instances to acquire a better cost-performance balance in the infrastructure. And second, it uses heuristic scheduling to deal with the unreliability of spot instances. Simulated experiments based on 4 scientific workflows showed substantial makespan and cost reductions of our strategy when compared with a reference strategy from the state of the art entitled Scaling First. These promising results represent a step towards new and better strategies for workflow autoscaling in the cloud. [ABSTRACT FROM AUTHOR]

Metadata for hydrologic models is rarely organized in machine-readable forms. This lack of formal metadata is important because it limits the ability to catalog, identify, attribute, and understand unique model software; ultimately, it hinders the ability to reproduce past computational studies. Researchers have recently proposed an ontology for scientific software metadata called OntoSoft for addressing this problem. The objective of this research is to evaluate OntoSoft for organizing the metadata associated with a data pre-processing software workflow used in association with the Variable Infiltration Capacity (VIC) hydrologic model. This is accomplished by exploring what metadata are available from online resources and how this metadata aligns with the OntoSoft Ontology. The results suggest that past efforts to document this software resulted in capturing key model metadata in unstructured files that could be formalized into a machine-readable form using the OntoSoft Ontology. [ABSTRACT FROM AUTHOR]

Workflows management systems (WfMS) are aimed for designing, scheduling, executing, reusing, and sharing workflows in distributed environments like the Cloud computing. With the emergence of e-science workflows, which are used in different domains like astronomy, life science, and physics, to model and execute vast series of dependents functionalities and a large amount of manipulated data, the workflow management systems are required to provide customizable programming environments to ease the programming effort required by scientists to orchestrate a computational science experiment. A key issue for e-science WfMS is how to deal with the change of the execution environment constraints and the variability and confliction of end users and cloud providers objectives for the execution of the same workflow or sub-workflow. They have to customize their management processes to insure the adaptability of the execution environment to the scientific workflows specificities, especially when dealing with large-scale (data, computing, I/O)-intensive workflows. In this paper, we propose a dynamically re-configurable framework for the deployment of scientific workflows in the Cloud (called DR-SWDF) that allows customizing the workflow deployment process according to a set of objectives and constraints of end users or cloud providers defined differently for the tasks or partitions of the same workflow. The DR-SWDF framework offers a K-means based algorithm that allows dynamically clustering the input workflows or sub-workflows in order to identify the most convenient techniques or algorithms to be applied for their scheduling and deployment in the cloud. The simulations results run on three examples large-scale scientific workflows show that our proposed framework can achieve better results than the use of a generic purpose approach. [ABSTRACT FROM AUTHOR]

Scientific workflows are designed to solve complex scientific problems and accelerate scientific progress. Ideally, scientific workflows should improve the reproducibility of scientific applications by making it easier to share and reuse workflows between scientists. However, scientists often find it difficult to reuse others’ workflows, which is known as workflow decay . In this paper, we explore the challenges in reproducing scientific workflows, and propose a framework for facilitating the reproducibility of scientific workflows at the task level by giving scientists complete control over the execution environments of the tasks in their workflows and integrating execution environment specifications into scientific workflow systems. Our framework allows dependencies to be archived in basic units of OS image, software and data instead of gigantic all-in-one images. We implement a prototype of our framework by integrating Umbrella , an execution environment creator, into Makeflow , a scientific workflow system. To evaluate our framework, we use it to run two bioinformatics scientific workflows, BLAST and BWA . The execution environment of the tasks in each workflow is specified as an Umbrella specification file, and sent to execution nodes where Umbrella is used to create the specified environment for running the tasks. For each workflow we evaluate the size of the Umbrella specification file, the time and space overheads of creating execution environments using Umbrella , and the heterogeneity of execution nodes contributing to each workflow. The evaluation results show that our framework improves the utilization of heterogeneous computing resources, and improves the portability and reproducibility of scientific workflows. [ABSTRACT FROM AUTHOR]

Geo-distributed clouds are becoming increasingly popular for cloud providers, and data centers with different regions often offer different prices, even for the same type of virtual machines. Resource provisioning in geo-distributed clouds is an important and complicated problem for budget and performance optimizations of scientific workflows. Scientists are facing the complexities resulted from various cloud offerings in the geo-distributed settings, severe cloud performance dynamics and evolving user requirements on performance and cost. To address those complexities, we propose a declarative optimization engine named Geco for resource provisioning of scientific workflows in geo-distributed clouds. Geco allows users to specify their workflow optimization goals and constraints of specific problems with an extended declarative language. We propose a novel probabilistic optimization approach for evaluating the declarative optimization goals and constraints to address the cloud dynamics. Additionally, we develop runtime optimizations to more effectively utilize the cloud resources at runtime. To accelerate the solution finding, Geco leverages the power of GPUs to find the solution in a fast and timely manner. Our evaluations with four common workflow provisioning problems demonstrate that, Geco is able to achieve more effective performance/cost optimizations in geo-distributed cloud environments than the state-of-the-art approaches. [ABSTRACT FROM PUBLISHER]

Now-a-days, Cloud computing is a technology which eludes provision cost while providing scalability and elasticity to accessible resources on a pay-per-use basis. To satisfy the increasing demand of the computing power to execute large scale scientific workflow applications, workflow scheduling is the main challenging issue in Infrastructure-as-a-Service (IaaS) clouds. As workflow scheduling belongs to NP-complete problem, so, meta-heuristic approaches are more preferred option. Users often specified deadline and budget constraint for scheduling these workflow applications over cloud resources. But these constraints are in conflict with each other, i.e., the cheaper resources are slow as compared to the expensive resources. Most of the existing studies try to optimize only one of the objectives, i.e., either time minimization or cost minimization under user specified Quality of Service (QoS) constraints. But due to the complexity of workflows and dynamic nature of cloud, a trade-off solution is required to make a balance between execution time and processing cost. To address these issues, this paper presents a non-dominance sort based Hybrid Particle Swarm Optimization (HPSO) algorithm to handle the workflow scheduling problem with multiple conflicting objective functions on IaaS clouds. The proposed algorithm is a hybrid of our previously proposed Budget and Deadline constrained Heterogeneous Earliest Finish Time (BDHEFT) algorithm and multi-objective PSO. The HPSO heuristic tries to optimize two conflicting objectives, namely, makespan and cost under the deadline and budget constraints. Along with these two conflicting objectives, energy consumed of created workflow schedule is also minimized. The proposed algorithm gives a set of Pareto Optimal solutions from which the user can choose the best solution. The performance of proposed heuristic is compared with state-of-art multi-objective meta-heuristics like NSGA-II, MOPSO, and ε -FDPSO. The simulation analysis substantiates that the solutions obtained with proposed heuristic deliver better convergence and uniform spacing among the solutions as compared to others. Hence it is applicable to solve a wide class of multi-objective optimization problems for scheduling scientific workflows over IaaS clouds. [ABSTRACT FROM AUTHOR]

Scientific workflows are used for dealing with complex problems in different e-science domains. These workflows are modeled and executed using Scientific Workflow Management Systems (SWfMSs). Generally, SWfMSs provide their own Workflow Specification Language, and this is a challenge considering the possibility of interchanging workflow specifications between different SWfMSs. In this paper, we propose the use of workflow patterns combined with software architecture concepts to capture the key semantics expressed in scientific workflows specified in different Workflow Specification Languages. These concepts are materialized in a novel specification interchange language (and associated tooling) called Workflow Interchange in Science with Patterns (WISP), which is the main contribution of our work. We have evaluated WISP based on real world specifications; in this paper, we present the results of such evaluation (including an assessment of the quality of the research material available), which has eventually led us to reformulate our initial proposal with more stringent assumptions on the input and output specifications. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Task clustering is a compute intensive method that will reduce execution overhead there by improving the computational granularity in clouds. Usually a job is composed of one or more tasks. The jobs with multiple tasks are always having higher risk of failures when compared to single task job. Clustering strategies can be used to reduce the impact of job failures. Fault tolerant clustering methods are used to enhance the runtime execution of workflow executions. Static task clustering is the widely employed clustering method for faulty environments. The proposed work utilizes Dynamic task clustering to improve the workflow make span by dynamically modifying the clustering granularity whenever there arises chances of failure. The proposed method performs well to adapt unexpected behaviours and provides better make-spans when compared to the static method. This paper discusses the implication of the rise of big data and especially that of high velocity data in the domain of high frequency trading (HFT), a growing niche of securities trading. We first take a brief look at the intricacies of HFT including some of the commonly used strategies used by HFT traders. The technological challenges in processing HFT and responding to the real time changes in the market conditions are also discussed. Some of the potential technological solutions to solve the issues thrown up by HFT are analyzed for their effectiveness to address the real time performance requirements of HFT. We identify Complex event processing (CEP) as a candidate to address the HFT problem. The paper is divided into 3 parts; part A deals with understanding HFT and the challenges that it poses to the technological processing. In Part B we look at complex Event Processing (CEP) and the types of problems it can be applied to. In Part C we show a framework to process HFT using techniques derived from CEP. [ABSTRACT FROM AUTHOR]

Quantitative knowledge of intracellular fluxes in metabolic networks is invaluable for inferring metabolic system behavior and the design principles of biological systems. However, intracellular reaction rates can not often be calculated directly but have to be estimated; for instance, via 13C-based metabolic flux analysis, a model-based interpretation of stable carbon isotope patterns in intermediates of metabolism. Existing software such as FiatFlux, OpenFLUX or 13CFLUX supports experts in this complex analysis, but requires several steps that have to be carried out manually, hence restricting the use of this software for data interpretation to a rather small number of experiments. In this paper, we present Flux-P as an approach to automate and standardize 13C-based metabolic flux analysis, using the Bio-jETI workflow framework. Exemplarily based on the FiatFlux software, it demonstrates how services can be created that carry out the different analysis steps autonomously and how these can subsequently be assembled into software workflows that perform automated, high-throughput intracellular flux analysis of high quality and reproducibility. Besides significant acceleration and standardization of the data analysis, the agile workflow-based realization supports flexible changes of the analysis workflows on the user level, making it easy to perform custom analyses.

Cloud computing is a technology wherein a network of remote servers is used to process large amount of data in real-time. The servers and data sources may be located in geographically distant regions. Scheduling of workflows is one of the major challenging issues in cloud computing. Workflows are used to express a wide variety of applications including scientific computing and multi-tier web applications. The Workflow scheduling problem is known to be NP-complete. No known traditional scheduling algorithm is able to provide an optimal solution in polynomial time for NP-complete problems. So, researchers rely on heuristics and meta-heuristics to achieve the most efficient solution. In this paper, a workflow scheduling algorithm is proposed to schedule large scientific workflows that are to be executed on IaaS clouds. The workflow scheduling algorithm generates a schedule with the task-to-resource mapping. The metaheuristic Catfish particle swarm optimization (C-PSO) technique is used to select the best schedule with the least makespan and execution cost. The performance of C-PSO is then compared with traditional PSO. The algorithm is simulated on the WorkFlowSim Simulator, an extension of CloudSim simulator. The solution is tested for different types of scientific workflows like Montage, Epigenome, CyberShake and Inspiral. It is observed from the experimental results that C-PSO gives better performance than traditional PSO in terms of execution cost and makespan. [ABSTRACT FROM AUTHOR]

The jABC is a framework for process modelling and execution according to the XMDD (eXtreme model-driven design) paradigm, which advocates the rigorous use of user-level models in the software development process and software life cycle. We have used the jABC in the domain of scientific workflows for more than a decade now-an occasion to look back and take stock of our experiences in the field. On the one hand, we discuss results from the analysis of a sample of nearly 100 scientific workflow applications that have been implemented with the jABC. On the other hand, we reflect on our experiences and observations regarding the workflow development process with the framework. We then derive and discuss ongoing further developments and future perspectives for the framework, all with an emphasis on simplicity for end users through increased domain specificity. Concretely, we describe how the use of the PROPHETS synthesis plugin can enable a semantics-based simplification of the workflow design process, how with the jABC4 and DyWA frameworks more attention is paid to the ease of data management, and how the Cinco SCCE Meta-Tooling Suite can be used to generate tailored workflow management tools. [ABSTRACT FROM AUTHOR]

Over the past decade process mining has emerged as a new analytical discipline able to answer a variety of questions based on event data. Event logs have a very particular structure; events have timestamps, refer to activities and resources, and need to be correlated to form process instances. Process mining results tend to be very different from classical data mining results, e.g., process discovery may yield end-to-end process models capturing different perspectives rather than decision trees or frequent patterns. A process-mining tool like ProM provides hundreds of different process mining techniques ranging from discovery and conformance checking to filtering and prediction. Typically, a combination of techniques is needed and, for every step, there are different techniques that may be very sensitive to parameter settings. Moreover, event logs may be huge and may need to be decomposed and distributed for analysis. These aspects make it very cumbersome to analyze event logs manually. Process mining should be repeatable and automated. Therefore, we propose a framework to support the analysis of process mining workflows. Existing scientific workflow systems and data mining tools are not tailored towards process mining and the artifacts used for analysis (process models and event logs). This paper structures the basic building blocks needed for process mining and describes various analysis scenarios. Based on these requirements we implemented RapidProM, a tool supporting scientific workflows for process mining. Examples illustrating the different scenarios are provided to show the feasibility of the approach. [ABSTRACT FROM AUTHOR]

Background: In Quantum Chemistry, many tasks are reoccurring frequently, e.g. geometry optimizations, benchmarking series etc. Here, workflows can help to reduce the time of manual job definition and output extraction. These workflows are executed on computing infrastructures and may require large computing and data resources. Scientific workflows hide these infrastructures and the resources needed to run them. It requires significant efforts and specific expertise to design, implement and test these workflows. Significance: Many of these workflows are complex and monolithic entities that can be used for particular scientific experiments. Hence, their modification is not straightforward and it makes almost impossible to share them. To address these issues we propose developing atomic workflows and embedding them in meta-workflows. Atomic workflows deliver a well-defined research domain specific function. Publishing workflows in repositories enables workflow sharing inside and/or among scientific communities. We formally specify atomic and meta-workflows in order to define data structures to be used in repositories for uploading and sharing them. Additionally, we present a formal description focused at orchestration of atomic workflows into meta-workflows. Conclusions: We investigated the operations that represent basic functionalities in Quantum Chemistry, developed the relevant atomic workflows and combined them into meta-workflows. Having these workflows we defined the structure of the Quantum Chemistry workflow library and uploaded these workflows in the SHIWA Workflow Repository. [ABSTRACT FROM AUTHOR]

Usually, scientists need to execute experiments that demand high performance computing environments and parallel techniques. This is the scenario found in many bioinformatics experiments modeled as scientific workflows, such as phylogenetic and phylogenomic analyses. To execute these experiments, scientists have adopted virtual machines (VMs) instantiated in clouds. Estimating the number of VMs to instantiate is a crucial task to avoid negative impacts on the execution performance and on the financial costs with under or overestimations. Previously, the necessary number of VMs to execute bioinformatics workflows have been estimated by a GRASP heuristic and have been coupled to a Cloud-based Parallel Scientific Workflow Management System. Although this work was a step forward, this approach only provided a static dimensioning. If the characteristics of the environment change (processing capacity, network speed), this static dimensioning may not be suitable. In this way, it is of interest that the dimensioning is adjusted at runtime. To achieve this, we developed a novel framework for monitoring and dynamically dimensioning resources during the execution of parallel scientific workflows in clouds, called Dynamic Dimensioning of Cloud Computing Framework (DDC-F). We have evaluated DDC-F in real executions of bioinformatics workflows. Experiments showed that DDC-F is able to efficiently calculate the number of VMs necessary to execute bioinformatics workflows of Comparative Genomics (CG), also reducing the financial costs, when compared with other works of the related literature. [ABSTRACT FROM AUTHOR]

Metabolic flux analysis (MFA) with 13 C labeling data is a high-precision technique to quantify intracellular reaction rates (fluxes). One of the major challenges of 13 C MFA is the interactivity of the computational workflow according to which the fluxes are determined from the input data (metabolic network model, labeling data, and physiological rates). Here, the workflow assembly is inevitably determined by the scientist who has to consider interacting biological, experimental, and computational aspects. Decision-making is context dependent and requires expertise, rendering an automated evaluation process hardly possible. Here, we present a scientific workflow framework (SWF) for creating, executing, and controlling on demand 13 C MFA workflows. 13 C MFA-specific tools and libraries, such as the high-performance simulation toolbox 13CFLUX2, are wrapped as web services and thereby integrated into a service-oriented architecture. Besides workflow steering, the SWF features transparent provenance collection and enables full flexibility for ad hoc scripting solutions. To handle compute-intensive tasks, cloud computing is supported. We demonstrate how the challenges posed by 13 C MFA workflows can be solved with our approach on the basis of two proof-of-concept use cases. [ABSTRACT FROM AUTHOR]