Search

Deadline-sensitive workflows require careful coordination of user constraints with resource availability. Current distributed resource access models provide varying degrees of resource control: from limited or none in grid batch systems to explicit in cloud systems. Additionally applications experience variability due to competing user loads, performance variations, failures, etc. These variations impact the quality of service (QoS) that goes unaccounted for in planning strategies. In this paper we propose Workflow ORchestrator for Distributed Systems (WORDS) architecture based on a least common denominator resource model that abstracts the differences and captures the QoS properties provided by grid and cloud systems. We investigate algorithms for effective orchestration (i.e., resource procurement and task mapping) for deadline-sensitive workflows atop the resource abstraction provided in WORDS. Our evaluation compares orchestration methodologies over TeraGrid and Amazon EC2 systems. Experimental results show that WORDS enables effective orchestration possible at reasonable costs on batch queue grid and cloud systems with or without explicit resource control.

Scientific workflow systems have become a necessary tool for many applications, enabling the composition and execution of complex analysis on distributed resources. Today there are many workflow systems, often with overlapping functionality. A key issue for potential users of workflow systems is the need to be able to compare the capabilities of the various available tools. There can be confusion about system functionality and the tools are often selected without a proper functional analysis. In this paper we extract a taxonomy of features from the way scientists make use of existing workflow systems and we illustrate this feature set by providing some examples taken from existing workflow systems. The taxonomy provides end users with a mechanism by which they can assess the suitability of workflow in general and how they might use these features to make an informed choice about which workflow system would be a good choice for their particular application.

Funded by the National Science Foundation (NSF), TeraGrid is one of the world's largest distributed cyberinfrastructures for open scientific research. The project began in 2001 as the Distributed Tera-scale Facility, which linked computers, visualization systems, and data at four sites through a dedicated 40-gigabit optical network. Today TeraGrid includes 25 platforms at 11 sites and provides access to more than a petaflop of computing power and petabytes of storage. TeraGrid has three primary focus areas. Its deep goal is to support the most challenging computational science activities those that cannot be achieved without TeraGrid facilities. TeraGrid's wide mission is to broaden its user base. The project's open goal is to achieve compatibility with peer grids and information services that allow development of programmatic interfaces to TeraGrid. The Science Gateways program seeks to provide researchers with easy access to TeraGrid's high-performance computing resources. A look at four successful gateways illustrates the program's goals, challenges, and opportunities.

We review the efforts of the Open Grid Computing Environments collaboration. By adopting a general three-tiered architecture based on common standards for portlets and Grid Web services, we can deliver numerous capabilities to science gateways from our diverse constituent efforts. In this paper, we discuss our support for standards-based Grid portlets using the Velocity development environment. Our Grid portlets are based on abstraction layers provided by the Java CoG kit, which hide the differences of different Grid toolkits. Sophisticated services are decoupled from the portal container using Web service strategies. We describe advance information, semantic data, collaboration, and science application services developed by our consortium. Copyright © 2006 John Wiley & Sons, Ltd.

Two closely linked projects aim to dramatically improve storm forecasting speed and accuracy. CASA is creating a distributed, collaborative, adaptive sensor network of low-power, high-resolution radars that respond to user needs. LEAD offers dynamic workflow orchestration and data management in a Web services framework designed to support on-demand, real-time, dynamically adaptive systems

Web service architectures have gained popularity in recent years within the scientific grid research community. One reason for this is that web services allow software and services from various organizations to be combined easily to provide integrated and distributed applications. However, most applications developed and used by scientific communities are not web-service-oriented, and there is a growing need to integrate them into grid applications based on service-oriented architectures. In this paper, we describe a framework that allows scientists to provide a web service interface to their existing applications as web services without having to write extra code or modify their applications in any way. In addition, application providers do not need to be experts in web services standards, such as Web Services Description Language, Web Services Addressing, Web Services Security, or secure authorization, because the framework automatically generates these details. The framework also enables users to discover these application services, interact with them, and compose scientific workflows from the convenience of a grid portal.

Within a decade after John von Neumann and colleagues conducted the first experimental weather forecast on the ENIAC computer in the late 1940s, numerical models of the atmosphere became the foundation of modern-day weather forecasting and one of the driving application areas in computer science. This article describes research that is enabling a major shift toward dynamically adaptive responses to rapidly changing environmental conditions.

The Linked Environments for Atmospheric Discovery (LEAD) project seeks to provide on-demand weather forecasting. A triad of cooperating services provides the core functionality needed to execute experiments and manage the data. In this article, we focus on three MyLEAD services - the metadata catalog service, notification service, and workflow service - that together form the core services for managing complex experimental meteorological investigations and managing the data products used in and generated during the computational experimentation. We show how the services work together on the user's behalf, easing the technological burden on the scientists and freeing them to focus on more of the science that compels them.

This work describes an approach to building Grid applications based on the premise that users who wish to access and run these applications prefer to do so without becoming experts on Grid technology. We describe an application architecture based on wrapping user applications and application workflows as Web services and Web service resources. These services are visible to the users and to resource providers through a family of Grid portal components that can be used to configure, launch, and monitor complex applications in the scientific language of the end user. The applications in this model are instantiated by an application factory service. The layered design of the architecture makes it possible for an expert to configure an application factory service with a custom user interface client that may be dynamically loaded into the portal.

Sophisticated data-distribution schemes and recent developments in sensors and instruments that can monitor the lower kilometers of the atmosphere at high levels of resolution have rapidly expanded the quantity of information available to mesoscale meteorology. The myLEAD personalized information-management tool helps geoscience users make sense of this vastly expanded information space. MyLEAD extends the general globus metadata catalog service and leverages a well-known general and extensible schema. Its orientation makes it an active player in large-scale distributed computation environments characterized by interacting grid and Web services.

Web services can be defined as loosely coupled, reusable software components that semantically encapsulate discrete functionality and are distributed and programmatically accessible over standard Internet protocols. Web services have received a lot of hype, the reasons for which are not easily determined. Some of their benefits might even seem to waste away, once we touch on the nitty-gritty details, because Web services per se do not offer a solution to underlying problems. The contributions included in this section delve into some of these issues, including: pitfalls of workflow issues; structuring procedural knowledge into problem-solving methods; discussing how a low initial entry barrier and simple technology are balanced against the long-term goal of easy integration; including semantics in a Web service modeling framework; and building on new kinds of applications such as grid enterprises.

As machines and programs have become more complex, the process of programming applications that can exploit the power of high-performance systems has become more difficult and correspondingly more labor-intensive. This has substantially widened the software gap?the discrepancy between the need for new software and the aggregate capacity of the workforce to produce it. This problem has been compounded by the slow growth of programming productivity, especially for high-performance programs, over the past two decades. One way to bridge this gap is to make it possible for end users to develop programs in high-level domain-specific programming systems. In the past, a major impediment to the acceptance of such systems has been the poor performance of the resulting applications. To address this problem, we are developing a new compiler-based infrastructure, called TeleGen, that will make it practical to construct efficient domain-specific high-level languages from annotated component libraries. We call these languages telescoping languages, because they can be nested within one another. For programs written in telescoping languages, high performance and reasonable compilation times can be achieved by exhaustively analyzing the component libraries in advance to produce a language processor that recognizes and optimizes library operations as primitives in the language. The key to making this strategy practical is to keep compile times low by generating a custom compiler with extensive built-in knowledge of the underlying libraries. The goal is to achieve compile times that are linearly proportional to the size of the program presented by the user, rather than to the aggregate size of that program plus the base libraries.

Advances in networking technologies will soon make it possible to use the global information infrastructure in a qualitatively different way—as a computational as well as an information resource. As described in the recent book The Grid: Blueprint for a New Computing Infrastructure, this Grid will connect the nation’s computers, databases, instruments, and people in a seamless web of computing and distributed intelligence, which can be used in an on-demand fashion as a problem-solving resource in many fields of human endeavor—and, in particular, science and engineering. The availability of grid resources will give rise to dramatically new classes of applications, in which computing resources are no longer localized but, rather, distributed, heterogeneous, and dynamic; computation is increasingly sophisticated and multidisciplinary; and computation is integrated into our daily lives and, hence, subject to stricter time constraints than at present. The impact of these new applications will be pervasive, ranging from new systems for scientific inquiry, through computing support for crisis management, to the use of ambient computing to enhance personal mobile computing environments. To realize this vision, significant scientific and technical obstacles must be overcome. Principal among these is usability. The goal of the Grid Application Development Software (GrADS) project is to simplify distributed heterogeneous computing in the same way that the World Wide Web simplified information sharing over the Internet. To that end, the project is exploring the scientific and technical problems that must be solved to make it easier for ordinary scientific users to develop, execute, and tune applications on the Grid. In this paper, the authors describe the vision and strategies underlying the GrADS project, including the base software architecture for grid execution and performance monitoring, strategies and tools for construction of applications from libraries of grid-aware components, and development of innovative new science and engineering applications that can exploit these new technologies to run effectively in grid environments.

This paper provides a manual for javab, a prototype tool that supports the automatic detection and exploitation of implicit loop parallelism in JVM bytecode. Implicit parallelism is made explicit by means of the multi-threading mechanism provided by the JVM. Automatically exploiting implicit parallelism at bytecode level can be done independently from the source program and platform from which the bytecode was obtained, and independently from the platform on which the bytecode eventually will run. The parallelized bytecode remains architecturally neutral and may exhibit speedup on any platform that supports the true parallel execution of JVM threads. This project is supported by DARPA under contract ARPA F19628–94–C–0057 through a sub-contract from Syracuse University. © 1998 John Wiley & Sons, Ltd.

Java RMI provides an elegant and powerful model for invoking member functions on objects that exist in remote address spaces. Unfortunately, it is a Java-to-Java communication model, and in many of the scientific applications we would like Java objects to interact with modules written in C++ or Fortran. This paper explores the design of RMI and extracts a subset of the RMI object model that is compatible with C++ and HPC++ remote object semantics. This RMI subset has been implemented to run over the Nexus runtime system and is being used as part of the LSA distributed linear system analyzer project. © 1998 John Wiley & Sons, Ltd.

HPC++ is a set of parallel programming tools that have been designed by the High Performance C++ Consortium. It consists of a modest set of compiler directives and a parallel extension to the C++ Standard Template Library (STL). In this short note we consider the problem of mixing the parallel STL model with the call reification style of parallel programming proposed by the Europa Consortium.

This paper describes a new approach to designing performance analysis tools for parallel processing systems. A primary contribution of this work is to explore the way in which application source code structure can be incorporated into an interactive, programmable event trace analyzer. Event histories are organized as a temporal, relational data base. The user interface is based on a simple two dimensional spreadsheet metaphor that provides a direct and flexible view of the data base and also provides a simple but powerful interactive graphics facility.

Executing large number of independent tasks or tasks that perform minimal inter-task communication in parallel is a common requirement in many domains. In this paper, we present our experience in applying two new Microsoft technologies Dryad and Azure to three bioinformatics applications. We also compare with traditional MPI and Apache Hadoop MapReduce implementation in one example. The applications are an EST (Expressed Sequence Tag) sequence assembly program, PhyloD statistical package to identify HLA-associated viral evolution, and a pairwise Alu gene alignment application. We give detailed performance discussion on a 768 core Windows HPC Server cluster and an Azure cloud. All the applications start with a "doubly data parallel step" involving independent data chosen from two similar (EST, Alu) or two different databases (PhyloD). There are different structures for final stages in each application.

Today's scientific workflows use distributed heterogeneous resources through diverse grid and cloud interfaces that are often hard to program. In addition, especially for time-sensitive critical applications, predictable quality of service is necessary across these distributed resources. VGrADS' virtual grid execution system (vgES) provides an uniform qualitative resource abstraction over grid and cloud systems. We apply vgES for scheduling a set of deadline sensitive weather forecasting workflows. Specifically, this paper reports on our experiences with (1) virtualized reservations for batchqueue systems, (2) coordinated usage of TeraGrid (batch queue), Amazon EC2 (cloud), our own clusters (batch queue) and Eucalyptus (cloud) resources, and (3) fault tolerance through automated task replication. The combined effect of these techniques was to enable a new workflow planning method to balance performance, reliability and cost considerations. The results point toward improved resource selection and execution management support for a variety of e-Science applications over grids and cloud systems.

The ubiquity of information technology, technological advances, and utility computing trends have motivated large-scale systems, but managing and sustaining these systems is far from trivial. Automatic or semi-automatic monitoring and control are a potential solution to this problem. However, since management scenarios differ from system to system, a generic management framework that can manage a wide variety of systems should support user-defined management logic. This paper proposes a novel architecture that can manage large-scale systems according to user-specified management logic that depends on both global and local assertions of the managed system. Furthermore, the paper demonstrates that despite having a global view of the managed system, a management framework can scale to manage most real world use cases.

Management architectures are well discussed in the literature, but their application in real life settings has not been as well covered. Automatic management of a system involves many more complexities than closing the control-loop by reacting to sensor data and executing corrective actions. In this paper, we discuss those complexities and propose solutions to those problems on top of Hasthi management framework, where Hasthi is a robust, scalable, and distributed management framework that enables users to manage a system by enforcing management logic authored by users themselves. Furthermore, we present in detail a real life case study, which uses Hasthi to manage a large, SOA based, E-Science Cyberinfrastructure.

With the emergence of service oriented architecture, service orchestration has been an important paradigm of building large-scale distribute systems. The performance of orchestrating services is critical to the entire system. Meanwhile,with the growing complexity the programmability of writing service orchestration program has became a pressing concern. In this paper, we presents a service orchestration library which provides C# function API for orchestration constructs derived from the web-service workflow language WS-BPEL. The library implements those constructs by mapping WS-BPEL concurrent semantic into asynchronous events and join-patterns provided in Microsoft CCR; meanwhile the library provides the abstraction to ease the service orchestration programming. We will demonstrate how to use this library together with Microsoft DSS, a lightweight service runtime, to program service workflows which can efficiently scale to platforms from single multicore CPU, to clusters of distributed computers.

Due to their scale, complexity, and dynamic nature, e-science cyber-infrastructures have raised major system management challenges. It has been argued that employing system management frameworks to monitor and control large scale systems is a possible solution to address these challenges. This paper presents an application of a management framework, known as Hasthi (Sanskrit name of Elephant which represents robustness), to manage an e-science project called linked environments for atmospheric discovery (LEAD). Hasthi is a generic framework that manages a system according to rules written by administrators, where rules define how Hasthi should respond to changes and failures. Using LEAD as an example, we demonstrate how such extensible management framework can be utilized to manage a complex system, while describing common problems and solutions.

In this paper we describe a web service oriented design for problem solving systems used by scientist to orchestrate complex computational experiments. Specifically we describe a programming model for users of a science gateway or participants in a virtual organization to express non-trivial tasks that operates equally well in a environment built from a single “many core” processor or one where the computation is completely remote.

This volume contains the papers presented at GADA 2008, the International Symposium on Grid Computing,High-Performance and Distributed Applications. The purpose of the GADA series of conferences, held within the framework of the OnTheMove Federated Conferences (OTM), is to bring together researchers, developers, professionals and students in order to advance research and development in the areas of grid computing and distributed systems and applications. This year's conference was held in Monterrey, Mexico, November 13-14, 2008.

An e-Science Grid Gateway is a portal that allows a scientific collaboration to use the resources of a Grid in a way that frees them from the complex details of Grid software and middleware. The goal of such a gateway is to allow the users access to community data and applications that can be used in the language of their science. Each user has a private data and metadata space, access to data provenance and tools to use or compose experimental workflows that combine standard data analysis, simulation and post-processing tools. In this talk we will describe the underlying Grid service architecture for such an eScience gateway. In this paper we will describe some of the challenges that confront the design of Grid Gateways and we will outline a few new research directions.

Web services-based event notification is an emerging technology that combines the asynchronous communication feature of event notification mechanisms and the interoperability feature of Web services technologies. Web services-based event notification systems are important components for service-oriented grid computing. WS-eventing and WS-notification are two major competing specifications for these systems. This paper is a comparative study of these specifications. The focuses of this research are on identifying the similarities and differences between these two specifications and identifying their evolutionary path from previous specifications. We found that competing Web services specifications take ideas and concepts from each other during the development progress, which is good for the maturity of the Web services-based event notification technology. We also identified several major changes from previous event notification systems to Web services-based event notification systems. In the end, we presented our WS-messenger project that supports both WS-eventing and WS-notification specifications and provides mediation between them.

Service oriented computing is a paradigm for accessing, integrating and coordinating loosely coupled software systems in a standardized way and has gained attention in the recent years within the scientific community. It is increasingly being employed by large scientific collaborations to "wrap" applications as Web services i.e. to create an additional "Web services layer" on top of existing scientific applications. This enables scientists to easily compose, monitor and run complex workflows consisting of scientific applications that are not only developed and managed by a distributed team of application developers but also run on a distributed set of heterogeneous resources. However, one of the biggest challenges for large scientific collaborations lies in keeping all the Web services persistent so that they can be accessed from scientific workflows whenever needed. In this paper, we discuss the architecture and implementation of a mechanism by which we can create the Web services on-demand, in the event that they are unavailable during the execution of a scientific workflow and thus obviate the need to keep them persistent.

Web services have become an integral part of workflow orchestration in scientific applications and many tools used by scientists including Kepler, Taverna and Triana incorporate Web services. However orchestration of complex workflows that involve services that exploit standards like WS-Addressing or WS-Security are not easily managed by these tools. Most complex use cases that involve ldquoadd onrdquo service standards use implementation specific assumptions, and as a result, the services and workflow composers become tightly coupled. This leads to stove-piped, noninteroperable implementations. This paper describes an effort to implement complex use cases that include asynchronous messaging and WS-Security, for a large application project called LEAD, while maintaining standard conformance and composer simplicity. The primary contribution of the paper is design of a Mediator and a generic Web service actor that allow the addition of new Web service standards to services in a workflow, without the need to make the workflow composer or enactor explicitly aware that these standards are being used. These concepts are demonstrated by an implementation that allows large workflows to be constructed using two different scientific workflow systems that were not designed with these extensions in mind.

Provenance about workflow executions and data derivations in scientific applications help estimate data quality, track resources, and validate in silico experiments. The Karma provenance framework provides a means to collect workflow, process, and data provenance from data-driven scientific workflows and is used in the Linked Environments for Atmospheric Discovery (LEAD) project. This article presents a performance analysis of the Karma service as compared against the contemporary PReServ provenance service. Our study finds that Karma scales exceedingly well for collecting and querying provenance records, showing linear or sub-linear scaling with increasing number of provenance records and clients when tested against workloads in the order of 10,000 application-service invocations and over 36 concurrent clients.

A Web services-based publish/subscribe system has the potential to create an Internet scale interoperable event notification system which is important for grid computing as it evolves a service-oriented architecture. WS-Messenger is designed to be a Web services-based message broker that can decouple event producers and event consumers and achieve scalable, reliable and efficient message delivery. In this paper, we discuss some challenges that are unique to Web services-based publish/subscribe systems and the key features that distinguish WS-Messenger from other existing message brokers. We then present the architecture and the technology used in WS-Messenger. Performance tests indicate WS-Messenger performs better than the WS-Notification implementation in Globus Toolkit 4 (GT4) and it can be used as a complement to GT4 to improve its scalability. We lastly describe its application to grid workflow orchestration in the LEAD project.

Data-driven computational science on community computational resources is frequently of a magnitude and scale that it requires that computations be done remotely, generating resulting data collections that are too large to be shipped back to a user's workstation. Service-oriented middleware is well equipped to carry out actions on behalf of a user, but SOA middleware does not address user trust in the privacy of their actions and security of their data. In this paper we develop a model that represents the trust relationship between the users and their remote resources in the grid system. We show how one can construct a trusted relationship from the model, with an emphasis on the importance of context to a specific trust relationship.

Event notification plays an important role in the orchestration of distributed systems. Web services-based event notification is an important step to achieve an interoperable messaging system on the Internet scale. However, different incompatible specifications have been proposed to specify the Web services interfaces for Web services-based event notification systems. The WS-Eventing specification and the WS-Notification specifications are two major initiatives. WS-Messenger is our implementation of both specifications. A mediation approach is used to reconcile the differences between these two specifications. In this paper, we propose a "normalization-processing-customization" (NPC) model for mediation among competing Web services and show how this model is used in WS-Messenger to reconcile the incompatibility between these two specifications. The NPC model is flexible and scalable. The mediation implementation in WS-Messenger is efficient with a very small overhead

Data-driven scientific applications utilize workflow frameworks to execute complex dataflows, resulting in derived data products of unknown quality. We discuss our on-going research on a quality model that provides users with an integrated estimate of the data quality that is tuned to their application needs and is available as a numerical quality score that enables uniform comparison of datasets, providing a way for the community to trust derived data.

Grid systems are now a standard approach to solving problems in large scale, multidisciplinary scientific endeavors. These research groups are often geographically distributed, and to conduct their research, they need to share access to physical resources such as supercomputers, large databases, on-line instruments and distributed applications. Grid infrastructure helps solve their problems because it can provide a layer of middleware that virtualizes the access to these resources. The users see a single coherent computer system instead of a complex network of distributed resources. In most cases, users enter the grid though a ”gateway” portal, which may be a web portal or a ”thick” desktop client. The gateway gives the user a way to browse metadata about computational experiments to access data products, to monitor active workflows and to run applications and share results. The user can focus on the problems of science and not computer systems. All of this is made possible because of the Service Oriented Architecture (SOA) that underlies the core Grid middleware. In this presentation, we will look at several examples of successful Scientific Grids and Gateways. We will also describe the fundamentals of the web service SOAs that works best in Grid systems. We will illustrate these ideas with an example called LEAD which is a Grid designed to improve our ability to predict meso-scale weather events such as hurricanes, typhoons and tornadoes. We will also describe how this entire approach to service virtualization is now being used in industry to better use the resources of a single, but distributed business enterprise. While a great deal of progress has been made there are many exciting and unsolved problems. As we go through the talk, we will highlight these challenges and research opportunities.

This chapter describes an approach to building large-scale, distributed applications based on a software component composition model that allows web services to be used as the basic units. The approach extends the Common Component Architecture used in many parallel supercomputer applications, from static composition of directly coupled processes to a system that incorporates mediated workflow between remote services. The system also allows legacy applications to be easily wrapped as a component and executed from a service factory. We motivate the work in terms of a large, distributed application for modeling severe storms. The entire system is based on a three-level architecture with a portal providing the user interface, a set of security and factory service utilities in the middle and the application services and components in the back-end.

The XML signature specification provides a rich and flexible message signature model for XML documents, and it has been adopted by SOAP applications to provide message-level security. However, the XML signature design introduces a number of complex processing steps, such as canonicalization and XPath filtering, that often lead to performance and scalability problems when encountering extremes of size and rate in the processing of XML. In this paper, we focus on the performance of validating large signed XML messages, as might be sent by a scientific application using grid Web services. We present the design and implementation of the GHPX/SSSV system for the streaming validation of SOAP digital signature. Our model consists of a streaming canonicalization and optimized SOAP signature validation. We present an empirical study of the performance characteristics of these streaming validation features. Based on our evaluations we conclude that the streaming validation model can not only provide high performance, but is also memory efficient.

Security is one of the most important features for grid services. There are several specifications used to add security to grid services, and some of them have been implemented and are in use. However, since most of the security mechanisms involve slow XML manipulations, adding security to grid services introduces a big performance penalty. In this paper, we introduce various security mechanisms and compare their features and performance. Our evaluation shows that transport level security (SSL) is faster than message level security, and should be used if there is no special requirement to use message level security. For message level security, WS-SecureConversation is generally fast, but has a scalability problem.

With the advent of grid computing, more and more high-end computational resources become available for use to a scientist. While this opens up new avenues for scientific research, it makes reliability and fault tolerance of such a system a non-trivial task, especially for long running distributed applications. In order to solve this problem, we present a distributed user-defined checkpointing mechanism within the XCAT3 system. XCAT3 is a framework for component component architecture (CCA) based components consistent with current Grid standards. We describe in detail the algorithms and APIs that are added to XCAT3 in order to support distributed checkpointing. Our approach ensures that the checkpoints are platform independent, minimal in size, and always available during component failures. In addition, our algorithms maintain correctness in the presence of failures and scale well with the number of components, and checkpoint size.

In this paper we test the suitability of Java to implement a scalable Web Service that solves a set of problems related to peer-to-peer interactions between Web Services that are behind firewalls or not generally accessible. In particular we describe how to enable reliable and long running conversations through firewalls between Web Service peers that have no accessible network endpoints. Our solution is to implement in Java a Web services dispatcher (WSD) that is an intermediary service that forwards messages and can facilitate message exchanges by supporting SOAP RPC over HTTP and WS-addressing for asynchronous messaging. We describe how Web service clients that have no network endpoints, such as applets, can become Web Service peers by using an additional message store-and-forward service ("mailbox"). Then we conduct a set of experiments to evaluate performance of Java implementation in realistic Web Service scenarios, involving intercontinental tests between France and the US.

Grid computing is about allocating distributed collections of resources including computers, storage systems, networks and instruments to form a coherent system devoted to a “virtual organization” of users who share a common interest in solving a complex problem or building an efficient agile enterprise. Service oriented architectures have emerged as the standard way to build Grids. This paper provides a brief look at the Open Grid Service Architecture, a standard being proposed by the Global Grid Forum, which provides the foundational concepts of most Grid systems. Above this Grid foundation is a layer of application-oriented services that are managed by workflow tools and “science gateway” portals that provide users transparent access to the applications that use the resources of a Grid. In this paper we will also describe these Gateway framework services and discuss how they relate to and use Grid services.

Software Component Frameworks are well known in the commercial business application world and now this technology is being explored with great interest as a way to build large-scale scientific applications on parallel computers. In the case of Grid systems, the current architectural model is based on the emerging web services framework. In this paper we describe progress that has been made on the Common Component Architecture model (CCA) and discuss its success and limitations when applied to problems in Grid computing. Our primary conclusion is that a component model fits very well with a services-oriented Grid, but the model of composition must allow for a very dynamic (both in space and in time) control of composition. We note that this adds a new dimension to conventional service workflow and it extends the "Inversion of Control" aspects of most component systems.

The benefits of component technologies are well known: they enable encapsulation, modular construction of applications and software reuse. The DOE sponsored common component architecture (CCA) as stated in R. Armstrong et al. (1999) project adopts a component-based approach for building large scale scientific applications. On the other hand, the Web services-based open grid service architecture (OGSA) and infrastructure (OGSI) based on I. Foster et al. (2002) come close to defining a component architecture for the Grid. Using an approach where a CCA component is modeled as a set of Grid services, the XCAT3 framework allows for CCA components to be compatible with the OGSI specification. This enables CCA components to be accessible via standard Grid clients, especially the ones that are portal-based. For CCA compatibility, XCAT3 uses interfaces generated by the Babel toolkit based CASC (2003), and for OGSI compatibility, it uses the Extreme GSX toolkit based on Indiana University Extreme Computing Lab. Grid Service Extensions (2003). In this paper, we describe our experience in implementing the XCAT3 system, and how it can be used to compose complex distributed applications on the Grid in a modular fashion.

Large scale Grid applications are often composed a distributed collection of parallel simulation codes, instrument monitors, data miners, rendering and visualization tools. For example, consider a severe storm prediction system driven by a grid of weather sensors. Typically these applications are very complex to build, so users interact with them through a Grid portal front end. This talk outlines an approach based on a web service component architecture for building these applications and portal interfaces. We illustrate how the traditional parallel application can be wrapped by a web service factory and integrated into complex workflows. Additional issues that are addressed include: grid security, web service tools and workflow composition tools. The talk will try to outline several important classes of unsolved problems and possible new research directions for building grid applications.