Search

There are many different options available to implement information technology (IT) today: enterprise data center legacy approaches or private cloud, public cloud, hybrid cloud, fog, device, mobile edge, and multicloud. Although definitions vary, for a variety of reasons, many companies are using multicloud approaches in a variety of patterns, at various layers such as infrastructure as a service (IaSS) and software as a service (SaaS).1 In some cases, multiple SaaS or infrastructure public or private clouds are used to support an end-to-end process, sometimes called workflow enactment.2 For example, a purchase order might be sent via a (cloud-based SaaS) email service such as Gmail, processed in a private cloud application, and then billed via a (cloud-based SaaS) billing service such as Zuora. Other use cases involve multiple public infrastructure clouds. For example, Netflix famously uses Amazon Web Services (AWS) for digital movie transcoding and its entertainment recommendation engine, but it also backs up data to Google’s cloud in the event of an AWS outage.

In cloud computing, "hybrid cloud" and "community cloud" are established jargon. These concepts, however, actually entail federation, that is, the ability to securely share compute resources. Cloud federation has been investigated at all levels of the system stack--infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). To better understand how to accomplish secure cloud federation, the author identifies six fundamental federation requirements, six deployment models, examines relevant existing standards, and discusses areas where further work needs to be done on best practices, tooling, and standards. The author notes that federation isn't specific to on-demand cloud resources and can be used to manage arbitrary services at any level in the system stack for a wide spectrum of application domains. Hence, federation has far-reaching implications for the scale and scope of cloud-based, and bare-metal-based, collaborations.

The Internet of Things needs for computing power and storage are expected to remain on the rise in the next decade. Consequently, the amount of data generated by devices at the edge of the network will also grow. While cloud computing has been an established and effective way of acquiring computation and storage as a service to many applications, it may not be suitable to handle the myriad of data from IoT devices and fulfill largely heterogeneous application requirements. Fog computing has been developed to lie between IoT and the cloud, providing a hierarchy of computing power that can collect, aggregate, and process data from/to IoT devices. Combining fog and cloud may reduce data transfers and communication bottlenecks to the cloud and also contribute to reduced latencies, as fog computing resources exist closer to the edge. This paper examines this IoT-Fog-Cloud ecosystem and provides a literature review from different facets of it: how it can be organized, how management is being addressed, and how applications can benefit from it. Lastly, we present challenging issues yet to be addressed in IoT-Fog-Cloud infrastructures., Comment: Preprint version - to be published in Elsevier's Internet of Things journal

We present a design for a general federation agent based on the OpenStack Keystone v3 service. We examine and choose an initial approach based on peer-to-peer methods that support the use of arbitrary application-level services in Virtual Organizations. We review general federation management requirements, deployment models, and current work, including those using Keystone. We then extend our work on KeyVOMS, a centralized third-party management system, to a distributed peer-to-peer system possibly using distributed hash tables. We also describe how OpenStack's Horizon dashboard could be extended to function as a Virtual Organization dashboard. While an initial approach is chosen, we catalog a number of implementation trade-offs and concerns including scalability, protocol security, network support, and adoption.

This paper describes approaches for supporting virtual organizations (VOs) in OpenStack. A VO provides a security and discovery context whereby collaboration across multiple sites can be enabled while enforcing joint security policies. VOs were developed in the grid computing arena to manage international scientific collaborations. However, the VO abstraction is not grid-specific and is applicable in any distributed computing environment, including inter-clouds. To address the need to securely manage on-demand data sharing in disaster response efforts, we prototyped VO support in OpenStack. To evaluate our practical implementation approach, and other related work, we systematically define the VO design space. This also allows us to clearly identify outstanding issues and recommendations for future work to realize effective cloud federations.

The time taken for a distracted driver to return to vigilance in the task of driving a semi-autonomous car is dependent on how engaged the driver is in other activities - such as finishing a text, phone call, etc. In a pilot study of multitasking, we found that auditory alerts that are given in fixed intervals (for example, every 2 seconds) tend to be ignored by or annoy the driver. Our method learns the statistical nature of the engagement of the driver to the particulars of the task using Markov Renewal Processes (MRP) and we use these statistics to optimize the timing of auditory alerts to make semi-autonomous driving safer and enjoyable. The virtue of the method also allows it to continually learn the driver's behavior at all times, even after the initial training period. Therefore, our method alerts the driver in a way that minimizes annoyance and increases effectiveness. More importantly however, it learns to alert each driver in an individualized manner, tailored to the driver's unique behavioral patterns learned over time.

The advent of affordable high performance computing (HPC) resources, such as computing clusters and grids, has transformed the computational capabilities of many organizations. However, the lack of convenient interfaces to such valuable resources limits their effectiveness, restricting their user base to a small set of highly skilled programmers and computer scientists. Without such an interface, domain specialists lacking exceptional knowledge of HPC systems are challenged in their ability to utilize these computing grids. This paper describes the design and implementation of a Web-based user interface aimed at increasing the usability and transparency of a 341-node computing cluster to multiple applications and user communities, by developing a streamlined Web interface. In discussion of the design, we address several common concerns including security, extensibility, and maintainability. We also outline two distinct solutions to this problem: our earliest approach using open-source grid computing tools such as the Globus Toolkit and Gridportlets, and our most current implementation, which uses a prototype Python-based Web application developed as a part of this paper. We then assess our experience with both approaches and conceptually evaluate them based on the common concerns mentioned above. Finally, we argue that, despite the flexibility afforded by the industrial-strength grid tools, a simpler, home-grown approach similar to our Python-based prototype is most effective in terms of time and maintainability for enterprises seeking to develop similar Web-based front ends for their computing resources. We conclude that our approach provides better value to a set of users as it increases in both number and diversity.

The integration of content-based event notification systems with workflow management is motivated by the need for dynamic, data-driven application systems which can dynamically discover, ingest data from, and interact with other application systems, including physical systems with online sensors and actuators. This requires workflows that can be dynamically reconfigured on-the-fly on the receipt of important events from both external physical systems and from other computational systems. Such a capability also supports fault tolerance, i.e., reconfiguring workflows on the receipt of failure events. When decentralized workflow management is considered, the need for a workflow agent framework becomes apparent. A key observation here is that systems providing truly autonomic, reconfigurable workflows cannot rely on any form of a priori knowledge, i.e., static information that is “compiled-in” to an application. Hence, applications will have to increasingly rely on a posteriori information that is discovered, understood, and ingested during run-time. In the most general case, this will require semantic analysis and planning to reach abstract goal states. These observations indicate that future generation grids could be pushed into more declarative programming methods and the use of artificial intelligence — topics that must be approached carefully to identify those problem domains where they can be successfully applied.

System management aspects are described for the process of grid-enabling a vibroacoustic analysis application using the Globus Toolkit 3.2.1. This is the first step in a project intended to grid-enable a suite of tools being developed as a service-oriented enterprise architecture for spacecraft telemetry analysis. Many of the applications in the suite are compute intensive and would benefit from significantly improved performance. In this paper we show the advantage of using Globus to grid-enable a single tool in a vibroacoustic analysis flow, with the result that using as few as eleven nodes, that tool's runtime improved by a factor of eight. While communication overhead does affect performance, these results also indicate that coordinated communication and execution scheduling as part of workflow management would be able to significantly improve overall efficiency. In the larger context, our experience also shows that the service-oriented architecture approach, using grid computing tools, can provide a more flexible system design, in addition to improved performance and increased utilization of resources. We also provide some lessons learned in using the Globus Toolkit.

Grid technology has matured considerably over the past few years. Progress in both implementation and standardization is reaching a level of robustness that enables production quality deployments of grid services in the academic research community with heightened interest and early adoption in the industrial community. Despite this progress, grid applications are far from ubiquitous, and new applications require an enormous amount of programming effort just to see first light. A key impediment to accelerated deployment of grid applications is the scarcity of high-level application programming abstractions that bridge the gap between existing grid middle- ware and application-level needs. The Simple API for Grid Applications (SAGA (1)) is a GGF standardization effort that addresses this particular gap by providing a simple, stable, and uniform programming interface that integrates the most common grid programming abstractions. These most common abstractions were identified through the analysis of several existing and emerging Grid applications. In this article, we present the SAGA effort, describe its relationship to other Grid API efforts within the GGF community, and introduce the first draft of the API using some application programming examples.

Grid computing represents the common vision of truly general distributed computing across a ubiquitous, open-ended infrastructure supporting a wide range of different application areas. Realizing this vision will require a long-term collaboration of fundamental and applied computer science, industry, commercial infrastructure providers, and many, many application domains. Each meeting like Euro-Par is an important contribution to this long-term effort. Grid computing has already made significant progress in the design, deployment and use of grids, but many challenges still remain before the vision can be realized. Many large scientific and engineering projects are adopting grids to support their project goals. Examples include the EU Data Grid project, NEESGrid, GriPhyN, and the NaReGI project, to name just a few. It is also possible to set-up small-scale grids to accomplish flexible scheduling and workflow management. Industrial organizations, such as Sun Microsystems, IBM, Microsoft, and Oracle already have or are starting to develop grid products and solutions. Standards bodies are pushing to bring stability to the marketplace with the development of standards such as the WS-* set of specifications. Even with this degree of progress, grids still suffer from issues of ease of deployment and use, fault tolerance and reliability, and the need for a true dominant best practice to emerge. Even though Globus is considered the de facto standard (and rightly so), there is still a wide “definition” of what constitutes a grid. Infrastructure developers, application domain experts and industry are all truly looking at an elephant. This is hampering the emergence of commonly accepted tools and best practices that can be codified in standards. To help address these issues, we present these papers at Euro-Par that document the best work on many aspects of grid computing. This year 40 papers were submitted to Topic 6. The chairs assembled a team of 78 reviewers that produced 158 reviews. All accepted papers had at least four reviews. Many of the papers “in the middle” in effect received more than four reviews as the chairs read, reread and debated their merits. In the end, ten papers were accepted – an acceptance rate of 10%. These papers represent work in the areas of peer-to-peer, grid brokering, grid scheduling, load balancing in grids, processor farms for grids, virtual grid workspaces, grid information services, grid modeling, data replication, fault tolerance, and grid applications. We invite you now to browse these papers and closely study the ones that spark your interest and will help you in your work as much as possible.

This paper describes the process of grid-enabling a vibroacoustic analysis application using the Globus Toolkit 3.2.1. This is the first step in a project intended to grid-enable a suite of tools being developed as a service-oriented architecture for spacecraft telemetry analysis. Many of the applications in the suite are compute intensive and would benefit from significantly improved performance. In this paper we show the advantage of using Globus to grid-enable a single tool in a vibroacoustic analysis flow, with the result that using as few as eleven nodes, that tool's runtime improved by a factor of eight. While communication overhead does affect performance, these results also indicate that coordinated communication and execution scheduling as part of workflow management would be able to significantly improve overall efficiency. In the larger context, our experience also shows that the service-oriented architecture approach, using grid computing tools, can provide a more flexible system design, in addition to improved performance and increased utilization of resources. We also provide some lessons learned in using the Globus Toolkit.

This paper describes the concept of optimistic grid computing. This allows applications to synchronize more loosely and better tolerate the dynamic and heterogeneous bandwidths and latencies that are seen in grid environments. Based on the observed performance of a world-wide grid testbed, we estimate target operating regions for grid applications. Mesh generation is the primary test application where boundary mesh cavities can be optimistically expanded in parallel. To manage the level of optimistic execution and stay within the application's operating region, we are integrating grid performance monitoring and prediction into the supporting runtime system. The ultimate goal of this project is to generalize the experience and knowledge of optimistic grid computing gained through mesh generation into a tool that can be applied to other tightly coupled computations in other application domains.

This position paper examines the use of topology-aware communication services to support message-passing in wide-area, distributed environments, i.e., grids. Grid computing promises great benefits in the flexible sharing of resources but poses equally great challenges for high-performance computing, that is to say, how to execute large-scale computations on a grid with reasonable utilization of the machines involved. For wide-area computations using a message-passing paradigm, these issues can be addressed by using topology-aware communication, i.e., communication services that are aware of and can exploit the topology of the network connecting all relevant machines. Such services can include augmented communication semantics (e.g., filtering), collective operations, content-based and policy-based routing, and managing communication scope to manage feasibility. While such services can be implemented and deployed in a variety of ways, we propose the use of a peer-to-peer, middleware forwarding and routing layer. In a related application domain (time management in distributed simulations) we provide emulation results showing that such topology-awareness can play a major role in performance and scalability. Besides these benefits, such communication services raise a host of implementation and integration issues for their operational deployment and use in grid environments. Hence, we discuss the need for proper APIs and high-level models.

This paper discusses preliminary work on standardizing and implementing a remote procedure call (RPC) mechanism for grid computing. The GridRPC API is designed to address the lack of a standardized, portable, and simple programming interface. Our initial work on GridRPC shows that client access to existing grid computing systems such as NetSolve and Ninf can be unified via a common API, a task that has proven to be problematic in the past.

In large distributed corporations, distributed design review offers the potential for cost savings, reduced time to market, and improved efficiency. It also has the potential to improve the design process by enabling wider expertise to be incorporated in design reviews. This paper describes the integration of several components to enable distributed virtual design review in mixed multi-party, heterogeneous multi-site 2D and immersive 3D environments. The system provides higher layers of support for collaboration including avatars, high fidelity audio, and shared artifact manipulation. The system functions across several interface environments ranging from CAVEs to Walls to desktop workstations. At the center of the software architecture is the Human Integrating Virtual Environment (HIVE) [6], a collaboration infrastructure and toolset to support research and development of multi-user, geographically distributed, 2D and 3D shared applications. The HIVE functions with VisualEyes software for visualizing 3D data in virtual environments. We also describe in detail the configuration and lessons learned in a two site, heterogeneous multi-user demonstration of the system between HRL Laboratories in Malibu, California and GM R&D in Warren, Michigan.

This paper examines the integration of grid computing and active networks. Resource management is a fundamental aspect of grid computing and should allow for the integration of new capabilities. Active networks will enable a range of higher-level abstractions that will be captured in middleware. This paper systematically examines the components of grid computing for areas of advantageous integration. Arguments are also made for performance models to understand the fundamental performance/flexibility trade-off that faces active networks.

In grid computing environments, network bandwidth discovery and allocation is a serious issue. Before their applications are running, grid users will need to choose hosts based on available bandwidth. Running applications may need to adapt to a changing set of hosts. Hence, a tool is needed for monitoring network performance that is integral to the grid environment. To address this need, Gloperf was developed as part of the Globus grid computing toolkit. Gloperf is designed for ease of deployment and makes simple, end-to-end TCP measurements requiring no special host permissions. Scalability is addressed by a hierarchy of measurements based on group membership and by limiting overhead to a small, acceptable, fixed percentage of the available bandwidth. Since this fixed overhead may push host-pair revisit time into the tens-of-hours, we also quantitatively examine the "trajectory" of the cost-error trade-off for measurement frequency.

A vibroacoustic analysis toolkit for launch vehicle telemetry was refactored as a service architecture using Globus Toolkit 4.0. We grid-enabled two tools in the analysis flow, managing their remote execution workflow from a desktop client. We examine the performance of the tools and the processing time for the entire tool chain. The results indicate that coordinated communication and execution scheduling as part of workflow management can significantly improve overall efficiency. Our experience also shows that the service-oriented architecture approach, using grid computing tools, can provide a more flexible system design, in addition to improved performance and increased utilisation of resources.

OpenStack is an open source cloud computing project that is enjoying wide. While many cloud deployments may be stand-alone, it is clear that secure federated community clouds, i.e., inter-clouds, are needed. Hence, there must be methods for federated identity management (FIM) that enable authentication and authorisation to be flexibly enforced across federated environments. Since there are many different FIM protocols either in use or in development today, this paper addresses the goal of adding protocol independent federated identity management to the OpenStack services. After giving a motivating example for secure cloud federation, and describing the conceptual design for protocol independent federated access, a detailed federated identity protocol sequence is presented. The paper then describes the implementation of the protocol independent system components, along with the incorporation of two different FIM protocols, namely SAML and Keystone proprietary. Finally performance measurements of the protocol independent components, and the two different protocols dependent components are presented, before the paper concludes with the current limitations.