Search

Computational chemists are using Common Component Architecture (CCA) technology to increase the parallel scalability of their application ten-fold. Combustion researchers are publishing science faster because the CCA manages software complexity for them. Both the solver and meshing communities in SciDAC are converging on community interface standards as a direct response to the novel level of interoperability that CCA presents. Yet, there is much more to do before component technology becomes mainstream computational science. This paper highlights the impact that the CCA has made on scientific applications, conveys some lessons learned from five years of the SciDAC program, and previews where applications could go with the additional capabilities that the CCA has planned for SciDAC 2.

Many large-scale scientific simulations require the solution of nonlinear partial differential equations (PDEs). The effective solution of such nonlinear PDEs depends to a large extent on efficient and robust sparse linear system solution. In this paper, we show how fast and reliable sparse linear solvers can be composed from several underlying linear solution methods. We present a combinatorial framework for developing optimal composite solvers using metrics such as the execution times and failure rates of base solution schemes. We demonstrate how such composites can be easily instantiated using advanced software environments. Our experiments indicate that overall simulation time can be reduced through highly reliable linear system solution using composite solvers.

The Portable, Extensible Toolkit for Scientific computing (PETSc), which focuses on the scalable solution of problems based on partial differential equations, now incorporates new components that allow full compos ability of solvers for multiphysics and multilevel methods. Through strong encapsulation, we achieve arbitrary, dynamic composition of hierarchical methods for coupled problems and allow customization of all components in composite solvers. For example, we support block decompositions with nested multigrid as well as multigrid on the fully coupled system with block-decomposed smoothers. This paper provides an overview of PETSc's new multiphysics capabilities, which have been used in parallel applications including lithosphere dynamics, subduction and mantle convection, ice sheet dynamics, subsurface reactive flow, fusion, mesoscale materials modeling, and power networks.

Coupling separately developed codes offers an attractive method for increasing the accuracy and fidelity of the computational models. Examples include the earth sciences and fusion integrated modeling. This paper describes the Framework Application for Core-Edge Transport Simulations (FACETS).

Current trends in microprocessor design indicate that chips are approaching their packaging thermal limits, and the power-related costs of high-performance clusters and multiprocessors continue to grow as a quadratic function of peak execution rates and clock frequencies. Although a faster scientific simulation, such as one obtained by exploiting quality-performance tradeoffs, is also often one that consumes less power by using fewer compute cycles, a major challenge is developing explicitly power-aware scientific computing tools that can exploit special energy-saving features of the circuit fabric. Such tools are perhaps most natural when scientific computing involved sparse or irregular computations, for example, simulations based on partial differential equations in two or three spatial dimensions solved using implicit or semi-implicit schemes. Sparse kernels typically cannot execute near peak rates of the CPU's, and there is potential for tuning them to co-manage both power and performance characteristics. Furthermore, each sparse kernel often has a variety of implementations offering a wide range of tradeoffs in solution quality (e.g., accuracy, reliability, and scalability) and performance (e.g., execution time/rate and parallel efficiency/speedup). Consequently, proper method selection to meet changing application quality-of-service requirements and changing technologies can potentially provide dramatic performance improvements and savings in energy by using circuit fabric features such as dynamic voltage scaling. Our goal is to design adaptive tools for sparse computations that deliver reduced-energy realizations without adversely impacting application performance. In this paper, we provide an overview of our project and discuss some initial results.

Typical scientific software designs make rigid assumptions regarding programming language and data structures, frustrating software interoperability and scientific collaboration. Component-based software engineering is an emerging approach to managing the increasing complexity of scientific software. Component technology facilitates code interoperability and reuse. Through the adoption of methodology and tools developed by the Common Component Architecture Forum, we have developed a component architecture for molecular structure optimization. Using the NWChem and Massively Parallel Quantum Chemistry packages, we have produced chemistry components that provide capacity for energy and energy derivative evaluation. We have constructed geometry optimization applications by integrating the Toolkit for Advanced Optimization, Portable Extensible Toolkit for Scientific Computation, and Global Arrays packages, which provide optimization and linear algebra capabilities. We present a brief overview of the component development process and a description of abstract interfaces for chemical optimizations. The components conforming to these abstract interfaces allow the construction of applications using different chemistry and mathematics packages interchangeably. Initial numerical results for the component software demonstrate good performance, and highlight potential research enabled by this platform. © 2004 Wiley Periodicals, Inc. J Comput Chem 14: 1717–1725, 2004

We survey techniques in the Toolkit for Advanced Optimization (TAO) for developing scalable algorithms for mesh-based optimization problems on distributed architectures. We discuss the distribution of the mesh, the computation of the gradient and the Hessian matrix, and the use of preconditioners. We show that these techniques, together with mesh sequencing, can produce results that scale with mesh size.

The solution of large-scale, nonlinear PDE-based simulations typically depends on the performance of sparse linear solvers, which may be invoked at each nonlinear iteration. We present a framework for using multi-method solvers in such simulations to potentially improve the execution time and reliability of linear system solution. We consider composite solvers, which provide reliable linear solution by using a sequence of preconditioned iterative methods on a given system until convergence is achieved. We also consider adaptive solvers, where the solution method is selected dynamically to match the attributes of linear systems as they change during the course of the nonlinear iterations. We demonstrate how such multi-method composite and adaptive solvers can be developed using advanced software architectures such as PETSc, and we report on their performance in a computational fluid dynamics application.

Developing portable and scalable software for the solution of large-scale optimization problems presents many challenges that traditional libraries do not adequately meet. Using object-oriented design in conjunction with other innovative techniques, we address these issues within the SNES (Scalable Nonlinear Equation Solvers) and SUMS (Scalable Unconstrained Minimization Solvers) packages, which are part of the multilevel PETSc (Portable, Extensible Tools for Scientific computation) library. The paper focuses on our design philosophy and its benefits in providing a uniform and versatile framework for developing optimization software and solving large-scale nonlinear problems. We also consider a three-dimensional anisotropic Ginzburg-Landau model as a representative application that exploits the packages' flexible interface with user specified data structures and customized routines for function evaluation and preconditioning. >

Often the most robust and efficient algorithms for the solution of large-scale problems involving nonlinear PDEs and optimization require the computation of derivatives. We examine the use of automatic differentiation (AD) for computing first and second derivatives in conjunction with two parallel toolkits, the Portable, Extensible Toolkit for Scientific Computing (PETSc) and the Toolkit for Advanced Optimization (TAO). We discuss how the use of mathematical abstractions in PETSc and TAO facilitates the use of AD to automatically generate derivative codes and present performance data demonstrating the suitability of this approach.

This manual describes the use of PETSc 2.0 for the numerical solution of partial differential equations and related problems on high-performance computers. The Portable, Extensible Toolkit for Scientific Computation (PETSc) is a suite of data structures and routines that provide the building blocks for the implementation of large-scale application codes on parallel (and serial) computers. PETSc 2.0 uses the MPI standard for all message-passing communication. PETSc includes an expanding suite of parallel linear and nonlinear equation solvers that may be used in application codes written in Fortran, C, and C++. PETSc provides many of the mechanisms needed thin parallel application codes, such as simple parallel matrix and vector assembly routines that allow the overlap of communication and computation. In addition, PETSc includes growing support for distributed arrays. The library is organized hierarchically, enabling users to employ the level of abstraction that is most appropriate for a particular problem. By using techniques of object-oriented programming, PETSc provides enormous flexibility for users. PETSc is a sophisticated set of software tools; as such, for some users it initially has a much steeper learning curve than a simple subroutine library. In particular, for individuals without some computer science background or experience programming in C, Pascal, or C++, it may require a large amount of time to take full advantage of the features that enable efficient software use. However, the power of the PETSc design and the algorithms it incorporates make the efficient implementation of many application codes much simpler than rolling them yourself. For many simple tasks a package such as Matlab is often the best tool; PETSc is not intended for the classes of problems for which effective Matlab code can be written. Since PETSc is still under development, small changes in usage and calling sequences of PETSc routines will continue to occur.

FACETS (Framework Application for Core-Edge Transport Simulations), is now in its third year. The FACETS team has developed a framework for concurrent coupling of parallel computational physics for use on Leadership Class Facilities (LCFs). In the course of the last year, FACETS has tackled many of the difficult problems of moving to parallel, integrated modeling by developing algorithms for coupled systems, extracting legacy applications as components, modifying them to run on LCFs, and improving the performance of all components. The development of FACETS abides by rigorous engineering standards, including cross platform build and test systems, with the latter covering regression, performance, and visualization. In addition, FACETS has demonstrated the ability to incorporate full turbulence computations for the highest fidelity transport computations. Early indications are that the framework, using such computations, scales to multiple tens of thousands of processors. These accomplishments were a result of an interdisciplinary collaboration among computational physics, computer scientists and applied mathematicians on the team.

Modern beam dynamics simulations require nontrivial implementations of multiple physics models. We discuss how component framework design in combination with the Common Component Architecture's component model and implementation eases the process of incorporation of existing state-of-the-art models with newly-developed models. We discuss current developments in componentized beam dynamics software, emphasizing design issues and distribution issues.

FACETS (Framework Application for Core-Edge Transport Simulations), now in its second year, has achieved its first coupled core-edge transport simulations. In the process, a number of accompanying accomplishments were achieved. These include a new parallel core component, a new wall component, improvements in edge and source components, and the framework for coupling all of this together. These accomplishments were a result of an interdisciplinary collaboration among computational physics, computer scientists, and applied mathematicians on the team.

This paper introduces three component technology initiatives within the SciDAC Center for Technology for Advanced Scientific Component Software (TASCS) that address ever- increasing productivity challenges in creating, managing, and applying simulation software to scientific discovery. By leveraging the Common Component Architecture (CCA), a new component standard for high-performance scientific computing, these initiatives tackle diculties at dierent but related levels in the development of component-based scientific software: (1) deploying applications on massively parallel and heterogeneous architectures, (2) investigating new approaches to the runtime enforcement of behavioral semantics, and (3) developing tools to facilitate dynamic composition, substitution, and reconfiguration of component implementations and parameters, so that application scientists can explore tradeos among factors such as accuracy, reliability, and performance.

Challenges in the scalable solution of large-scale optimization problems include the development of innovative algorithms and efficient tools for parallel data manipulation. This article discusses two complementary toolkits from the collection of Advanced CompuTational Software (ACTS), namely, Global Arrays (GA) for parallel data management and the Toolkit for Advanced Optimization (TAO), which have been integrated to support large-scale scientific applications of unconstrained and bound constrained minimization problems. Most likely to benefit are minimization problems arising in classical molecular dynamics, free energy simulations, and other applications where the coupling among variables requires dense data structures. TAO uses abstractions for vectors and matrices so that its optimization algorithms can easily interface to distributed data management and linear algebra capabilities implemented in the GA library. The GA/TAO interfaces are available both in the traditional library mode and as components compliant with the Common Component Architecture (CCA). We highlight the design of each toolkit, describe the interfaces between them, and demonstrate their use.