Search

Your search keyword '"Scott Klasky"' showing total 65 results
Start Over Author "Scott Klasky" Publication Year Range Last 3 years
65 results on '"Scott Klasky"'

1. MGARD: A multigrid framework for high-performance, error-controlled data compression and refactoring

Author

Qian Gong, Jieyang Chen, Ben Whitney, Xin Liang, Viktor Reshniak, Tania Banerjee, Jaemoon Lee, Anand Rangarajan, Lipeng Wan, Nicolas Vidal, Qing Liu, Ana Gainaru, Norbert Podhorszki, Richard Archibald, Sanjay Ranka, and Scott Klasky

Subjects
Error-controlled data compression, Data refactoring, I/O acceleration, Derived quantities preservation, Computer software, QA76.75-76.765
Abstract

We describe MGARD, a software providing MultiGrid Adaptive Reduction for floating-point scientific data on structured and unstructured grids. With exceptional data compression capability and precise error control, MGARD addresses a wide range of requirements, including storage reduction, high-performance I/O, and in-situ data analysis. It features a unified application programming interface (API) that seamlessly operates across diverse computing architectures. MGARD has been optimized with highly-tuned GPU kernels and efficient memory and device management mechanisms, ensuring scalable and rapid operations.

Published
2023
Full Text
View/download PDF

2. Error-Bounded Learned Scientific Data Compression with Preservation of Derived Quantities

Author

Jaemoon Lee, Qian Gong, Jong Choi, Tania Banerjee, Scott Klasky, Sanjay Ranka, and Anand Rangarajan

Subjects
data compression, autoencoders, error guarantees, moment preservation, constraint satisfaction, quantization, Technology, Engineering (General). Civil engineering (General), TA1-2040, Biology (General), QH301-705.5, Physics, QC1-999, Chemistry, QD1-999
Abstract

Scientific applications continue to grow and produce extremely large amounts of data, which require efficient compression algorithms for long-term storage. Compression errors in scientific applications can have a deleterious impact on downstream processing. Thus, it is crucial to preserve all the “known” Quantities of Interest (QoI) during compression. To address this issue, most existing approaches guarantee the reconstruction error of the original data or primary data (PD), but cannot directly control the problem of preserving the QoI. In this work, we propose a physics-informed compression technique that is composed of two parts: (i) reduction of the PD with bounded errors and (ii) preservation of the QoI. In the first step, we combine tensor decompositions, autoencoders, product quantizers, and error-bounded lossy compressors to bound the reconstruction error at high levels of compression. In the second step, we use constraint satisfaction post-processing followed by quantization to preserve the QoI. To illustrate the challenges of reducing the reconstruction errors of the PD and QoI, we focus on simulation data generated by a large-scale fusion code, XGC, which can produce tens of petabytes in a single day. The results show that our approach can achieve a high compression amount while accurately preserving the QoI within scientifically acceptable bounds.

Published
2022
Full Text
View/download PDF

21. Streaming Data in HPC Workflows Using ADIOS.

Author

Greg Eisenhauer, Norbert Podhorszki, Ana Gainaru, Scott Klasky, Philip E. Davis, Manish Parashar, Matthew Wolf, Eric Suchtya, Erick Fredj, Vicente Bolea, Franz Pöschel, Klaus Steiniger, Michael Bussmann, Richard Pausch, and Sunita Chandrasekaran

Published
2024
Full Text
View/download PDF

23. AI Surrogate Model for Distributed Computing Workloads.

Author

David K. Park, Yihui Ren 0001, Ozgur O. Kilic, Tatiana Korchuganova, Sairam Sri Vatsavai, Joseph Boudreau, Tasnuva Chowdhury, Shengyu Feng, Raees Khan, Jaehyung Kim, Scott Klasky, Tadashi Maeno, Paul Nilsson, Verena Ingrid Martinez Outschoorn, Norbert Podhorszki, Frédéric Suter, Wei Yang, Yiming Yang, Shinjae Yoo, Alexei Klimentov, and Adolfy Hoisie

Published
2024
Full Text
View/download PDF

40. The Exascale Framework for High Fidelity coupled Simulations (EFFIS): Enabling whole device modeling in fusion science.

Author

Eric Suchyta, Scott Klasky, Norbert Podhorszki, Matthew Wolf, Abolaji D. Adesoji, Choong-Seock Chang, Jong Choi 0001, Philip E. Davis, Julien Dominski, Stéphane Ethier, Ian T. Foster, Kai Germaschewski, Berk Geveci, Chris Harris 0004, Kevin A. Huck, Qing Liu 0002, Jeremy Logan, Kshitij Mehta, Gabriele Merlo, Shirley V. Moore, Todd S. Munson, Manish Parashar, David Pugmire, Mark S. Shephard, Cameron W. Smith, Pradeep Subedi, Lipeng Wan 0001, Ruonan Wang, and Shuangxi Zhang

Published
2022
Full Text
View/download PDF

49. MGARD+: Optimizing Multilevel Methods for Error-Bounded Scientific Data Reduction

Author

Lipeng Wan, David Pugmire, Xin Liang, Matthew Wolf, Dingwen Tao, Jieyang Chen, James Kress, Scott Klasky, Qing Liu, Norbert Podhorszki, and Ben Whitney

Subjects
FOS: Computer and information sciences, Computer science, Lossy compression, Theoretical Computer Science, Data modeling, Reduction (complexity), Computer Science - Distributed, Parallel, and Cluster Computing, Computational Theory and Mathematics, Computer engineering, Hardware and Architecture, Compression ratio, Decomposition (computer science), Distributed, Parallel, and Cluster Computing (cs.DC), Decomposition method (constraint satisfaction), Error detection and correction, Software, Data compression
Abstract

Data management is becoming increasingly important in dealing with the large amounts of data produced by large-scale scientific simulations and instruments. Existing multilevel compression algorithms offer a promising way to manage scientific data at scale, but may suffer from relatively low performance and reduction quality. In this paper, we propose MGARD+, a multilevel data reduction and refactoring framework drawing on previous multilevel methods, to achieve high-performance data decomposition and high-quality error-bounded lossy compression. Our contributions are four-fold: 1) We propose a level-wise coefficient quantization method, which uses different error tolerances to quantize the multilevel coefficients. 2) We propose an adaptive decomposition method which treats the multilevel decomposition as a preconditioner and terminates the decomposition process at an appropriate level. 3) We leverage a set of algorithmic optimization strategies to significantly improve the performance of multilevel decomposition/recomposition. 4) We evaluate our proposed method using four real-world scientific datasets and compare with several state-of-the-art lossy compressors. Experiments demonstrate that our optimizations improve the decomposition/recomposition performance of the existing multilevel method by up to 70X, and the proposed compression method can improve compression ratio by up to 2X compared with other state-of-the-art error-bounded lossy compressors under the same level of data distortion.

Published
2022
Full Text
View/download PDF

50. Improving I/O Performance for Exascale Applications Through Online Data Layout Reorganization

Author

Ruonan Wang, Lipeng Wan, Jean-Luc Vay, Scott Klasky, Jieyang Chen, Ian Foster, Todd Munson, Dmitry Ganyushin, Axel Huebl, Ana Gainaru, Xin Liang, Kesheng Wu, Junmin Gu, Norbert Podhorszki, and Franz Poeschel

Subjects
Large class, FOS: Computer and information sciences, Optimization, Distributed databases, Computer science, Layout, media_common.quotation_subject, Fidelity, IO performance, data access optimization, computer.software_genre, Computer Software, Heuristic algorithms, Arrays, Auxiliary memory, media_common, File system, data layout, Communications Technologies, WarpX, Distributed database, Data layout, Dynamic data, Computational modeling, Parallel IO, data layout IO, Exascale computing, Computational Theory and Mathematics, Computer architecture, Computer Science - Distributed, Parallel, and Cluster Computing, Hardware and Architecture, Signal Processing, Performance evaluation, Distributed, Parallel, and Cluster Computing (cs.DC), Distributed Computing, computer
Abstract

The applications being developed within the U.S. Exascale Computing Project (ECP) to run on imminent Exascale computers will generate scientific results with unprecedented fidelity and record turn-around time. Many of these codes are based on particle-mesh methods and use advanced algorithms, especially dynamic load-balancing and mesh-refinement, to achieve high performance on Exascale machines. Yet, as such algorithms improve parallel application efficiency, they raise new challenges for I/O logic due to their irregular and dynamic data distributions. Thus, while the enormous data rates of Exascale simulations already challenge existing file system write strategies, the need for efficient read and processing of generated data introduces additional constraints on the data layout strategies that can be used when writing data to secondary storage. We review these I/O challenges and introduce two online data layout reorganization approaches for achieving good tradeoffs between read and write performance. We demonstrate the benefits of using these two approaches for the ECP particle-in-cell simulation WarpX, which serves as a motif for a large class of important Exascale applications. We show that by understanding application I/O patterns and carefully designing data layouts we can increase read performance by more than 80%., 12 pages, 15 figures, accepted by IEEE Transactions on Parallel and Distributed Systems

Published
2022

Catalog

Books, media, physical & digital resources
See catalog results