Search

Disaggregated memory is a promising approach that addresses the limitations of traditional memory architectures by enabling memory to be decoupled from compute nodes and shared across a data center. Cloud platforms have deployed such systems to improve overall system memory utilization, but performance can vary across workloads. High-performance computing (HPC) is crucial in scientific and engineering applications, where HPC machines also face the issue of underutilized memory. As a result, improving system memory utilization while understanding workload performance is essential for HPC operators. Therefore, learning the potential of a disaggregated memory system before deployment is a critical step. This paper proposes a methodology for exploring the design space of a disaggregated memory system. It incorporates key metrics that affect performance on disaggregated memory systems: memory capacity, local and remote memory access ratio, injection bandwidth, and bisection bandwidth, providing an intuitive approach to guide machine configurations based on technology trends and workload characteristics. We apply our methodology to analyze thirteen diverse workloads, including AI training, data analysis, genomics, protein, fusion, atomic nuclei, and traditional HPC bookends. Our methodology demonstrates the ability to comprehend the potential and pitfalls of a disaggregated memory system and provides motivation for machine configurations. Our results show that eleven of our thirteen applications can leverage injection bandwidth disaggregated memory without affecting performance, while one pays a rack bisection bandwidth penalty and two pay the system-wide bisection bandwidth penalty. In addition, we also show that intra-rack memory disaggregation would meet the application's memory requirement and provide enough remote memory bandwidth., Comment: The submission builds on the following conference paper: N. Ding, S. Williams, H.A. Nam, et al. Methodology for Evaluating the Potential of Disaggregated Memory Systems,2nd International Workshop on RESource DISaggregation in High-Performance Computing (RESDIS), November 18, 2022. It is now submitted to the CCPE journal for review

2022 BSSw Fellowship Q&A Session Presentation

The primary mission of the IDEAS Productivity project is to help code teams supported by the U.S. Department of Energy to improve their software development practices in order to improve scientific productivity and produce software that is more sustainable. But we’re not a large project, and DOE comprises many code teams; moreover, these concerns resonate with projects throughout the broader computational science and engineering (CSE) community. Thus, we recognized from the start that we would benefit from extensive engagement with the CSE community to identify and tailor best practices for CSE software, including how best to assist CSE projects in implementing them. Consequently, the IDEAS project includes an extensive outreach component. We organize and deliver trainings and tutorials; we produce a monthly webinar series on Best Practices for HPC Software Developers and more recently we launched a panel discussion series on Strategies for Working Remotely; and we organize technical sessions at meetings like SIAM CSE21 to promote the discussion of experiences with software development and the exchange of ideas within the community. This poster will provide more information about these activities and how they support our core mission while engaging with and giving back to the broader community.A poster presented at the SIAM CSE21 conference, minisymposterium PP7: Software Productivity and Sustainability for CSE.

2022 BSSw Fellowship Q&A Session Presentation

Better Scientific Software does not happen without learning and implementing BETTER – Planning, Development, Performance, Reliability, Collaboration, and Skills. But how do we learn to be BETTER? The Better Scientific Software (BSSw) Fellowship program fosters and promotes practices, processes and tools to improve developer productivity and software sustainability of scientific codes. BSSw Fellows are chosen annually to develop a funded activity that promotes better scientific software, such as organizing a workshop, preparing a tutorial, or creating content to engage the scientific software community. Their expertise is diverse and they are often leaders in their own community. In this minisymposium, you will have an opportunity to meet BSSw Fellows and learn how their work will help your scientific software project be BETTER through: educating scientists on best practices for developing research software [Better Planning & Development], automating testing in scientific software [Better Reliability], improving the reliability and performance of numerical software [Better Reliability & Performance], and introducing agile scientific software development to underrepresented groups [Better Collaboration & Skills].

Description of the Better Scientific Software (BSSw) Community and BSSw Fellowship Program.

Through its Science Focus Area (SFA) projects the Subsurface and Biogeochemical Research program within the U.S. Department of Energy is tightly integrating observations, experiments, and modeling to advance a systems-level understanding of how watersheds function, and to translate that understanding into advanced science-based models of watershed systems. To broaden the impact of the existing SFAs, the IDEAS-Watersheds project builds on the success of a synergistic family of IDEAS projects initated in 2014. Specifically, it strives to improve wathershed modeling capacity by increasing software development productivity through an agile approach to creating a sustainable, reliable, parallel, software ecosystem with interoperable components.In this talk we highlight the unique structure of the IDEAS-Watersheds project and describe how it addresses many challenges of software development with interdisciplinary teams. Specifically, it is organized around six Research Activities that develop concrete use cases to drive advances in our software ecosystem. These use cases are designed to balance advancing software design and development practices for parallel architectures with the model and algorithmic development needed to address scientific challenges. In addition, we use a co-funding model to create a team of early career researchers that will be trained in modern software engineering, while acting as liaisons that contribute to shared deliverables.Presented at SIAM PP20 Minisymposium: Improving Productivity and Sustainability for Parallel Computing Software.

Description of the Better Scientific Software (BSSw) Community and the BSSw Fellowship program.