Search

Author(s): Paine, Drew; Poon, Sarah; Ramakrishnan, Lavanya | Abstract: Scientific exploration generates expanding volumes of data that commonly require High Performance Computing (HPC) systems to facilitate research. HPC systems are complex ecosystems of hardware and software that frequently are not user friendly. The Usable Data Abstractions (UDA) project set out to build usable software for scientific workflows in HPC environments by undertaking multiple rounds of qualitative user research. Qualitative research investigates how individuals accomplish their work and our interview-based study surfaced a variety of insights about the experiences of working in and with HPC ecosystems. This report examines multiple facets to the experiences of scientists and developers using and supporting HPC systems. We discuss how stakeholders grasp the design and configuration of these systems, the impacts of abstraction layers on their ability to successfully do work, and the varied perceptions of time that shape this work. Examining the adoption of the Cori HPC at NERSC we explore the anticipations and lived experiences of users interacting with this system’s novel storage feature, the Burst Buffer. We present lessons learned from across these insights to illustrate just some of the challenges HPC facilities and their stakeholders need to account for when procuring and supporting these essential scientific resources to ensure their usability and utility to a variety of scientific practices.

Science Capsule is a free open source software that allows researchers to automatically capture their end-to-end workflows including the scripts, data, and execution environment. Science Capsule monitors the workflow environment to capture the provenance at runtime. It provides a timeline view and a web interface to represent the workflow and data life cycle, and the associated provenance information. Science Capsule also leverages container technologies to provide a lightweight, executable package of the scripts and required dependencies, ensuring portability and reproducibility., This work is supported by the U.S. Department of Energy, Office of Science and Office of Advanced Scientific Computing Research (ASCR) under Contract No. DE-AC02-05CH11231.

The growth in scientific data volumes has resulted in a need to scale up processing and analysis pipelines using High Performance Computing (HPC) systems. These workflows need interactive, reproducible analytics at scale. The Jupyter platform provides core capabilities for interactivity but was not designed for HPC systems. In this paper, we outline our efforts that bring together core technologies based on the Jupyter Platform to create interactive, reproducible analytics at scale on HPC systems. Our work is grounded in a real world science use case - applying geophysical simulations and inversions for imaging the subsurface. Our core platform addresses three key areas of the scientific analysis workflow - reproducibility, scalability, and interactivity. We describe our implemention of a system, using Binder, Science Capsule, and Dask software. We demonstrate the use of this software to run our use case and interactively visualize real-time streams of HDF5 data.

Scientific software development processes are understood to be distinct from commercial software development practices due to uncertain and evolving states of scientific knowledge. Sustaining these software products is a recognized challenge, but under-examined is the usability and usefulness of such tools to their scientific end users. User research is a well-established set of techniques (e.g., interviews, mockups, usability tests) applied in commercial software projects to develop foundational, generative, and evaluative insights about products and the people who use them. Currently these approaches are not commonly applied and discussed in scientific software development work. The use of user research techniques in scientific environments can be challenging due to the nascent, fluid problem spaces of scientific work, varying scope of projects and their user communities, and funding/economic constraints on projects. In this paper, we reflect on our experiences undertaking a multi-method user research process in the Deduce project. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists make decisions around data change. There is a lack of common terminology since the concept of systematically measuring and managing data change is under explored in scientific environments. To bridge this gap we conducted user research that focuses on user practices, needs, and motivations to help us design and develop metrics and tools for data change. This paper contributes reflections and the lessons we have learned from our experiences. We offer key takeaways for scientific software project teams to effectively and flexibly incorporate similar processes into their projects.

Author(s): Paine, Drew; Ghoshal, Devarshi; Ramakrishnan, Lavanya | Abstract: Scientific datasets are continually expanding and changing due to fluctuations with instruments, quality assessment and quality control processes, and modifications to software pipelines. Datasets include minimal information about these changes or their effects requiring scientists manually assess modifications through a number of labor intensive and ad-hoc steps. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists systematically identify and make decisions around data changes. Currently, there is a lack of understanding, and common practices, for identifying and evaluating changes in datasets since systematically measuring and managing data change is under explored in scientific work. We are conducting user research to address this need by exploring scientist's conceptualizations, behaviors, needs, and motivations when dealing with changing datasets. Our user research utilizes multiple methods to produce foundational, generative insights and evaluate research products produced by our team. In this paper, we detail our user research process and outline our findings about data change that emerge from our studies. Our work illustrates how scientific software teams can push beyond just usability testing user interfaces or tools to better probe the underlying ideas they are developing solutions to address.

Reproducible research is becoming essential for science to ensure transparency and for building trust. Additionally, reproducibility provides the cornerstone for sharing of methodology that can improve efficiency. Although several tools and studies focus on computational reproducibility, we need a better understanding about the gaps, issues, and challenges for enabling reproducibility of scientific results beyond the computational stages of a scientific pipeline. In this paper, we present five different case studies that highlight the reproducibility needs and challenges under various system and environmental conditions. Through the case studies, we present our experiences in reproducing different types of data and methods that exist in an experimental or analysis pipeline. We examine the human aspects of reproducibility while highlighting the things that worked, that did not work, and that could have worked better for each of the cases. Our experiences capture a wide range of scenarios and are applicable to a much broader audience who aim to integrate reproducibility in their everyday pipelines.

Software is an integral element of the work of science yet it is not commonly an object of inquiry in studies of scientific infrastructures. This paper presents findings from an ethnographic study of a cosmology group's collaborative scientific software production. We demonstrate how these cosmologists use plots to simultaneously test their software and analyze data while interrogating multiple layers of infrastructural components. We broaden perspectives on scientific software development using a sociotechnical, software studies lens to examine this work of scientific discovery as a creative and embodied, yet exacting and methodical, activity that requires a 'human in the loop'. We offer a new reading of scientific software practices to convey how creating scientific software is often really the act of doing science itself--an intervention we believe is necessary to more successfully support scientific software sharing and infrastructure production.

Over the last several years, a growing body of work has examined the nature of large-scale virtual organizations for data-intensive cooperative science. These projects, known as Cyberinfrastructures (CI) in the United States, are established realms of inquiry for the eScience and Computer Supported Cooperative Work (CSCW) communities. Scholarship in these communities extends technology focused inquiries to investigate the sociotechnical concerns to such infrastructure creation and maintenance. In this paper we present findings from our qualitative study of a federated cyberinfrastructure organization known as GENI. We contribute to this body of scholarship by investigating how stakeholders in the GENI project position existing, and newly created, resources for use in educational settings. We examine how stakeholders acquaint new potential stakeholders with this CI in order to draw them into the community, and the ways in which stakeholder's roles evolve over time. Our findings illustrate several ways stakeholders leverage and align existing relationships and resources to expand the CI project's user base. Finally, this paper suggests avenues of further inquiry and implications for organizing future CI projects.

The CSCW community is reliant upon technology-centric models of groupware and collaboration that frame how we examine and design for cooperative work. This paper both reviews the CSCW literature to examine existing models of collaborative work and proposes a new, expanded conceptual model: the Model of Coordinated Action (MoCA). MoCA is a broader framework for describing complex collaborative situations and environments including, but not limited to, collaborations that have diverse, high-turnover memberships or emerging practices. We introduce MoCA's seven dimensions of coordinative action and illustrate their connection to past and current CSCW research. Finally, we discuss some ramifications of MoCA for our understanding of CSCW as a sociotechnical design space.

In recent years, the engineering education community has invested significant energy and resources in the creation of a rigorous empirical knowledge base to support the transformation of engineering education practice. In this work, we address the specific question: When researchers identify practical implications in their research publications, what specific actions are they suggesting be done, and who is expected to take these actions? We answer this question by identifying "implication for action" sentences in a sample of sixteen articles across two issues of the Journal of Engineering Education. We find that the actions offered are infrequently cognitive (15% in our dataset) and frequently related to courses (83% in our dataset); and the identification of an actor happens just under half of the time (43% in our dataset). We discuss these results through a pragmatic perspective and a critical perspective.

The objective of this work is to explore how implications for educational practice are represented in the scholarly literature of engineering education. This work is motivated by the increasing urgency to understand how research can be used to transform educational practice. Because scholarly writing (such as conference papers and journal articles) represents one place where researchers articulate ideas about how their research can be used (i.e., educational implications), an analysis of how such implications are represented can provide insight into ideas about the uses we imagine for our research. Once we have characterized the vision of use, we can step back and think about whether the collective vision we present may provide clues to the more general issue of transforming education.

While previous CSCW research has noted that computer scientists have their own research interests pertaining to cyberinfrastructure development projects, most have focused on the research imperatives of scientists. This qualitative, interview-based study investigates the perspective of computer scientists developing middleware software for cyberinfrastructures at two supercomputing centers. This paper examines how technologists develop and sustain middleware applications over time by leveraging expertise and partnering with different research domains in order to achieve long-term infrastructural goals.