Search

Author(s): Paine, Drew; Poon, Sarah; Ramakrishnan, Lavanya | Abstract: Scientific exploration generates expanding volumes of data that commonly require High Performance Computing (HPC) systems to facilitate research. HPC systems are complex ecosystems of hardware and software that frequently are not user friendly. The Usable Data Abstractions (UDA) project set out to build usable software for scientific workflows in HPC environments by undertaking multiple rounds of qualitative user research. Qualitative research investigates how individuals accomplish their work and our interview-based study surfaced a variety of insights about the experiences of working in and with HPC ecosystems. This report examines multiple facets to the experiences of scientists and developers using and supporting HPC systems. We discuss how stakeholders grasp the design and configuration of these systems, the impacts of abstraction layers on their ability to successfully do work, and the varied perceptions of time that shape this work. Examining the adoption of the Cori HPC at NERSC we explore the anticipations and lived experiences of users interacting with this system’s novel storage feature, the Burst Buffer. We present lessons learned from across these insights to illustrate just some of the challenges HPC facilities and their stakeholders need to account for when procuring and supporting these essential scientific resources to ensure their usability and utility to a variety of scientific practices.

Science Capsule is a free open source software that allows researchers to automatically capture their end-to-end workflows including the scripts, data, and execution environment. Science Capsule monitors the workflow environment to capture the provenance at runtime. It provides a timeline view and a web interface to represent the workflow and data life cycle, and the associated provenance information. Science Capsule also leverages container technologies to provide a lightweight, executable package of the scripts and required dependencies, ensuring portability and reproducibility., This work is supported by the U.S. Department of Energy, Office of Science and Office of Advanced Scientific Computing Research (ASCR) under Contract No. DE-AC02-05CH11231.

Digital and intelligent buildings are critical to realizing efficient building energy operations and a smart grid. With the increasing digitalization of processes throughout the life cycle of buildings, data exchanged between stakeholders and between building systems have grown significantly. However, a lack of semantic interoperability between data in different systems is still prevalent and hinders the development of energy-oriented applications that can be reused across buildings, limiting the scalability of innovative solutions. Addressing this challenge, our review paper systematically reviews metadata schemas and ontologies that are at the foundation of semantic interoperability necessary to move toward improved building energy operations. The review finds 40 schemas that span different phases of the building life cycle, most of which cover commercial building operations and, in particular, control and monitoring systems. The paper’s deeper review and analysis of five popular schemas identify several gaps in their ability to fully facilitate the work of a building modeler attempting to support three use cases: energy audits, automated fault detection and diagnosis, and optimal control. Our findings demonstrate that building modelers focused on energy use cases will find it difficult, labor intensive, and costly to create, sustain, and use semantic models with existing ontologies. This underscores the significant work still to be done to enable interoperable, usable, and maintainable building models. We make three recommendations for future work by the building modeling and energy communities: a centralized repository with a search engine for relevant schemas, the development of more use cases, and better harmonization and standardization of schemas in collaboration with industry to facilitate their adoption by stakeholders addressing varied energy-focused use cases.

The growth in scientific data volumes has resulted in a need to scale up processing and analysis pipelines using High Performance Computing (HPC) systems. These workflows need interactive, reproducible analytics at scale. The Jupyter platform provides core capabilities for interactivity but was not designed for HPC systems. In this paper, we outline our efforts that bring together core technologies based on the Jupyter Platform to create interactive, reproducible analytics at scale on HPC systems. Our work is grounded in a real world science use case - applying geophysical simulations and inversions for imaging the subsurface. Our core platform addresses three key areas of the scientific analysis workflow - reproducibility, scalability, and interactivity. We describe our implemention of a system, using Binder, Science Capsule, and Dask software. We demonstrate the use of this software to run our use case and interactively visualize real-time streams of HDF5 data.

Scientific software development processes are understood to be distinct from commercial software development practices due to uncertain and evolving states of scientific knowledge. Sustaining these software products is a recognized challenge, but under-examined is the usability and usefulness of such tools to their scientific end users. User research is a well-established set of techniques (e.g., interviews, mockups, usability tests) applied in commercial software projects to develop foundational, generative, and evaluative insights about products and the people who use them. Currently these approaches are not commonly applied and discussed in scientific software development work. The use of user research techniques in scientific environments can be challenging due to the nascent, fluid problem spaces of scientific work, varying scope of projects and their user communities, and funding/economic constraints on projects. In this paper, we reflect on our experiences undertaking a multi-method user research process in the Deduce project. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists make decisions around data change. There is a lack of common terminology since the concept of systematically measuring and managing data change is under explored in scientific environments. To bridge this gap we conducted user research that focuses on user practices, needs, and motivations to help us design and develop metrics and tools for data change. This paper contributes reflections and the lessons we have learned from our experiences. We offer key takeaways for scientific software project teams to effectively and flexibly incorporate similar processes into their projects.

Author(s): Paine, Drew; Ghoshal, Devarshi; Ramakrishnan, Lavanya | Abstract: Scientific datasets are continually expanding and changing due to fluctuations with instruments, quality assessment and quality control processes, and modifications to software pipelines. Datasets include minimal information about these changes or their effects requiring scientists manually assess modifications through a number of labor intensive and ad-hoc steps. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists systematically identify and make decisions around data changes. Currently, there is a lack of understanding, and common practices, for identifying and evaluating changes in datasets since systematically measuring and managing data change is under explored in scientific work. We are conducting user research to address this need by exploring scientist's conceptualizations, behaviors, needs, and motivations when dealing with changing datasets. Our user research utilizes multiple methods to produce foundational, generative insights and evaluate research products produced by our team. In this paper, we detail our user research process and outline our findings about data change that emerge from our studies. Our work illustrates how scientific software teams can push beyond just usability testing user interfaces or tools to better probe the underlying ideas they are developing solutions to address.

This paper considers sensemaking as it relates to everyday software engineering (SE) work practices and draws on a multi-year ethnographic study of SE projects at a large, global technology company building digital services infused with artificial intelligence (AI) and machine learning (ML) capabilities. Our findings highlight the breadth of sensemaking practices in AI/ML projects, noting developers' efforts to make sense of AI/ML environments (e.g., algorithms/methods and libraries), of AI/ML model ecosystems (e.g., pre-trained models and "upstream" models), and of business-AI relations (e.g., how the AI/ML service relates to the domain context and business problem at hand). This paper builds on recent scholarship drawing attention to the integral role of sensemaking in everyday SE practices by empirically investigating how and in what ways AI/ML projects present software teams with emergent sensemaking requirements and opportunities.

Process has been a concern since the beginning of CSCW. Developments in sociotechnical landscapes raise new challenges for studying processes (e.g., massive online communities bringing together vast crowds; Big Data technologies connecting many through the flow of data). This re-opens questions about how we study, document, conceptualize, and design to support processes in complex, contemporary sociotechnical systems. This one-day workshop will bring together researchers to discuss the CSCW community's unique focus and methodological toolkit for studying process and workflow; provide a collaborative space for the improvement and extension of research projects within this space; and catalyze a network of scholars with expertise and interest in addressing challenging methodological questions around studying process in contemporary, sociotechnical systems.

Software is an integral element of the work of science yet it is not commonly an object of inquiry in studies of scientific infrastructures. This paper presents findings from an ethnographic study of a cosmology group's collaborative scientific software production. We demonstrate how these cosmologists use plots to simultaneously test their software and analyze data while interrogating multiple layers of infrastructural components. We broaden perspectives on scientific software development using a sociotechnical, software studies lens to examine this work of scientific discovery as a creative and embodied, yet exacting and methodical, activity that requires a 'human in the loop'. We offer a new reading of scientific software practices to convey how creating scientific software is often really the act of doing science itself--an intervention we believe is necessary to more successfully support scientific software sharing and infrastructure production.

Over the last several years, a growing body of work has examined the nature of large-scale virtual organizations for data-intensive cooperative science. These projects, known as Cyberinfrastructures (CI) in the United States, are established realms of inquiry for the eScience and Computer Supported Cooperative Work (CSCW) communities. Scholarship in these communities extends technology focused inquiries to investigate the sociotechnical concerns to such infrastructure creation and maintenance. In this paper we present findings from our qualitative study of a federated cyberinfrastructure organization known as GENI. We contribute to this body of scholarship by investigating how stakeholders in the GENI project position existing, and newly created, resources for use in educational settings. We examine how stakeholders acquaint new potential stakeholders with this CI in order to draw them into the community, and the ways in which stakeholder's roles evolve over time. Our findings illustrate several ways stakeholders leverage and align existing relationships and resources to expand the CI project's user base. Finally, this paper suggests avenues of further inquiry and implications for organizing future CI projects.

Rob Kitchin’s The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences is a timely volume that knits together research and perspectives on the twenty-first century data deluge from multiple scholarly communities connected to CSCW. The book’s aim is threefold. First, to examine in detail and reflect upon the nature of data and the assemblages they are a part of. Second, to plot how data assemblages are Bshifting and mutating with the development of new data infrastructures, open data and big data.^ Third and finally, the volume offers an interrogation of the implications of these Bnew data assemblages with respect to how we make sense of and act in the world.^ Scholars new and old in CSCWwill benefit from Kitchin’s in-depth examination of the many facets of long-term and emerging research in data studies. Research in CSCW has long examined the sociotechnical work of infrastructuring through studies of scientific collaboration—e.g., cyberinfrastructure and eScience studies (cf. Jirotka, et al. 2006; Ribes and Lee 2010)—and in healthcare informatics, among other domains. The Data Revolution synthesizes this work while incorporating perspectives that may receive less immediate attention in this journal.

The CSCW community is reliant upon technology-centric models of groupware and collaboration that frame how we examine and design for cooperative work. This paper both reviews the CSCW literature to examine existing models of collaborative work and proposes a new, expanded conceptual model: the Model of Coordinated Action (MoCA). MoCA is a broader framework for describing complex collaborative situations and environments including, but not limited to, collaborations that have diverse, high-turnover memberships or emerging practices. We introduce MoCA's seven dimensions of coordinative action and illustrate their connection to past and current CSCW research. Finally, we discuss some ramifications of MoCA for our understanding of CSCW as a sociotechnical design space.

Coordinating Centers (CCs) are central bodies tasked with the work of coordination and operations management of a virtual organization whose purpose is to conduct multi-site research projects. We call these organizations Coordinating Center Enabled Networks (CCENs). This qualitative, interview-based study followed two CCs in the field of cancer epidemiology over seven months to answer the question: How does a CC facilitate the work of networked science in a CCEN? In order to answer the question of how CCs facilitate work, we first describe the complex ecology of CCEN work practices. We further discuss how various stakeholders engage in different work practices to facilitate scientific progress. Finally, we use the conceptual lenses of local articulation work and metawork together with the diversity of work practices to better understand what practices CCs actually coordinate.

The production and use of software pipelines is a key component of much modern scientific research. We present emerging findings from our qualitative, social science study of a radio astronomy group developing software pipelines as they produce a data processing infrastructure. This paper examines how these researchers co-produce data products and software pipelines to enact their research infrastructure. We investigate the work of co-producing data and software to illustrate that to better support data-intensive science, we need to understand the practices that enable and produce data products, software, and ultimately infrastructures.

Middleware software, which provides an abstraction layer between low-level computational services and domain-specific applications, is a key component of cyberinfrastructure. This paper presents a qualitative study of how cyberinfrastructure middleware development is accomplished in two supercomputing centers. Our investigation highlights key development phases in the lives of middleware projects. Middleware development is typically undertaken as part of collaborations between technologists and domain scientists, and middleware developers must balance the pressure to meet specific scientific needs and the desire to explore their own R&D agendas. We explore how developers work to sustain an ongoing development trajectory by aligning their own work with particular domain science projects and funding streams. However, we find that the key transition from being a component in a domain-specific project to a stand-alone system that is useful across domains is particularly challenging for middleware development. We provide organizational and national policy implications for how to better support this transition.

The objective of this work is to explore how implications for educational practice are represented in the scholarly literature of engineering education. This work is motivated by the increasing urgency to understand how research can be used to transform educational practice. Because scholarly writing (such as conference papers and journal articles) represents one place where researchers articulate ideas about how their research can be used (i.e., educational implications), an analysis of how such implications are represented can provide insight into ideas about the uses we imagine for our research. Once we have characterized the vision of use, we can step back and think about whether the collective vision we present may provide clues to the more general issue of transforming education.

While previous CSCW research has noted that computer scientists have their own research interests pertaining to cyberinfrastructure development projects, most have focused on the research imperatives of scientists. This qualitative, interview-based study investigates the perspective of computer scientists developing middleware software for cyberinfrastructures at two supercomputing centers. This paper examines how technologists develop and sustain middleware applications over time by leveraging expertise and partnering with different research domains in order to achieve long-term infrastructural goals.