Search

Organizations use command and control (C2) systems to collect, organize, and disseminate information in order to make decisions, impart instructions, and manage resources to accomplish a mission. C2 agility and robustness are critical to ensure that a C2 system can perform well in a variety of environments. One system design principle, decentralization, has been closely linked to desirable system characteristics such as agility and adaptability, but its relationship to system performance robustness is not well‐established. In this study, we explore C2 system architectures—ranging from fully centralized to fully decentralized archetypes—to assess their performance and robustness characteristics across a spectrum of operating environments. While the centralized archetype achieves high performance in favorable environmental conditions, its performance quickly degrades when conditions worsen, hindering its overall robustness. Conversely, the decentralized archetype achieves a lower but more stable performance profile resulting in more robustness when performance requirements are lower. Finally, we explore alternative, hybrid architectures with varying degrees of centralized and decentralized decision‐making. We find that by centralizing only certain system‐consequential functions such as resource allocation, and decentralizing more focused decision functions which can be performed suitably with only local information and resources, system performance and robustness are improved beyond that of the simple archetypes. [ABSTRACT FROM AUTHOR]

Standardized design approaches such as those embodied by concurrent design facilities have many benefits, such as increased efficiency of the design process, but may also have hidden costs. Specifically, when their standardized organizational decomposition is a poor fit for the particular design problem, important design trades might be missed or poor decisions made. Before we can understand how this lack of fit impacts the design process, we must be able to empirically observe and measure it. To that end, this paper identifies measures of "fit" from the literature along with attributes likely to impact design process performance, then evaluates the measures to determine how well the measures can detect and diagnose potential issues. The results provide comparative insights into the capabilities of existing fit measures, and also build guidance for how the systems engineering and design community can use insights from the "fit" literature to inform process improvement. [ABSTRACT FROM AUTHOR]

Engineering systems design is a dynamic socio-technical process where the social factors, such as interdisciplinary interactions, and technical factors, such as design interdependence and the design state, co-evolve. Understanding this co-evolution can lead to behavioral insights, resulting in efficient communication pathways and better designs. In that context, we investigate how to generate behavioral insights to inform effective structuring of interdisciplinary interactions in engineering systems design teams. We present an approach that combines the predictive capabilities of computational modeling with contextual information from empirical data. A stochastic network-behavior dynamics model quantifies the co-evolution of design interdependence, discipline-specific interaction decisions, and the changes in system performance. We employ two datasets, one of the student subjects designing an automotive engine and NASA engineers designing a spacecraft. Then, we apply Bayesian statistical inference to estimate model parameters and compare insights across the two datasets. The results indicate that design interdependence and social network factors such as reciprocity and popularity have strong positive effects on interdisciplinary interactions for the expert and student subjects alike. An additional modulating impact of system performance on the number of interactions is observed for the student subjects. Inversely, the total number of interactions, irrespective of their discipline-wise distribution, has a weak but statistically significant positive effect on system performance in both cases. However, we observe that excessive interactions mirrored with design interdependence and inflexibility in design exploration reduced the system performance. These insights support the case for open boundaries in systems design teams to improve system performance. [ABSTRACT FROM AUTHOR]

Meeting the United Nations Sustainable Development Goals (SDGs) will require adapting or redirecting a variety of very complex global and local human systems. It is essential that development scholars and practitioners have tools to understand the dynamics of these systems and the key drivers of their behavior, such as barriers to progress and leverage points for driving sustainable change. System dynamics tools are well suited to address this challenge, but they must first be adapted for the data‐poor and fragmented environment of development work. Our key contribution is to extend the causal loop diagram (CLD) with a data layer that describes the status of and change in each variable based on available data. By testing dynamic hypotheses against the system's actual behavior, it enables analysis of a system's dynamics and behavioral drivers without simulation. The data‐layered CLD was developed through a 4‐year engagement with USAID/Uganda. Its contributions are illustrated through an application to agricultural financing in Uganda. Our analysis identified a lack of demand for agricultural loans as a major barrier to broadening agricultural financing, partially refuting an existing hypothesis that access to credit was the main constraint. Our work extends system dynamics theory to meet the challenges of this practice environment, enabling analysis of the complex dynamics that are crucial to achieving the SDGs. [ABSTRACT FROM AUTHOR]

Testing is critical to mitigating the COVID-19 pandemic, but testing capacity has fallen short of the need in the United States and elsewhere, and long wait times have impeded rapid isolation of cases. Operational challenges such as supply problems and personnel shortages have led to these bottlenecks and inhibited the scale-up of testing to needed levels. This paper uses operational simulations to facilitate rapid scale-up of testing capacity during this public health emergency. Specifically, discrete event simulation models were developed to represent the RT-PCR testing process in a large University of Maryland testing center, which retrofitted high-throughput molecular testing capacity to meet pandemic demands in a partnership with the State of Maryland. The simulation models support analyses that identify process steps which create bottlenecks, and evaluate "what-if" scenarios for process changes that could expand testing capacity. This enables virtual experimentation to understand the trade-offs associated with different interventions that increase testing capacity, allowing the identification of solutions that have high leverage at a feasible and acceptable cost. For example, using a virucidal collection medium which enables safe discarding of swabs at the point of collection removed a time-consuming "deswabbing" step (a primary bottleneck in this laboratory) and nearly doubled the testing capacity. The models are also used to estimate the impact of demand variability on laboratory performance and the minimum equipment and personnel required to meet various target capacities, assisting in scale-up for any laboratories following the same process steps. In sum, the results demonstrate that by using simulation modeling of the operations of SARS-CoV-2 RT-PCR testing, preparedness planners are able to identify high-leverage process changes to increase testing capacity. [ABSTRACT FROM AUTHOR]

The engineering of complex systems, such as aircraft and spacecraft, involves large number of individuals within multiple organizations spanning multiple years. Since it is challenging to perform empirical studies directly on real organizations at scale, some researchers in systems engineering and design have begun relying on abstracted model worlds that aim to be representative of the reference socio-technical system, but only preserve some aspects of it. However, there is a lack of corresponding knowledge on how to design representative model worlds for socio-technical research. Our objective is to create such knowledge through a reflective case study of the development of a model world. This "inner" study examines how two factors influence interdisciplinary communication during a concurrent design process. The reference real world system is a mission design laboratory (MDL) at NASA, and the model world is a simplified engine design problem in an undergraduate classroom environment. Our analysis focuses on the thought process followed, the key model world design decisions made, and a critical assessment of the extent to which communication phenomena in the model world (engine experiment) are representative of the real world (NASA's MDL). We find that the engine experiment preserves some but not all of the communication patterns of interest, and we present case-specific lessons learned for achieving and increasing representativeness in this type of study. More generally, we find that representativeness depends not on matching subjects, tasks, and context separately, but rather on the behavior that emerges from the interplay of these three dimensions. [ABSTRACT FROM AUTHOR]

Systems engineering and design (SE&D) researchers increasingly tackle questions at the intersection of technical and social aspects of complex systems design. Practical challenges of access, limited observation scope, and long timescales limit empirical study of SE&D phenomena. As a result, studies are typically conducted in model world settings abstracted from the real world, such as behavioral experiments with student subjects. Model worlds must be representative of the phenomena being studied to ensure insights generalize to the real‐world settings. Currently, there is a lack of shared understanding and standards within the SE&D research community to evaluate representativeness of model worlds. This communication captures the results of ongoing efforts to build consensus on this topic: it defines the concept of model worlds, disambiguates representativeness from related concepts, and draws comparisons to other research domains. It outlines a potential path forward and calls for community participation in establishing shared standards for model world representativeness in SE&D research. [ABSTRACT FROM AUTHOR]

Disaster response is a challenging environment for data‐driven decision support, because of its rarity, uniqueness, and high stakes. This article develops and demonstrates practical decision support approaches for an important problem faced by the U.S. Federal Emergency Management Agency (FEMA), identifies challenges and successes during their partial implementation, and explores how such approaches can encourage the adoption of data‐driven decision‐making in crisis situations. Specifically, this article develops approaches for locating and staffing temporary Disaster Recovery Centers (DRCs), which provide in‐person service to disaster‐affected communities. Working with FEMA, we developed two models that aim to improve service to survivors and minimize costs. One model fits easily into FEMA's current decision‐making process, while the other further improves service by challenging some extant assumptions. By testing the models using data from three past disasters, we find that this decision support can result in cost savings of 75% on average by eliminating unnecessary DRCs and over‐staffing and, at the same time, maintain or reduce the maximum travel time required for the disaster‐affected population to access DRCs. Aspects of the models have already been used during disaster operations. FEMA's experience highlights the potential for data‐ and model‐driven analyses to improve resource allocation and demonstrates an approach to improve organizational decisions by developing models that either align with or challenge the decision‐making culture. [ABSTRACT FROM AUTHOR]

Abstract: This paper discusses the role that qualitative methods can and should play in engineering systems research and lays out the process of doing good qualitative research. As engineering research increasingly focuses on sociotechnical systems, in which human behavior and organizational context play important roles in system behavior, there is an increasing need for the insights qualitative research can provide. This paper synthesizes the literature on qualitative methods and lessons from the authors’ experience employing qualitative methods to study a variety of engineering systems. We hope that by framing the key issues clearly, other engineers who hope to join the qualitative path will build on what we have learned so far to enable greater insight into engineering systems. [ABSTRACT FROM AUTHOR]

The responder community must be ready to respond quickly and effectively in the event of a disaster. In order to maintain readiness, many disaster response communities exercise their response capabilities on a regular basis. The critical challenge is to design, conduct, and evaluate exercises in a manner that effectively tests responders' readiness and generates lessons that can improve readiness. This paper describes a framework to enable assessment of response readiness through evaluation of critical capabilities in exercises. It was developed for oil spill response based on the observation and analysis of four response exercises. The framework (1) identifies critical capabilities that lead to readiness for spill response, and maps them to (2) exercise design components that test each capability and (3) evaluation measures to evaluate each capability within an exercise. The framework enables continuous improvement by linking the evaluation of exercises to the critical capabilities required of an oil spill response organization; by evaluating the performance of specific capabilities, areas for improvement are clearly identified and can be re-tested in a future exercise. While the findings are necessarily specific to oil spill response, the principles apply to any disaster response context. [ABSTRACT FROM AUTHOR]

This paper develops the concept of representational uncertainty to frame a critical challenge in systems engineering. Representational uncertainty arises in complex systems problems when the correct system representation cannot practically be known until some initial work has been undertaken. Drawing on empirical evidence from two very different system design problems, we illustrate the nature and prevalence of representational uncertainty in systems engineering practice. Our findings show that errors in the system representation may lead to wasted design work that explores the wrong tradespaces, expects the wrong value from design choices, and organizes work on the wrong set of decomposed subproblems. We find that mitigating representational uncertainty requires design processes that incorporate discovery of the system properties through a "reality check" early in the design process. We consider the implications for systems engineering processes and tools, and highlight directions for future research. [ABSTRACT FROM AUTHOR]

When dealing with urgent, ill-defined problems, such as rapidly evolving emergency situations, operations managers have little time for problem formulation or solution. While the mechanisms by which humans formulate and solve problems have been described, mechanisms for rapid, concurrent formulating and solving are not well understood. This study investigates these mechanisms through a field study of transportation planning in a humanitarian response setting. The findings show that the problem is solved through greedy search and formulated through sensemaking, in which search enables updates to an evolving problem formulation, and the formulation directs and limits the search process. This study explores the implications of these findings for the development of better problem formulation processes and problem-solving strategies for urgent and ill-defined operations management problems. [ABSTRACT FROM AUTHOR]

Purpose - The purpose of this paper is to describe and analyze a successful training exercise in detail, through both a practical and a theoretical lens, in order to identify critical aspects of its success and enable others to build upon it; and to capture insights and lessons learned in a framework that will facilitate the design of future trainings for a variety of goals and audiences. Design/methodology/approach - The authors document and analyze the case study of a successful humanitarian logistics training exercise: the World Food Programme's Logistics Response Team (WFP's LRT) training. The LRT is described in detail in order to capture the extensive knowledge and experience that went into developing the full-scale, immersive exercise. Findings - The authors evaluate the LRT training through a theoretical lens, considering how it teaches the diverse set of skills required and identifying reasons for its success. The authors contrast the LRT with a light version developed for classroom use, and capture insights in a framework that highlights critical aspects of training design. Research limitations/implications - The requirements and design aspects highlighted in the framework are very high level, but they focus attention on key aspects that should be considered. Future research should develop more targeted metrics for evaluating what people learn from training exercises. More generally, a systematic approach to capturing knowledge and codifying good practices should be developed. Practical implications - The detailed case study and framework provide a basis for the design and improvement of simulated emergency training exercises, which are common in the humanitarian practice community. Originality/value - The case study of WFP's LRT training formally documents valuable knowledge and experience that went into its development. The humanitarian community can use the proposed framework to more systematically evaluate, improve, and extend training exercises. [ABSTRACT FROM AUTHOR]

Humanitarian aid agencies deliver emergency supplies and services to people affected by disasters. Scholars and practitioners have developed modeling approaches to support aid delivery planning, but they have used objective functions with little validation as to the trade-offs among the multiple goals of aid delivery. We develop a method to value the performance of aid delivery plans based on expert preferences over five key attributes: the amount of cargo delivered, the prioritization of aid by commodity type, the prioritization of aid by delivery location, the speed of delivery, and the operational cost. Through a conjoint analysis survey, we measure the preferences of 18 experienced humanitarian logisticians. The survey results quantify the importance of each attribute and enable the development of a piecewise linear utility function that can be used as an objective function in optimization models. The results show that the amount of cargo delivered is the most valued objective and cost the least important. In addition, experts prioritize more vulnerable communities and more critical commodities, but not to the exclusion of others. With these insights and the experts' utility functions, better humanitarian objective functions can be developed to enable better aid delivery in emergency response. [ABSTRACT FROM AUTHOR]

Designing international development projects is challenging because the complexity of the systems on which they act makes it difficult to identify the best leverage points for intervention. This paper seeks to identify the best combinations of interventions to increase the availability of and demand for quality seeds in Uganda and similar markets. A system dynamics model simulates the current dynamics in Ugandan seed markets based on data gathered by ongoing development projects. The findings show that one intervention is critical to enabling growth—investing in a system for verifying the quality of seeds—and that a combination of quality verification with education-oriented interventions is more powerful than quality verification alone. The results have implications for systems approaches to development: they suggest that a combination of interventions in different parts of the value chain enables larger changes than any single intervention alone. [ABSTRACT FROM AUTHOR]