Search

NPS NRP Executive Summary Navy leadership is interested in initiatives that can potentially increase the responsiveness of campaign analysis. Simulation-based campaign analysis is used to measure risk for investment options in how best to equip, organize, supply, maintain, train, and employ our naval forces. The Synthetic Theater Operations Research Model (STORM) is a stochastic simulation model used to support campaign analysis by the U.S. Navy, Marine Corps, and Air Force. Building, testing, running, and analyzing campaign scenarios in STORM is a complex, time-consuming process. A simulated campaign may span months, involve scores of ships and battalions, hundreds of aircraft and installations, all executing thousands of interconnected missions involving numerous events in time and space. Creating, testing, and approving the inputs for a single design point (DP) requires a significant investment in analysts’ time and computing resources. Consequently, there are limits on the number of DPs that can be produced, executed, and analyzed during a study’s timeframe. N8 - Integration of Capabilities & Resources This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). Approved for public release. Distribution is unlimited.

NPS NRP Executive Summary The Navy uses simulation-based campaign analysis to help measure risk for investment options for how best to equip, organize, supply, maintain, train, and employ our naval forces. The Synthetic Theater Operations Research Model (STORM) is a stochastic simulation model used to support campaign analysis by the U.S. Navy, Marine Corps, and Air Force. Building, testing, running, and analyzing campaign scenarios in STORM can be a complex, time-consuming process. The goal of this research is to apply Design Of Experiment (DOE) methods in the selection and creation of Design Points (DPs) to minimize the number of modeling runs required for meaningful comparisons. Another objective is to understand how best DOE methods can complement traditional baseline and excursion modeling. In addition to regular reviews, the research deliverables will include: (1) a final brief and/or technical report, in addition to student theses (if applicable); (2) all findings, methods, and data used in the study; and (3) appropriate conference or journal papers related to this research. N8 - Integration of Capabilities & Resources This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrp Chief of Naval Operations (CNO) Approved for public release. Distribution is unlimited.

NPS NRP Executive Summary The Navy uses simulation-based campaign analysis to help measure risk for investment options for how best to equip, organize, supply, maintain, train, and employ our naval forces. The Synthetic Theater Operations Research Model (STORM) is a stochastic simulation model used to support campaign analysis by the U.S. Navy, Marine Corps, and Air Force. Building, testing, running, and analyzing campaign scenarios in STORM can be a complex, time-consuming process. The goal of this research is to apply design of experiment (DOE) methods in the selection and creation of design points (DPs) to minimize the number of modeling runs required for meaningful comparisons. Another objective is to understand how best DOE methods can complement traditional baseline and excursion modeling. In addition to regular reviews, the research deliverables will include: (1) A final brief and/or technical report, in addition to student theses (if applicable) (2) all findings, methods, and data used in the study; and (3) appropriate conference or journal papers related to this research. N8 - Integration of Capabilities & Resources This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrp Chief of Naval Operations (CNO) Approved for public release. Distribution is unlimited.

NPS NRP Executive Summary The Marine Corps seeks to integrate analytical processes that develop actionable insights for decision makers across its enterprise. One supporting goal in this endeavor is to develop a methodology for implementing a wargame-analytic continuum by formally linking outcomes of wargames with more traditional constructive closed-form simulations. The primary research objective is to explore methods and tools that provide the greatest utility when integrating wargames with constructive simulations. As part of this, we will develop, assess, and document a translation/transformation process used to convert a human-in-the-loop wargaming activity into the analytic domain, highlighting judgment calls and issues, and how they were resolved. The research will be done in support of one or more current Marine Corps analysis issues. The SEED Center will focus on constructive simulations that augment other analyses by other organizations. Candidate analysis "partners" include OAD, MCWL, and NPS. Wargaming activities may include NPS, MCWL, and others. MCWL live-fire events may also be leveraged or informed. In addition to regular reviews, the research deliverables will include: (1) A final brief and/or technical report and student theses (if applicable); (2) All findings, methods, and data used in the study, as well as guidance on how the Marines can most efficiently acquire and organically use wargames within an analytic continuum; and (3) Appropriate conference or journal papers related to this research. Marine Corps Modeling & Simulation Office (MCMSO) HQMC Combat Development and Integration (CD&I) This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrp Chief of Naval Operations (CNO) Approved for public release. Distribution is unlimited.

The goal of this research was to improve upon the ability of OPNAV N1 analysts to quickly and efficiently obtain experiment-based information from their computational models. The enhanced information will enable N1’s analysts to better support Navy leadership in resource and policy decisions that shape the future Navy and help it retain and develop its most talented Sailors. This project built on previous collaborations with N1 using data farming to enhance the information gleaned from their Navy talent management models, such as the Officer Strategic Analysis Model (OSAM) model, the Production Resource Optimization (PRO) model, and the Navy Total Force Strength Model (NTFSM). During this research period, (1) Ensign William Desousa (2015) investigated the behavior of economic inputs in NTFSM; (2) Lieutenant Peter Bazalaki (2016) used the new data farming capabilities we developed in OSAM to investigate Surface Warfare Officer (SWO) inventory across a breadth of possibilities; and (3) Lieutenant Allison Hogarth (2016) built, tested, and demonstrated a user interface in Excel that enables users of the PRO model to automatically execute a sophisticated design of experiments—the tool that enables this new capability is known as Production Resource Optimization Model With Experimental Design (PROMWED). In addition to working with the student-officers, the faculty supporting this project performed an empirical study of statistical software packages that may provide better understanding of the high-dimensional behavior of manpower models in the future (Erickson, Ankenman, & Sanchez 2016). Naval Research Program Prepared for Topic Sponsor: OPNAV N1; Research POC Name: Mr. Ian Anderso NPS-N16-N154-A

The article of record as published may be found at http://dx.doi.org/10.1002/nav.21628 Decades ago, simulation was famously characterized as a “method of last resort,” to which analysts should turn only“when all else fails.” In those intervening decades, the technologies supporting simulation—computing hardware, simulation-modeling paradigms, simulation software, design-and-analysis methods—have all advanced dramatically. We offer an updated view that simulation is now a very appealing option for modeling and analysis. When applied properly, simulation can provide fully as much insight, with as much precision as desired, as can exact analytical methods that are based on more restrictive assumptions.The fundamental advantage of simulation is that it can tolerate far less restrictive modeling assumptions, leading to an underlying model that is more reflective of reality and thus more valid, leading to better decisions.

SEED Center Paper Final Report of Task Group MSG-088. Final Report of Task Group MSG-088 Data Farming is a process that has been developed to support decision-makers by answering questions that are not currently addressed. Data farming uses an inter-disciplinary approach that includes modelling and simulation, high performance computing, and statistical analysis to examine questions of interest with large number of alternatives. Data farming allows for the examination of uncertain events with numerous possible outcomes and provides the capability of executing enough experiments so that both overall and unexpected results may be captured and examined for insights. In 2010, the NATO Research and Technology Organization started the three-year Modeling and Simulation Task Group “Data Farming in Support of NATO” to assess and document the data farming methodology to be used for decision support. This final report documents the completed work of this Task Group, designated MSG-088. It includes a description of the six realms of data farming as well as the two case studies performed during the course of MSG-088. This report leverages the knowledge of data farming experts to codify data farming, describing how best to perform data farming to gain insight into questions in support of decision makers.

SEED Center Paper Simulation models are integral to modern scientific research, national defense, industry and manufacturing, and public policy debates. These models tend to be extremely complex, often with thousands of factors and many sources of uncertainty. To understand the impact these factors and their interactions have on model outcomes requires efficient, highdimensional design of experiments. All too often, many largescale simulation models are explored in ad hoc ways. This suggests that more simulation researchers and practitioners need to be aware of the power of experimental design in order to get the most from their simulation studies. In this tutorial, we demonstrate the basic concepts important for designing and conducting simulation experiments, and provide references to various resources for those wishing to learn more. This tutorial will prepare you to make effective use of designed experimentation in your next simulation study. Portions of this paper appeared or are updated from previous tutorials presented at the Winter Simulation Conference, most recently Sanchez and Wan (2012).

The article of record as published may be located at http://dx.doi.org/10.1016/j.ejor.2011.07.005 Analysts faced with conducting experiments involving quantitative factors have a variety of potential designs in their portfolio. However, in many experimental settings involving discrete-valued factors (particularly if the factors do not all have the same number of levels), none of these designs are suitable. In this paper, we present a mixed integer programming (MIP) method that is suitable for constructing orthogonal designs, or improving existing orthogonal arrays, for experiments involving quantitative factors with limited numbers of levels of interest. Our formulation makes use of a novel linearization of the correlation calculation. The orthogonal designs we construct do not satisfy the definition of an orthogonal array, so we do not advocate their use for qualitative factors. However, they do allow analysts to study, without sacrificing balance or orthogonality, a greater number of quantitative factors than it is possible to do with orthogonal arrays which have the same number of runs.

The article of record as published may be located at http://dx.doi.org/10.1016/j.ijpe.2005.06.007 We illustrate how an efficient methodology called frequency domain experimentation can be used to gain better insight into the behavior of production systems. With the full factorial designs commonly used for simulation experiments, the number of runs grows exponentially with the number of factors involved, while the run length remains constant. In frequency domain experiments, the number of runs is independent of the number of factors, while the required run lengths increase relatively slowly. We describe the method, illustrate its effectiveness at identifying important main effects, two-way interactions, and quadratic terms for a known model, demonstrate the approach by evaluating a kanban system involving 34 factors, and provide links to software. We also present computational requirements for running simulation experiments that combine a batch means approach with efficient run-oriented designs for a variety of systems. The results indicate that frequency domain experiments perform very favorably for systems, such as queueing networks, where the simulation’s output stream exhibits high positive autocorrelation.

As the world continues its rapid rise in urbanization, the battlefield of the future will likely be urban. This thesis explores the effective mix of three types of 155mm artillery munitions with varying accuracy (M982 Excalibur, M1156 Precision Guided Kit, and M795 dumb bomb) in two urban battlefield scenarios. Mission success corresponds to maximizing effects on enemies with minimal fratricides and collateral damage. Pythagoras, an agent-based simulation, together with efficient design of experiments (DOE), is used to study the effects and interactions of controllable factors (system capabilities) and uncontrollable factors (environment and terrain) on the intended measures of effectiveness. Regression metamodels from the DOE are then used to project a Pareto optimal frontier (POF) of the mixture that satisfies multi-objective functions, which includes the cost of munitions expended. A multi-objective evolutionary algorithm is also employed to identify a set of POFs or near-optimal alternatives to the DOE results. Although results from both battle scenarios favor high allocation of M982 (10–15%) for operational effectiveness, the effectiveness of M1156 only proves to be significant in a smaller terrain. A trade-off analysis between operational and cost effectiveness also shows that a substantial amount of M795 (>60%) is needed in the mixes. http://archive.org/details/exploringcharact1094560370 Outstanding Thesis Army, Singapore Approved for public release; distribution is unlimited.

This research examines the effectiveness of resupply systems in humanitarian assistance and disaster relief (HADR) operations by exploring different permutations of operational energy (OE)-focused assets and policies that a Marine Expeditionary Unit (MEU) employs to improve its throughput of resources to disaster victims. The basis for the modeled scenario is the support provided by the 31st MEU to the city of Hachinohe as part of Operation TOMODACHI. This thesis focuses on OE and only considers the medium tactical vehicle replacement (MTVR) as the baseline capability. An agent-based simulation is then used to model the effectiveness of OE-focused resupply strategies and capabilities. These options include (1) efficient driving techniques, (2) reducing idling time, (3) hybrid technologies, and (4) follower vehicles. To investigate their effectiveness, this research uses a design of experiments approach to efficiently examine a set of design factors for specified operational plans. Statistical results indicate that the operational plans employing shorter and quicker vehicle convoys that communicate with one another are most effective in resupplying isolated victims. This research also confirms that the employment of OE-focused assets and policies is effective in increasing timeliness of resupply. Taken together, these factors contribute toward increasing the operational reach of a MEU conducting HADR resupply. http://archive.org/details/anengineeredresu1094556144 Military Expert 5, Republic of Singapore Army Approved for public release; distribution is unlimited.

Marine Corps logistics always seeks to remain responsive, flexible, and sustainable to successfully support highly maneuverable units dispersed over large operational areas. The variety of Marine Corps battlespaces and the constant evolution of enemy weapons and tactics, however, makes logistic support increasingly difficult. Unmanned logistics systems (ULS) show potential to reduce Marine personnel risk and workload, and increase throughput, efficiency, and flexibility. To assist in the development of ULS operational concept and platform employment, this thesis uses discrete event simulation and a designed experiment to model and explore a ship-to-shore logistics process supporting dispersed units via three types of ULSs, which vary primarily in size. Major findings from the analysis illustrate the importance of the type of logistics method used in predicting successful re-supply and risk effects to the system. The hub-and-spoke re-supply method is less variable, returns higher ratios of delivered supplies, and performs better independent of risk when compared to the linear method. The observed method affects increase with the distance a unit is from the main logistics node. Small ULSs should be used for just-in-time re-supply, medium ULSs should be used for throughput, and all systems should be survivable to minimize risk. http://archive.org/details/simulatingsustai1094555593 Outstanding Thesis Captain, United States Marine Corps Approved for public release; distribution is unlimited.

There are several problems that limit the availability of modern-type helicopters that the German Army has been acquiring during the last ten years, including inspection policies, reliability, spare parts availability, and the number of personnel devoted to aircraft maintenance. The goal of this thesis is to identify factors that could lead to measurable improvement in operational availability and flight-hour supply of the German Army helicopter fleet. This research uses statistical analysis of failure-time data and a simulation model that emulates the usage and maintenance policies adopted by the fleet. The simulation model reflects normal daily operating and maintenance activities and manages individual aircraft with respect to flying operations and maintenance activities, including extensive scheduled inspections, non-recurring special inspections, and failure-driven unscheduled maintenance actions for each aircraft on a daily basis. The model reproduces recent historical trends accurately. It provides useful insights about the future availability of the German Army helicopter fleet by keeping current policies in place and making investments in the maintenance system. Without such investments, availability can be expected to decline over the next several years. By increasing maintenance assets, the decline can be reduced in a quantifiable manner. http://archive.org/details/annalysisoffacto1094555643 Outstanding Thesis Major, German Army Approved for public release; distribution is unlimited.

Before a newly built ship is brought into service, it has to undergo various trials as part of its delivery. The builder shipyard aims at completing the maximum number of trials in the minimum possible time. Many trials have one or more prerequisites and every trial may have certain environmental requirements for its conduct. At sea, the success of any specific trial cannot be guaranteed. A dynamic tool is needed to help decision makers rapidly construct alternative trial sequences after the failure of any trial, and aid them in deciding whether the trials can be continued, or the ship has to return to the harbor for repairs. This thesis develops such an adaptive tool, which generates an optimal sequence and feasible route for conduct of a given set of trials, minimizing the total time required in the absence of failures or adverse environmental conditions. The tool allows the user to generate alternate sequences of trials if an early trial fails. Simulation of the conduct of trials, under varying environmental conditions, reveals that the number of retries is the most important factor affecting the outcomes. It also identifies bottlenecks in the network, providing insight about onboard spare supportability for important systems. http://archive.org/details/andaptivemethodf1094547335 Outstanding Thesis Lieutenant Commander, Pakistan Navy Approved for public release; distribution is unlimited.

Taiwan is prone to many natural disasters, especially typhoons. This thesis adapts an existing stochastic prepositioning optimization model to create a tool for Taiwan military disaster recovery planners, and then uses experimental design techniques to systematically explore solutions. The goals are to minimize the expected number of casualties and unmet commodities demands, and to determine the average number of workers deployed in response to each scenario. A design of experiments methodology is applied to the optimization model to reveal how uncertainty in the parameters translates to uncertainty in objective function values. The approach can also identify the parameters with the greatest impact on the objective function, and result in more robust solutions. The analysis demonstrates that it is not always necessary to spend as much money and deploy as many workers as in the past in order to get the best results. Additionally, the approach shows how a decision maker, with more accurate and current weather reports, can refer to the path and intensity of typhoons while making rescue plans. In summary, this research shows that there is great potential for quantitative methods to improve the disaster-relief planning process. http://archive.org/details/improvingtaiwmil1094545891 1st Lieutenant, Republic of China Army Approved for public release; distribution is unlimited.

The current surface mine countermeasures (MCM) fleet is aging, yet there are no viable systems to replace it. The U.S. Navy requires an improved minehunting platform, and unmanned underwater vehicles (UUVs) can meet that need. In order to attain enough UUVs and operators to make these missions successful, the United States must rely on the participation of allies to provide these assets. This study assesses the key decision factors in mine clearance operations using UUVs of differing capabilities. It uses a discrete-event simulation to model the performance of UUVs in a large-scale MCM operation. Data is generated using a state-of-the- art design of experiments and analyzed to find the best tasking plan for the scenario. The results show that with proper tasking, UUVs with lesser ability levels can be used appropriately and still produce acceptable levels of mine clearance, usually more quickly than a smaller cadre of highly capable vehicles. This study finds UUV altitude, track spacing, number of passes, and search speed to be decision factors that influence minehunting results, while track spacing, number of passes, search speed, and resupply are influential factors that effect mission completion times. http://archive.org/details/evaluatingcombin1094545263 Lieutenant, United States Navy Approved for public release; distribution is unlimited.

According to Department of Defense (DOD) Instruction 2205.02 (June 23, 2014), DOD components must conduct humanitarian and civic assistance (HCA) activities in response to regional conflicts or natural disasters. The Under Secretary of Defense for Policy determines how HCA policy is coordinated and implemented within the DOD and delegates responsibility to the regional combatant commands. In past modeling efforts for disaster relief, stochastic optimization has been utilized and produced promising results; however, the deterministic nature of optimization models may not fully capture the uncertainty that is inherent in natural disasters and the demand created by them. In order to better understand the effects of the uncertainty surrounding natural disasters and realize a robust logistical response to these events, new approaches are necessary. This thesis develops an asset allocation optimization model for naval logistics, and then uses experimental design techniques to systematically explore solutions to the model. Our analysis reveals the importance of robust planning for natural disaster response to ensure that demand is met and a quick response is possible. Finally, we explore the use of unmanned aerial vehicles as logistics assets, and show that they have the potential to add much benefit to foreign humanitarian assistance. http://archive.org/details/investigatingnav1094545189 Lieutenant Commander, United States Navy Approved for public release; distribution is unlimited.