Search

Sequential Simulation Optimization is an online optimization framework where an operator iterates periodically between collecting data from a real-world system, using stochastic simulation to approximate the optimal values of some operational variables, and setting some choice of variables in the system for the next period. The aim is to converge to an optimum efficiently, as uncertainty due to finite data and finitely many simulations eventually reduces. Using Bike Sharing Systems (BSS) as a motivating example, we analyze a variant where data from the real-world system is subject to censoring, whose nature depends on the system variables selected by the operator. In the BSS setting, censoring is of customer demand, to pick up or drop off bikes. We show that a method built upon Sample Average Approximation attains asymptotically vanishing error in its parameter estimates and specification of the optimal operational variables.

The Marine Corps’ Expeditionary Advanced Base Operations (EABO) concept places small, distributed forces within a contested environment to achieve strategic effects. However, current sustainment platforms and infrastructure lack the ability to provide required support in the context of EABO. As such, the Marine Corps must develop a novel resilient and robust distribution network capable of providing responsive sustainment to forces conducting EABO. To solve this problem, we develop a two-stage stochastic mixed integer linear program that seeks to minimize the cost of instantiating and operating a sustainment network across a range of possible in-context scenarios. Among the network’s constraints are demand, cost, capacity, risk, and supply considerations. Of these constraints, demand introduces the most uncertainty. Fluctuations in demand come from a variety of factors to include intensity of conflict, attrition, and other combat dynamics. Through the explicit and judicious modeling of such uncertainty, our model provides solutions that are robust to a wide range of demand scenarios. By implementing our model in a notional operational scenario in the South China Sea, we provide results and insights that ultimately assist the Marine Corps by providing an analytical basis for determining an optimal network comprised of locations, capacities, and prepositioning quantities for sustaining forces conducting EABO in the western Pacific., Outstanding Thesis, Captain, United States Marine Corps, Approved for public release. Distribution is unlimited.

Operational availability of naval aircraft through material readiness is critical to ensuring combat power. Supportability of aircraft is a crucial aspect of readiness, influenced by several factors including access to 9B Cognizance Code (COG) aviation consumable repair parts at various supply echelons. Rapidly evolving additive manufacturing (AM) technologies are transforming supply chain dynamics and the traditional aircraft supportability construct. As of June 2022, there are 595 AM assets within the Navy’s inventory—all for research and development purposes. This report simulates 9B COG aviation consumable fulfillment strategies within the U.S. Indo-Pacific sustainment network for a three-year span, inclusive of traditional supply support avenues and a developed set of user-variable capability inputs. Simulated probabilistic demand configurations are modeled from historical trends that exploit a heuristic methodology to assign a “printability” score to each 9B COG requirement, accounting for uncertainty, machine failure rates, and other continuous characteristics of the simulated orders. The results measure simulated lead time across diverse planning horizons in both current and varied operationalized AM sustainment network configurations. This research indicates a measurable lead time reduction of approximately 10% across all 9B order lead times when AM is employed as an order fulfillment source for only 0.5% of orders., NPS Naval Research Program, This project was funded in part by the NPS Naval Research Program., Lieutenant Commander, United States Navy, Approved for public release. Distribution is unlimited.

Adversarial submarine activity in the Atlantic has steadily intensified over the past few years. Furthermore, strategic adversaries have developed sophisticated and stealthy submarines, making them much more difficult to locate. The heightened activity coupled with advanced platforms have allowed the United States' adversaries to challenge its dominance in the underwater domain. Though extensive research has been performed on optimized search strategies using Bayesian search methods, most methodologies in the open literature focus on searching for stationary objects rather than searching for a moving Red submarine conducted by a Blue submarine. Thusly motivated, we develop a model of an enemy submarine whose goal is to avoid detection. As the search effort is expended, a posterior probability distribution for the enemy submarine’s location is calculated based on negative search results. We present a methodology for finding a search pattern that attempts to maximize the probability of detection in a Bayesian framework utilizing Markovian properties. Specifically, we study three different running window methods: a simple network optimization model, a network optimization model that performs updates after every time period that is planning the entire route, and a dynamic program that only looks two time periods ahead.

The Marine Corps’ Expeditionary Advanced Base Operations (EABO) concept places small, distributed forces within a contested environment to achieve strategic effects. However, current sustainment platforms and infrastructure lack the ability to provide required support in the context of EABO. As such, the Marine Corps must develop a novel resilient and robust distribution network capable of providing responsive sustainment to forces conducting EABO. To solve this problem, we develop a two-stage stochastic mixed integer linear program that seeks to minimize the cost of instantiating and operating a sustainment network across a range of possible in-context scenarios. Among the network’s constraints are demand, cost, capacity, risk, and supply considerations. Of these constraints, demand introduces the most uncertainty. Fluctuations in demand come from a variety of factors to include intensity of conflict, attrition, and other combat dynamics. Through the explicit and judicious modeling of such uncertainty, our model provides solutions that are robust to a wide range of demand scenarios. By implementing our model in a notional operational scenario in the South China Sea, we provide results and insights that ultimately assist the Marine Corps by providing an analytical basis for determining an optimal network comprised of locations, capacities, and prepositioning quantities for sustaining forces conducting EABO in the western Pacific.

NPS NRP Technical Report, Modern submarines have smaller acoustic signatures that require new search planning approaches in short timeframe environments. In a typical scenario, the search assets are destroyers, maritime patrol aircraft, UUVs, and submarines (two or three of each), and the decision space of a commander comprises command and control, asset allocation, asset employment, and operation planning. Of these, the last three are germane to this project. The novel planning study should take advantage of past observations and intelligence to guide the employment of assets at the tactical level. The study also should account for targets that behave strategically -- for instance, where Red behaves in a certain way so that Blue believes they will continue doing that in the future, but then at some later point Red behaves differently. The starting point of our work will be a spreadsheet already developed by the sponsor., Naval Undersea Warfighting Development Center (UWDC), U.S. Fleet Forces Command (USFF), This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrp, Chief of Naval Operations (CNO), Approved for public release. Distribution is unlimited.

Computer networks are often the target of cyber attacks carried out by malevolent agents, to either disable critical system operations or to surreptitiously gain access to sensitive data. The asymmetric and covert nature of cyber attacks has led to their increased prevalence, where high-impact attacks on critical infrastructure can be launched with minimal resources. We consider the setup of a network switch and its connected nodes, and use multi-armed bandit models as a framework to formulate a network attack strategy to maximize expected rewards earned over time. Such models present upper confidence bound–based approaches on the cumulative regret, through optimal choice of possible attacker actions over a finite time horizon and bounded action space. We evaluate relevant multi-armed bandit models and develop our own algorithm. Numerical simulations consistently suggest that low cumulative regret is achieved over time for our algorithm in comparison to the other algorithms evaluated. We thus present a stylized model for strategic network exploitation, with the attacker having no prior knowledge of the rewards of various nodes in a network with a star topology. This enables effective network defenses to be continually developed, based on specific network topologies and reward feedback mechanisms., http://archive.org/details/multiarmedbandit1094568308, Civilian, Singapore Technologies Electronics, Singapore, Approved for public release. Distribution is unlimited.

As technology advances and nature becomes more unpredictable, hospitals need to focus on their response to specific threats and their capacity to handle them efficiently. Many studies have shown that due to underfunding, hospitals lack the basic size required to properly service populations. The purpose of this thesis is to build and analyze a model of a local hospital's emergency room operations under typical and surge conditions in terms of resource demand and utilization. The model will analyze emergency room bed capacity based on day-to-day operations and potential threats that could impact the surrounding area, thus creating a surge in bed capacity. This study shows where local resources may be under capacity and provides ways for them to improve their operations by planning a solid budget. Local community planners must understand the challenges and current limitations for their hospitals to receive assistance from state and federal funding sources. If planners do not grow their hospitals to coincide with their cities' population growth, they will continue to fail and provide increasingly inadequate care., Civilian, Department of the Navy, Approved for public release. Distribution is unlimited.

This thesis deals with online optimization of discrete performance measures in Markovian models with incomplete information. We consider a setting where a physical realization of the model is sequentially obtained over a number of periods. The information gathered to date is used in order to efficiently run the model in future days. The information is incomplete in two ways: (i) model parameters are initially unknown (the demand rates in our case), but can be estimated from the physical realizations; and (ii), the demands are censored when the system is in some boundary states. The method of Sample Average Approximation is used to solve the optimization problem. More precisely, in each period, sample paths are generated from the distributions estimated to date, and the best model configuration is determined with respect to these sample paths. Sequential observation of the system’s behavior allows for information to be gathered and a more informed decision to be made in each future round. The method developed in this thesis can be applied in a variety of contexts, where no information is known about the system beforehand, but can be observed at least partially in a sequential manner, such as assigning assets for surveillance of remote geographical regions for illicit activity. The motivating setting of this work is the operation of a bike-sharing system with fixed capacity stations, where an initial number of bikes must be set each day to minimize unsatisfied customers., Outstanding Thesis, Teniente Primero, Chilean Navy, Approved for public release. Distribution is unlimited.

The process of creating an intelligence status report requires a continuous collection of intelligence from various sources of varying inaccuracies and reliability. Consequently, managing many intelligence sources is not only a costly operation to establish but also to maintain continuously. Our objective is to use the Multi-armed Bandit (MAB) framework to model intelligence collection. The proposed framework generalizes the classical MAB model by accounting for censoring in sampled observations in a resource-constrained environment. We devise an online optimization framework, accompanied by rigorous analysis and comprehensive numerical experiments, that sheds light on this real-world problem., http://archive.org/details/anonlinelearning1094568305, Seren, Israel Defence Forces, Approved for public release. Distribution is unlimited.

The current best practices for minimizing the effects of COVID-19 are social distancing, the use of masks, the early detection of the virus among the population, and rapid quarantining of those who are infected. Early detection and rapid quarantining rely on testing, which is vital to the containment of the pandemic but also demands a large logistical effort in terms of testing sites, materials, labs, and staffing. This project develops tools to dynamically allocate these testing resources based on a model of the spread of COVID-19 on a Navy ship. Navy ships are highly compartmentalized environments and necessitate new models that relax the homogeneous-mixing assumption common to most epidemic models. We create a simulation model of the virus spread in the non-homogenous population, where people are spatially clustered into interconnected groups such as divisions or berthing on a Navy ship. The goal of this project is to develop testing protocols that minimize the impact of the virus to the ship’s operational capabilities. To do so, we create a simulation-optimization model, which focuses on detection, not mitigation and response., Ensign, United States Navy, Approved for public release. Distribution is unlimited.

This thesis explored the potential readiness benefits that could be gained from centralizing the Marine Corps’ KC-130J isochronal (ISO) maintenance special inspections. ISO inspections are scheduled for aircraft based solely on elapsed calendar time, and organic squadron maintenance personnel conduct the inspection in-house. Increases in squadron workloads combined with limited personnel have resulted in these ISO inspections taking considerably longer than expected to complete, which directly impacts aircraft readiness. By consolidating the maintenance effort at a central location, contract workers can be employed to handle the ISO inspections for the entire KC-130J fleet. The Air Force has employed a similar system for its special operations C-130 fleet, and the inspection timeline has improved dramatically. The inspection process was modeled via a stochastic discrete event simulation written in Python. The model provided an estimate on the overall improvement in readiness levels across several potential scenarios, and the minimum required turnaround time based on central facility capacity can be determined., Outstanding Thesis, Captain, United States Marine Corps, Approved for public release. Distribution is unlimited.

We consider a version of the continuum armed bandit where an action induces a filtered realisation of a non-homogeneous Poisson process. Point data in the filtered sample are then revealed to the decision-maker, whose reward is the total number of revealed points. Using knowledge of the function governing the filtering, but without knowledge of the Poisson intensity function, the decision-maker seeks to maximise the expected number of revealed points over T rounds. We propose an upper confidence bound algorithm for this problem utilising data-adaptive discretisation of the action space. This approach enjoys \tilde{O}(T^(2/3)) regret under a Lipschitz assumption on the reward function. We provide lower bounds on the regret of any algorithm for the problem, via new lower bounds for related finite-armed bandits, and show that the orders of the upper and lower bounds match up to a logarithmic factor.

Active shooter incidents in the United States are growing in frequency and scale. This increase has gained widespread attention, with mounting calls to authorities and policymakers to reduce the number and impact of such events. Unfortunately, studying active shooter incidents is particularly difficult. Unlike other social phenomena, the window for observation is brief, and realistic human experiments are practically unobtainable. The last decade has seen a flurry of computer simulation models attempt to fill this gap; however, without an abundance of data or corresponding analytic models for verification, their results are precarious. This thesis presents a series of analytic models that capture the central dynamics of active shooter incidents, allowing researchers to gain insight on which factors most affect the outcomes of these events. The models have potential to inform policy, enable analysis for decision-makers, and influence emergency response plans in order to ultimately save lives., Outstanding Thesis, Captain, United States Air Force, Approved for public release. distribution is unlimited

This research develops a principled method to efficiently collect intelligence from strategic targets in the absence of prior information. The resulting algorithms can be employed to geo-locate and characterize targets who attempt to avoid detection while crossing a perimeter. Our model considers the following scenario: agents operate in separate sectors of a perimeter, and a searcher attempts to interdict their activities. In every time period, each agent receives a demand of activities to perform; this demand is random, with a distribution that is known to the agent but not to the searcher. Each agent decides how many of the requested activities to perform, knowing that if the searcher looks into the agent’s sector a penalty proportional to the number of activities is incurred. From the searcher’s perspective, the objective is to detect as many malevolent activities as possible. To decide which sector to search, the searcher applies an adversarial bandit algorithm (like EXP3), which samples from a distribution that assigns more weight to the most lucrative sector so far. Knowing this, the agents balance their goal of performing activities versus the risk of getting caught. This narrative applies to perimeter protection settings more generally. The ultimate goal of the research is to design algorithms that optimally balance the exploration and exploitation of the data in the presence of strategic agents., http://archive.org/details/anadversarialban1094564842, Outstanding Thesis, Lieutenant Commander, Republic of Korea Navy, Approved for public release; distribution is unlimited.

We consider Monte Carlo simulation in a setting where the samples are subject to random censoring. Such censoring occurs in settings as varied and diverse as perimeter protection, survival analysis, and electro-magnetic spectrum monitoring. We introduce and analyze two estimators: one based on empirical likelihood methods and another rooted in control variates ideas. We show that the proposed estimators can dramatically reduce the estimator variance in relation to the crude Monte Carlo estimator while not sacrificing computational speed.

Surface Warfare Officers (SWOs) currently attend Basic Division Officer Courses (BDOC) and Junior Officer of the Deck (JOOD) courses prior to their first Division Officer (DIVO) tour. Due to the commissioning schedule, the schedule of these different courses and the number of seats available, there is typically a large wait time for SWOs during different times of the year between classes and before they enter the fleet. These wait times are costly and lower the military readiness of the fleet. With a growing Naval force, minimizing the friction that SWOs experience can produce more Department Heads in order to man the larger fleet by training more Junior Officers today. Current analysis is done in Excel and calculations must be repeated for different scenarios. These scenarios include different arrival rates and distributions, service rates, and population sizes. Using Simio and Python, simulation models of a queuing network can be created in order to find different queue lengths and waiting times automatically using historic commissioning rates and schedules. Once a proof-of-concept model has been created, the model inputs can be changed to match predicted inputs for a growing Naval force. Then allocations of training schedules and capacities can be recommended in order to optimize the SWO training pipeline.

Most existing solutions for protecting VMs assume known attack patterns or signatures and focus on detecting malicious manipulations of system files and kernel level memory structures. In this research we develop a system called ferify, which leverages VM introspection (VMI) to protect user files hosted on a VM against unauthorized access even after an attacker has managed to obtain root privileges on the VM. ferify maintains in the hypervisor domain a shadow file access control list (SACL) that is totally transparent to the VM. It uses the SACL to perform independent access control on all system calls that may operate on the target files. Further, ferify prevents kernel modification, ensures the integrity of process ownership, and supports hypervisor based user authentication. We have developed a ferify prototype for Linux and through a set of controlled experiments we show that the system is able to mitigate a range of zero-day attacks that otherwise may evade signature-based solutions. In addition, we analyze the root cause of the observed high processing overhead from trapping of system calls, and propose a general solution that can potentially cut that overhead by half.

In numerous settings in areas as diverse as security, ecology, astronomy, and logistics, it is desirable to optimally deploy a limited resource to observe events, which may be modelled as point data arising according to a Non-homogeneous Poisson process. Increasingly, thanks to developments in mobile and adaptive technologies, it is possible to update a deployment of such resource and gather feedback on the quality of multiple actions. Such a capability presents the opportunity to learn, and with it a classic problem in operations research and machine learning - the explorationexploitation dilemma. To perform optimally, how should investigative choices which explore the value of poorly understood actions and optimising choices which choose actions known to be of a high value be balanced? Effective techniques exist to resolve this dilemma in simpler settings, but the Poisson process data brings new challenges. In this thesis, effective solution methods for the problem of sequentially deploying resource are developed, via a combination of efficient inference schemes, bespoke optimisation approaches, and advanced sequential decision-making strategies. Furthermore, extensive theoretical work provides strong guarantees on the performance of the proposed solution methods and an understanding of the challenges of this problem and more complex extensions. In particular, Upper Confidence Bound and Thompson Sampling (TS) approaches are derived for combinatorial and continuum-armed bandit versions of the problem, with accompanying analysis displaying that the regret of the approaches is of optimal order. A broader understanding of the performance of TS based on non-parametric models for smooth reward functions is developed, and new posterior contraction results for the Gaussian Cox Process, a popular Bayesian non-parametric model of point data, are derived. These results point to effective strategies for more challenging variants of the event detection problem, and more genera

We consider the problem of sequentially choosing observation regions along a line, with an aim of maximising the detection of events of interest. Such a problem may arise when monitoring the movements of endangered or migratory species, detecting crossings of a border, policing activities at sea, and in many other settings. In each case, the key operational challenge is to learn an allocation of surveillance resources which maximises successful detection of events of interest. We present a combinatorial multiarmed bandit model with Poisson rewards and a novel filtered feedback mechanism - arising from the failure to detect certain intrusions - where reward distributions are dependent on the actions selected. Our solution method is an upper confidence bound approach and we derive upper and lower bounds on its expected performance. We prove that the gap between these bounds is of constant order, and demonstrate empirically that our approach is more reliable in simulated problems than competing algorithms.

Project Summary: This executive summary describes our results on the research project entitled "Sensor Fusion for Undersea Operations." The first part deals with efficient search with incomplete information, and is motivated by the problem of searching for objects over a large area. In the undersea domain, the goal is to learn the pattern of intrusions so as to efficiently allocate search resources. The model is complicated by the fact that intruders disappear if they are not detected within some window of time. Hence, the more cells the searcher opts to search, the less effective his search can be in any one area. The key operational question we wish to answer is: How should the perimeter be searched in order to maximize the expected number of events intruders over a finite time horizon? The second part of this project is motivated by the difficulty of wireless sensor networks (WSNs) to provide coverage under water. Due to difficulties in transitioning from WSNs to underwater sensor networks (UWSNs), it is critical to develop fast and accurate communication methods with sensors on the harsh environment of the oceans. In this work, we provide a comprehensive framework to study various approximation techniques in solving discrete-domain optimal sensor placement problems. We consider two general sensor placement problems and adapt some of the most commonly used approximation techniques in solving them.