Search

In this paper, we adapt computational design approaches, widely used by the engineering design community, to address the unique challenges associated with mission design using RTS games. Specifically, we present a modeling approach that combines experimental design techniques, meta-modeling using convolutional neural networks (CNNs), uncertainty quantification, and explainable AI (XAI). We illustrate the approach using an open-source real-time strategy (RTS) game called microRTS. The modeling approach consists of microRTS player agents (bots), design of experiments that arranges games between identical agents with asymmetric initial conditions, and an AI infused layer comprising CNNs, XAI, and uncertainty analysis through Monte Carlo Dropout Network analysis that allows analysis of game balance. A sample balanced game and corresponding predictions and SHapley Additive exPlanations (SHAP) are presented in this study. Three additional perturbations were introduced to this balanced gameplay and the observations about important features of the game using SHAP are presented. Our results show that this analysis can successfully predict probability of win for self-play microRTS games, as well as capture uncertainty in predictions that can be used to guide additional data collection to improve the model, or refine the game balance measure.

Test and evaluation is a necessary process for ensuring that engineered systems perform as intended under a variety of conditions, both expected and unexpected. In this work, we consider the unique challenges of developing a unifying test and evaluation framework for complex ensembles of cyber-physical systems with embedded artificial intelligence. We propose a framework that incorporates test and evaluation throughout not only the development life cycle, but continues into operation as the system learns and adapts in a noisy, changing, and contended environment. The framework accounts for the challenges of testing the integration of diverse systems at various hierarchical scales of composition while respecting that testing time and resources are limited. A generic use case is provided for illustrative purposes and research directions emerging as a result of exploring the use case via the framework are suggested.

Counterfeiting in metal part manufacturing has become a major global concern. Although significant effort has been made in detecting the implementation of such counterfeits, modern approaches suffer from high expense during production, invasiveness during manufacture, and unreliability in practice if parts are damaged during use. In this paper, a practical microstructure-based counterfeit detection methodology is proposed, which draws on inherent randomness present in the microstructure as a result of the manufacturing process. An optical Physically Unclonable Function (PUF) protocol is developed which takes a micrograph as input and outputs a compact, unique string representation of the micrograph. The uniqueness of the outputs and their robustness to moderate wear and tear is demonstrated by application of the methodology to brass samples. The protocol is shown to have good discriminatory power even between samples manufactured in the same batch, and runs on the order of several seconds per part on inexpensive machines.

We propose a method for ensuring traceability of metal goods in an efficient and secure manner that leverages data obtained from micrographs of a part’s surface that is instance specific (i.e., different for another instance of that same part). All stakeholders in modern supply chains face a growing need to ensure quality and trust in the goods they produce. Complex supply chains open many opportunities for counterfeiters, saboteurs, or other attackers to infiltrate supply networks, and existing methods for preventing such attacks can be costly, invasive, and ineffective. The proposed method extracts discriminatory-yet-robust intrinsic strings using features extracted from the two-point autocorrelation data of surface microstructures, as well as from local volume fraction data. By using a synthetic dataset of three-phase micrographs similar to those obtained from metal alloy systems using low-cost optical microscopy techniques, we discuss tailoring the method with respect to cost and security and discuss the performance of the method in the context of anticounterfeiting and how similar methods may be evaluated for performance. Cryptographic extensions of this methodology are also discussed.

In the context of globalized supply chains, counterfeiting of manufactured goods is a growing problem. The financial, legal, and reputational costs that counterfeit goods impose on legitimate enterprises have spurred investigation into efficient and robust anti-counterfeiting methodologies. In particular, physically unclonable functions (PUFs) have been applied effectively in several manufacturing areas, especially electronics. However, anti-counterfeiting solutions for generic manufactured goods are often expensive to make and implement, or not robust to minor damage that the goods may sustain during transport and use. In this paper, a framework for developing robust, efficient, and cost-effective optical PUFs for anti-counterfeiting of manufactured metallic goods is proposed, along with an example implementation for 4140-steel parts according to standard ASTM A29. For an input library of 50 steel micrographs, the proposed example PUF is shown to have good robustness to simulated part damage and an estimated classification error rate of less than 1%.