1. Test and Evaluation Framework for Multi-Agent Systems of Autonomous Intelligent Agents
- Author
-
Ivan Hernandez, Scott Welch, Laura J. Freeman, Jitesh H. Panchal, Adam Dachowicz, Melanie L. Grande, Andrew B. Lang, Erin Lanus, and Anthony Patrick
- Subjects
FOS: Computer and information sciences ,Computer Science - Artificial Intelligence ,Process (engineering) ,Computer science ,Multi-agent system ,Statistical model ,Combinatorial interaction testing ,Systems and Control (eess.SY) ,computer.software_genre ,Electrical Engineering and Systems Science - Systems and Control ,Variety (cybernetics) ,Test (assessment) ,Software development process ,Software Engineering (cs.SE) ,Computer Science - Software Engineering ,Intelligent agent ,Artificial Intelligence (cs.AI) ,Systems engineering ,FOS: Electrical engineering, electronic engineering, information engineering ,computer - Abstract
Test and evaluation is a necessary process for ensuring that engineered systems perform as intended under a variety of conditions, both expected and unexpected. In this work, we consider the unique challenges of developing a unifying test and evaluation framework for complex ensembles of cyber-physical systems with embedded artificial intelligence. We propose a framework that incorporates test and evaluation throughout not only the development life cycle, but continues into operation as the system learns and adapts in a noisy, changing, and contended environment. The framework accounts for the challenges of testing the integration of diverse systems at various hierarchical scales of composition while respecting that testing time and resources are limited. A generic use case is provided for illustrative purposes and research directions emerging as a result of exploring the use case via the framework are suggested.
- Published
- 2021
- Full Text
- View/download PDF