Background
Synthetic entities in current military simulations (e.g., OneSAF) have limited behaviors and are usually generated by rule-based/reactive computational models, which have minimal intelligence and are unable to adapt based on experience. Effective synthetic entities need human-inspired adaptive cognitive models that communicate, perceive their environment and others in it, draw conclusions, reason, and choose appropriate actions. Experiential learning is essential for developing such synthetic characters with credible behavior because attempting to achieve realistic and convincing behavior solely through strict behavioral control would be highly impractical.
Objectives
Build (proof-of-concept) autonomous synthetic characters or Non-Player Characters (NPCs) who are aware of human trainees’ needs for realistic and challenging learning experiences in military training simulations. Our work augments Multi-agent Reinforcement Learning (MARL) models with GNNs (Graph Neural Networks), drawing inspiration from operations research, human judgment and decision-making, game theory, graph theory, and cognitive architectures to better address the challenges.
Results
We have successfully created proof-of-concept behavior models for various military training scenarios leveraging our enhanced MARL framework. We utilize geo-specific terrains in Unity and abstractions derived from these terrains as simulation environments. A general overview paper of our group’s work can be found here. Our sample simulation environments and enabling libraries for running MARL experiments can be found at our group’s public repository: https://github.com/HATS-ICT
Next Steps
We continue working towards creating more realistic and challenging training experiences while reducing the cost and time required for their development. Our current research focuses on incorporating domain knowledge, such as tactics, military hierarchies, etc., to enhance and speed up our MARL experiments.
Published academic research papers are available here. For more information Contact Us