Simulations, Architectures &
Intelligent Agents Lab (SAIA)
Research Lead: Arno Hartholt
The SAIA Lab hardens research software, integrates academic and industry technologies into shared R&D platforms, and designs, develops, and transitions simulation-based intelligent systems.
The lab maintains the Rapid Integration & Development Environment (RIDE) and the Virtual Human Toolkit. These open integration platforms support rapid prototyping and evaluation of systems ranging from embodied conversational agents for training, education, and health to real-time military simulations on 3D geospatial terrain. Deployment targets include desktop, web, mobile, AR and VR, with integrated support for local and cloud AI services.
Projects
Virtual Human Toolkit
A modular R&D platform for creating real-time embodied conversational agents – virtual humans (VH) – that perceive and interact with users both verbally and nonverbally. Freely available for non-commercial use. Collaboration with all ICT VH labs.
RIDE
A rapid prototyping testbed unifying military simulation technologies to accelerate integration and experimentation for training and simulation systems, including human-machine teaming, drones, and autonomous vehicles. Freely available under Government Purpose Rights (GPR). Collaboration with all ICT labs.
Personalized Avatars
A research initiative developing high-fidelity digital double avatars that reflect individual soldiers’ attributes to study realism, engagement, and effectiveness in military simulation and training environments. Collaboration with ICT VHTL.
Previous Work and Collaborations
Modeling & Simulation
Design and development of agent-based models, simulation environments, and experimental platforms for studying and evaluating complex, integrated systems.
Hybrid Computing
Leveraging RIDE to stand up and evaluate an on-prem GeForce Now streaming server to support scalable, low-latency, multi-user training and experimentation. Collaboration with NVIDIA and MITRE, under DEVCOM Soldier Center STTC.
SAFARI
Multi-Domain Operations Electronic Warfare prototype that used RIDE to integrate Navy radar models with industry real-time visualization, developed by Unity, led by the Air Force Agency for Modeling and Simulation (AFAMS).
Generative Programming
DEVCOM Soldier Center STTC project to enable simulation-agnostic model reuse by representing behaviors and logic in canonical forms that can be automatically translated into executable models across multiple simulation environments, including RIDE.
TSS SimReady Behaviors
Developing agent behavior frameworks and content for the Synthetic Training Environment (STE) Training Simulation Software (TSS), including behavior trees and reinforcement learning. Collaboration with Cole Engineering Services, Inc. (CESI), and Dynamic Animation Systems (DAS), under DEVCOM Soldier Center STTC.
USMA Camouflage Study
Creation of visual stimuli in support of studying the effectiveness of various camouflage patterns. Collaboration with the U.S. Military Academy West Point, directed by the Office of the Secretary of Defense (OSD).
Wargaming Coalition
Exploring wargaming simulation and course-of-action (COA) analysis through WW2 scenarios in RIDE.
Human Dimension Avatar
2010 internal prototype with high-fidelity, real-time digital doubles of real people. Collaboration with ICT VGL.
RIDE Rapid Avatar
2021 internal prototype to showcase avatars captured in 10 minutes on 3D geospatial terrain. Collaboration with ICT VGL and Geospatial.
VH Fidelity Research
2021 internal prototype to showcase avatars captured in 10 minutes on 3D geospatial terrain. Collaboration with ICT VGL and Geospatial.
Training
Virtual humans and simulation-based systems that support operational readiness, skill acquisition, and performance assessment.
Immersive Training Environment
Life-sized conversational and kinetic virtual human prototype developed for USMC Camp Pendleton to train Marines. Developed in collaboration with Dignitas Technologies.
De-Escalation of Violence for Police
Virtual Reality prototype to train agents in de-escalation techniques. Developed in collaboration with RAND and the Los Angeles Police Department (LAPD). Collaboration with ICT MedVR.
VITA Magic Leap
Early VITA job interviewing AR prototype, developed with Magic Leap on their pre-release hardware, in collaboration with the Dan Marino Foundation. Collaboration with ICT MedVR.
Elect BiLat
A portable-PC based training program designed to provide students an immersive and compelling training environment to practice their skills in conducting meetings and negotiations in a specific cultural context.
CHAOS
The Combat Hunter Action and Observation Simulation (CHAOS) was part of the Future Immersive Training Environment (FITE) at USMA Camp Pendleton. The system consisted of virtual humans in a small town compound with which Marines interacted through a human translator. Collaboration with ICT Dialogue.
Social Wayfinding
Virtual reality testbed to support research into training navigation tasks that include virtual humans. Collaboration with ARL West.
STAT
Laptop training capability to help K-12 school administrators and psychologists practice conducting threat assessment interviews with students. Based on ELITE.
Miles
The Motivational Interviewing Learning Environment and Simulation system supports training social work counselors. Based on ELITE. Developed with ICT MedVR, MxR, and the USC School of Social Work.
Cicero
Virtual, configurable audience to teach and research public speaking. Collaboration with ICT MultiComp Lab.
Conflict Resolution Agent
Negotiation testbed in support of negotiation and conflict resolution research. Collaboration with ICT Affective Computing.
Mission Rehearsal Exercise (MRE)
The original USC ICT VH system, a pioneering, award-winning, immersive simulation that combined AI and interactive storytelling to train soldiers in decision-making and leadership in a simulated, high-pressure conflict scenario. Collaboration with all ICT VH labs.
Stability and Support Operations (SASO)
The follow-up to MRE: an advanced system of reasoning agents who exhibit emotions and basic reasoning skills, based on the SOAR cognitive architecture. Collaboration with all ICT VH labs.
Health
Virtual humans, interactive simulations, and AI-driven systems for healthcare training, behavioral health, information dissemination, assessment, and clinical support.
SimSensei
A basic research clinical decision support tool and interactive virtual agent-based healthcare dissemination system that is able to recognize and identify psychological distress from multimodal signals. Collaboration with ICT MedVR and all VH labs.
SimCoach
Web-based virtual humans designed to assist service members in breaking down barriers to care. Collaboration with ICT MedVR and all VH labs.
DocOn
Proof of concept mobile app with a virtual doctor avatar providing patient information. Collaboration with the USC Center for Body Computing.
Standard Patient Studio
A free online community where medical students, residents and physicians can improve their interview and diagnostic skills with a Virtual Standardized Patient. Collaboration with ICT MedVR.
RAM Replay
VR testbed in support of a DARPA-funded project aimed at improving memory and recall. Collaboration with Hughes Research Laboratories. Collaboration with ICT MedVR.
mHealth Coach
Proof of concept mobile app aimed at coaching young adult cancer survivors. Collaboration with the USC mHealth Collaboratory.
Education
Intelligent tutoring systems, learning simulations, and conversational agents that enhance instruction and learner engagement.
Canvas
Application developed for the Smithsonian Institute to semi-automatically generate and author educational content for classrooms.
Project Syria
Immersive journalism VR application that enables audiences to experience the plight of Syrian children at the civil war, led and designed by Nonny de la Peña.
Coach Mike
A virtual coach designed to be an enthusiastic proponent for STEM education as he helped museumgoers learn basic programming skills with a robot. Collaboration with ICT Learning Sciences.
PAL3
A Navy meta-tutor drone that accompanies a student or professional throughout their career, providing advice and recommendations along the way. Collaboration with ICT Learning Sciences.
Workforce Transformation
AI-enabled tools and simulation platforms that support workforce development, reskilling, and organizational adaptation.
Telecommuting Autonomous Personal Assistant (TAPA)
A prototype of an autonomous, personalized agent to represent a remote government worker in support of knowledge management advanced telecommuting technologies evaluation. Performed for the Defense Information Systems Agency (DISA).
Kingsman Telepresence
Evaluation of immersive academic and industry telepresence technologies in support of Army planning and acquisition, for DEVCOM Soldier Center STTC. Collaboration with ICT MxR.
BlueShark
Leveraged AR, VR, VH, and AI to create a vision for what future Navy workspaces and collaboration will look like. Collaboration with ICT MxR.
PRIMER
A VR prototype designed to show the potential of immersive dialogue agents and virtual environments that adapt and respond to both direct verbal input and indirect emotional input. Collaboration with ICT Dialogue.
Entertainment
Interactive experiences with virtual humans for games, theme parks, and museums.
Gunslinger
Combines VH technology with Hollywood storytelling and stagecraft, creating a mixed-reality experience where a single participant interacts with multiple virtual characters set in the Wild West. Collaboration with all ICT VH labs.
Monticello
Internal project to showcase integrated, multi-platform virtual humans for 2016 hardware, including mobile, web, desktop, AR (both headset and mobile), and VR, featuring a virtual Thomas Jefferson.