Background
The ICT VIRTUAL HUMAN TOOLKIT is a collection of modules, tools, and libraries designed to aid and support researchers and developers with the creation of synthetic yet believable and engaging characters. It forms the basis of all ICT’s research into embodied conversational AIs since its inception in 2009 and can be seen in projects including: ELITE SHARP trainers and VITA.
Objectives
The core technology emphasizes natural language interaction, nonverbal behavior, and perception and is broken
up into the following main modules:
- AcquireSpeech: Sends audio/text to speech recognizers and relays information to the entire system.
- MultiSense: Perception framework that enables multiple sensing and understanding modules to interoperate simultaneously.
- Non-Player Character Editor (NPCEditor): Suite of tools which work together to create appropriate dialogue responses to users’ inputs.
- NonVerbal Behavior Generator (NVBG): Rule-based behavior planner that infers communicative functions from the surface text and selects appropriate behaviors to augment/complement characters’ dialogue.
- SmartBody: Modular, controller-based character animation system using Behavior Markup Language (BML).
- vhtoolkitUnity: Renderer with custom authoring and debug tools based upon the Unity game engine.
Results
The Virtual Human Toolkit has been downloaded 6,000+ times and is used at Northeastern University, Carnegie Mellon University (CMU), Rensselaer Polytechnic Institute, University of Twente, and Princeton.
Next Steps
ICT has made the VIRTUAL HUMAN TOOLKIT available to download at: https://vhtoolkit.ict.usc.edu. It is free for Academics and the Military, with a commercial license available as a fully stand-alone Windows release. VHToolkit 2.0 is currently in development: more modular, multi-platform support (e.g., AR, VR, web, mobile), with cloud services support.
Published academic research papers are available here. For more information Contact Us
Learn more on the Virtual Human Toolkit website.