The ICT VIRTUAL HUMAN TOOLKIT is a collection of modules, tools, and libraries designed to aid and support researchers and developers with the creation of synthetic yet believable and engaging characters. It forms the basis of all ICT’s research into embodied conversational AIs since its inception in 2009 and can be seen in projects including: ELITE SHARP trainers and VITA.
The core technology emphasizes natural language interaction, nonverbal behavior, and perception and is broken
up into the following main modules:
- AcquireSpeech: Sends audio/text to speech recognizers and relays information to the entire system.
- MultiSense: Perception framework that enables multiple sensing and understanding modules to interoperate simultaneously.
- Non-Player Character Editor (NPCEditor): Suite of tools which work together to create appropriate dialogue responses to users’ inputs.
- NonVerbal Behavior Generator (NVBG): Rule-based behavior planner that infers communicative functions from the surface text and selects appropriate behaviors to augment/complement characters’ dialogue.
- SmartBody: Modular, controller-based character animation system using Behavior Markup Language (BML).
- vhtoolkitUnity: Renderer with custom authoring and debug tools based upon the Unity game engine.
The Virtual Human Toolkit has been downloaded 6,000+ times and is used at Northeastern University, Carnegie Mellon University (CMU), Rensselaer Polytechnic Institute, University of Twente, and Princeton.
ICT has made the VIRTUAL HUMAN TOOLKIT available to download at: https://vhtoolkit.ict.usc.edu. It is free for Academics and the Military, with a commercial license available as a fully stand-alone Windows release. VHToolkit 2.0 is currently in development: more modular, multi-platform support (e.g., AR, VR, web, mobile), with cloud services support.
Learn more on the Virtual Human Toolkit website.