STORY AND VIDEO: Since its founding in 1999, the institute has produced cutting-edge computing research that has been used by Hollywood, academia, tech and the military.

September 12, 2024

By Will Kwong

Virtual humans speaking with museumgoers about their experiences surviving the Holocaust. A phone app giving soldiers suffering from post-traumatic stress disorder a mental health specialist in their pocket. Computer-generated actors populating many top Hollywood films and bestselling video games.

The USC Institute for Creative Technologies has been at the forefront of augmented and virtual reality technology such as these since its founding in 1999. The institute conducts cutting-edge research in key computing areas such as artificial intelligence, computer graphics, geospatial sciences and many others.

These technological breakthroughs, positioned near Hollywood and the Silicon Beach tech corridor in west Los Angeles, have earned ICT numerous plaudits, including two Academy Awards, 29 patents and hundreds of thousands of citations. Three ICT researchers have been named fellows of the Association for the Advancement of Artificial Intelligence, the world’s foremost AI research organization.

These accomplishments and more were on display at the ICT 25th anniversary event held Aug. 19 at the center’s headquarters in Playa Vista within Silicon Beach.

“ICT is truly one of the most wonderful beacons for exploration and innovation,” USC President Carol Folt said at the event. She noted the institute’s history of trailblazing work and its ability to evolve and expand its research areas, including in virtual reality, augmented reality, graphics rendering and artificial intelligence.

Innovative history

Demonstrations in the institute’s core areas of research drew attendees, who were able to engage with numerous technologies developed at ICT during its history. Of special interest was research in natural language processing, computer vision and machine learning, which culminated in the development of virtual humans capable of engaging in realistic conversations.

At the event, Folt noted that “ICT is a front line for USC in our new frontiers of computing,” referring to the USC Frontiers of Computing “moonshot,” a $1 billion-plus, 10-year initiative to fast-track developments in advanced computing and expand the impact of scientific discovery in Los Angeles County’s growing tech corridor that encompasses USC Silicon Beach.

Human-centric technology

From its beginnings, ICT has developed technology-driven solutions to real-world applications for all walks of life. “ICT came into existence because of the convergence of outstanding technology, creativity and commitment to national security and our enduring human values at USC,” said Yannis C. Yortsos, dean of the USC Viterbi School of Engineering, which is home of ICT, at the event. Bill Swartout, chief science officer of ICT, noted ICT’s tremendous growth since its inception.

As one of only a handful of university affiliated research centers sponsored by the U.S. Department of Defense in the country, ICT-developed technology spans a wide array of everyday applications as well. Louis Caldera, the former U.S. secretary of the Army who spoke at the event, said ICT’s standing as a university affiliated research center demonstrates how its creative and technological experts have a deep understanding of which kind of technologies to bring to the Army.

ICT’s proximity to Hollywood and tech startups in L.A. also positioned it to play a role in advancing computer technology in the entertainment industry. “ICT has leveraged the best Hollywood immersive techniques and the best creative processes to envision and integrate the future,” ICT Executive Director Randall Hill said at the event. Graphic-rendering innovations from ICT have led to creating lifelike computer-generated characters in films such as Avatar and other Hollywood blockbusters.

One example that blends ICT’s research areas is the BRAVEMIND project, led by Skip Rizzo, Arno Hartholt and Sharon Mozgai. A virtual reality therapy system, this project has produced meaningful reductions in PTSD symptoms in clinical trials and has been adopted by Veteran Affairs medical centers and other sites that serve the needs of veterans.

“What we do here at ICT is look at what technologies are out there, and how can we apply those technologies to challenges in society,” said Rizzo, whose work in virtual reality-based PTSD exposure therapy received the American Psychological Association’s 2010 Award for Outstanding Contributions to the Treatment of Trauma. Mozgai, the director of the Virtual Human Therapeutics Lab at ICT, said, “While we are excited about the recent advances in AI, a core value of our work is to responsibly integrate this technology into our applications.”

Bridging the past and future

Other work includes the New Dimensions in Testimony project, a collaboration with USC Shoah Foundation — The Institute for Visual History and Education that gives visitors the opportunity to engage with an AI-powered video display of Holocaust survivor Pinchas Gutter. The specialized video display utilized advanced filming techniques developed using ICT’s Light Stage technology.

“Conversations are a powerful learning tool,” said Ron Artstein, a research scientist who demonstrated the interactive technology to attendees, engaging in a real-time virtual conversation with Gutter that made an emotional impact on many attendees. “When you talk to a person, you build an emotional connection. New Dimensions in Testimony lets people talk with historical witnesses in an immersive way.”

Researchers from the Visions & Graphics Lab demoed Text to Avatar, which lab director Yajie Zhao describes as “the front line of human-avatar technology.” Pairing sophisticated facial geometry and texture data in a digital environment, the project will utilize a deep learning model to “create numerous avatars that could fill entire environments,” Zhao said.

Developing computer-generated imagery for Hollywood is a central theme in ICT’s history: Visions & Graphics Lab researchers earned an Academy Award in 2010 for Best Achievement in Visual Effects, and the institute received a second award in 2018 for technical achievement. The applications stretch to other media as well. The hit video game The Last of Us also incorporated work developed through the institute’s motion-capture technology.

Creating the future

The two-day event and reception brought together key figures from the institute’s history to reflect on the progress made and to look to the future of ICT. Yortsos said the institute’s ability to bring together disparate industries including entertainment, academia, tech and the military has helped improved the lives of people worldwide.

Looking to the future, Hill said the institute would continue to bring together USC researchers and experts in immersive technologies to create the next generation of Department of Defense training tools and simulations: “The future is looking at how humans and machines can team together in all kinds of capacities.”