Written by Dava Casoni, Executive Producer, Force Writers Room and Kathleen Haase, Producer/Special Projects, VGL
The Department of Defense (DoD), the Entertainment Industry and Academia – an unlikely alliance born out of a 1996 workshop held by the National Research Council (at the request of the DoD) – explored whether Hollywood and game industry skills, particularly in modeling and simulation, could be used to improve military training.
That pivotal workshop attracted top Hollywood talent, including Richard Lindheim, EVP, Paramount Television Group, Ed Catmull, Co-Founder, Pixar; Alex Singer, Director, Star Trek: Voyager, and Jacki Morie, Head of Computer Technical and Artistic Training, Walt Disney Feature Animation.
Curiosity was piqued, relationships developed, and three short years later, the Institute of Creative Technologies (“ICT”) emerged in 1999, led by Richard Lindheim, its first Executive Director.
25 Years Later
Twenty-five years, and countless disruptive technologies and innovative lines of research later, ICT’s unique blend of seemingly antithetical partners – the DoD, entertainment industry and academia – has proved to have surprising commonality, in terms of operational tempos, secrecy, ingenuity, strategy, and deeply committed professionals playing at the top of their fields.
From our very early projects, Hollywood creatives, including Ron Cobb, production designer for Star Wars, Aliens, Total Recall and Back to the Future, Joseph Zito, Director, Delta Force One; Steven E. De Souza, screenwriter, Die Hard; David Fincher, Director, Fight Club; Spike Jonze, Writer/Director, Her; Mary Lambert, Director, The In Crowd; and David Ayer, Navy veteran and director of Suicide Squad (featuring Margot Robbie, long before she assumed the iconic Barbie role) ideated on the future of weapons, vehicles, equipment, uniforms, an “integrated [human/machine combat] system [to] monitor a soldier’s vital signs, the outside environment” temperature and combat systems, and, in the wake of 9/11, envisioned plausible terrorist scenarios to aid in defensive planning.”
Many of our current staffers have extensive Hollywood credentials, including our Director of Finance, Operations and HR, Cheryl Birch (Vice President, Business Affairs, Paramount Pictures); David Nelson, Director, MxR Lab (award-winning Producer and Director for 20th Century Fox, Disney, HBO and Sony Pictures Entertainment, winner of the Hugo Award for Excellence in Television Documentary), Kathleen Haase, Producer/Special Projects at our Vision and Graphics Lab, who holds producing and VFX credits on many major motion pictures including Detroit Rock City, Logan, Underworld: Blood Wars and Maleficent, and Dava Casoni (Association of Media and Entertainment Counsel, 2024 Public Counsel of the Year).
And, it works both ways – just as the DoD has benefitted from our entertainment industry partners, our “alumni” have gone on to be showrunners on major TV shows, published authors, and led innovative leaps at tech giants, including Microsoft, Google, Meta, Unreal, Nvidia, and Netflix. And our technologies, like those created by our Vision and Graphics Lab for visual and lighting effects, have set new industry standards.
Vision and Graphics Lab
Techniques developed in the Vision and Graphics Lab (“VGL”) have had a profound impact on the quality and realism of computer-generated imagery (CGI) as they enabled capturing and reproducing realistic lighting effects, which are crucial for creating immersive virtual environments. The VGL was awarded ACM’s “Significant New Research Award” in 2001 in the field of Image-based modeling and rendering and has been leading the field ever since. The VGL’s innovations in high dynamic range (HDR) imaging and image-based lighting (IBL) are now commonly used in computer graphics and computer vision everywhere.
Its Light Stage (‘LS”) technology pioneered high-resolution facial capture, generating sub-micron, pore-level detail and reflectance maps – techniques which have been broadly adopted in Hollywood over the last 25 years. The lab’s contributions are credited in almost fifty Hollywood productions, including Avatar, Suicide Squad, Blade Runner 2049, Furious 7, Space Jam and Ready Player One, among many others. These advancements and others have earned VGL two Academy Awards (Science & Technology) and VGL Founder and technology inventor, Paul Debevec, a 2022 Lifetime Achievement Emmy (Engineering, Science and Technology).
The VGL, co-led by ICT’s Kathleen Haase, also enjoys a collegial relationship with many industry partners and research groups such as Meta, Max Planck Institute, FlawlessAI, Sony Research, Nvidia, Digital Domain and WETA Digital to name a few.
Future R&D
Our VGL illumination and capture technologies allow for seamless VFX in every kind of digital production and have transported audiences to new worlds beyond imagination. We are building on this work taking it to new levels with ICT’s Rapid Face Asset Acquisition (ReFA), generating a realistic, personalized rigged facial model with multiview one-shot. And, as the race for greater immersion into the meta/omniverse mushrooms, the lab has been incorporating the latest AI-based techniques to generate all kinds of 3D assets from sparse input. For example, using just a single 2D selfie, our technique can generate a realistic 3D model of that person!
Further, VGL is introducing a novel 3D face generative model that allows for semantic control and creates high-quality face models by merely typing in a description of a target person. We are able to generate these models, in large part, due to our specifically designed normalization network that is able to utilize not only the high-quality LS scanned models, but also in-the-wild face models that are vast in quantity and semantic information.
VGL has recently expanded its know-how to improve the outcomes of training and simulation tasks that require real-life scene environments and scenarios. Our team excels in integrating geospatial research, generative AI for scenes and digital humans, along with computer vision and computer graphics. Having left an indelible mark on DoD systems and entertainment industry domain, the VGL is now turning to the future of content creation with our Neural Radiance Field (NeRF) technology that will enable the magic of editable 3D scenes with accurate depth and realism.
With NeRF combined with CG mesh, users can create, explore and manipulate immersive environments with control and creativity. As we explore all the possible applications for editable 3D scenes and our other ongoing research, spanning VFX production, efficient customization of digital avatars and the generation of synthetic training data for fundamental computer vision research, we are also unlocking limitless possibilities for digital communication, gaming, and virtual experiences with unparalleled realism and accuracy.
Force Writers Room
Likewise, the Force Writers Room (“FWR”), is enabling leaps in future-proofing Department of Defense R&D spending through its special blend of Hollywood creatives, multi-disciplinary teams, deep subject-matter expertise and “TV writers room” techniques. FWR is co-led by ICT’s Dava Casoni, who worked as an actor in Hollywood, and also served in the Army Reserves as a transportation and logistics officer, and Navy Captain Ric Arthur, TV writer (NCIS as Richard C. Arthur; The Last Ship, Hawaii Five-O), who originated and piloted the FWR process.
The FWR recently completed an 11-minute film, SeaStrike 2043, which depicts how the future U.S. Navy Fleet will fight a high intensity sea battle in 2043. SeaStrike 2043 was used as the basis of a war game among an invite-only group of the highest Naval flag officers, enabling discussion around the 33+ future technologies showcased in the film.
For SeaStrike 2043, the expanded FWR team included Jason Reed, award-winning director David Rosenbaum, Jayse Hansen, known for his future-forward design and animation of computer interfaces, holograms, HUDs, and medical simulations for blockbuster film franchises such as Iron Man, Guardians of the Galaxy, and Top Gun: Maverick, Carlton Eastlake, not to mention many former Imagineers, TV showrunners, award-winning illustrators, futurists and more.
The team is currently working on concept art for Navy Force Design 2045 to visually depict Force 2045 technology and conditions, and produced a series of graphic novels which explore emerging technology and the future impact and operation of battle. These products have drawn acclaim from the highest levels of the U.S. Navy, underscoring the relevance of the Force Writers Room’s unique approach and indispensable role in shaping strategic thought and preparation.
Redefining What’s Possible
While immersive technologies and modeling/simulation solutions are now developed the world over, the ICT remains singularly positioned with its unique combination of best-in-class scientists, engineers, and Hollywood creatives, as well as access to DoD experts and imperatives, resulting in an unparalleled perspective on advancing warfighter readiness (training, learning, mental and physical health, stress resilience) and force preparation (operational simulations, terrain and situational modeling, future force and technology visioning).
Whether strategically leveraging creativity and imagination for national security, or forging pre-eminent global technological advancements, ICT continues to push the boundaries of technology – redefining what’s possible in storytelling, visual effects, digital experiences, immersive training systems and beyond.
//
BIOS:
Dava Casoni, Executive Producer, Force Writers Room
Dava Casoni joined ICT in 2011, after a successful career as an attorney with senior roles at international law firms including Pillsbury (founded in 1868) and Milbank LLP (founded 1866), before founding her own boutique firm, Casoni Law, in 2009. She received her JD and LLM in General and International Law from Cornell Law School in 1996 and has substantial experience in negotiating and drafting complex transactions, financing, IP protection, and compliance matters, particularly in government and industry sponsored university research. Alongside her legal role at ICT, Dava Casoni also leads (with Navy Captain and TV writer Ric Arthur) the Force Writers Room, which brings multi-disciplinary expertise, Hollywood creativity and proven “TV writers room” techniques to strategically analyze and visualize the future, enabling warfighters to examine “unknowns” within a highly imaginative narrative framework.
Kathleen Haase, Producer/Special Projects, VGL
Kathleen Haase is a pivotal figure in the ongoing convergence of digital imaging within filmmaking, video game development, and military training simulations. At ICT, Haase directs a team of researchers and engineers at the Vision and Graphics Lab, overseeing technology innovations focused on generating photorealistic digital virtual humans, 2D to 3D terrain reconstruction and simulation, and working closely with military sponsors and industry leaders, as well as Hollywood directors and senior academics. In 2010, she was brought into ICT by Academy Award-winner Paul Debevec, a pioneer in digital imaging and computer graphics. Previously, Haase held positions in production with Lawrence Gordon (Die Hard, Predator, Field of Dreams) and was the Film Commissioner Specialist at Idaho Department of Commerce. Haase holds a Masters in Dispute Revolution from the University of Southern California, and a BA in Mass Communication, Media Studies, Fine Art and Business Administration from the University of Denver.