by Dr. Paul Rosenbloom, Professor Emeritus of Computer Science, USC and former Director for Cognitive Architecture Research at USC Institute for Creative Technologies
The ICT origin story does an excellent job of highlighting key players and activities that led to the call by the US Army for a University Affiliated Research Center (UARC) that would meld computing technology and entertainment expertise in transforming military simulation and training. Yet, from the perspective of USC’s proposal, it makes sense to augment this with the key people and activities that specifically enabled the proposal. As has been said in various forms from Tacitus to John F. Kennedy, although reformulated here in a gender-neutral fashion, “success has many parents.”
Four individuals who were particularly influential pre-proposal were Mike Zyda, Dick Lindheim, Alex Singer, and Martin Gunderson. Mike was a Professor at the Naval Postgraduate School (NPS) who chaired the National Research Council (NRC) study committee that identified the need for ICT. He further built on this to write a white paper that provided important background in writing the proposal. Mike would later come to USC as the founding director of its Computer Science Games Program.
Dick Lindheim was an executive vice president at Paramount Pictures and the founder of Paramount Digital Entertainment. He would later become ICT’s first executive director. Alex Singer was a director of movies and television, including a range of Star Trek episodes during the 1990s. Together they brought to the table a hypothesis concerning the importance of story and character for military simulation and training. With assistance from Martin Gundersen of USC’s School of Engineering – now the USC Viterbi School of Engineering – they recruited me to participate as a consultant to Paramount on the StoryDrive Engine, an attempt at exploring this notion while sidestepping the need for the new technologies that ICT would ultimately tackle. Dick also was a major advocate of seeking verisimilitude to enable participants in such simulations to behave as if they were real. The criticality of story, character, and verisimilitude became central to USC’s proposal.
In the words of the original proposal, “The vision for the UARC is to develop the art and technology for synthetic experiences that are so compelling participants will react as if they are real. … Their behavior will be propelled through engrossing stories stocked with engaging characters who may be either automated or manned — the high quality of the automated characters along with the provision of plug compatibility will make it impossible to distinguish. … In short, the UARC will provide a new meaning for ‘high fidelity’: verisimilitude.” And “Creating a true synthesis of art and technology and of the capabilities of the entertainment industry and the R&D community — all in service of verisimilitude — will revolutionize military training and mission rehearsal by making it more effective in terms of cost, time, the types of experiences that can be trained or rehearsed, and the quality of the result.”
Prior to the StoryDrive Engine, a second critical activity was a DARPA-funded effort that John Laird and I co-led through much of the 1990s at the University of Michigan and USC’s Information Sciences Institute (ISI), respectively, to develop intelligent automated forces (embodied as airplanes and helicopters) and commanders for large-scale, distributed military simulation. The push for humanlike artificial minds and plug compatibility in the UARC proposal was carried over from this earlier work, where the simulated entities were to be controllable seamlessly by either humans or artificial intelligence (AI).
Coming into this effort, my background was primarily in basic research on AI, with a focus on the Soar cognitive architecture – a hypotheses concerning the fixed structures and processes that enable general intelligent behavior in both natural and artificial systems. Soar was a joint effort between me and my group, John Laird and his group, Allen Newell and his group (at Carnegie Mellon University), and numerous other collaborators from around the world.
The minds of the simulated entities we built were all based on Soar. Jon Gratch and Randy Hill both played significant roles in developing these entities before going on to leadership positions at ICT and beyond. Milind Tambe was another critical member of the team, before going on to a distinguished career as a professor at USC and then Harvard, including being a pioneer in AI for social good.
The proposal effort itself began with a large open meeting of everyone at USC interested in the potential UARC. Given my background by this point in AI and military simulation, my connections with Paramount, and my active role at this time in new directions at ISI, I became the obvious choice to take the lead in writing the proposal. The overall effort, however, drew on a diverse group of key players from around the university, including from the School of Engineering (including ISI), the School of Cinema-Television (now the School of Cinematic Arts), and the Annenberg School for Communications (now the Annenberg School for Communications and Journalism).
A big part of USC’s overall credibility for the proposal came from these highly rated schools, plus a range of existing capabilities relevant to the UARC, such as the Entertainment Technology Center, the Annenberg Center for Communication, the Integrated Media Systems Center, and ISI. A further significant factor was USC’s status as a leading private research university located in Los Angeles, the hub of the Entertainment Industry and a major center for defense work, with good working relationships with both.
Herb Schorr, ISI’s Executive Director at the time, played an important role throughout the proposal process, working with USC’s leadership; and Bill Swartout, who was the director of ISI’s Intelligent Systems Division before becoming ICT’s director of technology, was very supportive as well. Our original thought was to stand up ICT as part of ISI. However, the Army ultimately wanted ICT to be independent of any one school, so it ended up reporting to the Provost’s office. This has recently changed though, with ICT now affiliated with USC’s new School of Advanced Computing, withinin the Viterbi School of Engineering, in a parallel relationship with ISI.
My own interest in the UARC was driven by a long-term goal of understanding and building humanlike minds – human minds and other natural and artificial minds that operate similarly to them – as embodied in architectures like Soar. During our work on military simulation, I also became increasingly intrigued with the idea of building virtual humans – graphically realistic simulations of human bodies – that would be controlled by such minds. Lewis Johnson, who had worked with us on aspects of the DARPA-funded effort, and Jeff Rickel did pioneering work in building a virtual human named Steve around Soar at ISI. Both Lewis and Bill Swartout were also involved in initial interactions with Paramount prior to the StoryDrive Engine, including being hidden with me in the Turbolift on the Star Trek: Voyager set to watch Alex Singer direct an episode. Jeff joined ICT early on and was central to the creation of its first virtual humans. Lewis also played a role in this before co-founding and becoming the CEO of Alelo.
Soar ended up being central to a number of efforts at ICT around virtual humans, at least partially due to the migration to ICT of so many of the researchers who had been working on it at ISI. Although I remained at ISI to continue work on new directions, a decade later I too joined ICT, at this point to develop a new cognitive architecture – Sigma – that retained what I thought was still right about Soar while generalizing the knowledge and skills it could use and shifting the kind of learning that was at its core.
The UARC proposal, more broadly, included a combination of basic and applied research, plus some educational activities. Six basic research thrusts were identified as crucial to verisimilitude: immersion, networking and databases, story, characters, setup, and direction. While many of these topics have been investigated at ICT over the years, immersion and character became the dual centerpieces. The former focused on graphics, including virtual human bodies and environments plus technologies for 3D immersion of humans into such worlds. The latter focused on virtual human minds. There is a big difference between a humanlike intelligent agent embedded within a simulated vehicle that could interact with others only through the vehicle – its sensors, mobility, weapons, and comms – versus a virtual human that was to interact face-to-face with real humans and other virtual humans via natural language, facial expression, body movement, etc.
Applied research was to focus on combining such capabilities in a way that would make a difference for military simulation and training. Such efforts were proposed to be organized around a small number of long-term themes, each involving a set of key projects and an integration architecture, with each project playing out via a sequence of prototypes of increasing functionality and level of integration. Although this is not exactly how the applied research played out, it has been extremely productive in both large and small ways, including demos, prototypes, and even deployed systems; and not only for the US Army but for a much broader range of organizations.
//
Author: Dr. Paul Rosenbloom
I am a Professor Emeritus of Computer Science at the University of Southern California.
Over most of my career, my research focused on understanding and building human-like minds, with work on cognitive architectures that were grounded in both artificial intelligence (AI) and cognitive science. More recently this research has extended to include more abstract methodologies for understanding minds, as well as comparably abstract approaches to understanding their encompassing disciplines (in particular, AI, cognitive science, and computing).
More details on this work can be found on my Bio, CV and Past Research page, on my Recent Publications page, and in two versions of a Memoir. The full personal intellectual memoir includes not only an intellectual theme – of seeking insight into mind and its encompassing disciplines via architectural exploration – that provides its overall focus, but also short descriptions of some of my other research and a smattering of more personal items. A reduced-in-scope intellectual memoir will be published soon, but a prepublication version can be found at the same link. You can also check out the Interviews tab, which includes a new one, as of January 2023, partially based on my memoir.