By Dr. Nik Gurney, Research Lead, Social Simulation Lab
Five years ago, I joined the Institute for Creative Technologies (ICT) as a postdoctoral researcher to ask ambitious questions about human reasoning and test them in computational settings.
ICT has always been a place where foundational science and applied technology meet in productive tension. My work has evolved within that space: from studying trust and decision-making in human-AI teams to building formal models of theory of mind (ToM), and finally to challenging assumptions about how we design frameworks for human-machine integration (HMI).
The trajectory reflects both continuity and change. Continuity in my commitment to understanding social reasoning, and then change in how I think about where and when such reasoning matters for artificial systems.
Modeling Theory of Mind
Much of my time at ICT has been devoted to developing computational models of theory of mind, as in the ability to infer and reason about the mental states of others. This ability underpins much of human social interaction, enabling us to explain, predict, and sometimes manipulate behavior.
While psychologists and philosophers have debated ToM for decades, computational scientists are still in the early stages of designing artificial agents that can reliably approximate it.
Working with David Pynadath and colleagues, I co-authored two papers that remain foundational to my research. The first, Operationalizing Theories of Theory of Mind, published in Lecture Notes in Computer Science (LNCS, volume 13775), provided a comprehensive review of ToM research and mapped theoretical constructs onto recursive agent models.
Using PsychSim as a case study, we demonstrated how minimal requirements for ToM could be implemented computationally, while highlighting persistent challenges around recursion depth, scalability, and contextual deployment.
The second, Robots with Theory of Mind for Humans, a survey in IEEE’s Robot and Human Interactive Communication (RO-MAN 2022), examined the fragmented landscape of ToM research for robotics. We argued for shared definitions, consistent benchmarking, and unified modeling frameworks, all prerequisites if our research community is to achieve progress comparable to other AI domains.
Both efforts emerged from the DARPA Artificial Social Intelligence for Successful Teams (ASIST) program, which pursued the ambitious goal of endowing artificial teammates with social intelligence. ASIST posed a fundamental question: can we design agents capable not only of task execution but of aligning with human partners by representing and maintaining shared mental models?
The challenge was formidable. Social reasoning in humans is nuanced, context-dependent, and often opaque. Yet ASIST made clear that progress in AI will remain constrained unless we can capture the “invisible variables” of belief, intention, and trust, that shape human teamwork.
Trust, Risk, and Human–AI Teams
ASIST also created space for me to investigate how human cognitive states affect collaboration with AI. I focused on trust, compliance, and risk perception—constructs that determine whether human operators heed or ignore AI recommendations. Too often, these factors are treated as afterthoughts in AI design. But in real-world contexts, from reconnaissance missions to clinical decision support, a human partner’s willingness to follow or question AI advice often proves decisive.
In Measuring and Predicting Human Trust in Recommendations from an AI Teammate (LNCS, LNAI volume 13336, HCII 2022), co-authored with David Pynadath and Ning Wang, we tested whether behavioral measures of trust could predict compliance more effectively than psychometric self-report inventories.
We found that behavioral indicators, as in what people actually did in response to AI recommendations, were stronger predictors of mission success than what they claimed about their predispositions. This finding suggested a new path for adaptive AI: systems that adjust to observed behavior rather than static self-reports.
We extended this work in Comparing Psychometric and Behavioral Predictors of Compliance During Human-AI Interactions (PERSUASIVE 2023). Across three datasets, we demonstrated that self-reported trust inventories consistently underperformed behavioral measures in predicting compliance.
The implication is straightforward but powerful: if we want AI to calibrate trust effectively, we need models that attend to what people do, not only what they say. This insight has fundamentally shaped my approach to designing socially intelligent systems, pushing me toward dynamic, interaction-based measures rather than static attitudinal assessments.
Social Simulation in Cybersecurity
While much of my research has centered on cooperation, I have also explored social simulation in adversarial domains. From 2023-2025, I contributed to the IARPA ReSCIND program, which asked how to leverage cognitive science against cyber attackers, through identifying vulnerabilities in human decision-making and designing defenses that exploited them.
Unlike traditional defenses that detect and block suspicious activity, ReSCIND sought to mislead, delay, and frustrate adversaries by introducing subtle perturbations aligned with known cognitive biases.
Though the program was ultimately discontinued, the work yielded valuable insights. For example, in Quantifying Loss Aversion in Cyber Adversaries via LLM Analysis (Hans, Gurney, Marsella, Hirschmann, 2025), we used large language models to analyze notes produced by hackers during simulated intrusions. We demonstrated that LLMs could segment hacker actions, link them to persistence mechanisms, and reveal patterns of loss aversion in real-time decision-making.
This study illustrated social simulation’s potential to move beyond descriptive accounts of cognition toward actionable defensive tools. Even in preliminary form, the work showed that adversarial behavior could be interpreted dynamically, opening new strategies for cyber defense.
Current Research: Contextual ToM and Paradigms SHIFT
Today, my work at ICT builds on these foundations while pushing in new directions. Two basic research projects define this next phase.
The first, Causal Theory of Mind Models to Support Human-Machine Integration, interrogates when social reasoning is necessary, sufficient, or counterproductive. Traditional approaches to modeling ToM assume it is universally desirable; I argue its value is deeply contextual.
When a social interaction can be described by a game-theoretic model with a clear solution, ToM may be superfluous. Conversely, when complexity renders analytic solutions intractable, ToM becomes not merely helpful but essential. We are testing these hypotheses in simulation environments designed to reflect human–machine teaming contexts, aiming to develop contextual rules that guide AI in deploying ToM selectively—improving efficiency without sacrificing adaptability.
The second project, Paradigms for Strategic Human–Machine Integration Framework Transformation (Paradigms SHIFT), challenges the assumption that models of human teams can be applied wholesale to human–machine partnerships.
Machines process information, learn, and adapt in ways that diverge fundamentally from human cognition. Early results using unsupervised machine learning on HMI data suggest we can identify team processes unique to human–machine formations, such as processes that traditional human-team frameworks fail to capture. Paradigms SHIFT seeks to formalize these findings into purpose-built HMI frameworks that leverage the complementary strengths of humans and machines without forcing either into the other’s mold.
Both projects represent a conceptual shift in how I approach social reasoning in AI. The goal is no longer to replicate human cognition in machines, but to understand when such reasoning adds value and when alternative models prove more effective. This research aims to yield both theoretical advances in social cognition and practical frameworks for designing AI that integrates seamlessly into human contexts.
Next Steps
Reflecting on five years at ICT, I see continuity in my focus on social intelligence but also significant evolution in scope and application. I began by mapping psychological constructs onto computational models; I now investigate when those constructs are useful, how they shape trust and compliance, and what new frameworks emerge when humans and machines collaborate as peers. From cooperative search-and-rescue simulations to adversarial cyber defense, this work has demonstrated that social reasoning is not a luxury for AI—it is a necessity, though not always in the same form it takes for humans.
The Social Simulation Lab‘s future trajectory follows these threads further. We will continue refining contextual models of ToM, testing new paradigms of human–machine teaming, and exploring how behavioral measures can anchor adaptive AI. At stake is not only more effective collaboration but also deeper insight into human cognition itself. Social reasoning, after all, is the bridge between individual minds and collective action. To model it well is to better understand both the promise and limits of collaboration; whether among people, among machines, or across the boundary between them.
Five years is brief in the span of scientific research, but long enough to witness significant shifts in the field. The central challenge endures: how do we build machines that understand us well enough to function as partners, without requiring them to think as we do?
My work at ICT has convinced me that answering this question demands more than technical innovation. It requires conceptual clarity, methodological rigor, and above all, a willingness to rethink our assumptions about social intelligence. That is the work ahead, and it is what excites me most about the next five years.
//