Army Award-Winning Research to Transform Soldier-Robot Communication

July 30, 2021

Army researchers developed ground-breaking technology that will enhance how Soldiers and robots communicate and carry out tasks in tactical environments.

This research sets out to develop a natural language understanding, or NLU, pipeline for robots that would be easily ported over to any computational system or agent and incrementally tames the variation that we see in natural language, said Army researcher Dr. Claire Bonial from the U.S. Army Combat Capabilities Development Command, known as DEVCOM, Army Research Laboratory.

This means that regardless of how a Soldier chooses to express him or herself to the robot, the underlying intent of that language is understood and can be acted on, given both the current conversational and environmental or situational context.

To do this, the NLU pipeline first automatically parses the input language into Abstract Meaning Representation, or AMR, which captures the basic meaning of the content of the language, Bonial said. It then converts and augments the AMR into Dialogue-AMR, which captures additional elements of meaning needed for two-way human robot dialogue in particular, such as what the person is trying to do with the utterance in the conversational context, for example give a command, ask a question, state a fact about the environment, etc.

This research was presented at the 14th International Conference on Computational Semantics, or IWCS 2021, where it received the Outstanding Paper Award.

The award citation noted that “The authors are not afraid of using old-fashioned hand-written rules when they do the job, something which is lacking in much current work in NLP,“ and that “Anyone who wants to work on human-robot dialogue will want to see this first go at parsing in this new domain.”

“This award was incredibly gratifying for several reasons,” Bonial said. “First, this paper represents research efforts that were planted and have been growing since I was a doctoral student. I was part of the first group of researchers establishing what has become one of the most widely used semantic representations in natural language processing, AMR.”

Bonial started work with this group in 2010, and has been actively involved in refining and extending the representation ever since. Thus, this paper represents a body of research spanning over a decade for Bonial.

This includes work to represent language expressed through semi-idiomatic constructions, for example “the higher you fly, the harder you’ll fall!,” and most recently extending AMR so that it can better capture two-way dialogue, and specifically task-oriented, situated dialogue between people and robots, in an augmented version of the representation called Dialogue-AMR, she said.

Efforts to develop Dialogue-AMR started in 2018 with dialogue expert Dr. David Traum as part of the University Affiliated Research Center previously established between DEVCOM ARL and the Institute for Creative Technologies at the University of Southern California.

Continue reading, via U.S. Army.