ARL 35 – Research Assistant, Creative Visual Storytelling

Project Name
Creative Visual Storytelling

Project Description
This project seeks to discover how humans tell stories about images, and to develop computational models to generate these stories. “Creative” visual storytelling goes beyond listing observable objects and their visual properties, and takes into consideration several aspects that influence the narrative: the environment and presentation of imagery, the narrative goals of the telling, and the audience who is listening. This work involves aspects of computer vision to visually analyze the image, commonsense reasoning to understand what is happening, and natural language generation and theories of narratives to describe it in a cohesive and engaging manner. We will work with low-quality images and non-canonical scenes. Paper reference:

Job Description
Possible projects include:
– Develop software framework for crowdsourcing the annotation of stories written about images
– Conduct manual and/or computational analysis of the narrative styles and properties of stories written about images
– Experiment with or combine existing natural language generation and/or computer vision software for creative visual storytelling
– Work with project mentor to design evaluation criteria for assessing the quality of stories written about images

Preferred Skills
Interest in and knowledge of some combination of the following:
– Programming expertise for language generation and/or computer vision
– Digital narratives and storytelling applied to images
– Experimental design and applied statistics for rating and evaluating stories

Apply now.

Go back to the summer research program list.