Events

Andrew Gordon, Reid Swanson: “Identifying Personal Stories in Millions of Weblog Entries”

May 20, 2009 | San Jose, CA

Speaker: Andrew Gordon, Reid Swanson
Host: Third International Conference on Weblogs and Social Media, Data Challenge Workshop

Stories of people’s everyday experiences have long been the focus of psychology and sociology research, and are increasingly being used in innovative knowledge-based technologies. However, continued research in this area is hindered by the lack of standard corpora of sufficient size and by the costs of creating one from scratch. In this paper, we describe our efforts to develop a standard corpus for researchers in this area by identifying personal stories in the tens of millions of blog posts in the ICWSM 2009 Spinn3r Dataset. Our approach was to employ statistical text classification technology on the content of blog entries, which required the creation of a sufficiently large set of annotated training examples. We describe the development and evaluation of this classification technology and how it was applied to the dataset in order to identify nearly a million personal stories.