Louis-Philippe Morency, Rada Mihalcea, Paval Doshi: “Towards Multimodal Sentiment Analysis: Harvesting Opinions from The Web”

November 16, 2011 | Alicante, Spain

Speaker: Louis-Philippe Morency, Rada Mihalcea, Paval Doshi
Host: 13th International Conference on Multimodal Interaction (ICMI 2011)

With more than 10,000 new videos posted online every day on social websites such as YouTube and Facebook, the in- ternet is becoming an almost in nite source of information. One crucial challenge for the coming decade is to be able to harvest relevant information from this constant ow of mul- timodal data. This paper addresses the task of multimodal sentiment analysis, and conducts proof-of-concept experi- ments that demonstrate that a joint model that integrates visual, audio, and textual features can be e ectively used to identify sentiment in Web videos. This paper makes three important contributions. First, it addresses for the rst time the task of tri-modal sentiment analysis, and shows that it is a feasible task that can bene t from the joint exploitation of visual, audio and textual modalities. Second, it identi es a subset of audio-visual features relevant to sentiment analy- sis and present guidelines on how to integrate these features. Finally, it introduces a new dataset consisting of real online data, which will be useful for future research in this area.