Dartmouth Events

Remote Sensing of Emotion and the Application to Large-scale and Longitudinal Studies

Abstract

Emotions play an important role in our everyday lives. They influence memory, decision-making and well-being.  In order to advance the fundamental understanding of human emotions, build smarter affective technology, and ultimately help people, we need to perform research in-situ. It is now possible to quantify emotional responses on a large scale using webcams and microphones in everyday environments.  I will present novel methods for physiological and behavioral measurement that allow for passive non-contact tracking of emotions via ubiquitous hardware. This will include highly scalable measurement of facial expressions, cardio-pulmonary signals and sympathetic nervous system activity.   I will follow this with insights from analysis from some of the world’s largest datasets of naturalistic human behavior (featuring examples from millions of individuals and 10,000’s of hours of data) and show how this data has allowed us to corroborate and extend the understanding of nonverbal behavior, including modeling gender and cultural differences.  Finally, I will show examples of new human-computer interfaces that leverage behavioral and physiological signals, including emotion-aware natural language conversation systems and real-time VR/AR visualization of vital signs (CardioLens).