ACII 2019 will host the following tutorials:

Integrating Theory-Driven and Data-Driven Approaches to Affective Computing via Deep Probabilistic Programming

Desmond Ong, National University of Singapore (NUS) & Agency for Science, Technology and Research (A*STAR)
Zhi-Xuan Tan, Agency for Science, Technology and Research (A*STAR); now at Massachusetts Institute of Technology (MIT)
Harold Soh, National University of Singapore (NUS)
Jamil Zaki, Stanford University
Noah Goodman, Stanford University

Presenter: Desmond Ong
Website: https://desmond-ong.github.io/pplAffComp
Research in affective computing has traditionally fallen into either theory-driven approaches that may not scale well to the complexities of naturalistic data, or atheoretic, data-driven approaches that learn to recognize complex patterns but still fall short of reasoning about emotions. In this tutorial, we introduce deep probabilistic programming, a new paradigm that models psychologically-grounded theories of emotion using stochastic programs. Specifically, this flexible framework combines the benefits of probabilistic models with those of deep learning models, marrying the advantages of both approaches. For example, when modelling someone’s emotions in context, we may choose to compose a probabilistic model of emotional appraisal with a deep learning model for recognizing emotions from faces—and we can do this all within a single unified framework for training and performing inference. By leveraging modern advances in deep probabilistic programming languages, researchers can easily scale these models up to larger, naturalistic datasets. Additionally, the lowered cost of model-building will accelerate rigorous theory-building and hypothesis-testing between competing theories of emotion.

The target audience comprises two groups of researchers. The first group includes researchers, such as cognitive psychologists, who favor theoretically-grounded models of affective phenomena, and who wish to scale their models up to complex, naturalistic datasets. The second group includes computer scientists, especially those who primarily use deep learning, who wish to add more emotion theory into their deep learning models, and in a principled manner. Deep probabilistic programming offers a way to combine the benefits of these two approaches to affective computing.

We will be learning from a webbook, using Jupyter notebooks. We will start with introductory primers to probabilistic programming concepts, such as stochastic primitives; compositionality and recursion; and stochastic variational inference. We will then transition into worked examples on previously-collected affective computing datasets. We will be using the open-sourced deep probabilistic programming language Pyro, first released in 2017. Tutorial participants will be able to download and run the code on their local machines as they follow along the material. We hope that by the end of this short tutorial, participants will be inspired—and equipped with some basic skills—to adopt approaches like deep probabilistic programming that merge theory- and data-driven approaches, and that such efforts will lead to greater collaboration between these historically-distinct paradigms in affective computing.

Thermal Imaging-based Physiological and Affective Computing

Presenters & Organizers:
Dr. Youngjun Cho, Assistant Professor, University College London (UCL), UK
Prof. Nadia Bianchi-Berthouze, Professor, University College London (UCL), UK
Website: TBA
As humans are homeothermic, our internal temperature is closely linked with numerous physiological and psychological mechanisms. Given this, human thermal patterns have been explored to improve understandings of our body for a couple of centuries. This tutorial is to bring traditional and advanced methods together into our community to discuss how to reliably interpret a person’s temperature into physiological signatures and how to use them to automatically infer our affective states, possibly in any situations. During the tutorial, we dive into existing methods, paradigms and physiological evidence around the thermal mechanism. First, we start looking at thermal measurements spanning from earlier thermometry to modern thermal imaging, and their use. Second, we review human physiology of heat production and physiological thermal signatures. Third, we discuss existing studies exploring the use of thermography in capturing a person’s physiological cues and understanding affective states through it. We also discuss computational aspects, for example, how to use advanced machine learning techniques for thermal imaging-based studies. Finally, we discuss the challenges and limitations emerged from the literature to explore research opportunities and directions in thermal imaging based physiological and affective computing.

Session I: Introduction to Thermal Imaging-based Physiological and Affective Computing
Session II: Practical Guide to Thermal Imaging-based Physiological and Affective Computing
Session III: Challenges, Opportunities and Applications


The Ambiguous and Uncertain World of Emotion Representation

Half day (3hrs with breaks)

Vidhyasaharan Sethu, The University of New South Wales
Julien Epps, The University of New South Wales
Nicholas Cummins, University of Augsburg
Shrikanth Narayanan, University of Southern California
Emily Mower Provost, University of Michigan
Carlos Busso, The University of Texas at Dallas

Vidhyasaharan Sethu, The University of New South Wales
Julien Epps, The University of New South Wales
Nicholas Cummins, University of Augsburg
Carlos Busso, The University of Texas at Dallas

This tutorial will focus on the topic of representing emotions in affective computing systems. Most commonly in computing systems, emotions have been represented with categorical labels such as ‘anger’ or ‘happiness’, or as values on numerical scales denoting affective attributes/dimensions, most typically arousal (activation) and valence (pleasantness). There has also been recent interest in ordinal representations of affect, which acknowledge the inherently ordinal characteristic of emotion perception, for example one emotional experience being ‘more’ pleasant than another. While it has been argued that the categorical representations of emotions do not capture every shade, as opposed to numerical or ordinal representations, all three are also single-valued representations that do not quantify the ambiguity present in perceived emotions.

Given how common it is for the expression of emotions to be ambiguous in naturally occurring human interactions, this is a key deficiency that needs to be addressed to develop emotionally aware machines. In order for the machine to exhibit richer, more human like responses it would need to quantify and incorporate this ambiguity into its internal representation of emotions and update it as more information is obtained. This would be hard to do with a chain of single estimates and demands a more nuanced representation which in turn needs the development of a suitably powerful framework within which emotion representation schemes can be described and developed.

This tutorial aims to present a framework, that was developed based on discussion across five research groups, within which properties for an optimal representation can be explicitly identified and specified. It will discuss a novel approach that seeks to bring together the myriad of emotion representations methods that are currently used under a common framework that also generalises to allow the ambiguity and uncertainty inherent in perceived affect to be quantified. In addition to presentations discussing affect representation methods, ambiguity in affect and a novel framework within which to analyse these methods, the session will also involve interactive group activity and discussions to brainstorm the challenges and shortcomings of current approaches as well as potential future direction.