An Affect Prediction Approach through Depression Severity Parameter Incorporation in Neural Networks

Rahul Gupta, Saurabh Sahu, Carol Espy-Wilson, Shrikanth Narayanan

Abstract:

Humans use emotional expressions to communicate their in-
ternal affective states. These behavioral expressions are often
multi-modal (e.g. facial expression, voice and gestures) and re-
searchers have proposed several schemes to predict the latent
affective states based on these expressions. The relationship
between the latent affective states and their expression is hy-
pothesized to be affected by several factors; depression disorder
being one of them. Despite a wide interest in affect prediction,
and several studies linking the effect of depression on affective
expressions, only a limited number of affect prediction models
account for the depression severity. In this work, we present
a novel scheme that incorporates depression severity as a pa-
rameter in Deep Neural Networks (DNNs). In order to predict
affective dimensions for an individual at hand, our scheme al-
ters the DNN activation function based on the subject’s depres-
sion severity. We perform experiments on affect prediction in
two different sessions of the Audio-Visual Depressive language
Corpus, which involves patients with varying degree of depres-
sion. Our results show improvements in arousal and valence
prediction on both the sessions using the proposed DNN model-
ing. We also present analysis of the impact of such an alteration
in DNNs during training and testing.