Publication result detail

Multimodal Emotion Recognition for AVEC 2016 Challenge

POVOLNÝ, F.; MATĚJKA, P.; HRADIŠ, M.; POPKOVÁ, A.; OTRUSINA, L.; SMRŽ, P.; WOOD, I.; ROBIN, C.; LAMEL, L.

Original Title

Multimodal Emotion Recognition for AVEC 2016 Challenge

English Title

Multimodal Emotion Recognition for AVEC 2016 Challenge

Type

Paper in proceedings (conference paper)

Original Abstract

This paper describes a systems for emotion recognition and its application on the dataset from the AV+EC 2016 Emotion Recognition Challenge. The realized system was produced and submitted to the AV+EC 2016 evaluation, making use of all three modalities (audio, video, and physiological data). Our work primarily focused on features derived from audio. The original audio features were complement with bottleneck features and also text-based emotion recognition which is based on transcribing audio by an automatic speech recognition system and applying resources such as word embedding models and sentiment lexicons. Our multimodal fusion reached CCC=0.855 on dev set for arousal and 0.713 for valence. CCC on test set is 0.719 and 0.596 for arousal and valence respectively.

English abstract

This paper describes a systems for emotion recognition and its application on the dataset from the AV+EC 2016 Emotion Recognition Challenge. The realized system was produced and submitted to the AV+EC 2016 evaluation, making use of all three modalities (audio, video, and physiological data). Our work primarily focused on features derived from audio. The original audio features were complement with bottleneck features and also text-based emotion recognition which is based on transcribing audio by an automatic speech recognition system and applying resources such as word embedding models and sentiment lexicons. Our multimodal fusion reached CCC=0.855 on dev set for arousal and 0.713 for valence. CCC on test set is 0.719 and 0.596 for arousal and valence respectively.

Keywords

emotion recognition, valence, arousal, bottleneck features, neural networks, regression, speech transcription, word embedding

Key words in English

emotion recognition, valence, arousal, bottleneck features, neural networks, regression, speech transcription, word embedding

Authors

POVOLNÝ, F.; MATĚJKA, P.; HRADIŠ, M.; POPKOVÁ, A.; OTRUSINA, L.; SMRŽ, P.; WOOD, I.; ROBIN, C.; LAMEL, L.

RIV year

2017

Released

16.10.2016

Publisher

Association for Computing Machinery

Location

Amsterdam

ISBN

978-1-4503-4516-3

Book

AVEC '16 Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge

Pages from

75

Pages to

82

Pages count

8

URL

http://dl.acm.org/citation.cfm?id=2988268

BibTex

@inproceedings{BUT133512,
  author="Filip {Povolný} and Pavel {Matějka} and Michal {Hradiš} and Anna {Popková} and Lubomír {Otrusina} and Pavel {Smrž} and Ian {Wood} and Cecile {Robin} and Lori {Lamel}",
  title="Multimodal Emotion Recognition for AVEC 2016 Challenge",
  booktitle="AVEC '16 Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge",
  year="2016",
  pages="75--82",
  publisher="Association for Computing Machinery",
  address="Amsterdam",
  doi="10.1145/2988257.2988268",
  isbn="978-1-4503-4516-3",
  url="http://dl.acm.org/citation.cfm?id=2988268"
}

Documents

Povolny2016