论文标题
基于卷积功能提取器和经常性神经网络的大型视频数据集上的情感识别
Emotion Recognition on large video dataset based on Convolutional Feature Extractor and Recurrent Neural Network
论文作者
论文摘要
多年来，情感识别任务一直是人类计算机互动领域中最有趣，最重要的问题之一。在这项研究中，我们将情感识别任务视为分类以及通过使用深度学习模型在不同数据集中处理编码的情绪的分类和回归任务。我们的模型将卷积神经网络（CNN）与复发性神经网络（RNN）结合在一起，以预测视频数据上的维度情绪。在第一步，CNN提取物具有视频帧的向量。在第二步中，我们为这些功能向量提供了训练RNN，以利用视频的时间动态。此外，我们分析了每个神经网络如何对系统的整体性能做出贡献。这些实验是在包括最大的现代AFF-WILD2数据库在内的公开数据集上进行的。它包含超过60个小时的视频数据。我们发现了使用混乱矩阵的说明性示例在不平衡数据集上过度拟合模型的问题。通过下采样技术来解决该问题以平衡数据集。通过显着减少培训数据，我们可以平衡数据集，从而提高了模型的整体性能。因此，该研究定性地描述了深度学习模型的能力，探索了足够数量的数据以预测面部情绪。我们提出的方法是使用TensorFlow Keras实现的。
For many years, the emotion recognition task has remained one of the most interesting and important problems in the field of human-computer interaction. In this study, we consider the emotion recognition task as a classification as well as a regression task by processing encoded emotions in different datasets using deep learning models. Our model combines convolutional neural network (CNN) with recurrent neural network (RNN) to predict dimensional emotions on video data. At the first step, CNN extracts feature vectors from video frames. In the second step, we fed these feature vectors to train RNN for exploiting the temporal dynamics of video. Furthermore, we analyzed how each neural network contributes to the system's overall performance. The experiments are performed on publicly available datasets including the largest modern Aff-Wild2 database. It contains over sixty hours of video data. We discovered the problem of overfitting of the model on an unbalanced dataset with an illustrative example using confusion matrices. The problem is solved by downsampling technique to balance the dataset. By significantly decreasing training data, we balance the dataset, thereby, the overall performance of the model is improved. Hence, the study qualitatively describes the abilities of deep learning models exploring enough amount of data to predict facial emotions. Our proposed method is implemented using Tensorflow Keras.
