Affective Analysis for Video Frames Using ConvLSTM Network

With the rapid development of various online video sharing platforms, large numbers of videos are produced every day. Video affective content analysis has become an active research area in recent years, since emotion plays an important role in the classification and retrieval of videos. In this work, we explore to train very deep convolutional networks using ConvLSTM layers to add more expressive power for video affective content analysis models. Network-in-network principles, batch normalization, and convolution auto-encoder are applied to ensure the effectiveness of the model. Then an extended emotional representation model is used as an emotional annotation. In addition, we set up a database containing two thousand fragments to validate the effectiveness of the proposed model. Experimental results on the proposed data set show that deep learning approach based on ConvLSTM outperforms the traditional baseline and reaches the state-of-the-art system.