Shared-Specific Feature Enhancement and Dual Distilled Contextual Graph Refinement Network for Multimodal Conversational Emotion Recognition

Emotion Recognition in conversation (ERC) is a crucial task in building empathetic machines. Previous studies usually treat multimodal (i.e., visual, audio, text) equally and rarely focus on the shared and specific features among different modalities, leading to the redundancy of multimodal features. Besides, the extraction of contextual information in a dialogue and multimodal knowledge transfer remains challenging, which results in the inadequacy of capturing relational and contextual semantics. To solve these issues, we propose a Shared-Specific Feature Enhancement and Dual Distilled Contextual Graph Refinement Network (S2FE-D2CGRN) for ERC task. Specifically, a shared-specific feature enhancement module (S2FEM) is first designed to enhance the text-guided shared semantics and make the specific features discriminative. Second, to refine and distill the contextual knowledge, a dual-distilled contextual graph refinement module (D2CGRM) is investigated, which includes a contextual graph construction and a two-stage knowledge distillation. Extensive experiments on two multimodal public datasets show the effectiveness of our proposed method compared with the state-of-the-art methods, indicating its potential application in conversational emotion recognition.