A label-anchored variational framework for air crisis event multi-modal recognition with missing modality

Air crisis event recognition from massive multi-modal social media data offers a promising avenue for enhancing emergency response efficiency, reducing crisis management costs, and uncovering critical situational insights. However, incomplete text-image pairs pose significant challenges, as the chaotic and damaged scenes typical of air accidents often hinder the collection of comprehensive information. To address this issue, we propose a label-anchored variational framework that shifts the paradigm from fusion-centric compensation to completion-driven discrimination. Specifically, we propose a general multi-modal recognition scheme for air crisis events with missing modalities that resorts to a Unimodal Knowledge Completion Variational Autoencoder (UKC-VAE) model. First, two separate VAE-based unimodal parallel encoders are presented to generate class-discriminative latent variables through topic-specific label embeddings acting as lightweight class anchors. Moreover, a contrastive learning-based semantic alignment module and a distribution alignment module are proposed to enhance the cross-modal knowledge transfer and ensure consistency across modalities. Extensive experiments demonstrate the superior performance of the proposed UKC-VAE model compared to several state-of-the-art baselines on the AirCrisisMMD and CrisisMMD datasets. The former is a new specialized multi-modal dataset that will be released soon.