Condition monitoring of train transmission systems based on multimodal fusion improved transformer network

The train transmission system is a critical component of railway operations, playing a pivotal role in ensuring service safety and reliability. However, existing condition monitoring approaches face two major challenges: (1) the coupling of rich multimodal signals, such as vibration, acoustics, current, and rotational speed, is often overlooked, limiting monitoring accuracy; (2) the small data problem in multimodal signals adversely affects the performance of neural networks. To address these issues, this paper proposes a Multimodal Fusion Improved Transformer Network for Condition Monitoring of Train Transmission Systems. The proposed network first explores interdependencies among different modalities of signals and compresses data to reduced dimensions through correlation analysis. It then infers global dependencies through computing self-attention scores based on Q, K, and V matrices. The approach is better than traditional CNN-based models in handling single-modality constraints, with the former demonstrated to be more accurate and trustworthy on publicly available datasets.