Multimodal Cloud-Edge Collaborative Optimization with Bidirectional Knowledge Flow

With the rapid evolution of smart grid technology, real-time monitoring and analysis of power system operation status face challenges in multimodal data processing in key scenarios such as transformer defect detection. Aiming at the dual technical bottlenecks of limited computing resources and cross-modal feature differences in existing cloud-edge collaborative analysis systems, this study proposes a heterogeneous cloud-edge collaborative multimodal analysis framework, which is innovative in dual-tower architecture design and bidirectional knowledge flow collaboration mechanism. In the cloud model, by extending the CLIP-ViT-L/14 visual encoder, integrating the dilated convolution module and the Transformer-GRU hybrid temporal modeling unit, fine-grained feature extraction and global temporal dependency capture of power equipment images are achieved; on the edge side, a lightweight MobileViT network and a temporal convolutional network (TCN) are used to adapt to the efficient reasoning requirements under the resource constraints of the edge side. Through the bidirectional knowledge flow interaction of the 'visual anchor cloud-to-edge optimization strategy' and the 'temporal consistency edge-to-cloud optimization mechanism', the co-evolution and performance complementarity of the cloud-edge model are achieved. Experiments show that the accuracy of this framework is 5.4% higher than that of traditional methods in multimodal defect detection tasks such as transformer oil chromatography and partial discharge, providing a high-precision, low-latency solution for multimodal analysis of smart grids.