TY - JOUR
T1 - Multimodal Reflective Synergy
T2 - Orchestrating Large-Small Model Collaboration for Knowledge-Based Visual Question Answering
AU - Sun, Zhongfan
AU - Guo, Kan
AU - Hu, Yongli
AU - Wang, Jiapu
AU - Tian, Daxin
AU - Yin, Baocai
N1 - Publisher Copyright:
© 1999 IEEE.
PY - 2026
Y1 - 2026
N2 - Large Vision-Language Models (VLMs) have demonstrated exceptional performance in knowledge-based visual question answering, while small-scale VLMs, owing to their distinct training paradigms, retain complementary strengths that large VLMs lack. Consequently, collaboration between large and small VLMs has emerged as a promising research direction. However, existing collaborative frameworks predominantly rely on unidirectional prompting, which limits the depth of synergistic reasoning across model scales. To address this limitation, we propose SyMRR, a large-small model Synergy framework based on Multimodal Reflective Reasoning, which enables deep, reflective, and visually grounded collaboration between compact and large VLMs. Specifically, we introduce a multimodal reflective chain-of-thought mechanism that guides the large VLM to critically assess the small model's predictions, generating both textual rationales and visually grounded evidence. These reflective signals are then integrated back into the small model's features via a dual reflective interaction strategy, utilizing modality-specific cross-attention to refine the initial multimodal cues. The resulting interaction-enhanced representations guide the large VLM toward more reliable and interpretable answer generation. Notably, SyMRR adopts a parameter-efficient design by freezing both backbone models and optimizing only lightweight interaction modules. Extensive experiments on the OK-VQA and A-OKVQA benchmarks demonstrate that SyMRR achieves state-of-the-art performance among collaborative frameworks, highlighting the effectiveness of deep reflective interaction for complex, knowledge-intensive multimodal reasoning.
AB - Large Vision-Language Models (VLMs) have demonstrated exceptional performance in knowledge-based visual question answering, while small-scale VLMs, owing to their distinct training paradigms, retain complementary strengths that large VLMs lack. Consequently, collaboration between large and small VLMs has emerged as a promising research direction. However, existing collaborative frameworks predominantly rely on unidirectional prompting, which limits the depth of synergistic reasoning across model scales. To address this limitation, we propose SyMRR, a large-small model Synergy framework based on Multimodal Reflective Reasoning, which enables deep, reflective, and visually grounded collaboration between compact and large VLMs. Specifically, we introduce a multimodal reflective chain-of-thought mechanism that guides the large VLM to critically assess the small model's predictions, generating both textual rationales and visually grounded evidence. These reflective signals are then integrated back into the small model's features via a dual reflective interaction strategy, utilizing modality-specific cross-attention to refine the initial multimodal cues. The resulting interaction-enhanced representations guide the large VLM toward more reliable and interpretable answer generation. Notably, SyMRR adopts a parameter-efficient design by freezing both backbone models and optimizing only lightweight interaction modules. Extensive experiments on the OK-VQA and A-OKVQA benchmarks demonstrate that SyMRR achieves state-of-the-art performance among collaborative frameworks, highlighting the effectiveness of deep reflective interaction for complex, knowledge-intensive multimodal reasoning.
KW - Dual Reflective Interaction
KW - Knowledge-Based Visual Question Answering
KW - Large-Small Model Synergy
KW - Multimodal Reflective Chain-of-Thought
UR - https://www.scopus.com/pages/publications/105050397926
U2 - 10.1109/TMM.2026.3731047
DO - 10.1109/TMM.2026.3731047
M3 - 文章
AN - SCOPUS:105050397926
SN - 1520-9210
JO - IEEE Transactions on Multimedia
JF - IEEE Transactions on Multimedia
ER -