Focus-then-fusion: Learning discriminative cross-modal prototypes for few-shot classification

Few-shot classification aims to recognize unseen classes using only few training samples. An effective approach for few-shot tasks is to measure the feature similarity between query instances and support prototypes. Considering the difficulty of constructing precise class prototypes with limited samples, class-relevant textual information is introduced to improve feature representations. However, under few-shot conditions, background noise severely disrupts textual-visual correlation modeling, resulting in semantic alignment deviations and inadequate modal complementarity. To address the problem, we propose a two-stage FocusFusion module, which introduces an explicit cross-modal correlation learning stage prior to feature fusion stage, aiming to focus on discriminative visual targets and alleviate background interference during textual-visual fusion process. Specifically, we learn the text-based correlations between class semantics and visual features to generate spatial position offsets towards discriminative targets, by which discriminative visual features are sampled to perform textual-visual feature fusion at local and global levels. Building upon the FocusFusion module, we further propose a discriminative cross-modal prototypical network, which utilizes the fused features to construct accurate class prototypes and improve metric-based few-shot classification. The proposed method achieves state-of-the-art results on five few-shot classification benchmarks.