Cross-Modal Proxy Prompt Alignment for Fine-Grained Image Classification

Fine-grained image classification remains challenging due to subtle inter-class differences, large intra-class variations, and lack of explicit semantic guidance in purely visual representations. Existing methods often struggle to construct stable and discriminative class prototypes, especially when textual annotations are unavailable. In this paper, we propose a cross-modal proxy prompt alignment (CPPA) method that introduces proxy prompts and obtains learnable class prototypes that function as auxiliary textual information to encode and inject fine-grained semantic cues into the model. The proxy prompts are directly optimized via multiple image-text alignment objectives, enabling them to autonomously acquire class-specific semantic structure without requiring additional annotations. Building on the aligned proxy prompts, we introduce a fusion mechanism that enables deep bidirectional interaction between visual features and textual proxies, effectively integrating local visual details with class-level semantic cues. Through a two-stage training procedure, our CPPA learns an enriched and more discriminative feature space while maintaining training stability. Experiments on CUB-200-2011, Stanford Dogs, and NABirds datasets show that our CPPA consistently enhances fine-grained image classification performance, demonstrating the effectiveness of proxy prompt alignment and cross-modal fusion for fine-grained image classification.