Research on pre-trained model compression mechanisms based on task matching similarity

In recent years, Transformer-based pre-trained models have achieved significant success in both computer vision (CV) and natural language processing (NLP) fields. However, due to their large size, model deployment has been constrained. To address this, we propose a pruning strategy based on task matching similarity. This strategy involves three key steps: 1) Task feature relevance analysis, which uses representation learning techniques to capture commonalities and differences between tasks; 2) Optimization of the model's dimensions using linear projection and iterative pruning algorithms to reduce parameters and computational complexity while preserving essential information; 3) Model retraining to ensure performance recovery and enhance generalization. Experimental results demonstrate that this strategy is effective on datasets such as Cifar-10 and CMRC2018, significantly reducing storage and training time while maintaining good performance. It shows great potential for widely applying Transformer models in resource-constrained environments.