Efficient structured pruning based on deep feature stabilization

The application of convolutional neural networks (CNNs) in computer vision highly depends on the consumption of computation and memory resources, which affects its development on resource-limited devices. Accordingly, CNN compression has attracted increasing attention. In this paper, we propose an efficient end-to-end pruning method based on feature stabilization (EPFS), which is feasible to be implemented for structured pruning such as filter pruning and block pruning. For block pruning, we introduce a mask to scale the output of structures and the ℓ₁-regularization term to sparsify the mask. For filter pruning, a novel ℓ₂-regularization term is proposed to constraint the mask along with the ℓ₁-regularization. Besides, we introduce the Center Loss to stabilize the deep feature and fast iterative shrinkage-thresholding algorithm (FISTA) to accelerate the convergence of mask. Extensive experiments demonstrate the superiority of our EPFS. On CIFAR-10, EPFS saves 47.5 % FLOPs on VGGNet with 1.17 % Top-1 accuracy increase. Furthermore, on ImageNet ILSVRC2012, EPFS reduces 55.2 % FLOPs on ResNet-18 with o.nly 1.63 % Top-1 accuracy decrease, which promotes the state-of-the-arts.