Enabling Efficient Bayesian Convolutional Neural Network with Feature Pre-extraction and Pre-pooling Strategies

Introducing Bayesian method into convolutional neural networks (CNNs) can effectively prevent neural networks from making over-confident decisions and overfitting. However, the large number of parameters in Bayesian convolutional neural networks (BCNNs) results in high computational costs, limiting the deployment in resource-sensitive computing systems. In this paper, two efficient strategies for BCNNs are proposed to optimize the inference process: 1) Feature Pre-extraction (F-P) strategy, which pre-extracts the feature maps in convolutional layers to reduce redundant computation introduced by repeated calculation, and 2) Pre-pooling (P-P) strategy, which performs pooling in advance to reduce the scale of feature maps involved in subsequent calculations. Compared with current efficient BCNN inference model, F-P strategy can eliminate around 92 % multiplications and accumulations, and P-P strategy can save 75% Gaussian random numbers requirement. To sum up, the proposed two optimization strategies can enable BCNN to save around 93% computational resources with 30.6% memory overhead together. The accuracy of LeNet-5 on CIFAR10 dataset has slightly decreased by 0.1%.