Adaptive Feature Compression and Resource Scheduling for End-Edge Co-Inference

Collaborative inference (Co-inference) across end devices and edge servers has emerged as a promising approach to satisfy the growing demand for computationally intensive and latency sensitive deep learning tasks. However, the limited computational capabilities and the bandwidth constraints of devices and edge servers pose significant challenges on co-inference. Current pertinent solutions either rely on predefined partition points and compression ratios or support only limited dimensions of dynamic adjustment, resulting in insufficient flexibility to adapt to diverse user requirements and network conditions. To address these challenges, we propose a novel framework that enhances co-inference through adaptive intermediate feature compression and efficient resource allocation. Specifically, we design a sophisticated feature compression method that incorporates channel pruning, spatial downsampling, and quantization, enabled by a weight-shared dynamic neural network architecture for efficient compression parameter switching without model reloading. Then, we formulate a constrained accuracy-maximization problem and develop a dynamic programming-based solution to jointly optimize partition points, compression parameters, and resource allocation, while meeting diverse user requirements. Experimental results show that our approach is capable of achieving up to a 22.8% improvement in inference accuracy compared to the state-of-the-art method in multiuser scenarios, demonstrating our superiority.