APM-CIM: An Array Partition Multi-Macro CIM System With Dynamic Sparse Approximation for Neural Network Edge Applications

Computing-in-memory (CIM) has been widely investigated as a solution to the von Neumann bottleneck to reduce data transfer between memory and processor. However, existing CIM macros for neural network (NN) edge applications still suffer from high latency and energy consumption due to the correlation between data precision and the number of multiplication and accumulation (MAC) cycles. Moreover, modern convolutional NN (CNN) models contain massive parameters that are distributed across multiple CIM macros. While multi-macro architectures enable large-scale deployment, they incur inefficient bandwidth utilization during input loading and frequent inter-module communication, both of which cause significant performance degradation. To address these issues, this article presents an array partition multi-macro CIM (APM-CIM) system with dynamic sparse approximation. First, a novel CIM macro incorporating double-input and sparse approximation mechanism is proposed to halve the number of computation cycles and skip insignificant operands during MAC. Second, an array partition mapping scheme for weights is proposed to reduce the bandwidth requirement of a CIM macro, thereby maximizing cache bandwidth utilization to implement high-parallelism computation. Finally, data and task managers are designed to minimize the performance degradation of the CIM macro at the system level. Through cross-level co-design of circuit, architecture, and system, the fabricated 40-nm CMOS chip achieves a system energy efficiency of 67.63 TOPS/W, which reduces the performance loss ratio by 3.7× and is 10.4× higher than state-of-the-art CIM-based systems. Using the VGG model on the CIFAR-10 dataset, the system delivers an accuracy of 89.95% at a power consumption of 34.0 mW.