PointISA: ISA-Extensions for Efficient Point Cloud Analytics via Architecture and Algorithm Co-Design

Point cloud analytics plays a crucial role in spatial machine vision for applications like autonomous driving, robotics and AR/VR. Recently, numerous domain-specific accelerators have been proposed to meet the stringent real-time and energy-efficiency demands of embedded and mobile platforms. However, most existing point cloud accelerators are kernel-independent hardware architectures, which face significant limitations and struggle to adapt to the rapidly evolving nature of point cloud applications. To address these challenges, we propose PointISA, a novel Instruction Set Architecture (ISA) extension designed for efficient point cloud analytics through architecture and algorithm co-design. First, we incorporate specialized instructions into PointISA for point cloud processing, such as Euclidean distance computation and multidimensional sorting. And then, we present an unified hardware architecture that efficiently executes both PointISA's specialized instructions and conventional matrix multiplication. Finally, we optimize key algorithms like farthest point sampling and k-nearest neighbors, adapting them into parallel multiple-points-to-multiple-points (MP2MP) patterns that fully exploit the architecture's parallel capabilities while maintaining algorithmic correctness. Evaluation shows that PointISA achieves an average 5.4 × speedup and 4.9 × improvement in power efficiency across diverse point cloud workloads while incurring a negligible area overhead of 0.9%.