LMFSNet: A Lightweight Multi-Level Fusion Hand Gesture Segmentation Network for Human-Robot Interaction

Traditional gesture-based human-robot interaction relies on one-to-one gesture-command mapping, requiring numerous gestures and imposing high cognitive load. Existing networks are often computationally heavy, limiting real-time deployment on resource-constrained robots. To significantly reduce the required gestures and improve the intuitiveness of user experience, we develop the Spatial Semantic Mapping framework to change the gesture-based control paradigm by assigning commands based on the spatial position of the hand, establishing a flexible one-to-many mapping. To achieve an optimal balance between accuracy and computational efficiency, we propose a Lightweight Multi-level Fusion Segmentation Network (LMFSNet). Firstly, to reduce computational costs greatly, we propose a lightweight Residual Axial Group Convolution as the core operation of the model. Secondly, to maintain high performance in the lightweight network, we design two modules: Dynamic Adaptive Attention Block (DAAB) and Long-Short Distance Extraction (LSDE) block. Specifically, the DAAB dynamically reweights features to focus on important information, and the LSDE effectively captures and fuses multi-scale features. Experimental results show that the proposed LMFSNet achieves state-of-the-art accuracy while maintaining real-time speed and a compact model size.