Human Activity Recognition by Fusing Ultrawideband Radar and Low-Resolution Infrared Information

The recognition of continuous daily human action is of great significance in smart homes, health management, and other areas. It is the key to achieving the 'Big Health' model. Both thermopile infrared array sensor (TPAS) and impulse radio ultra-wideband (IR-UWB) radar can detect human targets and provide relatively rich information without privacy invasion. This article builds a hardware platform based on these two types of sensors, collects data from actual indoor scenes, and proposes a human action recognition algorithm for achieving human action segmentation, human target localization, and recognition of multilevel activities. A~human action segmentation algorithm is proposed based on a finite state machine (FSM) and R-square value. Three data fusion schemes are designed based on a convolution neural network (CNN) network: data level fusion, feature level fusion, and decision level fusion to recognize human actions. A~positioning algorithm is proposed fusing TPAS and UWB information to determine orientation and distance. Finally, a scene model is established containing location, objects, spatial and temporal information, and a hidden Markov model (HMM) is trained to encode the information and predict high semantic human activities. The experimental results show that the accuracy for human dynamic recognition reaches 97.22%, the success rate of human positioning achieves 92.03% and the system can accurately recognize continuous human actions and high semantic activities in actual scenes.