Hierarchical residual stochastic networks for time series recognition

Due to the complex spatio-temporal variations of data, time series recognition remains a challenging problem for the present deep networks. In this paper, we propose end-to-end hierarchical residual stochastic (HRS) networks to effectively and efficiently describe spatio-temporal variations. Specifically, we design stochastic kernelized filters based on a hierarchical framework with a new correlation residual (CorrRes) block to align the spatio-temporal features of a sequence. We further encode complex sequence patterns with a stochastic convolution residual (SConvRes) block, which employs the stochastic kernelized filters and a dropout strategy to reconfigure the convolution filters for large-scale computing in deep networks. Experiments on large-scale datasets, namely NTU RGB+D, SYSU-3D, UT-Kinect and Radar Behavior show that HRS networks significantly boost the performance of time series recognition and improve the state-of-the-art of skeleton, action, and radar behavior recognition performance.