Echo state networks for multidimensional data: Exploiting noncircularity and widely linear models

The quaternion domain offers a convenient and unified means to process multidimensional data which are typically 3D and 4D, such as those measurements from 3D inertial sensors in body sensor networks and 3D wind modeling from 3D ultrasonic anemometers. To deal with the nonlinear and nonstationary characteristics of real-world multidimensional data, quaternion-valued nonlinear learning systems, like recurrent neural networks (RNNs), are highly desirable. To avoid current problems associated with the design of quaternion-valued RNNs, such as the computationally demanding training tasks and the stringent standard analyticity conditions in developing full quaternion-valued nonlinearities, quaternion-valued echo state networks (QESNs), built upon quaternion nonlinear activation functions with local analytic properties, are introduced. To further make QESNs second-order optimal for the generality of quaternion signals (both circular and noncircular), the standard widely linear model is modified so as to suit the properties of dynamical reservoir, typically realized by RNNs. This allows for a full exploitation of second-order information in the multidimensional data, contained both in the covariance and pseudocovariances. Simulations in the prediction setting on both benchmark 3D and 4D circular and noncircular signals and on noncircular, nonlinear and nonstationary real-world 3D body motion tracking and wind forecasting support the analysis.