Balance gate controlled deep neural network

By mimicking the information expression ways in human's brain, deep neural network refers to a deeply structured framework with multiple processing layers that attempt to model high-level abstractions of the data. It is becoming a mainstream technology for pattern recognition, data mining and intelligent control at industrial scale. Previous work on constructing very deep networks, such as convolutional neural network and recurrent neural network, makes complex tasks such as image classification feasible. However, they are very limited when dealing with irregular data whose features are unstructured or even unknown. The traditional fully connected neural network is contrarily too shallow to extract high-level features. In this paper, we present a balance gate controlled deep network structure to deepen the fully connected neural network. It uses a new gating strategy to control information flow and increase network stability. Experimental results on both irregular regression and time-series forecasting demonstrate that the proposed network out-performs other ad-hoc models and is easier to train in a deeper form than the fully connected neural network.