Learning Dynamic and Hierarchical Traffic Spatiotemporal Features with Transformer

Traffic forecasting has attracted considerable attention due to its importance in proactive urban traffic control and management. Scholars and engineers have exerted considerable efforts in improving the performance of traffic forecasting algorithms in terms of accuracy, reliability, and efficiency. Spatial feature representation of traffic flow is a core component that greatly influences traffic forecasting performance. In previous studies, several spatial attributes of traffic flow are ignored due to the following issues: a) traffic flow propagation does not comply with the road network, b) the spatial pattern of traffic flow varies over time, and c) single adjacent matrix cannot handle the complex and hierarchical urban traffic flow. To address the abovementioned issues, this study proposes a novel traffic forecasting algorithm called traffic transformer, which achieves great success in natural language processing. The multihead attention mechanism and stacking layers enable the transformer to learn dynamic and hierarchical features in sequential data. Two components, namely, global encoder and global-local decoder, are proposed to extract and fuse the spatial patterns globally and locally. Experimental results indicate that the proposed traffic transformer outperforms state-of-the-art methods. The learned dynamic and hierarchical features of traffic flow can help achieve a better understanding of spatial dependency of traffic flow for effective and efficient traffic control and management strategies.