Trans2Mamba: A Hybrid Attention-Mamba Architecture for Time Series Forecasting

Time series forecasting, particularly for data collected via Internet of Things (IoT) sensing innovations in transportation and energy sectors, plays a crucial role in understanding complex social behaviors and enhancing social intelligence. Accurate prediction of these sensor-driven signals is of great significance for optimizing societal resource allocation, proactively managing social dynamics, and solving complex urban challenges. However, because of the high nonlinearity, dynamism, and long-term dependence of time series, accurate prediction has become a challenging task. The existing models based on graph neural networks are challenging to adapt to dynamic dependencies that change over time, particularly when using static graph structures to represent dependency relationships. In addition, traditional structures have limitations in capturing long-term dependencies. Although attention-based methods have alleviated the above problems to some extent, their high computational complexity and memory overhead limit their practical application in large-scale scenarios. To this end, we propose a hybrid Attention-Mamba architecture named Trans2Mamba that combines attention and Mamba modules for efficient time series forecasting. This model can effectively capture the dynamic and remote dependency characteristics in time series. The model we propose consists of two main modules: an input embedding layer that adaptively characterizes the features of different time steps and variables through a learnable spatiotemporal embedding layer; a stacked ST Block consisting of an attention-based temporal block and a Mamba-based spatial block, aimed at accurately modeling spatiotemporal dependencies through division of labor and collaboration. We validated the effectiveness of the model on seven datasets covering electricity, meteorology, and transportation. In addition, the ablation experiment further verified the positive contribution and rational design of each key module in the model to the overall performance.