TrajPT: A trajectory data-based pre-trained transformer model for learning multi-vehicle interactions

Modeling and learning interactions with surrounding vehicles are critical for the safety and efficiency of autonomous vehicles. In this paper, we propose TrajPT, a Trajectory data-based Pre-trained Transformer model designed to learn spatial–temporal interactions among vehicles from large-scale real-world trajectory data. Inspired by pre-trained large language models, TrajPT adopts an autoregressive learning framework and a pre-training paradigm, and can be fine-tuned for different autonomous driving downstream tasks. To capture complex spatial–temporal interactions among vehicles, we utilize a spatial–temporal scene graph to encode observed vehicle trajectories and introduce a novel graph-based joint spatial–temporal attention module, which extracts spatial interactions within single frames and temporal dependencies across frames. TrajPT is pre-trained on pNEUMA, the largest publicly available vehicle trajectory dataset. We validate the performance of TrajPT by fine-tuning it on two downstream tasks: lane-changing prediction and trajectory prediction. Extensive experimental results demonstrate that the proposed TrajPT outperforms the baseline model and exhibits significant generalization performance across multiple datasets.