Collision risk prediction and takeover requirements assessment based on radar-video integrated sensors data: A system framework based on LLM

There are safety risks when drivers take over the control of autonomous driving vehicles, and reducing unnecessary takeovers is essential to improve driving safety. This study seeks to develop an interpretable system framework for collision risk prediction and takeover requirements analysis (CPTR-LLM) utilizing a large language model (LLM). The model's inference performance is enhanced through the collection of extensive perception data and the design of a two-stage training strategy, reasoning chain framework, and an error detection and correction mechanism. In terms of collision risk prediction, the experimental results show that the accuracy of CPTR-LLM can reach 0.88. The Cross-sectional-autoregressive-distributed lag (ARDL) model and Augmented Mean Groups (AMG) confirm the reliability of the model's predictive performance by revealing the association between different variables and collision risk. Regarding takeover requirement analysis, CPTR-LLM accurately comprehends the characteristics of the pre-takeover scene and comprehensively assesses the takeover requirement level in conjunction with collision risk, thereby effectively reducing unnecessary takeovers in simple driving scenarios and unsafe takeovers in scenarios with multiple moving targets. Overall, the findings of this paper offer significant insights into the application and takeover requirements of LLM in the domain of road safety.