RailDet: An End-to-End 3D Perception Network in Railway Environments

Autonomous trains have become a key development direction in the railway, offering the potential to significantly enhance operational efficiency while ensuring safety. In autonomous train perception systems, accurate 3D rail-track detection and 3D object detection are of paramount importance. However, existing methods typically address only one of these tasks, lacking an effective integration of both. To address these challenges, this paper proposes RailDet, an end-to-end 3D perception network for railway environments including 3D railtrack line detection and 3D object detection. RailDet consists of two components: a topology projection-based 3D Rail-track Detection (3DRD) module and a BEV-based 3D Object Detection (3DOD) module. First, image features are extracted in a unified manner. Then, the 3DRD module designs 3D anchor points and projects them onto the feature maps to obtain anchor features, which are subsequently used to predict the 3D positions of the railtrack. On the other hand, the 3DOD module transforms the image into the bird's-eye view (BEV), leveraging the inherent distance information to estimate the 3D positions of obstacles. Through this design, RailDet enables simultaneous 3D rail-track detection and object detection within a single network. Experimental results show that the proposed RailDet achieves a 3D object detection accuracy of 32.5% and a 3D rail-track detection accuracy of 82.5%. The results underscore the robustness and potential of our approach for accurate 3D rail-track and 3D object detection in complex railway environments.