StairNetV3: depth-aware stair modeling using deep learning

Vision-based stair modeling can help autonomous mobile robots deal with the challenge of climbing stairs, especially in unfamiliar environments. To address the problem that current monocular methods are difficult to model stairs accurately without depth information in scenes with fuzzy visual cues, this paper proposes a depth-aware stair modeling method for monocular vision. Specifically, we take the prediction of depth images and the extraction of stair geometric features as joint tasks in a convolutional neural network, with the designed information propagation architecture, we can achieve effective supervision for stair geometric feature learning by depth features. In addition, to complete the stair modeling, we take the convex lines, concave lines, tread surfaces and riser surfaces as stair geometric features and apply Gaussian kernels to enable StairNetV3 to predict contextual information within the stair lines. Combined with the depth information obtained by depth sensors, we propose a point cloud reconstruction method that can quickly segment point clouds of stair step surfaces. The experiments show that the proposed method has a significant improvement over the previous best monocular vision method, with an intersection over union increase of 3.4%, and the lightweight version has a fast detection speed and can meet the requirements of most real-time applications.