Polarmask-Tracker: Lightweight Multi-Object Tracking and Segmentation Model for Edge Device

The image or video input from the camera is one of the important data sources for unmanned vehicles to perceive the environment. However, the 2D/3D bounding box can only provide a very coarse approximation because one box often contains other targets and background. In order to solve the problem of precise target tracking and computing limitations of edge devices, this paper proposes Polarmask-Tracker, a lightweight segmentation-based multi-object tracking network for vehicular edge devices. Polarmask-Tracker extended the lightweight Polarmask segmentation head with tracking vector. The polar mask replaces the traditional mask prediction by regression of a group of fixed edge points in polar coordinate system, which can greatly optimize the computational complexity and regression difficulty of the mask. With an additional tracking vector branch generated based on mask, the model can learn tracking tasks in an end-to-end manner. Finally, we further accelerated the entire model based on TensorRT and achieve real-time tracking on mobile edge computing platform. Different from previous evaluations on the ImageNet and COCO datasets, this study uses the KITTI tracking dataset to extend the instance segmentation task to segmentation tracking, also called MOTS. At the same time, the target scales captured from the autonomous vehicle camera are usually smaller, which also brings additional challenges. Evaluations on NVidia Jetson AGX show that the final Polarmask-Tracker can achieve 122.55 FPS, 46.57 mAP for mask segmentation, 56.418 HOTA for tracking.