UAV-ENeRF: Text-Driven UAV Scene Editing With Neural Radiance Fields

The 3-D reconstruction of unmanned aerial vehicle (UAV) scenes is vital for agriculture, environmental protection, urban planning, and disaster response, to name a few. However, data acquisition can be constrained and hazardous under hostile environments, which limits the image data available in real-world applications. In this work, we propose a text-driven online editing framework for UAV scenes, which can generate novel views of existing scenes with abundant editing types. Compared with small single-object scenes, large-scale UAV scene editing suffers from several particular challenges. First, broader capturing scope exhibits illumination variation and complicated objects that reduce the 3-D scene consistency after editing. Second, high-resolution 2-D editing and 3-D reconstruction can be computationally expensive with tremendous graphical processing unit (GPU) memory. To tackle these issues, we first design a dual-branch compact neural radiance field (NeRF) structure to reduce memory usage and enhance accuracy for 3-D reconstruction. We then introduce a subpixel sampling scheme to expedite the generation of low-resolution images for 2-D editing, followed by a super-resolution (SR) module that restores the fine details of rendered images. Additionally, we develop a grouped content filtering mechanism to improve the 3-D scene consistency of the model by matching the rendering images and text descriptions, which also significantly reduces memory usage during editing. Extensive experiments demonstrate that the proposed method can achieve various editing effects, including different seasons, weather conditions, times of the day, and disaster scenarios. Our technique is computationally efficient and conveniently expandable for large-scale UAV scenes, alleviating data scarcity in harsh scenarios.