Dynamic scene representation in the era of neural rendering: from NeRFs to 3DGSs

With the development of deep neural networks and differentiable rendering techniques, neural rendering methods, represented by Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), have made significant progress. NeRF represents a 3D scene by encoding the appearance and geometry of the scene through neural networks, which are conditioned on both position and viewpoint. In contrast, 3DGS models the scene with a set of Gaussian ellipsoids, allowing for efficient rendering through the rasterization of these ellipsoids into images. However, both two methods are limited to representing static scenes. The rendering and reconstruction of dynamic scenes are critical in virtual reality and computer graphics. As such, extending neural rendering methods from static to dynamic scenes has become an important area of research. This survey organizes dynamic scene rendering methods based on NeRF and 3DGS and categorizes them according to different motion representations. Furthermore, it highlights the relevant applications of dynamic scene rendering, such as autonomous driving, digital humans, and 4D generation. Finally, we summarize the development of dynamic scene rendering and discuss the remaining limitations and open challenges.