MagFormer: Hybrid Video Motion Magnification Transformer from Eulerian and Lagrangian Perspectives

Video motion magnification methods attract much attention for their strong capability of capturing informative subtle signals from diverse engineering scenes. There are two main types of methods in this field, Eulerian and Lagrangian motion magnification, which have different advantages and perspectives. However, the combination of both remains largely unexplored. In this paper, we develop an end-to-end video motion magnification network, MagFormer, with a well-designed two-branch magnification module, which includes a convolutional neural network (CNN) for the Eulerian motion magnification branch and Transformer for the Lagrangian optical flow magnification branch. Our MagFormer can inherit the advantages of two perspectives, by leveraging both Eulerian global motion features from the camera-centered perspective and trajectories of the object-centered from the Lagrangian object perspective in a unified parallel framework. To validate the effectiveness of our method, we collect a new vibration dataset to measure video motion magnification methods via amplitude and frequency. More experiments are conducted on fixed-background subtle motion videos, constantly moving object videos and quantitative vibration videos. Experimental results show that our method achieves a favorable improvement compared to state-of-the-art methods. Codes will be released at https://github.com/Ree1s/MagFormer.