Physics-based Hand-object Interaction via Control Force in Virtual Reality

Physics-based hand-object interaction in VR/AR has been widely studied. Penetration-based dynamics, where the applied force is proportional to the degree of interpenetration, can effectively support hand-object grasping; however, they exhibit notable limitations when interacting with tiny objects and are restricted to a narrow range of interaction tasks. To address these issues, we introduce a control-based dynamics as an alternative. Specifically, we employ PD controllers to approximate the applied force based on the velocity and movements of the tracked hand. Unlike penetration-based methods, our approach relies solely on the hand's movements for force computation, eliminating issues related to insufficient penetration distance and more accurately reflecting real-world physics. To evaluate the effectiveness of the proposed method, we conducted a series of user studies involving various interaction tasks and virtual objects, in comparison with state-of-the-art approaches. The results verify the effectiveness of our approach in hand-object interactions and demonstrate its significant positive sense of agency compared to prior works.