Online Video Stabilization using Mesh Flow with Minimum Latency

Abstract

Most existing video stabilization techniques are used for post-processing, where previously recorded videos are given to the model to obtain stabilized versions. Online video stabilization usually relies on sensors like gyroscopes or assumes constant motion, which is not suitable for videos with changing motions. This work introduces a video stabilization technique with just one-frame latency. The algorithm operates at the spatial level in the infrequent domain, tracking the motion of mesh vertices. Motion tracks of feature marks are combined with the nearest mesh vertex using two median gauges, assigning each vertex a smooth motion track. The proposed approach, called anticipated foster track leveling, smoothes the motion profiles by utilizing previous motions and adapting accordingly for smoother results. This method can handle changes in movement in space and time and works in real-time, allowing applications in security systems, robotics, and unmanned aerial vehicles (UAVs). When evaluated against other models, MeshFlow gives an overall good performance in all comparison metrics evaluated. Hence MeshFlow can be used as a reliable low-latency technique for real-time video stabilization in remote devices. © 2023 IEEE.