Sub-Frame Appearance and 6D Pose Estimation of Fast Moving Objects

0533751 - ÚTIA 2021 RIV US eng C - Conference Paper (international conference)
Rozumnyi, D. - Kotera, Jan - Šroubek, Filip - Matas, J.
Sub-Frame Appearance and 6D Pose Estimation of Fast Moving Objects.
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020, s. 6777-6785. ISBN 978-1-7281-7169-2. ISSN 1063-6919. E-ISSN 2575-7075.
[2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle (US), 16.06.2020-18.06.2020]
R&D Projects: GA ČR GA18-05360S
Institutional support: RVO:67985556
Keywords : tracking * deblurring * matting * deblatting
OECD category: Computer hardware and architecture
http://library.utia.cas.cz/separaty/2020/ZOI/sroubek-0533751.pdf

We have proposed a method for sub-frame appearance and 6D pose estimation of fast moving objects. We propose a novel method that tracks fast moving objects, mainly non-uniform spherical, in full 6 degrees of freedom, estimating simultaneously their 3D motion trajectory, 3D pose and object appearance changes with a time step that is a fraction of the video frame exposure time. The sub-frame object localization and appearance estimation allows realistic temporal super-resolution and precise shape estimation. The method, called TbD-3D (Tracking by Deblatting in 3D) relies on a novel reconstruction algorithm which solves a piece-wise deblurring and matting problem. The 3D rotation is estimated by minimizing the reprojection error. As a second contribution, we present a new challenging dataset with fast moving objects that change their appearance and distance to the camera. High-speed camera recordings with zero lag between frame exposures were used to generate videos with different frame rates annotated with ground-truth trajectory and pose.
Permanent Link: http://hdl.handle.net/11104/0312101