Feature-aware uniform tessellations on video manifold for content-sensitive supervoxels

Yi, Ran, Ye, Zipeng, Zhao, Wang, Yu, Minjing, Lai, Yu-Kun ORCID: https://orcid.org/0000-0002-2094-5680 and Liu, Yong-Jin 2021. Feature-aware uniform tessellations on video manifold for content-sensitive supervoxels. IEEE Transactions on Pattern Analysis and Machine Intelligence 43 (9) , pp. 3183-3195. 10.1109/TPAMI.2020.2979714

Preview

PDF - Accepted Post-Print Version Download (8MB) | Preview

Abstract

Over-segmenting a video into supervoxels has strong potential to reduce the complexity of computer vision applications. Content-sensitive supervoxels (CSS) are typically smaller in content-dense regionsand larger in content-sparse regions. In this paper, we propose to compute feature-aware CSS (FCSS) that are regularly shaped 3D primitive volumes well aligned with local object/region/motion boundaries in video.To compute FCSS, we map a video to a 3-dimensional manifold, in which the volume elements of video manifold give a good measure of the video content density. Then any uniform tessellation on manifold can induce CSS. Our idea is that among all possible uniform tessellations, FCSS find one whose cell boundaries well align with local video boundaries. To achieve this goal, we propose a novel tessellation method that simultaneously minimizes the tessellation energy and maximizes the average boundary distance.Theoretically our method has an optimal competitive ratio O(1). We also present a simple extension of FCSS to streaming FCSS for processing long videos that cannot be loaded into main memory at once. We evaluate FCSS, streaming FCSS and ten representative supervoxel methods on four video datasets and two novel video applications. The results show that our method simultaneously achieves state-of-the-art performance with respect to various evaluation criteria.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Institute of Electrical and Electronics Engineers (IEEE)
ISSN:	0162-8828
Date of First Compliant Deposit:	30 March 2020
Date of Acceptance:	5 March 2020
Last Modified:	07 Nov 2023 06:43
URI:	https://orca.cardiff.ac.uk/id/eprint/130659

Citation Data

Cited 4 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item