Yi, Ran, Ye, Zipeng, Zhao, Wang, Yu, Minjing, Lai, Yu-Kun ORCID: https://orcid.org/0000-0002-2094-5680 and Liu, Yong-Jin 2021. Feature-aware uniform tessellations on video manifold for content-sensitive supervoxels. IEEE Transactions on Pattern Analysis and Machine Intelligence 43 (9) , pp. 3183-3195. 10.1109/TPAMI.2020.2979714 |
Preview |
PDF
- Accepted Post-Print Version
Download (8MB) | Preview |
Abstract
Over-segmenting a video into supervoxels has strong potential to reduce the complexity of computer vision applications. Content-sensitive supervoxels (CSS) are typically smaller in content-dense regionsand larger in content-sparse regions. In this paper, we propose to compute feature-aware CSS (FCSS) that are regularly shaped 3D primitive volumes well aligned with local object/region/motion boundaries in video.To compute FCSS, we map a video to a 3-dimensional manifold, in which the volume elements of video manifold give a good measure of the video content density. Then any uniform tessellation on manifold can induce CSS. Our idea is that among all possible uniform tessellations, FCSS find one whose cell boundaries well align with local video boundaries. To achieve this goal, we propose a novel tessellation method that simultaneously minimizes the tessellation energy and maximizes the average boundary distance.Theoretically our method has an optimal competitive ratio O(1). We also present a simple extension of FCSS to streaming FCSS for processing long videos that cannot be loaded into main memory at once. We evaluate FCSS, streaming FCSS and ten representative supervoxel methods on four video datasets and two novel video applications. The results show that our method simultaneously achieves state-of-the-art performance with respect to various evaluation criteria.
Item Type: | Article |
---|---|
Date Type: | Publication |
Status: | Published |
Schools: | Computer Science & Informatics |
Publisher: | Institute of Electrical and Electronics Engineers (IEEE) |
ISSN: | 0162-8828 |
Date of First Compliant Deposit: | 30 March 2020 |
Date of Acceptance: | 5 March 2020 |
Last Modified: | 26 Nov 2024 00:45 |
URI: | https://orca.cardiff.ac.uk/id/eprint/130659 |
Citation Data
Cited 4 times in Scopus. View in Scopus. Powered By Scopus® Data
Actions (repository staff only)
Edit Item |