Chen, Changjian, Wu, Jing ORCID: https://orcid.org/0000-0001-5123-9861, Wang, Xiaohan, Xiang, Shouxing, Zhang, Song-Hai, Tang, Qifeng and Liu, Shixia 2022. Towards better caption supervision for object detection. IEEE Transactions on Visualization and Computer Graphics 28 (4) , pp. 1941-1954. 10.1109/TVCG.2021.3138933 |
Preview |
PDF
- Accepted Post-Print Version
Download (5MB) | Preview |
Abstract
As training high-performance object detectors requires expensive bounding box annotations, recent methods resort to freeavailable image captions. However, detectors trained on caption supervision perform poorly because captions are usually noisy and cannot provide precise location information. To tackle this issue, we present a visual analysis method, which tightly integrates caption supervision with object detection to mutually enhance each other. In particular, object labels are first extracted from captions, which are utilized to train the detectors. Then, the label information from images is fed into caption supervision for further improvement. To effectively loop users into the object detection process, a node-link-based set visualization supported by a multi-type relational co-clustering algorithm is developed to explain the relationships between the extracted labels and the images with detected objects. The co-clustering algorithm clusters labels and images simultaneously by utilizing both their representations and their relationships. Quantitative evaluations and a case study are conducted to demonstrate the efficiency and effectiveness of the developed method in improving the performance of object detectors.
Item Type: | Article |
---|---|
Date Type: | Publication |
Status: | Published |
Schools: | Computer Science & Informatics |
Publisher: | Institute of Electrical and Electronics Engineers |
ISSN: | 1077-2626 |
Date of First Compliant Deposit: | 10 January 2022 |
Date of Acceptance: | 21 December 2021 |
Last Modified: | 28 Nov 2024 17:15 |
URI: | https://orca.cardiff.ac.uk/id/eprint/146505 |
Actions (repository staff only)
Edit Item |