Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

SceneSketcher-v2: Fine-grained scene-level sketch-based image retrieval using adaptive GCNs

Liu, Fang, Deng, Xiaoming, Zou, Changqing, Lai, Yu-Kun ORCID: https://orcid.org/0000-0002-2094-5680, Chen, Keqi, Zuo, Ran, Ma, Cuixia, Liu, Yong-Jin and Wang, Hongan 2022. SceneSketcher-v2: Fine-grained scene-level sketch-based image retrieval using adaptive GCNs. IEEE Transactions on Image Processing 31 , pp. 3737-3751. 10.1109/TIP.2022.3175403

[thumbnail of TIP_SceneSketcher-v2.pdf]
Preview
PDF - Accepted Post-Print Version
Download (19MB) | Preview

Abstract

Sketch-based image retrieval (SBIR) is a long-standing research topic in computer vision. Existing methods mainly focus on category-level or instance-level image retrieval. This paper investigates the fine-grained scene-level SBIR problem where a free-hand sketch depicting a scene is used to retrieve desired images. This problem is useful yet challenging mainly because of two entangled facts: 1) achieving an effective representation of the input query data and scene-level images is difficult as it requires to model the information across multiple modalities such as object layout, relative size and visual appearances, and 2) there is a great domain gap between the query sketch input and target images. We present SceneSketcher-v2, a Graph Convolutional Network (GCN) based architecture to address these challenges. SceneSketcher-v2 employs a carefully designed graph convolution network to fuse the multi-modality information in the query sketch and target images and uses a triplet training process and end-to-end training manner to alleviate the domain gap. Extensive experiments demonstrate SceneSketcher-v2 outperforms state-of-the-art scene-level SBIR models with a significant margin.

Item Type: Article
Date Type: Publication
Status: Published
Schools: Computer Science & Informatics
Publisher: Institute of Electrical and Electronics Engineers
ISSN: 1057-7149
Date of First Compliant Deposit: 14 June 2022
Date of Acceptance: 29 April 2022
Last Modified: 10 Nov 2022 12:16
URI: https://orca.cardiff.ac.uk/id/eprint/150486

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics