Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Stroke-based semantic segmentation for scene-level free-hand sketches

Zhang, Zhengming, Deng, Xiaoming, Li, Jinyao, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680, Ma, Cuixia, Liu, Yongjin and Wang, Hongan 2023. Stroke-based semantic segmentation for scene-level free-hand sketches. Visual Computer 39 , pp. 6309-6321. 10.1007/s00371-022-02731-8

[thumbnail of VisualComputer_strokesketch_SBIR.pdf]
Preview
PDF - Accepted Post-Print Version
Download (18MB) | Preview

Abstract

Sketching is a simple and efficient way for humans to express their perceptions of the world. Sketch semantic segmentation plays a key role in sketch understanding and is widely used in sketch recognition, sketch-based image retrieval, or editing. Due to modality difference between images and sketches, existing image segmentation methods may not perform best, which overlook the sparse nature and stroke-based representation in sketches. The existing sketch semantic segmentation methods are mainly designed for single-instance sketches. In this paper, we present a new stroke-based sequential-spatial neural network (S3NN) for scene-level free-hand sketch semantic segmentation, which leverages a bidirectional LSTM and graph convolutional network to capture the sequential and spatial features of sketches. In order to address the data lacking issue, we propose the first scene-level free-hand sketch dataset (SFSD). SFSD is composed of 12K sketch-photo pairs over 40 object categories, where the sketches were completely hand-drawn and each contains 7 objects on average. We conduct comparative and ablative experiments on SFSD to evaluate the effectiveness of our method. The experimental results demonstrate that our method outperforms state-of-the-art methods. The code, models, and dataset will be made public after acceptance.

Item Type: Article
Date Type: Publication
Status: Published
Schools: Computer Science & Informatics
Publisher: Springer
ISSN: 0178-2789
Date of First Compliant Deposit: 29 December 2022
Date of Acceptance: 14 November 2022
Last Modified: 07 Dec 2023 16:37
URI: https://orca.cardiff.ac.uk/id/eprint/155304

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics