SSPNet: Predicting visual saliency shifts

Wang, Huasheng, Lou, Jianxun, Liu, Xiaochang, Tan, Hongchen, Whitaker, Roger ORCID: https://orcid.org/0000-0002-8473-1913 and Liu, Hantao ORCID: https://orcid.org/0000-0003-4544-3481 2024. SSPNet: Predicting visual saliency shifts. IEEE Transactions on Multimedia 26 , pp. 4938-4949. 10.1109/TMM.2023.3327886

PDF - Accepted Post-Print Version Download (27MB)

Abstract

When images undergo quality degradation caused by editing, compression or transmission, their saliency tends to shift away from its original position. Saliency shifts indicate visual behaviour change and therefore contain vital information regarding perception of visual content and its distortions. Given a pristine image and its distorted format, we want to be able to detect saliency shifts induced by distortions. The resulting saliency shift map (SSM) can be used to identify the region and degree of visual distraction caused by distortions, and consequently to perceptually optimise image coding or enhancement algorithms. To this end, we first create a largest-of-its-kind eye-tracking database, comprising 60 pristine images and their associated 540 distorted formats viewed by 96 subjects. We then propose a computational model to predict the saliency shift map (SSM), utilising transformers and convolutional neural networks. Experimental results demonstrate that the proposed model is highly effective in detecting distortion-induced saliency shifts in natural images.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Institute of Electrical and Electronics Engineers
ISSN:	1520-9210
Date of First Compliant Deposit:	20 November 2023
Date of Acceptance:	18 October 2023
Last Modified:	20 May 2024 22:01
URI:	https://orca.cardiff.ac.uk/id/eprint/164090

Actions (repository staff only)

Edit Item