Wang, Huasheng, Lou, Jianxun, Liu, Xiaochang, Tan, Hongchen, Whitaker, Roger ORCID: https://orcid.org/0000-0002-8473-1913 and Liu, Hantao ORCID: https://orcid.org/0000-0003-4544-3481 2024. SSPNet: Predicting visual saliency shifts. IEEE Transactions on Multimedia 26 , pp. 4938-4949. 10.1109/TMM.2023.3327886 |
PDF
- Accepted Post-Print Version
Download (27MB) |
Abstract
When images undergo quality degradation caused by editing, compression or transmission, their saliency tends to shift away from its original position. Saliency shifts indicate visual behaviour change and therefore contain vital information regarding perception of visual content and its distortions. Given a pristine image and its distorted format, we want to be able to detect saliency shifts induced by distortions. The resulting saliency shift map (SSM) can be used to identify the region and degree of visual distraction caused by distortions, and consequently to perceptually optimise image coding or enhancement algorithms. To this end, we first create a largest-of-its-kind eye-tracking database, comprising 60 pristine images and their associated 540 distorted formats viewed by 96 subjects. We then propose a computational model to predict the saliency shift map (SSM), utilising transformers and convolutional neural networks. Experimental results demonstrate that the proposed model is highly effective in detecting distortion-induced saliency shifts in natural images.
Item Type: | Article |
---|---|
Date Type: | Publication |
Status: | Published |
Schools: | Computer Science & Informatics |
Publisher: | Institute of Electrical and Electronics Engineers |
ISSN: | 1520-9210 |
Date of First Compliant Deposit: | 20 November 2023 |
Date of Acceptance: | 18 October 2023 |
Last Modified: | 20 May 2024 22:01 |
URI: | https://orca.cardiff.ac.uk/id/eprint/164090 |
Actions (repository staff only)
Edit Item |