Predicting radiologists' gaze with computational saliency models in mammogram reading

Lou, Jianxun, Lin, Hanhe, Young, Philippa, White, Richard, Yang, Zelei, Shelmerdine, Susan, Marshall, David ORCID: https://orcid.org/0000-0003-2789-1395, Spezi, Emiliano ORCID: https://orcid.org/0000-0002-1452-8813, Palombo, Marco ORCID: https://orcid.org/0000-0003-4892-7967 and Liu, Hantao ORCID: https://orcid.org/0000-0003-4544-3481 2024. Predicting radiologists' gaze with computational saliency models in mammogram reading. IEEE Transactions on Multimedia 26 , pp. 256-269. 10.1109/TMM.2023.3263553

Preview

PDF - Accepted Post-Print Version Download (14MB) | Preview

Abstract

Previous studies have shown that there is a strong correlation between radiologists' diagnoses and their gaze when reading medical images. The extent to which gaze is attracted by content in a visual scene can be characterised as visual saliency. There is a potential for the use of visual saliency in computer-aided diagnosis in radiology. However, little is known about what methods are effective for diagnostic images, and how these methods could be adapted to address specific applications in diagnostic imaging. In this study, we investigate 20 state-of-the-art saliency models including 10 traditional models and 10 deep learning-based models in predicting radiologists' visual attention while reading 196 mammograms. We found that deep learning-based models represent the most effective type of methods for predicting radiologists' gaze in mammogram reading; and that the performance of these saliency models can be significantly improved by transfer learning. In particular, an enhanced model can be achieved by pre-training the model on a large-scale natural image saliency dataset and then fine-tuning it on the target medical image dataset. In addition, based on a systematic selection of backbone networks and network architectures, we proposed a parallel multi-stream encoded model which outperforms the state-of-the-art approaches for predicting saliency of mammograms.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Institute of Electrical and Electronics Engineers
ISSN:	1520-9210
Date of First Compliant Deposit:	1 April 2023
Date of Acceptance:	19 March 2023
Last Modified:	30 Jun 2024 22:44
URI:	https://orca.cardiff.ac.uk/id/eprint/158274

Citation Data

Cited 2 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item