Retrieving and classifying affective Images via deep metric learning

Yang, Jufeng, She, Dongyu, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680 and Yang, Ming-Hsuan 2017. Retrieving and classifying affective Images via deep metric learning. Presented at: AAAI Conference on Artificial Intelligence, New Orleans, Louisiana, USA, 2-7 February 2018.

Preview

PDF - Accepted Post-Print Version Download (1MB) | Preview

Abstract

Affective image understanding has been extensively studied in the last decade since more and more users express emotion via visual contents. While current algorithms based on convolutional neural networks aim to distinguish emotional categories in a discrete label space, the task is inherently ambiguous. This is mainly because emotional labels with the same polarity (i.e., positive or negative) are highly related, which is different from concrete object concepts such as cat, dog and bird. To the best of our knowledge, few methods focus on leveraging such characteristic of emotions for affective image understanding. In this work, we address the problem of understanding affective images via deep metric learning and propose a multi-task deep framework to optimize both retrieval and classification goals. We propose the sentiment constraints adapted from the triplet constraints, which are able to explore the hierarchical relation of emotion labels. We further exploit the sentiment vector as an effective representation to distinguish affective images utilizing the texture representation derived from convolutional layers. Extensive evaluations on four widely-used affective datasets, i.e., Flickr and Instagram, IAPSa, Art Photo, and Abstract Paintings, demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods on both affective image retrieval and classification tasks

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Completion
Status:	Unpublished
Schools:	Computer Science & Informatics
Related URLs:	Organisation
Date of First Compliant Deposit:	21 November 2017
Last Modified:	03 Nov 2022 10:05
URI:	https://orca.cardiff.ac.uk/id/eprint/106882

Actions (repository staff only)

Edit Item